CallCredit Gets Five Years of Performance Credit with Fusion ioMemory Solutions

The Challenge

Callcredit Information Group is one of three credit reference agencies in the UK that provides consumer credit reports to both businesses and consumers. It also provides pre-processing services and batch scoring of data for predictive risk analysis. This work requires running large and often complex batch processes against an MS SQL Server database backed by a 70-disk SAN.

Callcredit’s customers faced tight deadlines, but frequently delivered input files as late as possible to ensure the most current data. This made Callcredit’s task of quick turnaround extremely challenging. System Architect Adam Buckley knew that if Callcredit could decrease turnaround times, it would greatly improve the value of its services. Additionally, Callcredit’s rapidly growing client base required a significant increase in capacity.

Adam’s team analyzed the system and determined that improving batch-processing speeds would require eliminating bottlenecks in random read I/O and CPU processing. Callcredit could meet its immediate performance needs conventionally, by adding five racks of disks to its SAN. However, this was an expensive stopgap that would require additional purchases within six months. Adam and his team decided to investigate more innovative solutions that would future-proof performance without the hardware and operational cost scale out.

The SanDisk® Solution

Internet research convinced Adam’s team that Fusion ioMemory ioDrive® cards were the optimal option.

Benchmarking the System
Callcredit’s system consisted of a single database server backed by a conventional SAN. Adam benchmarked this system against a Fusion ioMemory-based system, consisting of a database server hosting four 640GB Fusion ioMemory ioDrive Duo cards backed by two additional servers to offload processing to 48 additional cores. The results are shown below.

Proof in Practice
The benchmark tests convinced Callcredit that the SanDisk solution would completely eliminate its I/O bottleneck, so Adam decided to put the ioDrive Duo cards to the real test—running an actual batch process against a production- ready system.

Callcredit’s SAN-backed system averaged 6,000 to 7,000 IOPS running the batch process and never broke 7,500 IOPS. The batch process on the new system sustained over 16,000 IOPS with peaks in excess of 25,000 to 35,000 IOPS.

But Adam quickly realized that with the I/O bottleneck removed, Callcredit could now fully utilize the 80 core CPU system. With just a few weeks of query optimization, Callcredit improved sustained performance to 23,000 IOPS and peak performance to 46,000 IOPS.

Adam said, “We run jobs two to four times faster than we used to. This means we can run more and more complex jobs in the same time, providing better service to our customers. In addition to improving batch-processing speed, SQL Server operations such as table scans (for analysis type queries and re-indexing) are substantially quicker. Also, certain file operations such as merging lots of fragmented text files (with many file extents) are about ten times quicker on the ioDrive cards.”

The phenomenal speed of the ioDrive Duo cards shifted the performance bottleneck to the CPU, maxing 80 AMD Opteron cores. This now gives Callcredit an easy upgrade path when it needs more performance.

“Our system’s primary constraint is now the SQL Server CPUs,” Adam continued. “And this can be cheaply addressed as processors improve and drop in price. We expect the ioDrive Duo cards can easily quadruple performance with this bottleneck removed, allowing us to just upgrade instead of scaling out our SAN with additional disks, racks, and so forth.”

Maximizing System Efficiency and Performance
Callcredit moved its entire database onto a single database server supported by two servers to which it could offload processing to alleviate the CPU bottleneck. The SanDisk-equipped system fully utilized 80 CPU cores, and Adam estimated it could max at least 80 more. Callcredit now utilizes this area on the SAN as second tier storage, providing a resilient hot standby to the high-performance primary tier provided by the ioDrive Duo cards.

The SAN upgrade Callcredit originally evaluated would have merely met, rather than exceeded performance needs, while adding five shelves of disk to its infrastructure. Adam expected Callcredit would need to add even more shelves of disks in the near future.

The Best Buy
As if doubling the performance of batch jobs, gaining an easy upgrade path, and providing performance headroom for 60 months weren’t enough, the ioDrive Duo card-equipped system cost Callcredit half as much as the SAN upgrade.

In addition to the upfront costs, Callcredit saved significantly on power, cooling, floor space, and maintenance overhead. Instead of adding five shelves of disks (and many more over the next 60 months), Callcredit added just two servers.

“The SanDisk solution cost 50% of the cost to upgrade the SAN,” said Adam. “The other solutions we looked at were considerably more expensive than a SAN upgrade. On top of that, since we’ve maxed out performance, we save big on development resources we no longer need to allocate to optimizing application performance.”

Summary

Implementing the Fusion ioMemory solution gave Callcredit the following benefits:

• 2–4x faster batch processing
• 1/2 the cost of a minimal SAN upgrade
• 5-year estimated growth headroom
• 25x higher IOPS
• 10x more bandwidth
• 4x capacity from SAN disks that no longer need striping

Adam is thrilled with the performance and performance headroom the ioDrive Duo cards provide. “The new ioDrive Duo tier takes the performance burden off our SAN, providing much more performance in a much smaller space. Our SAN is now a backup and storage device that serves as a failover system.”

About Callcredit

Callcredit Information Group enables organizations to make better decisions throughout the customer management lifecycle by combining a clear understanding of its clients’ needs, the latest technology, innovative products and sophisticated, real-time data models. This breadth of expertise and data enables Callcredit to offer comprehensive information solutions to a diverse range of international businesses and organizations.