IT professionals need to demand that storage vendors repeal the deduplication tax so organizations can finally benefit from the ever-decreasing cost of technology. The deduplication tax is a byproduct of executing a very heavy algorithm, in-line, to extract maximum capacity from storage media. Storage systems with deduplication require much more powerful storage servers and more RAM in those servers. These systems also require more drives to meet performance expectations than they should. The burden of deduplication means that the drives can’t perform anywhere close to the manufacturers’ specifications.
Can We Afford to Repeal the Deduplication Tax?
Deduplication enables all-flash vendors to make their arrays appear affordable to enterprise customers. When all-flash systems first appeared on the market, the cost of flash drives was 14 dollars per GB, but today, they cost less than 30 cents per GB. At $14 per GB, the deduplication tax might be justifiable, but at 30 cents per GB, it seems like now is the time to repeal the deduplication tax.
The Performance Impact of the Deduplication Tax
In our new white paper “Exposing the High Cost of Dedupe,” we take an in-depth look at the deduplication tax. In this entry, we will focus on the cost deduplication has on performance.
When vendors claim that deduplication doesn’t impact performance, they mean you won’t notice its impact. The reality is, you will. When deduplication is active, flash drives don’t perform as well as they should. Vendors will claim that it shouldn’t matter because their systems deliver hundreds of thousands more IOPS than, in theory, an organization will ever need. The problem is they provide performance by forcing you to purchase significantly more drives than you need.
Can You Have Too Much Performance?
The truth is that almost every organization can benefit from more performance and low latency. With excess performance, organizations can scale enterprise applications up, instead of out, reducing the “per-core” cost of software licensing. Organizations can also meet the demands of more workloads, thanks to excess performance, potentially reducing storage system count to one. Most importantly, without deduplication in the way, latency is reduced, and almost every application benefits from a reduction in latency.
The problem for AFA vendors is that counting on deduplication to make their storage system seem inexpensive means requiring 24 or more flash drives to deliver a couple hundred thousand IOPS. They force you to buy additional drives because the flash drives they install in their systems only extract about 20% of the drives’ performance potential, based on manufacturers’ specifications. While deduplication is just one of several reasons these vendors are only delivering 20% of the per drive performance, there is no doubt that it is a key factor.
Another problem for AFA vendors is to get even this meager performance, their systems must use a high-end server with large amounts of memory, which increases the deduplication tax rate even further.
Repeal the Deduplication Tax with Efficiency
To repeal the deduplication tax, customers need an efficient storage software solution that can generate maximum performance per drive from mid-range server hardware. While maximum drive performance requires more than just repealing the deduplication tax, it is a critical step.
Efficient storage software can deliver hundreds of thousands of IOPS but only needs 5 or 6 SAS flash drives. Efficiency means that you do not have to pay for 20 additional flash drives to meet your performance demands. While flash drives have come down in price, not having to buy 20 drives is going to save you more money than deduplication can ever hope to do.
Repealing the deduplication tax also saves money on the server and RAM spend. Efficient storage software can extract more performance per drive while also running on affordable mid-range server hardware with a minimal amount of RAM. The cost savings on a mid-range server versus a high-end server is significant.
Addressing the Capacity Issue
What if you need more capacity than what five or six drives will provide? First, remember that the density of flash drives is accelerating rapidly. 7.68TB flash drives are commonplace, 15.8 TB drives are readily available, and 50TB plus drives are less than a year away. A six drive system using 15.8TB drives delivers over 94TBs of raw capacity and still provides more than 300,000 IOPS.
Other AFA vendors are reluctant to leverage higher capacity drives because they can’t deliver the per drive performance potential. They need more than the minimum drives to provide acceptable performance, but they know you will question all the excess capacity. If the vendor went with twenty-four 15.8TB drives, their starter system would deliver over 379TBs of raw capacity, much more than many data centers need. Unfortunately, the system would still only provide about 20% of its performance potential. In other words, they don’t want you to repeal the deduplication tax.
As a result, inefficient storage software means that AFA vendors force you to buy smaller capacity drives, claiming they haven’t tested higher capacity drives yet. As a result, you end up with 24 or more low capacity drives in a system, and it still doesn’t perform up to its per drive performance potential.
Forcing you to use lower capacity drives also means restricting your expansion capabilities. If you are using a 24-bay storage server and you can use 12 15.8TB drives instead of 24 7.68TB drives, this means the system can expand your capacity by just adding drives instead of immediately adding another storage shelf. Avoiding adding a storage shelf is yet another cost savings. The deduplication tax actually makes upgrades much more expensive.
Conclusion
The time is now to repeal the deduplication tax. Deduplication makes a significant impact on flash drive performance and latency. In an attempt to counteract that impact, vendors force you to buy high-end servers, maximum RAM, and far more flash drives than you need. Efficiency enables you to repeal the deduplication tax, realize better performance, and surprisingly lower costs. EffIciency also enables features like unlimited snapshots that helps reduce the cost of backup infrastructure in addition to primary storage.
The impact on performance is just one of the problems with deduplication, to learn the rest download our white paper “Exposing the High Cost of Dedupe.“
