By all indications, today’s data centers are actively embracing snapshots for backup.  While most are using this to augment traditional backups, several are actively replacing their traditional backup solutions with snapshot-based backup and recovery solutions.

Check out these survey results from a survey Gartner conducted. Nearly two-thirds of the respondents stated that they have plans to augment backup software with snapshot and replication solutions, and one-third of those polled indicated that they plan to replace backup software with snapshots and replication:

Source: “The Future of Backup May Not Be Backup”
Gartner ID Number: G00218917 Author: Dave Russell Date: 22 Sep, 2011

What triggers this trend?  Simply put, snapshots can be generated quickly and frequently, yet, there is none of the overhead of resource requirements typical of traditional backup. Most of all, recoveries are instantaneous and painless.  Combining snapshots with bandwidth-efficient replication provides additional protection, safeguarding against local site outages. For a detailed discussion on why traditional backup methods are failing to meet recovery needs, refer to this discussion

What’s not to like?  Well, here’s the big caveat. NOT ALL SNAPSHOTS ARE CREATED EQUAL. Moving the onus of backups to a storage system that isn’t optimized for this foundational capability would be like leaping from the frying pan to the fire.

Firstly, in order to allow frequent recovery points, snapshots should be inexpensive both in terms of storage capacity and performance.  Otherwise, you could end up consuming expensive storage capacity. Not to mention bogging down performance of the array.  Secondly, the implementation needs to be able to support a large number of snapshots to truly support frequent recovery points. (See here for further discussion on snapshot implementations.) So make sure to read the fine print and ask the hard questions.

Rounding out the Solution

While snapshots can greatly improve backup efficiency, there is tangible value in combining this foundational capability with an intelligent backup management system for an end-to-end solution.

This is what the Nimble-Commvault partnership is centered around.

Commvault has been one of the first vendors to recognize and embrace the industry trend towards snapshots. The Commvault IntelliSnap Connect program facilitates a data and information management approach that leverages hardware based snapshot and replication technologies.

Following is what the joint solution would deliver to end customers:

• Centralized management that allows admins to configure, execute, and monitor backup and restore operations, as well as perform monitoring and auditing from a single console
• Support for long-term retention, compliance and e-discovery through archival to tape and virtual tape media
• Integrated catalog that allows indexing and tracking of snapshot copies across primary, secondary, and other forms of media
• Granular recoveries of snapshots, application objects, files, VMs, and volumes
• Reduced cost and complexity through elimination of redundant server, networking and storage resources required by traditional backup architectures
• Frequent recovery points irrespective of the volume of data to be protected
• Rapid restores that require no data movement
• Support for app and VM consistent recoveries speeding up application and VM restore times

Stay tuned for more on this front.

Ajay Singh
Vice President, Product Management

The Escher Stairs of Efficiency Claims

An end user bombarded by the many efficiency claims made by storage vendors might be forgiven for being confused, skeptical, or both. How is it possible for so many vendors to claim they deliver storage with X% lower cost than other vendors? For all these claims to be true, the storage world would have to be the real world equivalent of M.C. Escher’s  mind-bending Penrose Stairs. What’s really going on here?

Comparing Storage Efficiency

Well, the problem is that most such claims are based on simplistic comparisons, such as only comparing capacity efficiency (usable capacity/raw capacity). Or just comparing raw performance.  And even these are often inflated with unrealistic assumptions.

While interesting, such one-dimensional comparisons are typically only useful for niche applications such as archiving or HPC.  For mainstream applications you typically care about multiple dimensions of a storage solution such as price/performance, data protection, availability and capacity efficiency. Knowing this, the question then is – how does one construct more meaningful comparisons?

A Better Comparison

Assuming many solutions meet your threshold of reliability and availability, here are some dimensions of storage efficiency you might consider in comparing them:

· Capacity AND Performance Efficiency

A basic definition of capacity efficiency (usable capacity/raw capacity) can be too simplistic for a couple of reasons. Often it ignores capacity savings techniques like inline compression and cloning. More importantly, it ignores the inherent performance differences between architectures. If you could get 50% compression without a performance impact, that’s certainly nice. But if you could get the performance of high performance drives (15K rpm disks, or better, flash SSDs) and the capacity of high density drives (7.2K rpm disks) in a single tier of storage – that’s HUGE! When you consider 15K RPM drives cost 500% more per GB than 7.2K RPM drives, the above example translates to a 500% capacity advantage from the get go!  To capture such differences, a meaningful comparison of efficiency ought to consider both $/GB AND $/IOPS.

· Data Protection Efficiency

The most visible elements of efficient data protection are the capacity efficiency of backup storage (e.g. dedupe ratios), and the bandwidth and capacity efficiency of DR storage. It’s less common to see quantitative comparisons of the level of data protection – namely the RPOs and RTOs enabled by the system although these translate to very real and potentially big costs. And then there’s another part which is sometimes overlooked and typically harder to quantify: operational efficiency, in other words how easy is it to setup and manage backups and DR on a day to day basis. More on this topic next.

· Operational Efficiency (i.e. Simplicity)

This is the dimension that is hardest to measure, but no less important to consider. Operational Efficiency encompasses qualitative attributes like simplicity – can an admin just install and start using a storage technology without days of training, professional services and years of experience? Does the performance adapt quickly to changing workloads? Quantitative measures might be the time (or number of steps) required for common tasks.

There’s another reason to pay close attention to operational efficiency – it helps you distinguish truly efficiently designed storage solutions from less efficiently “bundled” ones. Here’s a hypothetical example to illustrate:

What if you had a shrink-wrapped solution that bundled a small amount of expensive but fast storage together with a lot of cheap but slow storage. And also threw in some software to slowly move data back and forth – to relocate the right data on the right tier. And some more software to do the same for backup purposes. On paper such a solution can appear to have it all– good $/IOPS, good $/GB and automation to simplify management. So what could be missing – potentially a LOT!

If the data transfer process is slow and heavy duty – it might take hours to complete and impact performance while it’s happening. And since application workloads change dynamically, you’d be constantly monitoring workloads and over-allocating performance tiers to ensure bursty applications don’t experience bad performance for extended periods. Despite this, it’s virtually certain that some applications would experience poor performance. As for backups/restores – you’d be constantly battling backup windows and dealing with poor recovery points and slow, painful restores. So in reality, such a package would deliver much less than the sum of its parts.

What This Means for You

Not every application needs a multi-dimensional, well balanced storage solution. Perhaps for an archive tier $/GB is the one over-riding concern. Or maybe for a critical application you’re willing to pay a lot for performance, even if it means compromising on capacity and efficient data protection.  However the vast majority of mainstream applications need more versatile storage solutions.

One approach to picking the right one is to assign explicit weights to your criteria: for example capacity efficiency, performance efficiency, data protection efficiency and operational efficiency might be all equally important in your environment and deserve equal weights. You can then compare storage solutions under each of these four criteria and rate each on a scale of 1-5. The overall weighted rating would give you a much better measure of storage efficiency for your applications than anything vendor marketing materials could. In upcoming blogs we will share real world data on how Nimble does on each of these criteria.

Umesh Maheshwari
Co-Founder and CTO

Flash memory shines on reads: it reads 100 times faster than a disk. But its performance advantage is much weaker on writes, and its write endurance is much lower than disk’s. Therefore, Nimble OS uses flash only for accelerating reads, aka “read caching”. It uses NVRAM (a DRAM-based device) for accelerating writes, aka “write caching”.

On the other hand, a few storage systems use flash memory for write caching. Here I describe what compels these systems to use flash in this manner and the cost-benefit tradeoff it entails.

In general, storage systems implement write caching using a non-volatile “write buffer.” On a write request, the system stores the data into the write buffer anacknowledges the request. In the background, as the buffer fills up, the system drains the buffer to the underlying storage. The speed at which the write buffer can be drained to underlying storage constrains the sustainable write throughput.

The write buffer helps in following ways:

1. It enables the storage system to acknowledge a write request with very low latency.
2. It can absorb a high-throughput burst of writes, while it drains less speedily to disk-based storage over a longer period of time.
3. It absorbs overwrites (multiple writes to the same blocks), thereby reducing the amount of drainage, which may support a higher write throughput.
4. It allows the data being drained to be sorted by logical addresses, thereby improving the sequentiality of drainage, which may improve the speed of draining and support a higher write throughput.

The latency advantage depends on the buffering medium. NVRAM (DRAM made non-volatile with battery backup or flash backup) provides latency of a few tens of microseconds. Flash a few hundreds of microseconds. Disk a few milliseconds. Most storage systems use NVRAM for write buffering. However, file systems that are not tied to a hardware platform cannot assume the availability of NVRAM, and may buffer writes on flash or even on disk. E.g., the write buffer in ZFS, called ZFS Intent Log (ZIL), is generally stored on flash or disk.

A few storage systems now use flash as a secondary write buffer in addition to using NVRAM. E.g., EMC “FAST cache” uses flash as both a read cache and a write buffer. In such systems, written data is staged through the NVRAM-based buffer, the flash-based buffer, and finally to disk. The flash-based buffer is much bigger than the NVRAM-based buffer, and therefore provides higher levels of burst absorption, overwrite absorption, and sequentiality improvement, which in turn may support a higher write throughput. These advantages are based on the assumption that the NVRAM-based buffer cannot be drained directly to disk-based storage at high throughput.

Most storage systems employ a simplistic disk layout such that draining the write buffer results in random writes on disk. Furthermore, these systems amplify the IO load in order to support parity RAID and copy-on-write snapshots. The resulting load cripples the speed at which data can be drained to disk. (NetApp’s WAFL performs better by concatenating random data blocks and writing them into free space, but it too degenerates gradually as the free space becomes fragmented.) Because these systems cannot drain to disk at high speed, they stand to benefit from adding a larger write buffer. Even so, this benefit is limited because it does not eliminate random writes to disk—it only reduces them by some modest amount.

Furthermore, many of these storage systems could instead use a disk-based write buffer, which would be similar to a write-ahead log used in database systems. The log is written sequentially, which disks perform just as well as flash drives (about 100MB/s per drive). One advantage of a flash-based buffer over a disk-based buffer is that it also serves as a read cache for newly written data. However, as described later, there are cheaper ways of building a read cache. Another advantage is that the draining process can read the flash-based buffer in random order, so it supports a more thorough sorting of the data, thereby extracting more sequentiality.

Now consider the cost of write buffering. A flash-based buffer is expensive. First, because it holds the only copy of newly written data, it must employ the more expensive forms of flash and controllers, and also some RAID-like redundancy in the form of parity or mirroring. (In fact, a flash-based buffer needs to be even more reliable than an NVRAM-based buffer, because it is larger and the overwrite-absorption and re-sorting might make it difficult to recover the system to a consistent state upon loss.) On the other hand, a read cache does not ever store the only copy of any data, so it can be constructed inexpensively without sacrificing reliability: add a checksum to every block, verify the checksum on every read, and toss the cached block if the checksum does not match. Second, pushing the writes through flash burns through its limited write endurance, again requiring expensive, high-endurance, flash. Third, to obtain a significant edge over NVRAM-based log, the flash-based log must be much bigger. E.g., it may need to be large enough to absorb all writes during a busy period lasting hours.

The questionability of using flash as a write cache for disk is epitomized by a research paper, Extending SSD Lifetimes with Disk-Based Write Caches, which states the following:

“We present Griffin, a hybrid storage device that uses a hard disk drive (HDD) as a write cache for a Solid State Device (SSD).”

In other words, the authors are proposing just the opposite of using a flash-based write cache for disk! These authors are reputable researchers from the academia and Microsoft Research, and they exhibit a deep understanding of flash characteristics as a storage medium. There are practical issues with following their proposal, but the mere existence of this proposal questions the wisdom of using flash for write caching.

Nimble’s CASL™ filesystem uses the entire disk storage as a log, and always writes data to disk in large sequential chunks. This enables it to drain data from NVRAM buffer to disk storage at high throughput. This avoids the need for a secondary write buffer. It is as if the entire disk subsystem is at once a write buffer and the end point of storage.

In summary, flash-based write caching addresses burst throughput but only partially improves sustained throughput, while a write-optimized disk layout addresses both with little cost. However, systems with legacy disk layouts are forced to cache writes in flash as a costly fix to improve their write performance partially.

Nimble recently completed a survey of 599 respondents regarding business drivers and challenges in deploying VDI. While the interest in VDI continues to grow, costs and performance were flagged as the biggest storage-related challenges impeding VDI deployments.

Those of us who are familiar with VDI would agree that this is not all that surprising. Storage performance heavily determines the responsiveness of virtual desktops and if the user experience is diminished, users will not accept VDI. And in these times of tightening budgets, costs are always subject to close scrutiny.

On closer analysis though, you realize that addressing both cost and performance together is a paradoxical problem to overcome with traditional storage solutions. Let’s see why.

VDI has some unique workload characteristics. At steady state, VDI behaves predictably with IOPS tied closely to the profile of desktop workloads being run. However, in the course of a normal day, VDI infrastructure also goes through boot-storms and login storms (the period when multiple desktop users try to boot or log in at the same time) which cause a peak in read IOs. There are also virus scanning and OS upgrade operations that occur from time to time, which triggers a spike in writes.

Traditional storage wisdom recommends throwing flash and expensive high-RPM drives to provision for these peak scenarios.

Wouldn’t that then cause storage costs to shoot up? So how does one get around this conundrum?

It would seem the crux of the problem comes down to efficiency. The Oxford dictionary defines efficiency as achieving maximum productivity with minimum wasted effort or expense. In other words if we can meet the performance demands of VDI “efficiently,” that would by definition mitigate the cost challenge.

While efficiency has been used in the storage industry predominantly in the context of capacity efficiency (i.e. $/GB), it is equally critical to focus on performance efficiency as well (i.e. $/IOPS). A combination of those two sets of efficiencies would result in “affordable performance.” Unfortunately, most storage systems today tend to be optimized for one or the other, not both.

But wait—that’s not all. We just talked about the tendency of VDI IO’s to fluctuate throughout the day. Clearly there is more to performance efficiency that goes beyond purely $/IOPS. A truly efficient solution needs to be able to deliver performance when needed without incurring high overheads i.e. “adaptive” performance. In essence, what you really need is “affordable” and “adaptive” performance.

Unfortunately traditional tiered solutions fall short in this regard owing to the complexity and overhead associated with data classification, data movement, and the granularity of data movement. One could construct a system that is highly optimized around $/GB and $/IOPS, but has data trapped in low performance tier, rendering the system unresponsive to fluctuating workload demands.

Can you architect a system that truly delivers “affordable” and “adaptive” performance? The answer is yes and it comes down to not just what resources the storage solution leverages for performance, but how it leverages those resources.

Let’s shift gears and look at real-world examples of customers who have successfully deployed VDI, and what has worked for them. One approach is to start out with VDI by consolidating virtual desktop workloads with other workloads over the same storage infrastructure.

No longer do you have to purchase silo’d infrastructure that has to be managed separately, thus cutting down on both capex and opex costs.

Of course, the storage array needs to be able to deliver “adaptive, affordable performance” and simplified management to effectively handle the consolidated workload.

And once you are ready to scale up to higher desktop numbers, rinse, repeat.

One of our customers called the other day and asked me to explain our Non-Disruptive Software Updates. I gave him the usual basics about our two-click process of download and update directly from the Array management GUI; you know, no downloading to your PC and copying files here or there – just a couple clicks.

Then I continued with a technical explanation about the update process itself. The first step being a set of high-availability health-checks run by the update process to ensure the system and networking is in proper working order. Then how the standby controller unpacks the software image in a new location and reboots the new version into standby mode. Then the active controller unpacks, reboots and is taken-over by the standby. All unbeknownst to the applications.

Since he was a new customer, I felt compelled to give him a few stats on our latest version of GA software to help put him at ease. We announced our latest version of Nimble OS two weeks earlier and had 170 systems already updated when the customer called me. Of those systems, 55% were updated during their prime time while serving production data. And 100% of all systems had no service disruption to any application.

Now, I had to be completely honest and let him know there were 10 systems that took an extra step to update. A built-in high-availability health-check process had determined network connectivity mismatches on the standby controller could have caused interruption or degradation of service to applications after failover. A quick reconfiguration of the customer’s network and the updates were once again off to the races.

By the way, the customer hit the update button about halfway through that last sentence…