Joerg's Storage Blog

NetApp Deduplication An In-depth Look

2011-06-12T12:51:00.000-07:00

There has been a lot of discussion lately about the NetApp deduplication technology, especially on twitter. We had a lot of misinformation and FUD flying around, so I thought that a blog entry that takes a close look at the technology was in order.

But first a bit of disclosure, I currently work for a storage reseller that sells NetApp as well as other storage. The information in this blog posting is derived from NetApp documents, as well as my own personal experience with the technology at our customer sites. This posting is not intended to promote the technology as much as it is to explain it. The intent here is to provide information from an independent perspective. Those reading this blog post are, of course, free to interpret it the way they choose.

How NetApp writes data to disk.

First lets talk about how the technology works. For those who aren't familiar with how a NetApp array stores data on disk, here's the key to understanding how NetApp approaches writes. NetApp stores data on disk using a simple file system called WAFL (Write Anywhere File Layout). The file system stores metadata which contains information about the data blocks, has inodes that point to indirect blocks, and indirect blocks point to the data blocks. One other thing that should be noted about the way that NetApp writes data is that the controller will coalesce writes into full stripes when ever possible. Furthermore, the concept of updating a block is unknown in the NetApp world. Block updates are simply handled as new writes, and the pointers to the updated blocks are moved to point to the new "updated" block.

How deduplication works.

First, it should be noted that NetApp deduplication operates on a volume level. In other words,all of the data within a single NetApp volume is a candidate for deduplication. This includes both file data, and block (LUN) data that is stored within that Netapp volume. NetApp deduplication is a post-process that occurs based on either a watermark for the volume, or on a schedule. For example, if the volume exceeds 80% of it's capacity a deduplication run can be started automatically. Or, a deduplication run can be started at a particular time of day, usually at a time when the user thinks the array will be less utilized.

The maximum sharing for a block is 255. This means that if there are 500 duplicate blocks,there will be 2 blocks actually stored with 1/2 of the pointers pointing to the first block and 1/2 of the pointers pointing to the second block. Note that this 255 maximum is separate from the 255 maximum for snapshots.

When deduplication runs for the first time on a NetApp volume with existing data, it scans the blocks in the volume and creates a fingerprint database, which contains a sorted list of all fingerprints for used blocks in the volume. After the fingerprint file is created, fingerprints are checked for duplicates, and, when found, first a byte- by-byte comparison of the blocks is done to make sure that the blocks are indeed identical. If they are found to be identical, the block‘s pointer is updated to the already existing data block, and the new (duplicate) data block is released. Releasing a duplicate data block entails updating the indirect inode pointing to it, incrementing the block reference count for the already existing data block, and freeing the duplicate data block.

As new data is written to the deduplicated volume, a fingerprint is created for each new block and written to a change log file. When deduplication is run subsequently, the change log is sorted, its sorted fingerprints are merged with those in the fingerprint file, and then the deduplication processing occurs as described above. There are two change log files, so that as deduplication is running and merging the new blocks from one change log file into the fingerprint file, new data that is being written to the flexible volume is causing fingerprints for these new blocks to be written to the second change log file. The roles of the two files are then reversed the next time that deduplication is run. (For those familiar with Data ONTAP usage of NVRAM, this is analogous to when it switches from one half to the other to create a consistency point.) Note that when deduplication is run an an empty volume, the fingerprint file is still created from the log file.

Performance of NetApp deduplication.

There has been a lot of discussion about the performance of Netapp deduplication. In general, deduplication will use CPU and memory in the controller. How much CPU will be ustilied is very had to determine ahead of time, however in general you can expect to use from 0% to 15% of the CPU in most cases, but as much as 50% has been observed in some cases. The impact of deduplication on a host or application can very significantly and depends on a number of different factors including:

    •    The application and the type of dataset being used
    •    The data access pattern (for example, sequential versus random access, the size and pattern of the
    •    I/O)
    •    The amount of duplicate data, the compressibility of the data, the amount of total data, and the
    •    average file size
    •    The nature of the data layout in the volume
    •    The amount of changed data between deduplication runs
    •    The number of concurrent deduplication processes and compression scanners running
    •    The number of volumes that have compression/deduplication enabled on the system
    •    The hardware platform—the amount of CPU/memory in the system
    •    The amount of load on the system
    •    Disk types ATA/FC, and the RPM of the disk
    •    The number of disk spindles in the aggregate

The deduplication is a low priority process, so host I/O will take precedence over dedupllication. However, all of the items above will effect the performance of the deduplication process itself. In general you can expect to get somewhere between 100MB/sec to 200/MB/sec of data dedupication from a NetApp controller.

The effect of deduplication on the write performance of a system is very dependent on the model of controller and the amont of load that is being put on the system. For deduplicated volumes, if the load on a system is low—that is, for systems where the CPU utilization is around 50% or lower—there is a negligible difference in performance when writing data to a deduplicated volume, and there is no noticeable impact on other applications running on the system. On heavily used systems, however, where the system is nearly saturated, the impact on write performance can be expected to be around 15% for most models of controllers.

Read performance of a deduplicated volume depends on the type of reads being performed. The implicit on random reads is negligible. In early versions of ONTAP the impact of deduplication was noticeable with heavy sequential read applications. However with version 7.3.1 and above NetApp added something they called "intelligent cache" to ONTAP specifically to help with the performance of sequential reads on deduplicated volumes and were able to mitigate the performance impact of sequential reads nearly completely. Finally, with the addition of FlashCache cards to a controller, performance of deduplicated volumes can actually be better than non-deduplicated volumes.

Deuplication Interoperability with Snapshots.

Snapshots and their interoperability with deduplication has been a hotly debated topic on the internet lately. Snapshot copies lock blocks on disk that cannot be freed until the Snapshot copy expires or is deleted. On any volume, once a Snapshot copy of data is made, any subsequent changes to that data temporarily require additional disk space, until the snapshot is deleted or expires. The is true with deduplicated volumes as well as non-deduplicated volumes. Thus the space savings from deuplication for any data held by a snapshot prior to a deduplication run will not be recognized until after that snapshot expires or is deleted.

Some best practices to achieve the best space savings from deduplication-enabled volumes that contain Snapshot copies include:

    •    Run deduplication before creating new Snapshot copies.
    •    Limit the number of Snapshot copies you maintain.
    •    If possible, reduce the retention duration of Snapshot copies.
    •    Schedule deduplication only after significant new data has been written to the volume.
    •    Configure appropriate reserve space for the Snapshot copies.

Some Application Best Practices

VMWare

In general VMware deduplicates well, especially if a few best practices in laying out the VMDK files are considered. The following best practices should be considered for VMware implementations:

    •    Operating system data deduplicates very well therefore you should stack as many OS's onto the same volume as possible.
    •    Keep VM swap files, pagefiles, user and system temp directories on separate VMDK files.
    •    Utilize FlashCache where ever possible to cache frequently accessed blocks (like those from the OS).
    •    Always perform proper alignment of your VM's on the NetApp 4K boundaries.
    •

Microsoft Exchange

In general deduplication provides little benefit for versions of Microsoft Exchange prior to Exchange 2010. Starting with Exchange 2010 Microsoft has eliminated single instance storage and deduplication can reclaim much of the additional space created by this change.

Backups (NDMP, SnapMirror and SnapVault)

The following are some best practices to consider for backups of deduplicated volumes:

    •    Ensure deduplication operations initiate only after your backup completes.
    •    Deduplication operations on the destination volume complete prior to initiating the next backup.
    •    If backing up data from multiple volumes to a single volume you may achieve significant space savings from deduplication beyond that of the deduplication savings from the source volumes. This is because you are able to run deduplication on the destination volume which could contain duplicate
    •    data from multiple source volumes.
    •    If you are backing up data from your backup disk to tape consider using SMTape to preserve the deduplication/compression savings. Utilizing NDMP to tape will not preserve the deduplication savings on tape.
    •    Data compression can affect the throughput of your backups. The amount of impact is dependent upon the type of data, compressibility, storage system type and available resources on the destination storage system. It is important to test the affect on your environment before implementing
    •    into production.
    •    If the application that you are using to perform backups already does compression, NetApp data compression will not add significant additional savings.

Conclusions

In general, NetApp deduplication can help drive down the TCO of your storage systems significantly, especially when combined with FlashCache in a VMware or Virtual Desktop environment. If best practices are followed carefully, the performance impact of deduplication is negligible, and the space savings for some applications can be considerable. Some careful planning and testing in the customers environment are necessary to ensure that maximum advantage is taken of deduplication, however the ability to schedule when the operations take place combined with the ability to turn on and off deduplication provide significant flexibility in to tune the environment for a customer's particular application profile.

EMC FAST and NetApp FlashCache a Comparison

2011-05-30T22:51:00.000-07:00

Introduction

This article is intended to provide the reader with an introduction to two technologies, EMC FAST and NetApp FlashCache. Both of these technologies are intended to improve the performance of storage arrays, while also helping to bend the cost curve of storage downward. With the amount of data that needs to be stored increasing on a daily basis, anything that addresses the cost of storage is a welcome addition to the data center portfolio.

EMC FAST

EMC FAST (Fully Automated Storage Tiering) is actually a suite made of of two different products. the first, called FAST Cache operates by keeping a copy of "hot" blocks of data on SSD drives. In effect it acts as a very fast disk cache for data that is currently being accessed while the data itself is being stored on either 15K SAS or 7200 RPM NL-SAS (SATA) drives.

FAST Cache provides the ability to improve the performance of SATA drives, as well as to turbo charge the performance of fiber channel and SAS drives as well. In general, this kind of technology helps to divide performance from spindle count, which helps drive down the number of drives required for many workloads, thus driving down the cost of storage, and the overall TCO of storage.

The other product in the FAST suite is FAST Virtual Pool. This is the product that most people associate with FAST since it is the one that leverages three different disk technologies, SSD, high speed drives such as 15K RPM SAS, and slower high capacity drives such as 7200 RPM NL-SAS. By placing only data that requires high speed access on the SSD drives, data that is receiving a moderate amount of access on the 15K SAS drives, and putting the rest on the slower, high capacity disks EMC FAST is able to drive the TCO of storage downward.

NetApp FlashCache

NetApp approaches the overall issue of improved performance while simultaneously driving down the TCO of storage in a different way. NetApp believes that using fewer disks to store the same amount of data is the best way to drive down TCO. Therefore NetApp has spent a significant amount of time developing storage efficiency tools to help their customer's store more data in less space. For example, they developed a variant of RAID-6 called RAID-DP which provides the protection and performance of RAID-10, while utilizing significantly less space. NetApp has also developed block level de-duplication which can be utilized with primary production data.

However, as with many technologies of this type there could be a performance penalty paid for it's utilization. Therefore, Netapp needed to develop a way to improve the performance if it's arrays while also supporting it's storage efficiency technology. With the advent of Flash memory, Netapp found a way to do this without any need for significant changes in the architecture of it's arrays. Thus was born FlashCache.

FlashCahce provides a secondary read cache for hot blocks of data. This proves a way to separate performance from spindle count, and thus not only allows workloads intended for Fiber Channel or SAS drives to potentially run on SATA drives, but it also addresses some of the performance issues with the storage efficiency technologies that NetApp developed. For example, with FlashCache utilized in a virtual desktop environment Netapp de-duplication allows many individual Windows images to be represented in a very small footprint on disk. However a problem arrises when a large numer of desktops all try to access their Windows image at once. However with the addition of FlashCache, most, if not all of the Windows image would end up being storage in Flash memory, thus avoiding the performance issue of a boot storm, virus checking storm, etc.

Conclusion

Both EMC and Netapp have developed ways to help both improve the performance, and drive the TCO of storage downward. the two vendors approached the problem is somewhat different ways, but in the end they have both solved the problem in unique and effective ways.

The NetApp technology requires that the user buy-in completely to the NetApp vision of storage efficiency. If the user ignores the advantages of de-dupication in particular, or has data or workloads that simply don't allow for the application of the NetApp storage efficiency technology then the TCO saving that NetApp promises will not be achieved. Utilizing FlashCache to seperate performance from spindle count is also critical in maintaining the performance of the array. This separation of performance from spindle count also in and of itself drives dwn the number ofd drives needed to support a workload, and thus also drives down the TCO.

The EMC technology requires a very good understanding of your application workloads, and careful planning and sizing of the different tiers of storage. EMC could do more to make the two sub-products work together so that a single solution could provide both the TCO and the performance improvements at the same time. However, EMC FAST is a product that provides the TCO improvement promised, and doe it with a clean and elegant solution.

Finally, a little on the future. With the cost of Flash memory coming down 50% year over year, it will soon reach the same price point that we currently see 15K HDD's at. Once that happens one has to wonder what role 15K HHDs will fill? If 15K HDDs are, indeed, squeezed out of existence by this reduction in the price of Flash memory, what purpose will 3 tiered automated storage tiering fill? Or, will the future simply be 2 tiers of storage, one that provides bulk capacity, and one that accelerates the performance of this bult capacity? if that predication is correct, then FAST VP will have a limited life, and FAST Cache and FlashCache will be the longer surviving technology.

Flash Storage and Automated Storage Tiering

2011-05-20T11:12:00.000-07:00

In recent years, a move toward automated storage tiering has begun in the data center. This move has been inspired by the desire to continue to drive down the cost of storage, as well as the introduction of faster, but more expensive storage in the form of Flash memory in the storage array marketplace. Flash memory is significantly faster than spinning disks, and thus it’s ability to provide very high performance storage has been of interest. However, its cost is considerable, and therefore a way to utilize it and still bend the cost curve downward was needed. Note that Flash memory has been implemented in different ways. It can be obtained as a card for the storage array controller, or as SSD disk drives, and even, as cache on regular spinning disks. However it is implemented, it’s speed and expense remains the same.

Enter the concept of tiered storage again. The idea was to place only that data which absolutely required the very high performance of Flash on Flash, and to leave the remaining data on spinning disk. The challenge with tiered storage in the way that it has been defined in the past was that it meant that too much data would be placed on very expensive Flash since traditionally an entire application would have all it’s data placed on a single tier. Even if only specific parts of the data at the file, or LUN level were placed on Flash, the quantity needed would still be very high, thus driving the costs of for a particular application up. It was quickly recognized that the only way to make Flash cost effective would be to place only the blocks which are “hot” for an application in Flash storage, thereby minimizing the footprint of Flash storage.

The issue addressed by automated storage tiering is that you no longer need to know ahead of time what the proper tier of storage for a particular application’s data needs to be. Furthermore the classification of the data can occur at a much more fine-grained block level rather than the file or the LUN as with some earlier automated storage tiering implementations.

Flash has changed the landscape of storage for the enterprise. Currently, Fash/SSD storage can cost 16-20X what Fiber channel, SAS, or SATA storage can cost. The dollars per GB model ends up looking something like the following:

However the IOPS per $ model looks more like this:

The impact on the tiered storage architectural model of Flash storage has been, in effect, to add a tier-0 level of storage where application data is placed that requires extremely fast random I/O performance. Typical examples of such data are database index tables or key lookup tables, etc. Placing this kind of data, which may only be part of an application’s data, on Flash storage can often have a dramatically positive effect on the performance of an application. However, due to the cost of Flash storage the question is often raised, how can data centers ensure that only data that requires this level of performance resides on SSD or Flash storage so that they can continue to contain costs? Furthermore, is there a way to put only the “hot” parts of the data in the very expensive tier-0 capacity, and leave less hot, and cold data in slower, less expensive capacity? Block based automated storage tiering is the answer to these questions.

Different storage array vendors have approached this problem in different ways. However, in all cases, the object is to place data at a block level, on tier-0 or Flash storage only while that data is actually being accessed, and then to store the rest of the data on lower tiered storage while the data is at rest. Note that this movement must be done at the block level in order to avoid performance issues, and to truly minimize the capacity of the tier-0 storage.

One approach used by several storage vendors is to move blocks of data between multiple tiers of storage via a policy. For example, the policy might dictate that writes always occur to tier-0, and then if that data is not read immediately it is moved to tier-1. Then if the data isn’t read for 3 months that data is then moved to tier-2. The policy might also dictate that if the data is then read from the tier-2 disk then it is placed back on tier-0 in case additional reads are required and the entire process starts all over again. Logically this mechanism provides what enterprises are looking for, minimizing tier-0 storage and placing blocks of data on the lowest-cost storage possible. The challenge with this approach is that the I/O profile of the application needs to be well understood when the policies are developed in order to avoid accessing data from tier-2 storage too frequently and generally moving data up and down the stack too often since this movement is not “free” from a performance perspective. Additionally, EVT has found that for most customers, data rarely needs to spend time in tier-1 (FC or SAS) storage, that most of the data ends up spending most of it’s live on the SATA storage.

Therefore as the cost of Flash storage continues to come down, the need for the SAS or Fiber Channel storage will continue to decline, and eventually disappear leaving just Flash and SATA storage in most arrays.

Another approach that at least one storage vendor is using is to avoid all the policy based movement and to treat the Flash storage as a large read cache. This places the blocks that are most used on tier-0, and leaves the rest on spinning disk. When the fact that the sequential write performance of Flash, SAS/FC, and SATA is similar is taken into consideration along with a controller that orders its random writes, this approach can provide a much more robust way to implement Flash storage. In some cases, it allows an application that would not normally be considered a good candidate for SAS or Fiber Channel storage to be able to utilize SATA disks instead. In general, this technique de-couples spindle count from performance thus providing more subtle advantages as well. For example, applications which has traditionally required very small disk drives so that the spindle could would be might (many, many 146GB FC drives, for example) can now be run on much higher capacity 600GB SAS drives and still provide the same, or better performance.

Overall, automated storage tiering is becoming a de-facto standard in the storage industry. However different storage array vendors have taken very different approaches to the implementation of automated tiering, but in the end the result is uniformly the same. The ability of the enterprise to purchase Flash storage to help improve the performance of their applications while at the same time continuing to bend the cost curve of storage downward.

Dell Buys 3PAR and Monolithic vs. Modular Storage

2010-08-16T12:58:00.000-07:00

Well, it’s been a while since I blogged, but something happened today that warrants comment.Dell has offered to buy 3PAR for about $1.1 billion. So, a number of my customers have called and emailed me asking what this all means? They want to know how I view the addition of 3PAR to Dell’s storage portfolio? What does this mean for the storage industry, and should they seriously start/stop looking at 3PAR? What about all this discussion about monolithic vs. modular storage? Is 3PAR really Tier-1 storage?

From a Sales Perspective

So, what does the fact that Dell has paid a lot of money to get 3PAR mean to those who are buying storage out there? Certainly 3PAR has been one of the innovators in storage ever since it appear back in 1999 bring things like thin provisioning and tiered storage to market. The question is, will Dell leave 3PAR alone as a business unit to continue to operate pretty much as they have in the past?

Obviously, the fact that 3PAR was on the block for sale says that they weren’t exactly burning it up, so I would expect Dell to make some changes. For example, 3PAR wasn’t the most channel friendly storage company in the world. They preferred to sell direct, especially to larger customers. I expect that this might change once Dell management starts to make more of the decisions at 3PAR. Dell depends a lot on the channel, and certainly they expect integrated sales. In other words, Dell expects that sales to their bigger clients be integrated between servers, storage, and desktops where possible, etc. HP and IBM tend to do the same thing. Once you let in the IBM server guy, for example, expect IBM storage to be right behind, and that and “integrated offering of servers and storage” will get pushed at the highest (CIO) levels of your organization.

My view of this is that it’s never a good thing, since HP, IBM, and now Dell have strengths and weaknesses in their different lines, and just because I happen to think that, say, HP servers are the best technical fit for me, doesn’t mean that HP storage is as well. I might think that Dell/3PAR is the right storage, but that doesn’t mean that Dell’s servers are really what I need. Don’t misunderstand here, I think that HP, IBM, and now Dell will have a lot of success selling an integrated solution to the top by touting cost savings, having a single throat to choke, and “integration” between the technologies. I think that this is a topic for another blog posting, so I’ll leave it here for now.

Where does 3PAR fit? Is it an Enterprise Array?

3PAR has traditionally marketed themselves as an Enterprise array which brings up a lot of discussion about what is and isn’t an Enterprise array. Some people have suggested that in order to be truly Enterprise an array needs to be Tier-1, monolithic, and be capable of supporting mainframe storage. Based on that definition, 3Par doesn’t qualify on a number of counts since it is a modular array that doesn’t support mainframe. Many people suggest that 3PAR fits in a new category called Tier 1.5. But certainly 3PAR plays at the upper end of the storage array space and competes with the EMC, IBM, HP, and HDS’s of the world for block based storage.

This begs the question is Tier 1.5 “good enough”? I’ve been arguing for some time, that for a lot of applications in today’s economic climate, that yes, Tier 1.5 is fine. That monolithic Tier 1 storage arrays are overkill for the vast majority of applications, and that the cost savings of a Tier 1.5 array is enough that for many, many, applications it is very attractive for customers who are looking to save on storage expenditures. There is also a school of thought that modular, perhaps federated, arrays are the wave of the future. That monolithic arrays will be around for some time, but that their share of the overall market will shrink down to a very small percentage. Again, this is a great topic for a future blog posting.

Will There Be Synergy?

Certainly the addition of 3PAR to the Dell fold fills a major gap in Dell’s storage portfolio. But it also might help 3PAR play in areas that it hasn’t been able to play in before. For example, 3PAR has never has a NAS offering, so the question is, can a combination of products from Dell, including 3PAR as the block storage underneath, provide a high end NAS solution from Dell? Also, now that Dell owns Ocarina, will this mean that 3PAR will have a de-dupe solution available? But it also raises some questions, such as what about Exanet? Will Dell turn them into just software that sits on top of 3PAR or EqualLogic hardware? Lots of questions to be answered here going forward, but certainly Dell has the pieces in place to provide added value to each of the individual components.

What about the EMC/Dell Relationship?

A lot of people predicted the end of the EMC/Dell relationship when Dell bought EqualLogic. That didn’t happen, Dell is still a major storage partner for EMC, and they still sell a lot of EMC arrays. So that begs the question, is the 3PAR purchase the death nell for the EMC/Dell partnership? Only time will tell, but certainly Dell is now in a much stronger competitive position against EMC than they were after the EqualLogic acquisition.

Path Management Software Recommendations

2010-04-14T13:56:00.000-07:00

I haven’t posted in a while; it’s been pretty nuts, which is a good thing if you’re in the business of selling computer hardware/software, but it puts a crimp on my free time to post blog entries.

Lately I have been asked a lot about a topic in storage management that I thought most companies have a solid handle on already. But I’m being asked more and more what my recommendations are around path management. I suspect that this has something to do with the path management changes in VMware vSphere causing people to readdress this topic for VMware, and ask if they should look at it on a wider level in the data center.

In the following blog entry I describe the current state of path management and outline some recommendations for the use/implementation of path management software in the data center.

Background

The history of path management is very wrapped up with the history of the storage array vendors. Prior to the advent of PowerPath from EMC, path management was very much a part of the operating system. OS’s such as IBM’s VM, DEC’s VMS, and other mainframe and mini-computer operating systems all had the ability to communicate with their storage via more than one path and to load balance the I/O across those paths. However, when EMC and other storage vendors started to move into the open systems world, where the OS’s had a “small system” background, they found that path management was not something offered by the operating system. Early version of MS Windows, and various flavors of UNIX all had no way to address storage across more than one path, or if they did, it was simply a failover path without the ability to load-balance the I/Os. In response to this situation EMC developed PowerPath, and other storage vendors such as Hitachi soon followed suit. At that time, the path management software developed by the storage vendors only supported that storage vendor’s particular arrays. In other words EMC’s PowerPath only supported EMC arrays, Hitachi’s HDLM software only supported Hitachi arrays, and so forth. While this allowed a customer to optimize the connections between their hosts and their storage, it also had the effect of locking the customer into a single vendors arrays simply because it became very difficult to support more than one vendor’s array on the same host, and switching vendors also became more difficult by adding a significant level of effort to the migration process since all of the path management software all had to be replaced as well.

This situation remained the same for a number of years, until EMC announced support for non-EMC vendors in their PowerPath product. This announcement was a part of EMC’s plans at the time to move from a pure hardware company to a more software driven business. Along with the announcement of PowerPath for other storage vendors, EMC also announced a set of APIs that would allow management of non-EMC arrays from their flagship Control Center product and several other smaller changes to help position EMC as a software company.

Unfortunately, these changes were never fully recognized in the EMC software, nor were the EMC Sales teams particularly enthusiastic about the move away from a focus on “big iron” hardware that they had made so much money with for such a long time. This left some of the EMC products, such as PowerPath, in a position where they had some support for other vendors arrays, but that support was not complete from either an array specific feature perspective, or in the case of PowerPath, from a array vendor perspective. For example, aside from support for their own arrays, EMC PowerPath provides support for HDS (and some of the OEMs of HDS products such as HP) and some IBM arrays as well. However, support for other significant array vendors in the market such as 3PAR, Compellant, NetApp, etc. is notably missing. As a matter of fact, EMC has not added support for any array vendor since their original announcement in 2003 of support for HDS, IBM, and HP.

Things began to change when Microsoft introduced MPIO as part of the Windows operating system starting with the Windows 2000/2003 versions of the software. Microsoft, having learned from those that went before them, decided to provide a standardized mechanism for path management, but at the same time they also allow the storage array vendors to provide a plug-in (call a DSM) if they wanted to add additional capabilities for path management to MPIO. SUN developed it’s version of MPIO, called MPxIO in 2003, IBM introduced it’s version of MPIO in 2002, Linux announced Device Mapper in 2005, and the last vendor to announce MPIO software as part of the operating system was HP which announced their version of MPIO in 2007. Note that HP had a non-OS integrated version of path management called PVLinks available since the early 1990s. Therefore, by 2005 virtually every operating system in use in the data center had path management built in and the need for array vendor supported software simply no longer existed in order to provide path management.

At the same time as all of the above was happening, a company called VERITAS as part of their VERIAS Volume Manager was developing one true independent piece of path management software. VERITAS was positioning itself as an OS and array independent storage management company, and therefore it developed it’s own suite of tools for volume management, a file system, and path management software called DMP (Dynamic Multi Pathing). DMP has had its issues over the years, but was particularly popular with SUN customers, if for no other reason than it came with Foundation Suite which was very popular with SUN Solaris customers.

VMware Path Management

All of the above addresses path management purely from a host and storage array perspective. However, with the introduction of VMware another player entered the path management landscape. From the beginning, VMware required users to utilize the path management tat was built into VMware. The path management software built into VMware 3.5 and older provided basic path management feature, specifically path failover, but did not provide load balancing or any array specific features. This provided users of VMware some ability to tolerate path outages, but defiantly limited VMware’s ability to provide for high I/O applications. This limit wasn’t the only reason that early versions of VMware didn’t support high I/O applications, but it certainly needed to be addressed should VMware ever want to be able to support these high I/O applications. With the introduction of vSphere, VMware has finally addressed this issue, and more. Much like with MPIO VMware has now introduced a mechanism for storage array vendors to provide a “plug-in” into the path management functions built into vSphere called a storage array type plug-in (SATP) for the new Native Multipathing Plugins (NMP) module. One of the first vendors to take advantage of this capability was EMC with their PowerPath/VE product which provides support for EMC arrays.

Recommendations for Current Approach

There currently is no panacea when it comes to picking an approach to path management in the data center. Once again it boils down to two philosophies. Do you want to try to utilize a single path management software, or do you want to utilize the path management software that is built into the operating system?

Option #1 – Utilize a single path management software

For this option, a single piece of software is chosen and then loaded on every host to provide path management. There are really only two options for this software. You can utilize Symantec (once VERITAS) DMP, or you can use EMC PowerPath. These are the only two products in wide distribution that provide multi-vendor array support and a wide variety of host support as well.

PROs

The main advantage to utilizing a single piece of software is management of the path management software. You have a single product that is well understood provided by a single vendor to manage. Theoretically this should reduce your management costs, and provide for a more reliable and stable environment.

CONS

The down side to this approach is the cost involved. Since the software is a purchased product, a license for every host in the datacenter must be purchased from the vendor, or some kind of enterprise license obtained. In either case, tracking of the software licenses, and general management of the software often offsets the cost benefits of having a single vendor for path management. The second major disadvantage of utilizing a single path management software product is that you are locked into the list of supported hosts and arrays that the vendor chooses to provide. In the case of PowerPath the list of non-EMC arrays is very short, and appears to show no signs of ever growing any larger. In the case of DMP, there is also a question of where that product is heading, and how much development in the form of additional array support Symantec intends to provide. This creates a situation where the path management policy can prevent the organization from purchasing a new array that might provide significant cost benefit advantages. Finally, the management of versions of the path management software, the switch firmware, the operating system, and the array firmware create a complex matrix and increase the cost of support significantly. It can also slow down the rollout of newer models of arrays, switches, and host operating systems delaying cost saving.

Option #2 – Utilize the path management software built into the operating system

For this option, the built in path management software supplied with the operating system is utilized, along with the DSM (where available) to provide path management for all of the arrays in the datacenter.

PROS

With this approach dependence on third party software to provide path management is eliminated, and the support matrix for the host OS, switch and array is reduced making support much more straightforward. With the addition of a DSM to provide array specific features, all of the capabilities of the array are made available, without the need for third part software or vendor lock in. The operating system vendor, with whom a close support relationship already exists, provides support for the path management software reducing the possibility of three way finger pointing in the case of a problem. Finally, the ability to support any array which you feel provides you with a business advantage in terms of saving costs, providing additional features, and/or additional speed is once again on the table rather than the short list of arrays supported by the third party path management software vendors.

CONS

Since each operating system provides it’s own path management software some additional learning and understanding of those different path management software products would need to be maintained by the appropriate support team (storage or OS). This could add some additional costs in terms of training and support.

RECOMMENDATIONS

At this point in time, it is my recommendation that path management software integrated with the operating system be utilized rather than third party software. I believe that the flexibility to utilize nearly any array on the market today to address any storage issues you might face without being locked into a particular vendor, or even a short list of vendors, far outweighs any minor added overhead involved with learning multiple path management software interfaces.

Virtual Computing Environment Coalition

2009-11-05T07:46:00.000-08:00

I was wondering how long it would take before these three decided to get together and try to push out the competition. For a customer that hasn’t really done anything with Virtualization due to the perceived risk with implementation this would be a tempting way to go since it would appear to reduce those risks. There are really three main parts to this announcement.

The Vblock Infrastructure Packages
This is the combined hardware and software offered by Cisco, EMC, and VMware which are made up of pre-packaged VMware, UCS, MDS, and EMC storage solutions and called a Vblock. As part of the announcement, three Vblock solutions were introduced:

Vblock 0 (available mid-year 2010) is an entry level system which includes Cisco compute and networking, EMC Unified Storage, and VMware vSphere software.
Vblock 1 is a mid-sized configuration that includes Cisco UCS, Nexus 1000v and MDS swicthes, EMC CLARiiON CX-4 storage, and VMware vSphere.
Vblock 2 is a high end solution that includes Cisco UCS, Nexus 1000v and MDS swicthes, EMC Symmetrix V-Max storage, and VMware vSphere.

VCE Services
There are a number of "pre-packaged" services offerings ranging from high level strategy, to actual implimentation services for the Vblock. These servcices will be delivered by a new company called Acadia which is a joint venture between Cisco and EMC. Note that a quick trip to Acadia.com indicates that Acadia will not be fully functional until 2010.

VCE Seamless Support
The three vendors have created a vutual support center which is a combination of people from each of the three. They have also created joint test labs and cooperative engineering groups, etc. to try and provide the customer with a single point of contact for support.

My experience has been that the support for Vmware solutions tends to break down around the interfaces between the components. For example, where the storage meets the server and VMware itself. When there is a problem, who do you call first? As it turns out managing the “interfaces” tends to fall on the heads of the customer's infrastructure team. They in turn need the skills and the knowledge to fill those gaps. THAT is a lot of risk to take on, especially in the beginning. The consolidated support part of this coalition would mitigate those risks since there is only one virtual support organization. The question in my mind as a customer would be, how up to speed are these support guys going to be in regards to those “gaps” on day one?

You also have to wonder how much leverage a customer ends up having when they are buying this kinds of unified solution? Since everything is unified, and there is only one game in town, you have to wonder how expensive something like this is going to be? Sure, EMC, CISCO, and Vmware are going to tell you that all of the goodness of the unification means that the solution is going to be a little more expense ... But look at what you’re getting for that extra money! ;-)

What I would be interesting in knowing is what other vendors like Microsoft, Brocade, IBM, NetApp and Hitachi are planning as a response, if anything? Perhaps a collation between VMware, DELL, Brocade, and NetApp to offer similar prepackaged solutions? Would Vmware refuse to join such a collation and turn away business? Or even if they didn’t join the collation there’s still nothing preventing the rest of those vendors from forming a collation and just buying Vmware as they need it or joing with Microsoft and offereing something based on Hyper-V instead of VMware. Could this be the start of “Coalition Wars”?

The real cost of storage

2009-04-05T14:27:00.001-07:00

What's the real cost of storage? I get asked this question all the time, and it's so difficult to answer because it really does depend on so many factors from storage team to storage team. What's really surprising to me is that I'm being asked the question at all. You would think that everyone who runs a storage organization would know exactly what that number is. Some people simply look at their budget and say "here you go, this is what it costs". But can you break it down? Do you know where all of that money is going, and why? I think that's really what people are asking. They know how much they are spending, but they want to know why and how they can save money. Certainly in these economic times storage managers are asked to do more with less while the data continues to grow. So that leaves them asking, how? How do I manage to address this growing pile of data with fewer people, less CAPEX budget, and more demands from the business around things like disaster recovery?

So how do you address the question? How do you do more with less? A lot of storage managers are looking at the cost per GB of their disks and asking, can I get this number down? I think that they can, but it may mean doing some things in different ways than they have in the past. Specifically, here are some things to look at.

Tiered Storage

Yup, I'm recycling that idea again. Getting data off expensive spinning disk and onto cheaper disk saves money, I think that's been well established and taking another look at how you are classifying your data is a worthwhile endeavor at this point in time. Why? Because things have changed in the last year or two, and those changes might have an impact on your data classification policies, so I think a review might be in order. For example, a few years ago when I was classifying data I used SATA disk pretty much just for dev/test and archive data. But things have changed, and now there's technology out there that will allow you to use SATA disk for some of your production workload. Some technology that will even allow you to use SATA drives for all but your most demanding workloads for that matter the IBM's new XIV are now available. So, another look at your tiering policies and the SATA technology that's available today is probably a good use of your time if you're looking to save some money.

Cost of Managing Your Storage

What does it really cost to manage your storage on a per GB basis? This is really the age old question of "how many TB of storage can a single storage admin administer?" that we have been asking for a long time. The answer to this question is critical since you probably aren't getting a whole lot more headcount right now, and you might even be asked to give some up. So how do you manage more disk space with the same or fewer people? First, you have to keep in mind all of the things that go into managing a TB of space. There's a lot more to it than just provisioning a TB to an application and then walking away, right? Here are a few examples of the kinds of things that go into managing a TB of space based on my experience:

Provisioning – This one is obvious, right? But you would be surprised how many people have immature processes and procedures around disk provisioning. How many people still manage their disks based on spreadsheets and command-line scripts making the process time consuming and error prone.
Backup/recovery – So you have to make sure that your data is protected, and that you can get it back should the need arise. This can be a time consuming effort, and one place that you can look for efficiencies that will save you money. It's also a place that people sometimes forget to account for when they are buying more disks. Don't forget that as you add disk capacity, you also have to add backup/restore capacity, and that means more tape, or backup disk, etc but it also means that you have to account for the increased load on your backup admins as well.
Disaster recovery – All of the same things I talked about above with backup/recovery also applies to DR.
Data migration – Sooner or later you're going to have to move this data around. Whether it's because the lease is up on an array, or you need to re-tier the data doesn't matter, what matters is that this can be a costly process in perms of people time, and sooner or later you're going to have to do it.
Performance management – At some point you always get that call "hey, our database is slow and we've looked at everything else and haven't found the problem, can you look and see if it's the disks?" Unless you have some very mature performance management processes in place, this tends to turn into a huge people time sink.
Capacity management – We all know that our data is growing, that's a given, so that means that we need to spend some time planning how we are going to address that growth. When are we going to have to make those new disk purchases, when will we have to buy a whole new array? What about the switches? Are we going to need to expand that environment when we bring in that new array as well?
Documentation – yes, that's right, I said it, documentation is an important part of managing your storage, and it can take up quite a bit of the storage admins time, but it has to be done.

So the question I always ask is, "how mature and efficient are your processes?" Do you have a high degree of automation around all of the above? What use are you making of technology to help you manage the processes above? If you have very mature processes, employ a high degree of automation, and make good use of technology to help you automate as many of those processes as possible, then you probably have done everything you can to drive down the cost of managing your storage. But now is a good time to take a look and see if you can improve any of those areas. For example, does my disk vendor really provide tools to make managing my disk arrays easier? Not just from a provisioning standpoint, but from the standpoint of all of the above. If not, maybe it's time to consider looking at another vendor, one that has better tools.

Let me leave you with a final thought in this area based on my experience. What I found when I was managing storage was that the cost of managing a TB of disk could easily meet or exceed the cost of buying that disk over the 3-4 year life of that disk. So, a myopic focus on who has the cheapest disks on a per GB basis may not make much sense. Perhaps what we should focus on is how much it costs to manage a TB of a particular vendor's disk. In other words, the 3-4 year TCO for any storage acquisition needs to include the cost of management, not just the per GB cost of the space.

SSD vs. Wide Striping

So, what's this got to do with the topic at hand? Well, I think that a lot of the argument around this is really an argument around the cost of managing disks. Both technologies have their places, and both can help you address certain performance issues, and both can help you save money. The difference is that SSDs only help with a very small percentage of cases, whereas wide striping can help you with the vast majority of cases. What's more, wide striping can help you address those management costs and drive down that 3-4 year TCO I keep talking about, where-as SSDs really don't help there at all, and in a lot of cases, I believe that the 3-4 year TCO goes way up with SSDs. That's not to say that for those cases where you need the performance, that using SSDs in a targeted way isn't a good idea. But just keep in mind what I said about the cost of managing a TB of storage perhaps exceeding the cost of purchasing it in the first place. In the end, I think we need both, but I think that the bulk of your storage should be on a side striped array where your storage admins don't have to spend a lot of time trying to figure out exactly where they should place the data so that the new LUNs will perform, and the added load doesn't negatively impact existing applications.

My vision

So, ideally, I think that the storage team should have a vast majority of their data on an array that does wide striping, manage that space though some kind of virtualization engine, and purchase SSDs very tactically to address specific performance issues, again managing everything through the virtualization engine thus allowing re-tiering of the data should that be necessary, and making migrations when they are needed quicker, easier, and less impactful to the business. You also need to deploy software to help you with performance management as well as capacity management, and something to help automate the documentation process. This means that there is very likely not a single vendor that can provide all of the technology, but rather you will need to put together a "best of breed" approach you your storage environment. Here's an example of one set of technologies that I think can help get you to where you want to be.

IBM XIV storage – The XIV provides wide striped storage on SATA disks and makes it all very easy to manage. This is where I would put the bulk of my data since my admins wouldn't have to sit there and try and figure out where to place the data, etc.

EMC CLARiion – Put some flash drives in a CLARiiON and I think you have a great platform for those few LUNs you need that require the kind of performance that SSDs offer if you have that kind of need.

Datacore SANSymphony - A software approach to SAN virtualization which allows you to move data around to different arrays without the users being aware that it's going on. This is the way that you address things like re-tiering of your data as well.

Akorri – This is a software tool that helps you to manage your entire storage infrastructure find the bottlenecks, and generally free up storage admin time.

Quantum DXi 7500 – This is a deduplicating VTL that will help you reduce the amount of time that your backup admins spend troubleshooting failed backups.

Aptare Storage Console – This is software that will help you manage your backups. It will report on things like what backups failed, which of those were on SOX systems, etc.

The above are just a few examples of what's available out there to help you to create a more mature, automated, easier to manage storage environment, but they certainly aren't the only ones, just some good examples of what's available, and why you should be looking at that kind of technology. In the end, whatever you choose, just making sure that you are truly addressing the 3-4 year TCO of your environment is the key to getting those management costs under control and allowing your storage/backup admins to manage larger and larger environments.

Storage Shangri-La

2009-02-23T18:58:00.001-08:00

Cloud Computing

I don't know about you, but I've spent a lot of time reading about "Cloud Computing" lately. A lot of space has been devoted to the topic in the blogosphere, that's for sure. Some people think it's the "next big thing", others say not on your life. But don't worry; I'm not going to bore you with another prediction. Personally, I think that the truth lies somewhere in the middle. By the end of this year, or the beginning of next I think we will see some people adopting "Cloud Computing", mostly in the SMB space. The enterprise customers will pretty much stick to their data centers, with a few exceptions for certain applications.

Ok, so now that I've bored you with a prediction after I said I wouldn't, here's why I did it. If I'm right, and enterprise customers do stick to their internal data centers it begs the question what are those data centers going to look like? How are these companies going to address the simultaneous issues of an uncertain economy, increasing demands on IT, and storage in particular, that confront them? For now, I'll stick with the storage team, since I think that they have a particularly difficult task. Data volumes continue to grow, no matter what is happening with the economy. Maybe those volumes won't grow quite as fast as they did when things were booming, but they will continue to grow. This means that the issues of increased capacity will continue to challenge the storage team. What will be new is that they will have to address those challenges with fewer dollars. As I indicated in my last blog, entitled Storage Efficiency, that means an ever more myopic focus on "storage efficiency" for most companies. But as I said, this can also present an opportunity for forward thinking leaders to implement changes in IT, and in storage in particular, that will provide not only long term cost savings, but also provide better service to the business.

Everyone into the pool!

So, what is my vision for the storage team that will do these amazing things? It's just as simple as applying what seems to be working for the server team to storage, virtualization. Actually, it's a bit more than that. It's creating a pool of storage which can be managed as a single entity and delivered in different ways (NAS, SAN, FCoE, etc.), easily backed up, and protected with a proper DR solution. I realize that some of you reading this are saying "he's talking about storage Shangri-La"! Well, maybe I am, but I think that it's something that technology today might just allow me to do. It won't necessarily come from a single vendor, but I think that it's doable, but it means some changes to the way that organizations purchase storage, and the kind of storage that they purchase. It also means that some money will need to be expended in order to create that Shangri-La. It's because of those expenditures that it's going to take forward looking leadership. The fearful and the visionless need not apply.

If you are going to use heterogeneous storage (and I think you should at least be able to) in your storage pool, then you need some way to do things like SNAPs, Replication, and DR which is not vendor dependant. Personally, I think that the virtualization engine itself should provide those features, but you could use a third party tool to perform those functions as well. The key point here is that you separate these functions from the storage array so that you aren't dependant on what's available from a single storage vendor, or a single storage vendor's array for this functionality. That is unless you pick a storage vendor who provides virtualization in the array itself as your virtualization engine. For example, if you use the Netapp V Series of virtualization engines, or the Hitachi USP or USP-VM to perform your virtualization. Those engines provide you with the ability to use the vendor's tools for replication, etc. with many other vendors' storage. They key is to find a virtualization engine which allows you to perform storage moves in a completely transparent manner to the hosts that consume that storage. This is important not only for reducing the impact of changes in your storage vendor, for example, but also when you want to re-tier you data. We often take data for certain applications which we consider borderline, and put it on tier-1 storage just to be safe. Now we can put that data on tier-2 storage (SATA), and if the performance turns out not to be what we need, we can move it to tier-1 without any disruption to the application. Thus saving CAPEX costs for the organization as well as OPEX costs.

This also means that there would be a change over time in the kind of storage I would buy. I would prefer to buy storage arrays that have little in the way of the kind of features that I describe above. Basically, just something that lets me configure different protection levels, and provides the storage out more than one port so that I can provide some high availability. All this should save on the per GB cost of the storage, and since I can use any vendor I want, my ability to negotiate price is enhanced. Again, more savings on CAPEX costs.

Storage Delivery

Once we have this pool of disk available, we need to make sure that we can deliver this storage in different ways. We need to make sure that the storage network is flexible enough to deliver the storage using iSCSI, Fiber Channel, and NAS (NFS or CIFS). Again, if you can get this from a single source, like Netapp, that's one way to go. However, if you go a different route with your virtualization engine, then you need to make sure that your NAS engines are gateways, not appliances, so that you can deliver any vendors storage out of the pool. The same is true for any other storage consumers other than your applications hosts. For example, if you want to do backup to disk using something like a Data domain box, then, again, make sure that you are using their gateway so that you can utilize any kind of disk from the storage pool with your Data Domain solution.

Backup and DR

Finally, backups and DR need to be addressed. As I mentioned above, these services need to be available in the pool regardless of the mix storage vendors used. But at some point you may need to take things to tape, and that's OK. Again, as long as the tape management system you use will play well with the virtual disk pool you have created. But more importantly, I recommend that daily backups be done to disk. The cost is within reason when you consider some of the deduplicating devices available today. This relegates tape to just an offsite (DR) role. You can even replicate some of these deduplicating devices, potentially eliminating tape entirely and saving yourself a lot of OPEX costs.

Wrap-up

So, I really believe that now is the time, under the guise of cost savings, to introduce things like storage virtualization, backup to disk, SNAPs, etc. If you have forward thinking leadership they will recognize that the ROI for the costs is reasonably short, and when it's done, the ability of the storage team to manage more storage, provision storage more quickly, and reduce the cost of a managed GB of storage will be greatly enhanced going into the future. It will also position the storage team to handle the onslaught of storage growth that we are going to see once the economy turns around.

GestaltIT

I just want to mention that this blog is now being syndicated on http://gestaltit.com/. I want to say what an honor it is for me to be associated with GestaltIT. Stephen, and all the other authors are much better known and much smarter folks, so I'm hoping to be able to provide some content that doesn't embarrass. Take a look if you get a chance, there some great stuff there.

--joerg

Storage Efficiency

2009-01-31T19:20:00.001-08:00

So, I've been sitting here thinking that with the current economic distress everyone is looking to save money. In the storage business, this means an almost myopic focus on something called "storage efficiency". Everyone wants to get the most "bang for the buck" that they can right now, and they really don't want to talk about much else, and that's really too bad.

I say it's too bad, because for those few who are bigger thinkers, people who are willing to go out on a limb and take a more strategic view of things, right now is a great time to make some changes that will, at the end of all this, leave their business with a stronger, better, more sustainable storage infrastructure. Or better yet, should those at the top of the IT pyramid actually have magically found some stones, they could create an entire IT organization that's better, stronger, and faster than it is now and one that even operates more efficiently than the one they have today.

Unfortunately, what I'm seeing is fear and the result of that is that people are pulling back. They are dragging out or postponing projects, turning the screws on their vendors to reduce costs, and some are laying off people or even going so far as to outsource. I won't even go into why I think that anyone who outsources today is both a fool and a traitor to this county, that's for another time/post.

To those few who have the courage to build instead of tear down. For those who recognize opportunity in the current economic climate, I say bravo. To the rest, I give the Bronx Cheer.

But back to the topic at hand. What I find interesting is that this myopic focus on "Storage Efficiency" on the part of both the consumers of storage and the resulting response from the vendors of storage. All of the big storage vendors have some kind of "Storage Efficiency" marketing strategy going. The blogosphere is full of arguments about how vendor A's storage is very inefficient, and the supporters of vendor A defending that vendor's storage efficiency. In the end, I don't think that any vendor's storage hardware in inherently more efficient, or less efficient, than any other vendors. It's all about how you lay out your applications on that array, how well you manage the space, and how you are able to properly tier the data. In other words, in the end, it's about people. In this case, Storage Architects and Storage Admins who do the grunt work of managing a company's storage infrastructure on a day to day basis. If they are good and are allowed to obtain the tools that they need, you get efficient storage utilization. Otherwise, you end up with very low utilization rates. My fear, however, is that with all of this focus on "Storage Efficiency" from a hardware perspective that those folks in the trenches won't be allowed to get what they need in order to truly make a company's storage more efficient than it is today. Management will fall prey to all that marketing hype and think that if they just switch from vendor A to vendor B that all of their problems will be solved. Oh, and to pay for that switch and since it's going to be soooo much easier to manager vendor B's storage, lets lay off a couple of those Storage Admins we aren't going to need anymore. Again, for those folks I have no sympathy, and they deserve the disaster that's waiting for them just around the corner.

In the end, I think that given the opportunity to do some storage virtualization in conjunction with server virtualization and network virtualization that storage could become very efficient. When you do all three together, you end up with a very efficient data center, as well as a very green data center. Yes, that's right, I said green data center. I fully realize that green sooooo 2008 and no one wants to talk about it anymore (back to that myopic focus on "Storage Efficiency"). But I think that if you look at the big picture, that the more efficient your storage/servers/networks are, the greener they are. That means reall dollar savings folks, so let's not stop talking about "green" just yet.

So, in my opinion, for those that are willing to invest in the future, I say build a "virtual datacenter". Some call it "Unified Computing", some call it "Cloud Computing", and some have other names for it. But as I see it, it's just creating an environment in which business users can run the applications they need in order to operate the business. I think that the "virtual datacenter" would allow for containerized applications. This means that the user's applications including the code and the data, would be in some kind of portable container that could be easily moved, expanded, shrunk, spun up or spun down, depending on the needs of the business. Add to this a way for business users to deploy their own applications into the environment and you completely change the relationship between IT and the business.

Yes, I know this concept isn't for the faint of heart, especially in today's economic climate. But in the end I truly believe what you would have is a much more efficient, flexible, responsive IT organization which has a much better relationship with the business. Heck you might even end up with IT being viewed by the business as something other than just a cost center which needs to be controlled! Yeah, I know, fat chance, but I can dream, can't I?

--joerg

Wide striping is a two edged sword

2009-01-28T18:33:00.001-08:00

I have spent a lot of time lately talking with some of my coworkers, friends, etc. on the topic of wide striping. This topic keeps coming up since there are now a number of vendors selling storage arrays with SATA drives that claim to have "the same performance as fiber channel". Some of the Sales folks I work with keep asking how we are supposed to dissuade people from that idea, or if it's true. One of the prime offenders in this regard is IBM with their new XIV array. The XIV uses wide striping and SATA drives and they claim to have "enterprise performance" at a very low price point. But they aren't the only ones; you have Dell telling people the same thing about their EqualLogic line of storage as well, and there are other too. For an excellent article about the XIV and its performance claims, take a look at http://thestorageanarchist.typepad.com/weblog/2009/01/1037-xiv-does-hitachi-math-with-roman-numbers.html.

What I usually tell them is that the statement is true; you can get fiber channel performance by striping across a large number of SATA drives. The only problem is that you have to give up a lot of usable disk space in order to keep it that way. A quick example usually illustrates the point quite well. Let's say that for the sake of easy math the average application in your environment uses about 5TB of space (I'm sure some are a lot more, and some a lot less, but we are talking average here). Let's also say that you need about 2,000 IOPS per application in order to maintain the 20ms max response time you need in order to meet the SLAs you have with your customers. Finally, let's also assume that your SATA array has about 90TB of useable space using 180 750GB SATA drives and you can get about 20,000 IOPS in total from the array. So, let's do some basic math here. That means that you can run about 10 applications at 5 TB apiece which will take up about 50TB. So, your array will perform well, right up until you cross the ½ full barrier. After that, performance will slowly decline as you add more application/data to the array.

So, what does this mean? It means that the cost per GB of these arrays is really about twice what the vendors would have you believe. OK, but considering how much cheaper SATA drives are than 15K fiber channel drives, that's still OK, right? Sure, as long as you are willing to run your XIV at ½ capacity. In today's' economic climate, that's going to be tough to do. I can just imagine the conversation between your typical CIO and his Storage Manager.

Storage Manager – "I need to buy some more disk space."

CIO – "What are you talking about, you're only at 50% used in theses capacity reports you send me and we didn't budget for a storage expansion in the first year after purchase!"

Storage Manager – "Well, you know all that money we are saving by using SATA drives? Well, it means I can't fill up the array; I have to add space once I reach 50% or performance will suffer."

CIO – "So let performance suffer! We don't have budget for more disk this year. Why didn't you tell me this when you came to me with that 'great idea' of replacing our 'enterprise' arrays with a XIV?!?!"

Storage Manager – "Ahhh … ummmmm … gee, I didn't know, IBM didn't tell me! But we had some performance issues early on, and figured this out. Do you really want to tell the SAP folks that their response time is going to double over the next year?"

CIO – "WHAT! We can't let that happen, we have an SLA with the SAP folks and my bonus is tied to keeping our SLAs! How could you let something like this happen! Maybe I should use the money for your raise to pay for the disks!"

Storage Manager – "Um, well, actually, we need to buy an entire new XIV, the one we have is already full."

OK, enough fun, you get the idea … make sure you understand what wide striping really buys you and if you decide that the TCO and ROI make sense, make sure you communicate that up the management tree in the clearest possible terms. Look at the applications that you currently run, see how much space they require, but don't base the sizing of your EqualLogic (see, I'm not just bashing the XIV) just on your space requirements. Base them more on your IOPS requirements. With SATA drives chances are pretty good that if you size for IOPS, you'll have more than enough space.

--joerg

2009 Outlook

2009-01-27T14:56:00.001-08:00

Like everyone else I'm looking at the business climate in 2009, and it makes me nervous. I listen to the news reports of more layoffs and cutbacks that come almost nightly, and wonder what that means to me and to the storage business. I have coworkers who suggest that storage is recession-proof. That no matter what the economy is doing, that data will continue to grow, and thus companies will have to continue to grow their storage infrastructure. I'm not sure that I buy it, but that just might be my nerves talking. Perhaps it's just that I tend to believe that the truth typically lies somewhere in the middle. So, I thought I'd take a minute and describe what I think is going to happen this year. No guarantees, I can't predict the future, but a little speculation is always fun.

Storage will continue to grow just not as fast
Yup, I do believe that the amounts of data that companies keep on hand will continue to grow. Just not at the same rate it has in the past. Depending on whom you want to believe, year to year growth for storage has been growing at 40-60% CAGAR or even more. I'm guessing that in 2009 we are not going to see that kind of growth. Due to the reduced sales volume that most companies will see due to the recession, there's got to be an attendant reduction in the amount of data that gets created. How much is the $64,000.00 question. I suspect that the growth rate might be cut in half, or even more. Add to this the fact that budgets are getting slashed and storage managers are going to be looking to expend the useful life of storage that they have on hand and it makes me think that this year the average growth rate for storage is going to sit somewhere between 5-10%. So, overall I believe that the volume of raw disk sales is going to drop dramatically. I'm probably not the only one looking at things that way, look at the major storage vendors, they are all cutting forecasts, laying off people, and generally cutting back.

It's not all doom and gloom
I think that in this situation, however, there is some opportunity. Storage providers that can help the storage managers at their clients address the issues of their budget reductions and to find ways to do more with less will get quite a bit of business. I also think that companies, like the one I work for, that can package best of breed hardware and software into solutions that are very cost effective will also do well. Vendor loyalty, however, is going to go out the window and companies that were once locked into a single vendor will look at other vendors if they perceive that other vendor as being more cost effective. Again, this means opportunity for vendors to get into companies that they had previously been locked out of. I predict that we are going to see some of the major storage users leave the "big four" (EMC, NepApp, Hitachi, IBM) and moving to storage from smaller players in an effort to reduce both CAPEX and OPEX costs.

The year of storage efficiency and virtualization
Finally, this year it will all be about efficiency and virtualization. I'm betting that CIOs will actually accelerate any server virtualization projects that they currently have in the works in order to get those reduced costs as quickly as they can get them. However, what they will find is that unless they are quite careful, their server virtualization project might result in increased spending on storage, backup/recovery, and DR that they hadn't planned for. This can be overcome to some extent by partnering with storage suppliers that understand the issues involved when dealing in a virtualized world. I also predict that sales of things like data deduplication, and thin provisioning are going to accelerate this year. Again, all of this is in an effort to "do more with less" on the part of storage consumers.
So, overall, I'm cautiously optimistic that for those that can show their customers how to "do more with less" this year will be just a challenge, but in the end the will survive. For those who continue to try and do business as usual, well, they mind for this year to be more difficult.

--joerg

IBM XIV Could Be Hazardous to Your Career

2009-01-06T16:56:00.001-08:00

So, I haven't blogged in a while. I guess I should make all of the usual excuses about being busy (which is true), etc. But the fact of the matter is that I really haven't had a whole heck of a lot that I thought would be of interest, certainly there wasn't a lot that interested me!

But now, I have something that really get my juices flowing. The new IBM XIV. I don't know if you've heard about this wonderful new storage platform from the folks at IBM, but I'm starting to bump into a lot of flolks that are either looking seriously at one, or have one, or more, on the floor now. It's got some great pluses:

It's dirt cheap. On top of that, I heard that IBM is willing to do whatever it takes on price to get you to buy one of these boxes, to the point that they are practically giving them away. And, as someone I know and love once said "what part of free, isn't free"?
Fiber channel performance from a SATA box. I guess that's one of the ways that they are using to keep the price so low.
Teir 1 performance and reliability at a significantly lower price point.

So, that's the deal, but like with everything in this world, there's no free lunch. Yes, that's right, I hate to break it to you folks, but you really can't get something for nothing. The question to ask yourself is, is the XIV really too good to be true? The answer is yes, it is.

But the title of this blog is pretty harsh, don't you think? Well, I think that once you understand that the real price you are paying for the "almost free' XIV could be your career, or at least your job, then you might start to understand where I'm coming from. How can that be? Well, I think that in most shops, if you are the person who brought in a storage array which eventually causes a multi-day outage in your most critical systems that your job is going to be in jeopardy. And that's what could happen to you if you buy into all of the above from IBM regarding the XIV.

What are you talking about Joerg?!? IBM says that the XIV is "self healing", and that it can rebuild the lost data on a failed drive in 30 minutes or less. So how can what your said be true? Well folks, here's the dirty little secret that IBM doesn't want you to know about the XIV. Due to its architecture if you ever lose two drives in the entire box (not a shelf, not a RAID group, the whole box all 180 drives) within 30 minutes of each other, you lose all of the data on the entire array. Yup, that's right, all your tier 1 applications are now down, and you will be reloading them from tape. This is a process that could take you quite some time, I'm betting days if not weeks to complete. That's right, SAP down for a week, Exchange down for 3 days, etc. Again, do you think that if you brought that box in after something like that your career at this company wouldn't be limited?

So, IBM will tell you that the likely hood of that happening is very small, almost infinitesimal. And they are right, but it's not zero, so you are the one taking on that risk. Here's another thing to keep in mind. Studies done at large data centers have show that disk drives don't fail in a completely random way. They actually fail in clusters, so the chances of a second drive failing within the 30 minute window after that first drive failed are actually a lot higher than IBM would like you to believe. But, hey, let's keep in mind that we play the risk game all the time with RAID protected arrays, right? But the big difference here is that the scope of the data loss is so much greater. If I have a failure in a 4+1 RAID-5 raid group, I'm going to lose some LUNs, and I'm going to have to reload that data from tape. However, it's not the entire array! So I've had a much smaller impact across my Tier 1 applications, and the recovery from that should be much quicker. With the XIV, all my Teir 1 applications are down, and they have to all be reloaded from tape.

Just so you don't think that I'm entirely negative about the XIV let me say that what I really object to here is the use of a XIV with Tier 1 applications or even Tier 2 applications. If you want to use one for Tier 3 applications (i.e. archive data) I think that makes a lot of sense. Having your archive down for a week or two won't have much in the way of a negative impact on your business, unlike having your Tier 1 or Tier 2 applications down. The once exception to that I can think of is VTL. I would never use a XIV as the disks behind a VTL. Ca you imagine what would happen if you lost all of the data in your VTL? Let's hope that you have second copies of the data!

Finally, one of the responses from IBM to all of this is "just replicate the XIV if your that worried". They right, but that doubles the cost of storage, right?

The Future of storage, or is the disk drive dead?

2008-07-30T07:11:00.000-07:00

We are running out of places to put things.

Data continues to grow at a frightening rate. According to an IDC study there was about 281 Exabytes of data stored on disk in 2007 word wide. This data is growing at CAGR of about 70%. At this rate, in 3 years there will be about 1400 Exabytes of data sitting on disk.
Now, a lot of this data is sitting on people's desktops, laptops, ipods, phones, digital cameras, etc. right now. However, things like cloud storage will change all of that. Heck, we are seeing some of the change right now with things like social networking sites, photo sharing sites, etc. IDC says that for 85% of that data a corporate entity will be responsible for the protection and security of the data.
So, in the future, we are going to have to store a lot more data than we do today, a LOT more data. How are we going to do that? Just the physical aspect of getting exabytes of data on the floor is going to be a challenge. I don't even want to talk about protecting and managing that much data. But for now, I want to talk about the density of the hard disk drive since that's going to soon become the physical limit of what we can store on the floor of our data centers.

The bits are getting too small!

Enterprise disk drive capacity has obeyed Moore's Law and doubled every 18 months for quite a few years. However, this growth has appears to be slowing down over the last 5 years, and it is now taking approximately 29-30 months to double the capacity of an Enterprise disk Drive.
This shows that we are nearing the maximum areal density (max capacity) of current disk drive technology called the superparamagnetic limit. Areal density as it refers to disk drives is measured by the number of bits per inch (bpi) times the number of tracks per inch (tpi).
The areal density of disk storage devices has increased dramatically since IBM introduced the RAMAC in 1956. RAMAC had an areal density of two thousand bits per square inch, while current-day disks have reached 100 billion bits (100 gigabits per square inch). Perpendicular recording is expected to increase storage capacity even more over time, but we do appear to be approaching the limit.
As the magnetic bits get smaller, at some point they no longer hold their charge. Thermal fluctuations reduce the signal strength and render the bits unstable. However, this ultimate areal density keeps changing as researchers find new techniques for recording and sensing the bit. Years ago the limit was thought to be 20 gigabits per square inch. Today, the limit is several hundred gigabits per square inch, and more than a terabit is expected soon. But that's about all you can get out of the technology.

Denser is faster.

Increasing the density of hard disk drives has a side benefit. It makes the drives faster as well. This is really quite logical when you think about it. Since the closer things are together on the drive, the more data passes by a read/write head in the same period of time thus making the drive faster.

Shorter term solution.

So, if the disk drive is not going to be able to continue to provide us with the kinds of capacities we are going to need in the future, what will? Well, there are a number of things that are being looked at by a lot of folks who are a lot smarter than me! But in the short term, things like SSD look promising once we work out some of the kinks. Specifically, the write speed issue. Until we can get that up I'm not sure how much general acceptance SSD technology is going to get. Price, I am convinced, will take care of itself as the scales of economy kick in. Holographic storage, some people have been working on this for a very long time and it seems like such a promising technology, but it has yet to come to fruition. There is one company out there that's trying to ship a product, but they recently pushed off their release date until the end of this year. Still, if they can work out the kinks, it definitely has promise, especially for media applications. But what about beyond that? What technologies are the researchers looking at that sound really cool? I look at some of those next.

Sci-Fi data storage.

So, this is where it gets fun. Some of the technologies that researches are currently looking into really do sound like something out of a Sci-Fi movie. Here are some examples of the stuff I'm talking about:

Nanodots - A nanodot has north and south poles like a tiny bar magnet and switches back and forth (or between 0 and 1) in response to a strong magnetic field. Generally, the smaller the dot, the stronger the field required to induce the switch. Until now researchers have been unable to understand and control a wide variation in nanodot switching response. A NIST team significantly reduced the variation to less than 5 percent of the average switching field and also identified what is believed to be the key cause of variability. Nanodots, as small as 50 nanometers (nm) wide could be used to storage data.

Array's of magnetic snakes - According to a weekly digest from the American Physical Society (APS), physicists at Argonne National Laboratory (ANL) have found that under certain conditions, magnetic particles could form magnetic ‘snakes' able to control fluids. According to the researchers, this magnetic self-assembly phenomena may be used to make the next generation of magnetic recording media or transparent conductors based on self-assembled conducting networks of magnetic micro-particles.

Nanowires - Switchable fluorescent proteins, able to move reversibly between two optical states, have been known from some years. But now, German researchers have discovered the mechanism behind this optical switch in a protein found on the tentacles of a sea anemone. According to the researchers from the University of Pennsylvania, Drexel University and Harvard University, barium titanium oxide nanowires suspended in water could hold 12.8 million GB per square centimeter. If the memory density can be realized commercially, "a device the size of an iPod Nano could hold enough MP3 music to play for 300,000 years without repeating a song or enough DVD-quality video to play movies for 10,000 years without repetition," the University of Pennsylvania researchers said.

Is the disk drive dead?

So, does this mean that the disk drive is dead in the future? I don't think so. I believe that the disk drive we know and love will simply move from one tier of storage to another. We are already seeing some of this movement with the implementation is backup to disk. Technologies such as data deduplication will continue to accelerate this process, and the addition of new primary data storage technologies will simply end the process by pushing hard disk drives from on-line primary storage to what will be considered near-line storage in the future. Long live the disk drive!

VMWare and how it effects Storage

2008-06-02T17:29:00.000-07:00

"VMWare Changes Everything"

That's a lovely marketing phrase, but when it comes to storage, it does, and it doesn't. What you really need to understand is how VMWare can effect your storage environment as well as the effects that storage has on your VMWare environment. Once you do, you'll realize that it's really just a slightly different take on what storage administrators have always battled. First some background.

Some Server Virtualization Facts

The trend of server virtualization is well under way and it's moving rapidly from test/dev environments into production environments. Some people are implementing in a very aggressive way. For example, I know one company who's basic philosophy is "it goes in a VM unless it absolutely can be proven it won't work, and even then we will try it there first."
While a lot of people think that server consolidation is the primary motivating factor in the WMVware trend, I have found that many companies are also driven by Disaster Recovery since replicating VMs is so much easier then building duplicate servers at a DR site.
85% of all virtual environments are connected to a SAN, that's down from nearly 100% a short time ago. Why? Because NFS is making a lot of headway, and that makes a lot of sense since it's easier to address some of the VMWare storage challenges with NFS than it is with traditional fiber channel LUNs.
VMWare changes the way that servers talk to the storage. For example, they force the use of more advanced file systems like VMFS. VMFS is basically a clustered file system and that's needed in order to perform some of the more attractive/advanced things you want to do with VMWare like VMotion.

Storage Challenges in a VMWare Environment

Application performance is dependant on storage performance. This isn't news for most storage administrators. However, what's different is that since VMWare can combine a number of different workloads all talking through the same HBA(s), the result is that the workload as seen by the storage array turns into a highly random, usually small block I/O workload. These kinds of workloads are typically very sensitive to latency much more than they require a great deal of bandwidth. Therefore the storage design in a VMWare environment needs to be able to provide for this type of workload across multiple servers. Again, something that storage administrators have done in the past for Exchange servers, for example, but on a much larger scale.
End to end visibility from VM to physical disk is very difficult to obtain for storage admins with current SRM software tools. These tools were typically designed with the assumption that there was a one-to-one correspondence between a server and the application that ran on that server. Obviously this isn't the case with VMWare, so reporting for things like chargeback becomes a challenge. This also effects troubleshooting and change management as well since the clear lines of demarcation between server administration and storage administration are now blurred by things like VMFS, VMotion, etc.
Storage utilization can be significantly decreased. This is due to a couple of factors, the first of which is that VMWare requires more storage overhead to hold all of the memory, etc. so that it can perform things like VMotion. The second reason that VMWare uses more storage is that VMWare admins tend to want very large LUNs assigned to them to hold their VMFS file systems and to have a pool of storage that they can use to rapidly deploy a new VM. This means that there is a large pool of unused storage sitting around on the VMWare servers waiting to be allocated to a new VM. Finally, there is a ton of redundancy in the VMs. Think about how many copies of Windows are sitting around in all those VMs. This isn't new, but VMware sure shows it to be an issue.

Some Solutions to these Challenges

As I see it there are three technical solutions to the challenges posed above.

Advanced storage virtualization - Things like thin provisioning to help with the issue of empty storage pools on the VMWare servers. Block storage virtualization to provide the flexibility to move VMWare's underlying storage around to address issues of performance, storage array end of lease, etc. Data de-dupulication to reduce the redundancy inherent in the environment.
Cross domain management tools - Tools that have the ability to view storage all the way from the VM to the physical disk and to correlate issues between the VM, server, network, SAN, and storage array are beginning to come onto the market and will be a necessary part of any successful large VMWare rollout.
Virtual HBAs - These are beginning to make their way onto the market and will help existing tools to work in a VMWare environment.

Conclusion

Organizations need to come to the realization that with added complexity comes added management challenges and that cross domain teams that encompass VMWare Admins, Network Admins, and SAN/Storage Admins will be necessary in order for any large VMWare rollout to be successful. However, the promise of server virtualization to reduce hardware costs and make Disaster Recovery easier is just too attractive to ignore for many companies and the move to server virtualization over the last year shows that a lot of folks are being drawn in. Unfortunately, unless they understand some of the challenges I outlined above, they may be in for some tough times and learn these leassons the hard way.

--joerg

EMC World 2008

2008-05-24T12:28:00.001-07:00

Well folks, I just got back from EMC World 2008 in Las Vegas. It was a fun trip, but man am I tired. There was a lot of walking at the concference as well as a lot of late nights having fun after the conference sessions were over each day.

I'll have some more detailed postings on what I think about some of the technology I saw at EMC World a little later. Right now I just wanted to talk a a bit about the general trends and feelings I got from the convention.

First and foremost, EMC has finally awakened to the fact that people want de-duplicating products, and they want them now. EMC has really been behind the eight ball when it comes to dedupe. I don't know if it was because of their close relationship with FalconStor in the past, or what, but they really didn't have much of a story to tel l when it came to dedupe, and start-ups like Data Domain where definatly eating EMC's lunch in that market. But the EMC giant has definitely awakened from it's slumber, and introduced some interesting new products.

Basically, the new products fall into two categories, first to a software addition to the existing DL400 line which provides deduplication. The second is a new line of deduplication engines that provide much the same capabilities as Data Domain does. The main differences are that EMC's appliances provide the users with a choice between in-line, post processing, or no deduplication at all. They also have a well designed VTL feature which is an area that Data Domain has been struggling in.

The other area that EMC was emphasizing was "green computing". A lot of this was nothing more than marketing hype and spin on existing products. However, they did mention a feature that they would provide soon that really was "green computing". The idea was to spin down drives that weren't currently in use. Now when EMC didn't introduce and specific products yet, they did suggest that we would see this technology first in the VTLs, but that it could make an appearance in the overall CLARiiON line in the not too distant future.

Overall, a lot of EMC marketing around "green", but some new technology and a good opportunity to talk with the folks at EMC about where they are going with some of the products. I got to spend a little sime talk ing with the folks who work on StorageScope about reporting, and support for AIX VIO in Control Center in general.

Finally, I took my wife along so she could have some fun as well, and I think she ended up having more fun than I did. Las Vegas is a great place for shopping, hanging out in the SPA, and generally having a good time. All of which she did while she was there. We also went to see Phantom of the Opera, which was great. Overall, a good trip for both of us. More details on a latter posting.

--joerg

Storage Sea Change

2008-05-13T12:07:00.001-07:00

Folks,

After re-reading yesterday's posting, I had one of those "well DUH" moments. It seems obvious now, but it hit me like a ton of bricks. Block Storage Virtualization (BSV) is creating a sea change for how people are going to buy their storage.

Once we are in a virtual world, then we no longer need "intelligent storage". All we really want is cheap storage, the intelligence will be elsewhere (i.e. in the virtualization engine). Of course, this is the reason that so many vendors (NetApp and HDS spring directly to mind) have put virtualization right into their array. They are really just trying to hold back the tide.

But it really is holding back the tide. Why would I want to commit to a vendor like HDS as my front end virtualization engine? Why wouldn't I want a completely independent engine? Well, at least one reason springs to mind. It might be easier to get there from here. What I mean by that is if I have an existing vendor's product already in house, have processes built around it, have people trained, etc. then it makes some sense to be able to leverage all of that knowledge and all those processes. However, if I'm not a NetApp or HDS shop, then why would I bring them in just to virtualize my existing storage? It's no easier from a training/process perspective to do that than to go with something that's a pure virtualization play like SVC, Invista, or Yadda Yadda.

The difficulties involved in virtualizing your existing storage/application are something you should seriously consider. Picking a virtualization engine that will allow you to "encapsulate" your existing LUNs, for example, might make the process of rolling out the virtualization engine a lot less painful for your users than allocating all net new storage that's been "virtualized" and then copying your data to the new "virtualized" LUNs.

So what does all this lead to? I suspect that what we will see from the storage vendors are more "dumb" array products and increased sales of arrays like EMC's CLARiiON AX lines of storage. Why pay for all of that expensive smarts in something like a Symmetrix when all you really need is something that can serve up LUNs that perform well. So the sea change I predict is coming is not the complete demise of the storage "big iron", no, it's more like they will go the way of the mainframe. There will still be a business there, it just won't be as big a business as it once was. Sure, the vendors will fight against it, just like IBM did with the mainframe, but in the end I think that the results will be the same, storage "big iron" will get marginalized.

--joerg

Block Storage Virtualization

2008-05-12T11:26:00.000-07:00

For my first posting I really want to talk about block storage virtualization. I really think that 2008 will be the year that people start to roll this out in production in a serious way. Why? It's the money stupid!

Yes, that's right, with the economy getting tight, I suspect that IT budgets, even those for storage, are going to get slashed. So, how are storage managers going to do more with less? You don't think that with the budget cuts there will also be a reduction in the growth of storage/data do you? Of cource not! The business will simply expect the storage team to do more with less, that's all. Simple really, don't you think?

What this will mean is that storage managers are going to be looking for a way to drive the per-GB cost of storage down even more. For many I think that the answer will be block storage virtualization.

Why? Well, I think that there are a couple of answers to that. First off, one direct way to reduce CAPEX will be to drive down the cost of the array's themselves. How? Easy, more competition. If I virtulize the storage, then the array becomes even more of a comodity than it is today, thus driving down the price. It's basic economics really. The more vendors I allow to bid on my next 100TB storage purchase, the lower the price per GB should be, right?

Also, if the real "smarts" is in the virtualization controller, then I don't need it in the disk array, so I can save money on licencing the software in the array. I no longer need to buy replication software from each storage vendor, I have a single replication mechanism which is probably in the virtualization controller itself. More in this in a later post, I think it's going to have a huge impact on the storage vendors going forward.

I also think that I can achieve some OPEX savings by having more efficent operations and fewer outages. Think about it, if all of my storage admins work with a single tool for provisioning, replication, etc. then I have more people with the same skill set, all working in the same interface. That's got to be more efficient and less error prone than having a couple of folks who know the HDS stuff well, and a couple more that know the EMC stuff well, etc.

You had this option before by just buying all of your storage from a single vendor, the trouble with that way of approaching things was that I also had vendor "lock-in". The vendor knew that they had me by the short hairs. Where this really showed up was not in the per/GB price of my storage, or my storage software. I mean, anyone with two brain-cells to rub together knows that if you are going to get everything from a single vendor you better lock in your discount up front, and it better be big. But trust me, the vendors made up for those big discounts via things you didn't have them locked in on. Professional Services, for example. At any rate, virtualization gets me out from under all of that, makes provisioning something that anyone on the team can do at any time following the exact same processes and procedures. You have to believe that will have a posative effect on you OPEX costs.

So, if 2008 is the year of block storage virtualization, what about file virtualization? We all still do NAS right? More on that next time.

--joerg

Welcome!

2008-05-12T11:07:00.000-07:00

Welcome to my storage blog!

I've been thinking about doing something like this for some time, but never got around to it with all of the pressing things going on at work and home, etc. But I just really need to get some things off my chest when it comes to this topic, so here I am!

What I plan to write about here is simple, it's what I know best, computers and storage. I work in the storage business, but I grew up in the Systems Administration side of things. I think that this gives me a bit of a bias, although I prefer to call it a perspective.

I've actually worked on both sides of the equation so to speak. I've worked for manufactures such as CDC (yes, you need to be old like me in order to recognise that they actually built computers back in the day) and EMC. I've also worked implimenting systems and storage at companies in the helathcare industry as well as media companies.

This has been a long time coming, and I have a lot of pent up demand on topics, so I suspect that there will probably be a lot of entries really quick.

Climb in, strap in, and hang on, cause here we go!

--joerg