The Storage Architect

A Dying Industry (slightly off topic)

noreply@blogger.com (Chris M Evans) — Thu, 24 Jul 2008 12:00:00 +0000

The BBC reported today of the next stages of the government and record companies' attempts to crack down on illegal file sharing. Persistent offenders will have their internet connections cut off. Unfortunately, the genie is out of the bottle in terms of the ability for music to be copied and shared.

In the late 90's, I founded a music company that distributed music by CD. You chose the tracks (from our catalogue) and we cut them to disk and shipped them to you. We also had digital downloads DRM protected with Microsoft's encryption. This was *before* Apple release their iTunes store. At the time, the record labels would not give us any new material and we were restricted to back catalogue and inferior content. This ultimately was the demise of the business as our customers couldn't understand why we didn't have access to any music track in the world.

At the time, the record companies' view was that providing us content that could be distributed on CD was risky and would allow people to copy the music. Well, of course it would, in exactly the same way people were *already* copying CDs purchased from record stores! Their principled approach changed when Apple rocked up with large volumes of cash for access to their catalogue and the digital download era started for real.

Now I see a music industry which for years acted in a protectionist fashion, overcharged for their content and milked the customer with countless re-releases and compilations, trying to target anyone they can in order to protect their revenue stream (a bit like when SCO sued IBM).

Unfortunately the genie I mentioned earlier is out there, alive and well and doesn't need the Internet. Music content can easily be swapped on portable hard drives, CD & DVD-ROMs, memory sticks, flash memory cards and so on. The Internet just provides the opportunity to spread music wider than physical media can.

Ultimately I think all copyrighting and protection systems, whether electronic or legal, will fail or be circumvented. We already know that artists are changing their revenue models away from the recorded content into concerts and other revenue streams. Most record companies have moved back from DRM protected content on digital downloads and Sony scored the biggest faux pas with the rootkit they deployed on CDs in 2005.

There will always be people who want something for nothing, as the release of Radiohead's recent album "In Rainbows" shows. However it also shows that people will pay (the average pay price for the album in a survey was believed to be around £4) for decent content and perhaps that represents the crux of the problem - not selling stuff people don't actually want.

Recycling Drives - Update

noreply@blogger.com (Chris M Evans) — Wed, 23 Jul 2008 16:52:00 +0000

Last week I posted about wasted hard drives, removed from arrays and crushed to prevent the leak of sensitive data.

I contacted HGST and Seagate to get some additional background. Here are their responses, slightly edited to correct any spelling mistakes but otherwise intact.

Seagate

(a) when will the technology be deployed in Enterprise FC drives? Our OEMs are currently developing with the Cheetah 15K.6 FDE, a drive that Seagate has already in production.

(b) is the technology proprietary to Seagate? - No, this will becompliant with the Trusted Computing Group's spec. All hard drive vendors are participating in this Trusted Computing Group and we expect that they will have self-encrypting drives that will be inter-operable with ours.

(c) is DriveTrust accepted by the US Government and other similar organisations as secure enough to treat a drive as "wiped" if the encryption keys are removed? Endorsement from National Security Agency (NSA) has already been received for the 1st Self-Encrypting Drive Model-the Momentus(r) 5400 FDE hard drive, for protection of information in computers deployed by U.S. government agencies and contractors for national security purposes.

(d) are any of the "big" manufacturers (EMC/HDS/IBM) looking to deploy DriveTrust enabled drives in storage arrays? IBM and LSI have both publicly announced that they will do so. Note that Hitachi has also just announced a self-encrypting drive, the Deskstar E7K1000, a drive designed for business critical storage systems.

(e) Where do the drives go when they're wiped for final disposal? Extra shipping is involved to ship a drive to a special data destruction service facility, where it can be degaussed or shredded, and then the drive must be shipped to [be] environmentally disposed of. Alternatively, a drive may be over written, a process that takes hours and hours, using energy and tying up system resources, and then may be re-purposed.

HGST

My name is Masaru Masuda, working on product planning for Hitachi GST. Let me try to answer your question. Like Raj mentioned below, we have already supported bulk encryption feature for 2.5" and 3.5"and will support it to Enterprise product next year. With the bulk encryption feature, user data on the HDD media is automatically and always encrypted by the SoC inside [the] HDD. The security feature has two basic functions. One is active protection of data (encryption with secret key) and secure erase of the drive by deleting the encryption key for repurposing or disposal. As you pointed out, Standardization is a key for security. Therefore, a non profit security organization called TCG (Trusted Computing Group) was formed as described in the page 5 and 6 of the attached package. We have been very actively involved in the activities of TCG and plan to pick up security feature based on TCG standards which will be implemented from next year.The security market is still small but it has been growing steadily due to the data security concern and also as a fast and cheap solution for repurposing of drives in Server applications or disposal of failed drives. Also we have had a recycling process for drives failed in the internal testing and for drives returned from the field.

Thanks to both companies for their responses.

So it seems to me that in the future there will be no excuse for scrapping drives. I think the retirement process for HDDs should form part of the "green measurement" of storage.

Five Storage Strategies that May Save You Money

noreply@blogger.com (Chris M Evans) — Tue, 22 Jul 2008 12:00:00 +0000

Infoworld have a great article here discussing how to save money on storage in these tight times we are experiencing. Here's a summary - with my opinions of course!

Play Hardball with Vendors. Ah if it was only that simple. It may be possible to find another vendor selling hardware marginally cheaper, but are you ready for it? There are few companies who have got their storage deployment to a level where interoperability allows them to take storage from any of a range of vendors. Fewer still who have calculated the cost of migration or the full operating expense for each vendor's product; it isn't all about the hardware cost alone.

Avoid New Purchases By Reclaiming What You Have. So what tools will you use to achieve this? Do you know how and why you are missing storage which can be reclaimed? Storage reclamation is usually an ad-hoc process run when storage gets tight or when admins have the time. Some places may have written scripts to automate the discovery of wastage but it isn't easy. I use my own software tool and can highlight about 10 separate categories but you need to be careful of the law of diminishing returns.

Audit Backup and Replication Configurations To Cut Waste. Right, so throw away some backups. Are you *sure* you can do that? Surely the data owner needs to validate whether that backup copy isn't needed any longer...

Rethink Storage Network Decisions. I like this one. Basically, find a cheaper piece of hardware - which may include DAS! Again, interoperability and migration costs have a big impact here. Any marginal savings may be wiped out by the cost of moving to another platform.

Use a Tiering Methodology That Delivers Results Simply. Finally a point I agree with. Tiering can be implemented easily by taking a common sense approach to moving data to a cheaper layer of disk. This doesn't have to involve a migration project but can be achieved as users request more storage.

All of these options are great, however they fail to attack the underlying issue of rising costs - that fact that more data is being generated each day. Simply asking users to question whether new storage is needed or existing allocations can be re-used is as easy as implementing new technical solutions.

Remember - keep it simple!

The Defrag Debate

noreply@blogger.com (Chris M Evans) — Mon, 21 Jul 2008 21:00:00 +0000

I was asked again this week whether defragging of hard drives on Windows servers is really necessary. This is quite pertinent as the cost of deploying an enterprise-wide defrag tool can be significant and any opportunity to save money has to be a good one.

I discussed fragmentation last year (here) when looking into a problem which turned out to be the lack of free space consolidation. however I didn't discuss the potential performance impact.

So, reflecting on whether defragmentation is required or not, I'd say that in most cases the benefits are minimal. Here's why...

Firstly, hard drives have on-board cache; the larger and more "enterprise" the drive, then the larger the cache. We are also likely to see more and more hybrid drives on the market, which will have large amounts of fast memory fronting access to the old moveable components. Cache will mask the impact of fragmented files during writes as data will be written to cache and confirmed as written by the drive. The data can then be written to disk asynchronously afterwards. Obviously if cache becomes overrun, then the benefit will be negated.

Second, operating systems have built in file system performance techniques, including lazy writing and file prefetch. These features will try and minimise the latency issues of reading and writing to disk.

Third, if the data being accessed is a database, the database itself will also have lazy writer processes to asynchronously write data to disk.

Now all of the above applies directly to "traditional hard drives". In systems which have RAID controllers with onboard cache, then the issues will be less. Where storage is taken from a SAN with an enterprise or modular array, all reads and writes will occur through cache, giving the best options for performance and masking the underlying hard drive mechanics.

So, can fragmentation actually help? In fact, I have seen one instance when this occurred and it required slightly special circumstances. The file system was made up from a number of concatenated LUNs using a volume manager. As the server had multiple CPUs and the storage was SAN connected, then multiple I/Os could be issued to the volume. With more fragmentation, then the I/Os are spread across all of the LUNs and performance was increased.

Why Tape Technology Just Doesn't Cut It

noreply@blogger.com (Chris M Evans) — Thu, 17 Jul 2008 08:17:00 +0000

There have been a raft of tape announcements in the last week, most notably the two 1TB wannabee's IBM and Sun. For around a mere $37,000, plus the cost of a cartridge, I can backup 1TB of my most precious data. HP have also announced plans to extend the life of the DAT/DDS tape drive.

If you are a large enterprise customer then the cost of these drives may be justified (although I struggle to see how, when LTO4 drives can be had for about $5000 a piece) and I'm sure actual versus list price will be much lower.

The thing is, hard drives just continue to outpace tape growth. With 1.5TB drives on the way, and 1TB SATA drives available for less than $200, then disk-to-disk is much more appealing than tape at this rate. Obviously I'm riding roughshod over the issues of disk power consumption and portability but my point is that tape just isn't keeping up the pace in either capacity or throughput.

The whole issue is especially true in the small business area where it is easy to purchase terabytes of primary storage but backup to tape is really time consuming.

Why can't tape produce the equivalent bit density of disk? Is it the more fragile nature of the medium? Clearly tape is more flimsy than a rotating sheet of metal; the T10000 cartridge tape is 6.5 microns thick and the tape itself covers approximately 11.5 square metres, much more than the total surface area of the spinning plates in a hard drive.

I guess we will just have to accept tape capacity will never be good enough. That's just the way it is.

By the way, Sun get a big fat zero in the RSS ratings for not providing their news in RSS feed format....

Destroying hard drives, what a waste

noreply@blogger.com (Chris M Evans) — Wed, 16 Jul 2008 21:15:00 +0000

Many financial and government organisations choose to destroy the hard drives that are declared as failing and removed from their arrays. They use products like this which make the hard drive unusable.

What happens to these hard drives? I presume they just end up in landfill and aren't recycled. Is it beyond the wit of man to find another solution?

First of all, a large number of these drives haven't actually failed. They've been marked as having a potential to fail by the array and before a hard failure occurs, the data is moved off to a hot spare. Naturally it is more efficient to copy parity generated data like RAID-5/6 to another drive than to read all drives in the parity group and rebuild the data.

Second, we are imbedding encryption directly into the drive itself. Can't we simply create a drive where the keys can be wiped in the event that the drive needs to be recycled? This seems to me to be the simplest and most elegant solution.

Incidentally, I checked the Hitachi Global Storage Technologies (HGST) website and could only see some nice words about caring about the environment but nothing specific relating to recycling itself.

Green IT with HP

noreply@blogger.com (Chris M Evans) — Tue, 15 Jul 2008 11:21:00 +0000

Last Wednesday I passed a pleasant evening chatting with a number of people from HP on the subject of "Green IT". I happen to think that "Green IT" is an oxymoron as IT is never going to deliver computing power using 100% recyclable energy and components. However, IT can certainly improve its green credentials from the position it occupies now.

The HP representatives included the EMEA VP for Marketing, one of the Sales Managers in HP's Power and Cooling Solutions division, the EMEA Environmental Strategies and Sustainability Manager and the UK and Ireland head of Innovation and Sustainable Computing. As you can imagine this gave the opportunities for plenty of lively debate.

For me, there were a number of highlights; firstly HP admitted and recognises that almost all organisations are attacking the green issue not for a sense of altruism but because being green has a direct impact to the bottom line, whether that is in reducing costs or in acquiring new business.

Second, there's the degree of how complex and unstructured the whole green debate is. Is the aim to reduce carbon footprint or to recycle precious resources (like metals)? How should all of these initiatives be measured? What's a good or bad measure? I think I need more time to mull it over.

An interesting side issue of the discussions relates to how HP have selected the bloggers with which to interact. This is being achieved in conjunction with external agencies who obviously follow the market. My concern is how to HP will determine who is an influencer and who is simply spouting hot air. There's got to be a scientific (ish) basis for this; perhaps it's readership size, perhaps it is references to their blog, perhaps it's the level of comments. Perhaps it is based on keyword count and/or other semantic scanning.

However it is achieved, companies like HP will need to ensure that the tranche of their marketing spend directed at bloggers is appropriately spent. It will be really interesting to see how this develops.

Three Degrees of Bill Gates

noreply@blogger.com (Chris M Evans) — Mon, 14 Jul 2008 22:09:00 +0000

I heard on the wireless today that Bill Gates has a LinkedIn Profile (well hasn't everyone). Just out of interest, I thought I'd see how far I am from the man himself and I was surprised to find I was only 3 steps away, despite Bill only having 5 connections. I was impressed. Much better than the Six Degrees of Kevin Bacon.

On a related note, I've set up a "Storage Bloggers" group on LinkedIn for those of us who run Storage Blogs online. Drop me a note/comment if you want to join the group (I won't publish those comments). I'll also be putting up a page to list and link the blogs too.

Is the Oyster Failure a Red Herring?

noreply@blogger.com (Chris M Evans) — Mon, 14 Jul 2008 15:17:00 +0000

This weekend there was a "fault" on the UK London Underground Oyster card system which caused chaos for many travellers and rendered a large number of cards inoperable and requiring replacement.

This follows the recent cloning of the Oyster card by a group of "Dutch Boffins" (The Register's term, not mine) who had previously cracked the encryption used on the Amsterdam metro (GVB).

I found this story interesting as I travel regularly using both services. It also piqued my curiousity because of the implications on data encryption. It shows that given enough time and ingenuity, it is possible to hack/crack almost all, if not all encryption methods. This has significant impacts for the use of Cloud Storage as a location to store what I would term "Data at Risk" (not Data in Flight or Data at Rest) and re-inforces the need for organisations with valuable data (governments etc) to store their information in secure locations.

Anyway, getting back to this weekend's failure, the conspiracy theorist in me says that TFL decided to change the encryption method used by Oyster to fix the successful Duch crack. This inevitably rendered a number of cards invalid as the cards were/are fixed in nature and couldn't be changed. TFL decided it was better to take the bad publicity of a number of rejected cards than compromise the entire system, because you can be sure the workaround for getting the card recharged would eventually find itself in the public domain.

What do you think?

Seagate Raises the Bar with 1.5TB Drive

noreply@blogger.com (Chris M Evans) — Fri, 11 Jul 2008 09:03:00 +0000

Seagate announced yesterday version 11 of their Barracuda hard drive range, to be released next month (August 2008) with a maximum capacity of 1.5TB. The news link has all the speeds and feeds if you're interested in how they have achieved this remarkable milestone.

I've trawled the 'net to plot the release of previous versions of the drive and their capacities at the time. Here's a graph of the releases I could find, going back to 2002. Trending the growth (totally un-scientifically of course), then we can expect to see 2TB drives by December 2008, 3TB drives by November 2009 and 4TB drives by June 2010. This may be a little optimistic as the trending is skewed slightly by the recent advances perpendicular recording has brought to capacity growth, but maybe not, as my recent post on Hitachi 5TB drives shows.

Unfortunately, sustained transfer rates for these drives have remained around the 100MB/s mark, so offloading a complete drive sequentially takes around 250 hours, by my calculations. I'd love to know how long a RAID rebuild would take (Seagate if you fancy loaning me some drives, I'll find out for you!).

As previously discussed, the increased capacities are good as they increase the GB/Watt and GB/cm3 density but we're going to be increasingly challenged by how we get data on and off them - especially when the drives fail.

Distributed Computing Nirvana

noreply@blogger.com (Chris M Evans) — Wed, 09 Jul 2008 09:38:00 +0000

Last year I blogged about the concept of storage futures (or options - I'd have to go back and check which) which would allow storage charges to be based on a forward pricing model. The logic for this was to penalise those "customers" who don't bother to take the time and plan their storage demand requirements. By charging more closer to the delivery time of the storage, customers are dis-incentivised to ask for new storage at the last minute.

The evolution of cloud computing has the opportunity to deliver on my original idea. I'm sure I don't have to explain Cloud computing to anyone, but in case you're not aware, it is effectively distributed computing for the Web age. Amazon Web Services is probably the most popular service (in terms of awareness). The Amazon services provide the ability to create a virtual machine, perform database services, manage queues and of course store data in their S3 Simple Storage Service. I'll discuss my experiences on Cloud Storage in more detail in another post.

Many large organisations are facing issues with meeting the power and cooling demands in their datacentres. This is being driven by the increase in computing power and storage density, achieved by the use of blade server technology and virtualisation. Although more computing is being achieved in the same physical space, for some organisations their business model demands more computing to take place in order to gain business advantage. Think of pharmaceutical companies who are using software to model organic chemical interactions rather than perform the experiments in the lab.

I suspect if you investigate the use of computing in a lot of these datacentres, then only a small percentage of the computing power will be dedicated to core business operations. There will be many applications providing anciliary services such as reporting, financials, batch processing, reconciliation, inventory and so on. Many will not be time or location dependent and could easily be removed from the core datacentres for processing elsewhere.

Obviously this change of processing requires a different operating model. A key trend in the industry is to consolidate into a small number of large (and expensive) datacentres, but by operating in this way, companies are artifically constraining their growth into the size of these datacentres and setting a timeline which will require new datacentres to be built before expansion can continue.

So what is the answer? Computing could to move to be location independent except for only those critical components which can't suffer the effects of latency. As an example, take file archiving. If data has been unreferenced for more than a specific time (say 3-6 months) then move it into the storage cloud. The data can be duplicated in multiple locations automatically to provide redundancy. Note that I'm assuming all the issues of security have been investigated, discussed, resolved and implemented.

Immediately redundant data is out of the datacentre and the cost of storage reduced to a service charge, which is likely to be signficantly lower than the cost in the primary location.

Some organisations may decide that the cloud is too unsafe for their data. In this instance this is where a more appropriate datacentre strategy needs to be developed. Rather than having a small number of "megacentres", smaller location critical sites can be built for primary data, with other sites developed in locations offering the cheapest space/power costs. In this way, large organisations could effectively operate their own computing (and storage) cloud.

I think the options for real distributed computing are really exciting. They provide the opportunity to "green" the storage environment over and above the simple task of deploying larger disk drives and bigger storage systems.

Shock - Escargots n'est pas francais!

noreply@blogger.com (Chris M Evans) — Sun, 06 Jul 2008 17:43:00 +0000

The BBC has reported that prices in France for snails are set to rocket. I lived and worked in France for a few years and loved tucking into the garlic covered gastropods. But horror of horrors, apparently most snails served in France are now not French! Is nothing sacred?

5TB drives

noreply@blogger.com (Chris M Evans) — Fri, 04 Jul 2008 21:31:00 +0000

I just read this on The Register. 5TB drives! Can you imagine it! The HDD manufacturers continue to push the envelope even further.

Now I have a concern about drives getting to this size and that's the ability to get data on/off the drive itself. With 73/146/300GB drives, the capacity to response time ratio is still within a tolerance that means adequate random access throughput can be achieved. But with larger drives the number of different concurrent accesses will increase and if response time doesn't decrease then very large HDDs will start to operate like sequential devices.

I think I need an illustration to make my point. Imagine a 73GB drive is receiving 200 random I/Os per second, each with an average 5ms response time. Scale the capacity up to a 5TB drive and that's about 69 times the capacity. The scaled up drive would have to cope with 13800 I/Os a second and provide an average response time of 0.07ms!

Firstly, it is unlikely 5TB drives will be expected to perform like today's 73GB drives but it serves to illustrate that we can't expect to simply consolidate and shrink the number of drives installed into an array. We need something more.

I think we need a more innovative approach to the design of the drive interface. This may simply be shed loads of cache, to improve the overall average response time, or perhaps multiple virtual interfaces per drive or independently mobile read/write heads which don't need to read/write a cylinder at the same time. It could even be drives that dynamically reallocate their data to make read/write quicker (for example, put frequently read/write blocks in the same physical area of the drive).

Who knows what the solution is, but rest assured something needs to happen to make 5TB drives useful devices.

Monster Mash (Up)

noreply@blogger.com (Chris M Evans) — Wed, 02 Jul 2008 20:57:00 +0000

The UK Government is running a competition offering participants up to £20,000 if they can create new uses for existing free sources of government data by combining the data into new and useful information (a mash-up). You can find a link to the data here.

The volume of data is immense. I started trawling the UK census from 2001, burrowing down to the small village I live in. Of the 4700 or so residents, there are three buddhists but disappointingly no Jedi (there's an urban myth that Jedi is put down by so many people who have no specific religious persuasion). Apparently there are no 1-room properties, so no studio flats and no basement flats (no-one lives below ground level). 2.4% of houses have no central heating and only 5.5% of the population are not in good health.

I'm not sure what this indicates other than don't try selling central heating or life insurance policies where I live! But being less facetious, the power in this data will be in deriving new value. I can see two immediate uses/methods.

Firstly, most of this data is useful to people looking for new places to live; schools information, services information, crime levels and so on. So, develop a website and put a postcode in to see how your prospective area rates.

Second, trawl the data and find the ends of the spectrum - the good and bad, best and worst of each metric. For example, which area has the highest population density? Which has the worst crime? This information could be great for business planning; don't set up a locksmiths in an area with the lowest crime rate and so on.

Of course the hardest part will not be to correlate different data sources but in bringing together a consistent view of the information. Some data is accessed via APIs, some by XML, some in Excel format. What "common" point of reference can be used? Postcode? Address?

Whatever, the availability of more and more data content will be absolutely invaluable, but for me, I'd like to see more real time information to be mashed up. For instance, at the airport a live XML feed of flight arrivals/departures that I can read in the taxi when I'm running late (I don't want to have to log onto their website); same for the train; a feed showing my nearest tube station or restaurant as I travel; a feed of the waiting time at all the Disnet rides, so I can pick the shortest queue without having to find a status board; I'm sure there are many many more.

Rich data is a wonderful thing; bring it on!

Keep Your Data for 200 Years - Why?

noreply@blogger.com (Chris M Evans) — Sat, 28 Jun 2008 15:42:00 +0000

Courtesy of The Register, I followed their link to a company called Delkin (the data Belkin?) who are touting their premium Blu-Ray disks (BD-R) with a lifetime of 200 years (and 100 years for their DVD-R disks).

Now, this all sounds wonderful; a "guaranteed protection over time" (whatever that means) for your "wedding photos, tax documents etc". The trouble is, and we've been down this road before, having media that survives 200 years is great, but (a) what's going to be around to read it and (b) will the data format still be understandable by the latest software?

Attacking the first, it is conceivable that Blu-Ray compatible drives will be around in 10-20 years' time. After all, we can still read CD-ROMs 20 years after they were introduced and Blu-Ray is already a mass-marked storage platform, not just for data, but for media content too. However, 200 years is a bit hopeful. The 20 years since the introduction of the CD format has seen DVD, Blu-Ray, HD-DVD (!) plus countless other solid state formats.

Data format is more of an issue. I discussed this issue in a recent post.

Another great example of hardware and data compatibility issues can be seen with the BBC's Domesday Project, which used Laserdiscs and non-standard graphical images for displaying information. What's to say that in 50 years time we won't think JPEG just as archaic?

So, don't waste your money on $27 BD-R disks. Buy them cheap, keep multiple copies and refresh your data regularly.

Incipient Revisited

noreply@blogger.com (Chris M Evans) — Mon, 23 Jun 2008 07:48:00 +0000

You will remember that I recently posted a comment about migration costs, specifically with relation to Incipient. My view was (and still is) that the majority of migration costs come from preparatory and remedial work rather than execution of the migration. Well, Incipient asked for the right of reply and I had a call last week with Robert Infantino, their Marketing and Alliances Sr VP.

The $5000/TB figure they were quoting was an average they had seen in the industry for certain vendors' professional services time to come in and perform the migration work on behalf of the customer. Incipient's take was that they could provide their appliance/software expertise to provide the same service but at a significantly reduced cost (I won't quote specific numbers here, but the number quoted was much lower than the equivalent cost from "a vendor"). So, I guess with clarification, it is more clear that Incipient were comparing the vendor costs versus their product costs and not including any internal customer costs (project management, preparation work etc) in the calculation. This seems a more appropriate comparison in my opinion.

Getting back to the vendor discussion, there's a real issue here. If vendor X wants to sell you their latest technology, they need to accept and take the hit on helping with migration to their new array. This should be even more so where the vendor doesn't change as this should be a "no brainer" and built into the technology.

In a world where hardware is becoming a commodity, one differentiator will be the vendor who can minimise the effort/cost and impact of migrating from one technology to another. Until then, products like SVC and those from Incipient will continue to have a market position - oh and humble consultants like yours truly!

The Rise of SSDs

noreply@blogger.com (Chris M Evans) — Tue, 17 Jun 2008 20:06:00 +0000

Sun recently announced that they will be putting solid state disks into all of their server and storage range of hardware. EMC already have solid state drives for DMX-4, which was announced in January this year. EMC have also stated that they think SSDs will reach a price parity with high end FC drives by 2010.

All of a sudden (and I'm sure plenty of people will claim it isn't sudden) solid state disks are all the rage. For servers, I can see the logic. It's another step in keeping the power and cooling demands of servers down; it also extends primary memory further and will definitely increase performance.

But what about storage arrays? I can see the benefit of putting a tier of SSD drives into DMX arrays, especially in the way EMC have chosen to implement it. It allows those targeted applications to get the performance they require at a manageable price point without a drastic reconfiguration of the array. But an entire array of SSD? That's just the same as existing products like Tera-RamSan.

If SSD prices are driven down, then surely so will the price of standard hard drives. HDD manufacturers aren't going to lie down and let solid state take away their business. We've seen their response already with Seagate taking STEC to court over patent infringements.

So where will it end? Well, tape didn't go away as many forecast it would. I don't see spinning drives going away any time soon either. What I'd like to see is the rise of intelligent storage systems that learn the busy and quiet blocks and move the data between SSD and HDD to keep optimal performance. Meantime, HDD prices will continue to fall and the battle will be between cheap (but fast) HDDs and balancing their cost against the power/cooling they need.

FC Enhancements

noreply@blogger.com (Chris M Evans) — Fri, 13 Jun 2008 17:58:00 +0000

A comment posted to my previous blog entry reminds me of a requirement I've had for some time from Fibre Channel. In the "Good Old Days" in my first working life as a mainframe systems programmer, I could very easily see a breakdown of response time against each storage device on an LPAR. Now, the passing years may have given me "rose tinted spectacles" (or more accurately now, contact lenses) of that time, but I seem to remember the reason that I could see seek time, disconnect and connect time was due to the design of the (then) MVS I/O subsystem. As each I/O (CCW) was processed, the hardware must have been adding a consistent timestamp to each part of the process; the I/O initiation, the connect, the disconnect and subsequent seek and then the reconnect and data transfer time to complete the I/O (if none of this makes sense, don't worry, it probably means you are under 40 years old and never wore sandals to work).

Nowadays, the I/O infrastructure is a different kettle of fish. Each part of the infrastructure (host, HBA, fabric, array) are provided by different vendors and have no consistent time reference, therefore tracking the time to execute a storage "exchange" is very difficult. There is (as far as I am aware) nowhere within a fibre channel packet to track this response time at each stage of the journey from host to storage.

If we want the next generation of storage networks to scale, then without a doubt we need to be able to track the journey of the I/O at each stage and use this information to provide better I/O profiling.

Now, just how do I become a member of the t11 committee.....

Storage Migration Costs

noreply@blogger.com (Chris M Evans) — Thu, 12 Jun 2008 20:18:00 +0000

I’ve not paid much attention to Incipient (their news page doesn’t provide an RSS feed, so there’s no chance of me seeing their press releases easily), but my attention was recently drawn to a recent release relating to their iADM and iNSP products (catchy names, those).

Now, if you want to know about their products, have a look at their website for yourself. Rather, my interest was sparked by a claim in their press release, quoted below:

The High Cost of Today's Data Migration

Industry estimates and field data captured by Incipient indicate that SAN storage is growing at 40 - 60 percent annually and 25 percent of data under management is moved annually at an average cost of $5,000 per terabyte. Based on these estimates, a data center with one petabyte of storage under management today spends $1.25 million annually on data migration operations. Two years later, the data center is likely to grow to nearly two petabytes increasing the annual data migration cost to nearly $2.5 million.

Source: Incipient Press Release 11 June 2008

So the estimate is $5000 per TB of data movement and 25% of data being moved each year. I can understand the latter; it’s simple logic that if you have a 3-4 year lifecycle on technology then on average 25% of your estate will be being refreshed each year (although that figure is slightly distorted by the fact that you’re also deploying an additional 40-60% each year). Now, how to get to a $5000 per TB calculation...

Excluding new storage acquisition, network bandwidth, etc, I’d assume that the majority of migration costs will be people time. That would include planning and execution of migrations. In environments of 1PB or more, I could (almost) bet my house on the fact that there will be a significant amount of the storage infrastructure which is (a) not understood (b) badly deployed (c) backlevel amongst many other issues. $5000/TB would therefore seem quite reasonable, based on the amount of work needed to refresh. The only problem, though, is that a majority of the manpower cannot be solved by software alone. This will include documenting the environment, bringing server O/S, firmware and drivers up to date, negotiating with customers for data migrations, migration schedule planning, clearing up wastage, new server hardware and so on.

It would be an interesting exercise to determine what percentage of the $5000/TB cost is actually attributable to data movement work (i.e. having someone sitting at a screen issuing data replication commands). I suspect it is quite low. From experience, I’ve been able to move large volumes of data in quite short timespans. In fact assuming sensible preparation and planning, most of the time doing migrations is sitting around (previous employers disregard this statement).

So how much money would Incipient save? My bet is not much.

Ah this is so accurate...

noreply@blogger.com (Chris M Evans) — Wed, 11 Jun 2008 19:48:00 +0000

As a contractor/consultant I can totally relate to this list; I'm sure many of you can too.

Simple is Good

noreply@blogger.com (Chris M Evans) — Wed, 11 Jun 2008 14:02:00 +0000

I've been doing a lot of travelling recently (rather a lot in fact), mostly in Europe, with a little in the UK between airports. European trains are much better than their UK counterparts - they are reliable, clean, comfortable (note I didn't claim they were fast) and their cost structure is simple to understand. No restrictions about time of day travel, booking in advance or all that nonsense. No. Simply turn up at the station and buy a ticket.

The UK on the other hand must have one of the most complex ticketing systems, especially around London. As an example, if I travel into London from where I live and want to return home between 4pm and 7pm then I can't buy a cheap day return. Presumably that's because they can fleece travellers who don't realise this rule exists. However if I am already in London and want to travel out, I can buy a cheap day single and travel on it between 4pm and 7pm! Even the people selling the tickets think it is crazy. I could give you dozen's of other similar examples, but life's too short.

So it is with storage. Keep it simple. Take tiering as an example. You could spend days and weeks developing the most finely detailed tiering strategy but in reality you will find most data will sit on a small number of tiers, the bulk of it being in the middle range. Developing complex tiering structures, just like complicated train pricing structures just leads to confusion and in the end additional cost. All that's needed is a simple strategy with most of the data on cost efficient storage.

Remember - simple is good.

Dealing With The Consequences

noreply@blogger.com (Chris M Evans) — Tue, 03 Jun 2008 16:00:00 +0000

In a remarkable piece of coincidence, two WWII unexploded bombs were found today and caused air traffic delays. The first, at Stratford in east London, temporarily closed London City Airport. The second closed a runway at Amsterdam’s Schiphol airport. It’s amazing that over 60 years after these bombs were dropped, they are still being found; fortunately this time with no injury to anyone. I found out about both incidents because colleagues of mine flying from both City Airport and into Schiphol were delayed or had to alter their plans.

On a less serious level, but important nonetheless, we are having to live with the consequences of our data storage policies. We will have data being stored now which must be accessed in 60 years time. This will include medical and financial information, directly affecting individuals if the information cannot be retrieved. Obviously good data management practices are essential, but they will go beyond the normal storage of data we’ve been used to up until now.

Take the use of medical imagery; x-rays, CAT and PET scans, MRI scans. These all now produce complex digital images. If we store them using today’s format, how will the format and presentation of the images change in the future as we move to more complex display technology, higher resolutions and possibly 3D television screens?

If you want examples of what I mean, think of your old word processing documents. They could have been stored in an early version of Microsoft Word, WordPerfect, WordStar or other similar now defunct technology. You may be lucky and still be able to read them; you may not. Fortunately, apart from some formatting codes, most word processing documents can be opened in Notepad, WordPad or some raw file viewer which will at least allow you to recover the content. Things won’t be so easy with imaging files as their binary nature will render them useless if the software isn’t retained to open them.

I can see one of our future storage challenges will be to ensure all of our data is retained in a readable format for the future. XML addresses some of these problems and the adoption of file format standards will help. However, data will need to be refreshed as it ages. More metadata referring to the content format of files will have to be produced and software written to detect and convert unstructured files as file formats become defunct.

For many of us, we can afford to lose a few files here and there or perhaps print out the most important of our documents. For large organisations, the data management lifecycle has only just begun.

HP Give It Large

noreply@blogger.com (Chris M Evans) — Fri, 30 May 2008 20:46:00 +0000

Yesterday afternoon I had an opportunity to meet with HP as part of an informal session to make contact with storage bloggers. HP are obviously interested in the possible benefits keeping the blogging community well informed could bring, however my blog is not to act as a mouthpiece for the HP marketing department and I'd suggest if you want to keep abreast of their technology releases, use this XML link.

What's more interesting is where HP storage is headed. Take for example their new Extreme Storage solution. A scalable NAS product which reaches the heady heights of 820TB in a single unit. Fantastic you may think, and I guess if you have a real need for this volume of data in a single unit, then it's the one for you.

However, apart from the obvious issues like whether your raised floor can actually take the weight of a fully configured device (and how do you cool this kind of beast), what troubles me more is how much data on a system like this is actually of any use.

Although the ExDS9100 is aimed at delivering storage for high performance solutions, I think there is a risk of arrays like this being deployed to defer the hard work of actually classifying and setting sensible deletion policies, which, let's face it, for most companies has sat as a task in the "too hard box" for as long as NAS storage has been around. It may well be that some customers see this product as a way to defer the inevitable and actually start managing their data.

Anyway, fair play to HP for entering the market and making use of their Polyserve acquisition and fair play to them for wanting to talk to the blogging community too. If I get any juicy nuggets of information (like whether HP have a position on cloud storage), you can be sure I'll share it here.

UK Email and Phone Database

noreply@blogger.com (Chris M Evans) — Tue, 20 May 2008 18:56:00 +0000

The BBC reported today that the UK government is planning a tracking database for all phone calls made and emails sent in the UK.

This strikes me as an unbelievably stupid plan. In the first place, every UK citizen will be baulking at this incredible intrusion into civil liberties. Second, it is highly unlikely that the government could ever deliver such a database based on their previous track record with developing and deploying large scale IT projects - think of the UK tax system and new NHS IT system (which will apparently be 4 years behind schedule). Will this database contain all the content of the calls or just a list of who called/emailed who?

Assuming the former, let's do some "back of the fag packet" calculations...

According to the Mobile Operators Association, there were 70 million mobile subscribers at the end of 2006, with an average 100 minutes per user per month and 12 text messages per week. That's 84,000,000,000 minutes and 43,680,000,000 text messages in a 12 month period, the timescale the database is expected to hold data for.

Using the following forum posting referring to Skype, a conservative estimate of bandwidth is 30kb/s or about 225KB per minute. A text message is a maximum of 160 characters. This means the government database would need a 12 month capacity of only 6.36TB to store the text messages (assuming no database or filesystem overhead) but a whopping 17.6PB of storage to hold the voice calls. Let's assume those calls are made consistently over the course of a year, then the system would need to ingest about 600MB/s of data.

Now, I would imagine even the UK government wouldn't be stupid enough to attempt to store the content of all those calls (I didn't even attempt to calculate the email traffic). If they don't, let's face it, you have to question what the point is, if the content isn't being recorded as anyone with any nefarious intent will simply use anonymous pay-as-you-go SIM cards and bypass all the tracking. Still, at an efficiency of say, 40%, the 44PB potentially needed could set up the lucky EMC or HDS salesperson for life! I wonder if any of them have done the same calculation as me....?

Nexsan going public

noreply@blogger.com (Chris M Evans) — Fri, 09 May 2008 06:54:00 +0000

I've been reading the Nexsan statement announcing their intention to go public. I don't know much about their products other than they sell high density storage systems. What intrigued me about reading the announcement is the amount of exposure to the internal operation of the business occurs when statements are made to SEC as part of a flotation.

For instance, Nexsan have never made a profit, although their losses are declining year on year. They list their competitors, including all the usual suspects like EMC, HDS and HP, but quote Dell and Equallogic (now Dell) at the beginning of the list; this gives you an idea of the position in the market they see themselves in.

Nexsan don't run a direct sales force; they rely on Kodak for their support arm; they are reliant on Bell Microproducts for their disk drives; they rely on three subcontractors to manufacture their equipment; they are reliant on channel partners to install products at client sites.

So if I invested, what exactly would I be getting? Well I guess IP is the key value and that's exactly what the company will be rated on the value of. The more Nexsan can reduce costs by removing the dependency on external suppliers and the more they can acquire and use their own IP, then the more chance the company has of reaching that profit goal.

Remember 3Par? They launched last November at $14 a share and are now trading at a more $8.37. I wouldn't be betting the kids' college fund just yet....