Back in those days the "Route 128 High-Tech Beltway" was the east-coast precursor of Silly-Con Valley, where the long-haired preppies hippies cut their hair, donned suits and ties, and collectively laid the foundation for the technology transition from the monolithic mainframe to the departmental computer and eventually to the desktop PC.

Back then I had as customers likes of Polaroid Corporation who used the Prime 300 RTOS to monitor the self-contained instant film packaging machines (Polaroid was the largest manufacturer of batteries at the time – go figure!). I also supported Dan Bricklin and Bob Frankston of Software Arts, inventors of VisiCalc – inarguably the "tipping point" application to bring desktop computing out of the denizens of home hobbyists and word processors and onto the desktop of knowledge workers worldwide (few know that Dan and Bob used a Prime system to develop and maintain VisiCalc – Bob also introduced me to the first port of Emacs to the Prime platform).

I also helped bring to market the Prime 50 Series, complete with the first-in-the-industry green LED "power on" lamp. All the LEDs on most every minicomputer were red prior to that (mostly because red was the cheapest – or only – color). At my suggestion (which was based on my experience as a computer operator in Prime's IT department) beginning with the 50 Series, green LEDs came to mean "all good," red meant "serious problem" and yellow indicated "caution" (as in, remote access enabled). From then on operators could tell at a glance the state of their systems.

Those were the days, my friend.

This past week we have witnessed the passing of three great icons of the 80's: Michael, Farah and Ed McMahon, and these events caused me pause to reminisce about my life back then. Coincidentally, a fellow co-worker from Prime sent me this link to a series of Dr. Who television ads that Prime had developed for the Australian market, and this post was borne of that inspiration.

They just don't make 'em like that anymore.

If you're not a Dr. Who fan, getting Tom Baker to do adverts for your product back then was huge, as the show was insanely popular in the UK and Australia. Cult-like popular, even – the longest running science fiction television show in the world, according to Guinness World Records.

Thanks for the memories...

P.S.: I'm still looking for the videos of the Prime Australia "Keep Computers Confusing" campaign which featured a "robot" named Dr. Primestein. Leave a comment if you have a link.

technorati tags: Prime Computer,Dr. Who,1980's,Dan Bricklin,Bob Frankston,VisiCal,Software Arts

Claus I agree with your approach here, and I do wonder if our Boy Wonder, Barry, is a full time blogger for EMC without anything else to do.

After that slap-in-the-face, both Claus and Michael have has chosen to censor my comments on their respective blogs his blog, and it appears that Christophe Bertrand will no longer publish my comments either. [UPDATE 21 June 2009: Although he obviously agreed with Claus’ decision to censor me, Michael now says he hasn’t received any of my comments on his blog- I’ll give him the benefit of the doubt – for now.]

Was it something I said? 

Fortunately, the storage anarchist does have a day job.

In fact, the latest issue of the EMC.now magazine includes an article about how the Symmetrix Product Group stays closely connected with the requirements and future vision of its customers and prospects, and provides some insight about what I really do here at EMC. If you're interested, the article is How to mind the future of a mission-critical world and it can be found on page 18 of the on-line version or on page 10 of the PDF version of the EMC.now magazine.

In fact, this close customer interaction that the Symmetrix management team maintains is the real reason why the words "from a unique perspective" is included in my blog's masthead. I get to see the future of storage technology through the eyes of customers dealing with the here and now.

Customer insight is also why I can ask EMC's competitors the tough questions so quickly and precisely whenever they make an announcement – I actually DO live and breathe customer requirements for storage, and it really IS part of my job to understand if, when and how competitors are addressing the customer requirements I learn about daily. So as irritating as my questions are, I know that the competitors are getting these same questions from their prospects. And their bloggers have come to know that  that I won't hesitate to call them out on a BS answer – especially when they make stuff up or misrepresent the facts.

And if that makes me persona non grata, so be it. Anarchy cannot be censored!

By the way, there are lots of other interesting articles in this issue of EMC.now. Whether you are an EMC customer, partner, prospect or competitor, I encourage you to give it a thorough read. And if you'd like to discuss any of the content, feel free to post your thoughts and questions here.

I promise you won't be censored, even if I might not be able to answer all of your questions.

This is another post from the storage anarchist.

technorati tags: EMC,strategic planning,Symmetrix,V-Max,EMC.now,voice of the customer,TCE,mission-critical,storage strategy,HDS censorship

Carrying over the theme from my last post, it seems that it isn't just HDS bloggers and competitive marketing teams who like to make stuff up. In fact, it seems to come straight from the top, as Beth Pariseau found when she dug into assertions being made by HDS's vice president of corporate marketing Eric-Jan Schmidt:

IDC: HDS market share numbers not accurate

Caught 'em red-handed.

The fundamental error in the whole discussion is that IDC does not report revenues by product or platform – their quarterly data only reports revenues by end-user vendor bucketed into several different price bands.

Mr. Schmidt apparently used the IDC buckets US$300,000 and above as the cut-off for "enterprise storage," but the reality is that there are indeed "enterprise" array sales below that mark as well as "mid-tier" sales above it. That shouldn't come as a shocker when the list price for an entry-level Symmetrix is around US$240,000, and CLARiiON systems can scale as large as 960 disk drives.

But then again, as I have been observing here for several years now, Misleading Marketing (and Hitachi Math) are hallmarks of HDS PR.

This is an original post from the storage anarchist.

technorati tags: Hitachi Math,IDC,storage market share,HDS,Hitachi,EMC,Symmetrix,CLARiiON,Beth Pariseau

V-Max sure has gotten under the skin of the HDS and their bloggers.

Not only has the pitiful HDS marketing machine rushed out yet another overhyped and underwhelming (green eggs and HAM) announcement, but every HDS blogger seems determined to take as many uninformed pot-shots of FUD at a product they clearly have not even yet begun to comprehend.

And it’s not just the bloggers who clearly don’t get it: a customer recently told me about some Hitachi marketing materials he has seen that attacked V-Max based entirely upon a Hitachi “suspicion” about the architectural utility of the Virtual Matrix. Seems based on that (mistaken) “suspicion” Hitachi’s conclusion is that V-Max simply cannot work. PERIOD.

When you don’t understand how something works, I guess all you CAN do is make sh*t up!

The latest blatantly uninformed attempt to discredit V-Max comes from HDS’ Christophe Bertrand as he delves deep into the FUD-bucket. In his latest post he tries to cast aspersions against V-Max while trying to deflect several of my very, shall-we-say, PESKY observations about the limitations of TSM – especially when it comes to relocating volumes that are being replicated.

Historically, Chris tends to mislead through incompletely reasoned logic and abject blind bias (I’ve suggested to him on more than one occasion that he is insulting the intelligence of his audience, but he still persists with his blissfully ignorant attacks). And he doesn’t fail to follow form with his latest…

In fact, it’s almost as if Christophe is Mr. T reincarnated (remember THOSE silly adverts?)!
 

quit yo’ jibba jabba

For those of you who can’t decipher what Chris is REALLY saying, here's the translation into the King’s English:

1. As Chris confirms, TSM cannot currently relocate LUNs while they are being replicated. Customers are indeed out of compliance during any LUN migration (not so with V-Max’s Virtual LUN migration).
2. And did we all know before Chris explained it that it isn’t even possible to relocate a LUN that is being Synchronously replicated until the latest USP-V Beta?
3. And lets not forget, TSM limits you to QUEUING a maximum of 64 LUN migrations, but only executes them in groups of 8 at a time, with a 15-minute pause in between groups. If' you’re needing to relocate a large # of volumes (like say, during a tech refresh), you are in for a long wait. And a massive impact on performance for the duration of the move to boot. 
   
   On the other hand, EMC’s Open Replicator/Live Migration and Virtual LUN both can migrate hundreds or even thousands of LUNs concurrently, without disrupting replication and WITHOUT impact on response times of running applications!
4. And yes indeed folks, here we are a year later, and Hitachi is only just now shipping the exact same Flash Drives as EMC, with the exact same MTBF ratings as the drives EMC began shipping a year ago. Chris makes it sound like the MTBF suddenly got better, which is anything but true.
   
   And indeed EMC took the leadership position in driving down costs AND ensuring enterprise-class reliability of EFDs. But lacking EMC’s experience and understanding of the technology, Hitachi configures these EFDs with less usable capacity than EMC, resulting in a higher $/GB than EMC.
5. Despite all the assertions to the contrary, Hitachi has nothing like either LUN-level or sub-LUN Fully Automated Storage Tiering (FAST) in their portfolio nor on their roadmap – and Chris confirms by not responding to my challenge to demonstrate otherwise. EMC is charting a course to a new way of deploying storage that will capitalize on the lowest $/GB as offered by SATA drives combined with the lowest $/IOPS as enabled by solid-state Enterprise Flash.
   
   Meanwhile Hitachi is caught flat-footed and indeed has no response – nothing, nada, zip and zilch. Nothing but crickets…
6. In marketing-land, there is an eternity of difference between "capable of" and "actually doing". A massive subset of - dare I say MAJORITY - of Hitachi/ HP/ Sun installed base wisely chooses NOT to use Hitachi’s external storage virtualization CAPABILITY, and thereby they avoid suffering the performance impact, risks of data loss and vendor Trojan Horse lock-in that is inherently introduced by adding another layer to their storage infrastructure. And with V-Max today supporting over TWO PETABYTES of usable capacity, many are now asking themselves whether all the UVM hype is really necessary (much less realistic).
7. As IDC and Gartner both note this quarter, a mere 12,500 USP-V+USP-VMs in over 4 years is barely enough to beat out the DS8000 for 2nd place in enterprise storage market share, leaving Symmetrix in sole command of #1 for what – the 16th consecutive year?
8. As the smart people over at Wikibon point out, utilizing storage that is over 4 years old is not only increasing the risk of data loss (NEWS FLASH: disk drives DO wear out, which is why maintenance contract prices increase as the drives get older), Running old storage also has a disgracefully negative impact on energy efficiency. New storage is more reliable, more efficient, and has less impact on our environment.
9. Upon closer inspection (and a weeks worth of prying eyes), HAM is today near-universally recognized as more marketing hype than practical reality: it requires proprietary host software and operational intervention of many hosts, it does not provide the promised active-active data clustering, and it doesn't work for mainframe or Oracle RAC or most other clustered server environments. Thus, UVM customers remain trapped with no way to non-disruptively upgrade to a next-gen USP-V/VM platform (if such a thing ever shall exist).
10. Fortunately, EMC’s Open Replicator/Live Migration coupled with PowerPath Migration Enabler offers customers a more efficient and non-disruptive means to migrate off of older storage onto either DMX or V-Max…and not just from Symmetrix storage – EMC’s solution provides non-disruptive or minimally disruptive migration off of most Fibre Channel storage…TODAY. And SRDF has provided HA with array fail-over for years…

Given the growing rumors that HP will soon replace USP-V with their own native clustered EVA storage arrays, plus the obvious demise of Sun as a reseller once Oracle takes over, we should probably be asking ourselves: How soon the USP-V will go the way of the Hitachi Mainframe!

Needless to say, you can all rest assured that the jungle cat that is V-Max will be increasingly painful in HDS' future – however short that future may be!

Another post by the storage anarchist!

technorati tags: EMC, V-Max, Symmetrix, Hitachi, USP-V, HAM, UVM, Open Replicator, PPME, PowerPath, HHAM, EFDs, Enterprise Flash Drives, FAST, Flash, Green IT, DMX, SATA

Seems Claus (and HDS) don't want to answer any tough questions about HAM.

What are they hiding?

background

This started when I posted these questions on Claus's "Eggs in a Basket" post:

the storage anarchist on 28 May 2009 at 1:05 pm

I’ll happily focus where you’d like: but first understand that we are all still waiting for you to explain the host requirements.

Does HAM require host-resident code? Does the host have to have HBAs configured to point at both old- and new- storage? Can all of this host “preparation” be implemented non-disruptively? Are the two data paths truly active-active - writes down both, or active-passive? If the second member of the USP-V cluster is remote (say, 80KM away), does the host still have to have a connection to both arrays? If the “local” array fails, all of the hosts’ I/Os will be serviced by the remote - right? How much standby communications bandwidth would a customer have to have to utilize this? And since the two arrays are essentially mirrors for each other, does that mean the customer requires 2x replication and other SW licenses to use in a HA scenario? What about if just doing a migration. And are the WWN and SCSI ID of a given LUN the same on both sides of the HAM cluster?

And since remote replication has to be suspended whenever you actually relocate a LUN (and you can actually only relocate 8 LUNs at a time from a maximum queue of 64 LUNs) - and THAT’s either within the USP-V or to/from externally virtualized storage, how is the remote site kept in sync with the primary during LUN migrations? And doesn’t the storage admin actually have to perform TWO separate migrations - one on the primary and one on the remote - should they want to relocate a LUN? During that relocation period of time, aren’t customers out of compliance - and in fact still at risk of data loss should the primary fail?

Oh - and when does HAM ship? How much does it cost? Is it even in Beta (I know HDS customers who were told that the planned May Beta had been rescheduled for August - is that true)?

Enquiring minds wanna know.

Claus actually answered a selective subset of those questions in his mildly disrespectful Response to Barry Burke post (I actually thought the 'TIS A STRANGE ROACH" anagram of storage anarchist was pretty clever).

But when I reposted the unanswered questions from the quoted section above, he decided to censor the inquiry as irrelevant.

I'll leave it to you to decide if my questions are out of line – but I suspect that customers really do need to know if HAM is going to corrupt data or take them out of compliance while they migrate/relocate their data. And that what Hitachi announced in May '09 isn't actually shipping until "Q4" (and I'm not even sure if that's Calendar Q4'09, or Hitachi Limited's fiscal Q4 which ends in 2010!) Not that pre-announcing is bad (EMC did the same thing for FAST), but the difference is that EMC didn't try to hide the facts about FAST – and I in fact answered any and all  V-Max and FAST related questions that came in on my blog.

Finally, although Hitachi claims that Thomson Reuters is actively using HAM (they were the HAM reference site), there are strong rumors that other prominent Hitachi stronghold customers have been told that the scheduled May beta won't actually start until August.

Anyway, if anyone else has answers to my questions, please feel free to send them my way.

technorati tags: Claus Mikkelsen,blog censorship,green eggs,HAM

EMC Proposes to Acquire Data Domain for $30.00 Per Share in Cash

A cool $1.8 billion in cash, immediate tender offer.

How do you like THEM apples?

Here's a quick rundown of early discussions:

• Chuck Hollis has posted a thought provoking commentary on the counter offer.
• Stephen Foskett opines over at Gestalt IT.
• Steve Duplessie explores the ulterior motives on his blog.
• David Vellante draws parallels to the Steinbrenner wars over at Wikibon.

Like Chuck said – Never a Dull Moment!

technorati tags: EMC,Data Domain,NetApp,Joe Tucci

Nick is not going to quit, nor are we. Everything we can imagine that could even possibly help him – we (as in the collective WE) will do. It’s in our DNA:

Failure is not an option.

And Nick undoubtedly appreciates our efforts:

Cancer survivor Steve Duplessie upped the ante today, and EMC is among the first to join in his pledge challenge. I know Steve – he has beat the odds once already, and now he’s putting his money where his mouth is to help Nick beat the odds.

I’d like to think that none of us need cash as a motivator, but if it sends even one more person to be registered as a donor, it will be money well spent.

Asian, Caucasian, Indian, African or not – please, PLEASE consider registering as a donor – time is running out – not just for Nick, but for the thousands of Leukemia patients waiting for a bone marrow donor.

Sign up now, and BE THE MATCH! The registry is woefully short on Asian/Caucasian donors – visit AADP.org if you can help improve the odds. And visit Mark Fredrickson’s blog to see how to short-circuit the system to get your samples to Nick as quickly as possible!

Thanks, everyone!

technorati tags: Nick Glasgow, EMC, Leukemia, bone marrow

The campaign to find a life-saving bone marrow match for Nick is accelerating, with broad support from customers and competitors alike.

In order to avoid this campaign taking on the characteristics of blog-o-spam, we EMC bloggers are collectively centralizing our communications about the progress of the campaign and the status of Nick's situation over on Mark Fredrickson's (new) blog: markfredrickson.wordpress.com.

Further, we are encouraging the use of the #helpnick tag on Twitter, and we are deploying the banner graphic above on our future posts, including those not specifically related to Nick.

All bloggers worldwide – EMC or not – are encouraged to use this banner and to link it back to Mark's blog. If you'd like to blog Nick's story, feel free, but we'd appreciate your linking to Mark's posts to ensure your/our respective readers know where to go for the latest status.

That said, I'd like to personally thank all the companies, partners, customers and competitors who have joined in the campaign. The broad support is truly appreciated by Nick and his family and friends, as you can see for yourself over at markfredrickson.wordpress.com.

Thanks!

technorati tags: Nick Glasgow,bone marrow,Luekemia

Enjoy!

This post is from the storage anarchist.

technorati tags: EMC,30th Anniversary,RunEMC

A life we can all save if we try. Help save Nick!

And the actual ReTweet request:

A life we can all save if we try. Help save Nick! - http://bit.ly/duO6W - PLEASE RETWEET!!!

Please help spread the word – as far and as quickly as possible!

technorati tags: Nick Glasgow, Lukemia, bone marrow

Which is all they can do, I guess, unless they are going to publicly explain in concrete terms why IBM is unable (or unwilling) to support the larger-capacity STEC ZeusIOPS drives in the DS8K that EMC has been shipping for Symmetrix since February 2009.

In the interest of those who really don't want to sift through the cruft to get to the reality behind the discussion, I outline for you here the simple facts of the debate:

1. EMC is shipping today the two largest-capacity enterprise-class flash drives available in the market – the STEC ZeusIOPS 4Gb/s Fibre Channel SLC-based drives in 200GB and 400GB capacities.
2. EMC refers to these drives as "Enterprise Flash Drives" (EFD) in recognition of their specific designs to support the availability and data integrity requirements of enterprise storage, and as opposed to the more common drives targeted at the server or laptop markets.
3. IBM reports to be shipping today the STEC ZeusIOPS 4Gb/s Fibre Channel SLC-based drives in 73GB and 146GB capacities only.
4. IBM calls its flash drives simply Solid State Drives (SSDs).
5. EMC's 200GB EFD and IBM's 146GB SSD are the same physical STEC ZeusIOPS drive, with 256GB of internal SLC NAND flash – the only difference between the two is that the EMC version provides more usable capacity from the same amount of flash.
6. EMC alone ships STEC's newest and largest ZeusIOPS 4GB/s FC drive with 512GB of SLC NAND, formatted for 400GB usable capacity.
7. EMC's 400GB EFD further reduces customer cost per usable GB, enabling customers to get more than twice the usable capacity from the same number of drives as IBM's largest SSD, or to use fewer 400GB EFDs to meet their capacity targets and thereby enjoy not only lower acquisition costs vs. the IBM DS8K, but reduced power, cooling and space requirements as well.
8. EMC asserts that the 200GB and 400GB formatting does not significantly reduce the practical life of either drive in any workload when used in EMC arrays, including pathological/artificial write-intensive workloads.
9. EMC stands behind this assertion with the same replacement and service warranty as is offered for both Fibre Channel and SATA-based hard disk drives in EMC storage arrays.
10. IBM has not yet explained publicly why it can not (or will not) offer similar capacities and the corollary cost savings on the DS8K.

Just the facts , ma'am.

This post is from the storage anarchist.

technorati tags: EMC,enterprise flash drive,EFD,SSD,solid state storage,STEC,ZeusIOPS,Symmetrix,DMX-4,V-Max,Tony Pearson,Barry Whyte,IBM,DS8000

And I have been trying so hard not to be The Storage Antagonist ;-}

Word to the wise, though – if you don't understand something, don't blog about it as if you do.

I've tried to get IBM's Tony Pearson to understand this repeatedly over the years, and he just keeps making the same mistakes. Probably has him despising me as much as that other blogger with the same first name, because every time he slips up, I'm usually there to correct him before his misinformation gets any traction.

This week TonyP is trying to wax intelligent on Flash Drives for the DS8K, but in his attempts to discredit my previous post, he removes any lingering doubt that IBM doesn't "get" flash.

Be sure to take the time to read the comments, and you'll see that TonyP clearly didn't take the time to understand the STEC ZeusIOPS drive or its wear-leveling algorithms. As a result, he pretty much embarrasses himself and his employer (not to mention the IBM Distinguished Engineers he throws under the bus) in the process.

At least he didn't try to drag Master Scientist BarryW down with him!

So, knowing that TonyP wouldn't dare to actually do the math for his readers, I will…
 

hey tony! here are the answers to the quiz!

Using the architectural definitions and modeling tools for the STEC ZeusIOPS wear-leveling algorithms and assuming that the SLC NAND flash will tolerate exactly 100,000 Program/Erase (P/E) cycles, the math says that the latest version of the 256GB (raw) STEC ZeusIOPS drive will wear out below it's rated usable capacity when exposed to a 100% 8KB write workload with 0% internal cache hit at a constant arrival rate of 5000 IOs per second in 4.92 years when configured at 200GB, and in 8.91 years configured at 146GB (yeah, I was off by .08 years).

Unfortunately, I cannot share the actual data or spreadsheet used to compute these numbers because they contain STEC proprietary information about their architecture and wear-leveling algorithms. So you'll have to trust me on this, and trust that IBM and EMC are in fact using the same STEC drives with the identical wear-leveling algorithms, just formatted at different capacities.

At a mix of 50/50 Read/write, the projected life of the drive is 9.84 years @ 200GB, and 17.8 years @ 146GB. And for what TonyP asserts is the "traditional business workload" (70% read / 30% write) the projected life expectancy is a healthy 16 years @200GB and 30 years @146GB.

Now, that's long enough for the drives to be downright ancient - more likely they will have been replaced with newer/faster technology long before the drive is even half-through its P/E life expectancy under those conditions.

So in the Real World that we all actually live in, nothing is ever 100% write – even database logs (which are not recommended for Flash drives) will not typically generate a 100% constant write workload at max drive IOPS. And the current generation of SLC NAND has been observed to easily exceed 100,000 P/E cycles, so even the above numbers are extremely conservative.

No, the truth is, the difference between the projected life at 146GB and 200GB on a 256GB (raw) ZeusIOPS is truly insignificant...and your data is no more at risk for the expected life of the drive either way.

Unless, of course, your array can't adequately buffer writes or frequently writes smaller than 8k blocks which will drive up the write amplification factor...two issues I suspect the DS8K in fact suffers from. Which, of course, would explain why IBM's Distinguished Engineers wouldn't want to take the risk with the DS8K. They don't get to be DEs by leaving things to chance, to be sure.

Symmetrix, on the other hand, isn't subject to these risk factors. Writes are more deeply buffered and delayed by the larger write cache of Symmetrix (DS8K is limited to 4GB or 8GB of non-volatile write cache vs. 80% of 256GB on DMX4 and 80% of 512GB on V-Max). Symmetrix writes are always aligned to the ZeusIOPS' logical page size to minimize write amplification, and the P/E cycles experienced by the NAND in the drive is proactively monitored to enable pre-emptive replacement should a drive exhibit premature or runaway wear-out.

Not so the DS8K, apparently…hence the conservative approach.

But don't be fooled – the deficiencies of the DS8K mean you will pay more Dollars per Usable GB of SSD on a DS8K than for Symmetrix DMX4 or V-Max EFDs.

Oh – and for the record TonyP, I don't think I ever said EMC was using a newer or different EFDs than IBM. I just asserted that EMC knows more than IBM about these EFDs and how they actually work in a storage array under real-world workloads. Thus, EMC are able to ship drives configured with more usable capacity per device without increasing the risks to customer data.

See, while IBM was playing catch-up,

EMC DID THE MATH!

technorati tags: EMC,Symmetrix,V-Max,DMX,Flash,EFD,SSD,NAND,IBM,DS8K,DS8000,STEC,ZeusIOPS,DMX4,Tony Pearson,performance,availability,wear-levelling,Program / Erase,P/E

I complained then that IBM was throwing cold water on a very key new technology; the fact that the errors in that white paper STILL haven't been corrected after nearly TWO MONTHS underscores my observation that IBM is totally out of touch with reality, and no longer the "trusted advisor" they once were.

(What happened, BarryW – I know you were working on getting those errors corrected!)

But today's news takes the cake: instead of doing it themselves (today IS IBM-Announcement-Tuesday, after all), IBM let STEC be the one to announce IBM's support for flash SSDs.

In my book, when you trivialize the importance of ANY technology to the point of having your supplier announce GA and availability rather than doing it yourself, it means something. And when IBM's sales force is to this day telling prospects that flash SSDs are "unproven technology" and "not ready for the enterprise," I can only conclude that IBM is embarrassed to admit some huge limitation or inadequacy of their products when used with Flash.

So, I asked myself…
 

…what is ibm trying to hide?

Now, I know better than to ask either BarryW or TonyP for an honest answer to a question like that.

So instead I turned to the paragon of accuracy for all things IBM – RedBooks and RedPapers.

Lo and behold, there is actually a RedPaper titled "DS8000: Introducing Solid State Drives!" Looks like exactly the resource I'd need to understand how IBM is supporting flash in their array…and possibly some insights as to why they aren't making any noise about it.

And indeed it is, even though the current version is plainly labeled "Draft Document for Review April 28, 2009 3:02pm." While there are likely to be some minor grammatical edits, I'm reasonably confident that the actual implementation and configuration details won't change.

Here's what I learned about their DS8K support for STEC SSDs from this RedPaper:

• SSDs are available only on new DS8K systems – they cannot be added to existing systems (EFDs can be added, non-disruptively, to any CLARiiON CX4-series, Symmetrix DMX4 or V-Max storage array, new or as an upgrade)
• The DS8K does not support RAID-6 or RAID-10 on their SSDs - only RAID- 5 is allowed at this time (All of EMC's arrays today support RAID 5, RAID 6 or RAID 1/10 on EMC EFDs)
• DS8K SSD’s must be installed on a DA pair without any other drive types on the DA (CLARiiON, Symmetrix DMX and V-Max allow EFDs to be placed on any drive channel and to be intermixed with any other drive type)
• A maximum of 16 SSD's can be installed on any DS8K DA pair, for a total maximum of 128 SSDs (Symmetrix DMX4 supports128 EFDs standard and more via RPQ; V-Max supports 256 EFDs and more via RPQ)
• If you install ANY SSDs in a DS8K, your maximum configuration is limited to 2 expansion bays and your maximum number of drives (SSDs + HDDs combined) is limited to 256 (128 for the smaller DS8K) (EFDs have no impact on the total number of drives or expansion cabinets you can configure on with CLARiiON, DMX or V-Max, and if you want a V-Max with 2400 EFDs, all you have to do is ask).
• You cannot order a DS8K with only SSDs (both Symmetrix DMX-4 and V-Max can be RPQ'd as flash-only configurations, although using HDDs for vault drives is more cost-effective).
• Although STEC asserts that IBM is using the latest ZeusIOPS drives, IBM is only offering the 73GB and 146GB STEC drives (EMC is shipping the latest ZeusIOPS drives in 200GB and 400GB capacities for DMX4 and V-Max, affording customers a lower $/GB, higher density and lower power/footprint per usable GB.)
• The only way to relocate data from HDD to SSD on a DS8K is via host-based copy utilities, which consumes host MIPS and adds load to both server and storage while potentially requiring application downtime to effect the relocation (Symmetrix DMX4, V-Max and CLARiiON all offer array-based non-disruptive Virtual LUN relocation; VLUN on Symmetrix supports both Open Systems and Mainframe volumes, and V-Max also allows for changing the RAID type when relocating a LUN to a different tier)
• z/OS support of Solid State Drives in the DS8K will be available on z/OS V1.8, or later. (Both Symmetrix DMX4 and V-Max support EFDs for mainframes running any version of z/OS, z/VM or TPF. Additionally, V-Max supports EFDs for use by iSeries and AS/400 servers)

And finally, despite all the bluster and fanfare of IBM's Quicksilver demo last year, there has been no mention to date of anything truly innovative in IBM's use of flash. In particular, not a peep out of IBM (or anyone else, for that matter) that they have anything even REMOTELY similar to EMC's Fully Automated Storage Tiering (FAST) that was announced on April 14 along with the V-Max.

For while flash as a "Tier 0" is interesting and useful ("Tier 0" is an EMC term that IBM has copied, by the way), the true full power of flash will only be realized when the system can automatically and dynamically utilize flash capacity to accelerate the LUNs and sub-LUN data chunks that are used most frequently and/or that achieve the greatest benefits from significantly reduced response times.

Come to think of it, the DS8K doesn't even offer thin provisioning – with Symmetrix and CLARiiON, you can use Virtual Provisioning on EFD capacity to maximize the utilization of that expensive capacity!

the true meaning – ibm is abandoning storage

Or at least, they clearly aren't taking storage seriously.

More importantly, the aging, decrepit DS8K is no longer a viable storage platform for the enterprise. Later this year, IBM will finally get around to upgrading the aging DS8K processors to the p6, but it won't be enough to catch up with the Symmetrix V-Max.

And the XIV debacle is almost over, as customers learn first hand that everything I've said about the data corruption and performance is in fact true (thanks to all the free eval units seeded over the past 12 month). Moshe has cashed all his checks, and the shine has worn off for that "special XIV sales overlay," as they all get hit with real quotas, the end of their draws, and the ultimate insult of getting folded back into the mainstream sales force.

And while IBM has been handing cold cash to Moshe, it has been starving the DS8K. Meanwhile, Joe Tucci has upped funding on storage platforms, and the Symmetrix V-Max has totally re-defined enterprise-class storage. True scale-out architecture, built on industry standard components, and leveraging the time-proven Enginuity operating system (instead of cobbling together a loosely connected hodge-podge of Linux-based storage services), Symmetrix V-Max has defined a storage future that inevitably won't include any IBM-manufactured storage.

IMHO, at least.

This post is from the storage anarchist.

technorati tags: EMC,Enterprise Flash Drivesr,STEC,Zeus-IOPS,IBM,DS8000,DS8K,EFD,SSD,Solid State Storage,RedPaper,FAST,Fully Automated Storage Tiering

Time sure flies when you're having fun.

I've been all kinds of busy lately, what with all the global customer interest in Symmetrix V-Max and FAST. It is truly refreshing to take a step back from the blog wars and share perspectives and insights with customers and prospects. And because the V-Max addresses so many of the efficiency, cost and ease-of-use concerns facing IT today, while setting a unique and differentiated path for the future, I'll tell you that the conversations are truly exciting.

I won't do a retrospective of the past year, but if you want to take a walk down memory lane, start here with my one-dot-oh post. Looking back, a lot of the things I discussed a year ago are still true today, which should be sorta obvious, since V-Max is the most significant new technology in the storage industry since EMC introduced Flash Drives at the beginning of 2008.

Not much else has changed (insert token competitor-backhand slap here :-).

But anyway, here's to another year in the world of the storage blog-o-sphere, and to the growing social network of customers and competitors that will undoubtedly continue to underscore the conversations of our industry. With all the new Twitter, FaceBook, FriendFeed, BackType and LinkedIn stuff intermingled, we've created a pretty broad spectrum of interconnected people. And it's kinda fun hearing from my non-techie friends that they find my blog entertaining, even though they have nothing to do with the storage industry.

So to all my readers: thanks for your continued support-slash-debate (as the case may be).

TTFN!

technorati tags: the storage anarchist,EMC,Symmetrix V-Max

After a week that saw its flagship product superseded by EMC’s Symmetrix V-Max, its sole product differentiation obsoleted by VMware’s vSphere, and the remains of it’s second-largest reseller literally swept out from under their feet, Hitachi Data Systems has decided to call it quits in the information storage market.

In the midst of the global economy rattled by recession, parent company Hitachi Ltd. (Japan) was apparently no longer able (or willing) to support the foolish acquisitions and free-falling margins delivered by its Santa Clara-based Hitachi Data Systems subsidiary as they struggled to challenge 18-year market leader EMC and its VMware virtualization juggernaut division.

Not surprisingly, and even though the myriad of misguided marketing campaigns over the past several years have clearly been an embarrassment to the mother ship, the latest instantiation of the HDS marketing machine is going out with one last hurrah.

Launched yesterday with fanfare not seen since Circuit City’s going out of business close-out scam, the HDS “Switch IT Off!" Liquidation Sale leverages the Earth Day platform as it aims to unload the massive inventories of unsold USP-V enterprise-class-wanna-bee storage arrays on unsuspecting consumers world-wide.

Although companies in the IT space come and go all the time, the demise of HDS is unique, if only in its timeline. Riding high just a week ago when they announced version 2.0 of their VMware SRM adapter, HDS executives had no reason to suspect that things would get so bad so fast.

But on the very day of their SRM announcement, they found themselves caught in the beginnings of what can only be called a “perfect storm”: the undeniable superiority of Symmetrix V-Max, being locked out of the vSphere virtual data center of the future by VMware, and Oracle's unexpected total eclipse of the Sun. Despite the valiant efforts of the HDS bloggers (and bloggers-for-hire) to mislead and misdirect, the aftermath of these events has apparently left HDS leadership with no choice but to cut their losses and refocus on construction equipment consumer electronics set-top cable boxes whatever's next.

Neither HDS nor Hitachi Ltd. officials were available for comment on this story.

I know, it isn't April Fool's Day. But I missed it this year, and I just couldn't resist! 

technorati tags: EMC,Symmetrix,V-Max,VMware,vSphere,Hitachi Ltd.,HDS,Hitachi Data Systems,Liquidation Sale,Switch IT On!,Switch IT Off!,virtualization,virtual data center of the future,Inspire the Next

The virtual data center becomes real.

And if I may be so humble, more important to customers than the announcements themselves is the Day 1 integration between and across the products and companies.

Case in point: EMC has so many integration points with vSphere 4 that it takes two press releases to include everything:

• EMC Integrates with VMware vSphere 4 for Next-Generation Virtualized Data Centers
  Aligns Products, Solutions and Services with VMware vSphere 4 for Next-Gen Levels of IT Choice, Control and Flexibility
• EMC Delivers High Availability Path Management for VMware Vsphere 4 Data Centers
  New EMC PowerPath/VE Enables Customers to Improve Performance, and Simplify, Standardize and Automate Path Management Across Physical and Virtual Environments

Many of the V-Max ease-of-use features announced last week are targeted specifically for massive-scale vSphere environments, while things like the EMC Storage Viewer vCenter Plugin,   EMC's adapters for Site Recovery Manager, and now the new EMC PowerPath/VE work equally well with all of EMC's arrays, including both DMX and V-Max.

Chad Sakac, EMC's resident VMware evangelist-extraordinaire provides his take on today's announcements over on his Virtual Geek blog – I imagine he'll have more to say (he's the only storage blogger that writes longer articles than me, BTW – you were warned!).
 

hey – where is everybody?

A quick web scan for press releases this morning at 9:30am EDT reveals that neither HDS nor IBM have anything to say about vSphere. IBM is probably busy explaining their earnings report (20% decline in storage revenues alone) and trying to figure out how they let Lucky Larry steal their future in broad Sun-light!

On the other coast, the poor HDS bloggers are clearly still trying to figure out what hit them last Tuesday. They are aggressively defending their "we were first" pride while actively and repeatedly demonstrating their ignorance of what was actually announced. In fact several of them are spewing misinformation so vapidly that they appear be stuck in a time warp, competing against a circa 1999 Symmetrix instead of the 2009 Symmetrix V-Max .

The misinformation is understandable, I guess, so I'll cut them some slack. I'm sure their attention was distracted by the April 14th HDS press release announcing v2 of Hitachi's SRM adapter for Vmware. Which reminds me – hey, HDS marketing brainiacs:

Shouldn't you have WAITED A WEEK? 

I mean seriously, get a clue! Speaking of which, here's a hint for Hu, Claus, Christophe and Michael: everything you need to know about Symmetrix V-Max is right here on my blog.

And as I keep reminding people inside and outside of EMC, unlike in a race, when it comes to technology "first" only matters while it is also "only." Once competition arrives, the only one that matters any more is "best." (Chuck, you might want to add that to your friendly coaching post)

Symmetrix V-Max: Better, Faster and Easier.

And purpose-built for the vSphere 4 virtual data center!

technorati tags: EMC,Symmetrix,Symmetrix V-Max,V-Max,VMware,vSphere,vSphere 4,virtual data center,PowerPath/VE,virtual matrix architecture,IBM,HDS,Hitachi,SRM,Site Recovery Manager,vCenter plug-in,EMC Storage Viewer

For those of you who weren't able to participate in this week's Symmetrix V-Max virtual launch, here are most of the video's used in the event:

emc's vision for the virtual data center

delivering on emc's vision: the virtual matrix architecture

vmware and symmetrix v-Max

cisco and Symmetrix v-max

intel and symmetrix v-max

microsoft adcenter and v-max

technorati tags: EMC,Symmetrix V-Max,Virtual Matrix Architecture,virtual data center,Joe Tucci,Dave Donatelli,VMware Integration,Paul Maritz,Cisco,Rob Lloyd,Intel,Pat Gelsinger,Microsoft,AdCenter,Mike Anderson,Bong Kang

techmute.com: Response to Tony’s V-Max Questions

Thanks, Matt – There's clearly room for improvement on EMC's communications to some audiences, but I think you received almost 100% of the intended message.

In answer to Matt's lingering uncertainty in questions 9 & 10 at the end of his 1st comment response to TonyA:

9. Being purpose-built for the its current (and future) functionality, rest assured that the performance of Symmetrix V-Max wide-striping and SRDF are not restricted by the Virtual Matrix Architecture…in fact, most customers will realize improvements to both relative to DMX4.
10. Best practices for Symmetrix V-Max were made available simultaneously with General Availability of the system. Although not radically different from DMX4 for existing features, best practices for new capabilities were developed and validated with the assistance of Beta Sites and in-house CSE expertise.

technorati tags: EMC,Symmetrix V-Max,Virtual Matrix Architecture,blogger-for-hire,Tony Asaro

But wait – there's more!

OK, not really more than has already been announced, but there were several parts of yesterday's announcements that I didn't personally cover here on my own blog.

So I thought I should highlight a few of them so that readers and visitors get a complete picture (for a complete link page of relevant launch articles, visit my first Overtake the future post):

• Auto-Provisioning – a key new feature that streamlines and accelerates the process of storage allocation to clustered servers, as in VMware ESX clusters. Both Steve Todd (Information Playground) and Mark Twomey (Storagezilla) provide in-depth coverage (Mark even links the demo video).
• Symmetrix Management Console (SMC) Templates & Wizards – new features that automate a wide variety of common storage management functions to make them more easily repeatable and delegable. Steve included insights into these in the post linked above as well.
• VMware integration – beyond Auto Provisioning and the continued Symmetrix integration with VMware's Site Recovery Manager, there is also new EMC Storage Viewer plugin for VMware's vCenter. This plugin visually bridges the gap between VMware admins and storage admins. Chad-the-Virtual-Geek-Sakac answers perhaps the most frequently-asked question in the press yesterday: "So, just how *IS* V-Max integrated with VMware?"
• New SRDF/EDP (Extended Distance Protection) – essentially, zero-data-loss multi-hop Asynchronous long-distance replication, without the need for a full copy of the data volumes in the middle site – reducing the cost of extended distance business continuity. 'Zillaman does the honors on this new SRDF option as well…
• V-Max Quality – a common concern about any new product is how complete and stable the product is at initial release. Given the significance of switching to Intel processors and changing the underlying architecture required to deliver Symmetrix V-Max (not to mention the incredible scale of the new system), the focus on delivering Quality on Day One has never been higher. Although not a V-Max developer himself, Steve Todd went behind the scenes to explore some of the innovative strategies for quality taken by the V-Max development organization.

All in all, there is a TON of information here – I know I personally wrote over 10,000 words about the new products and architecture in support of the launch.

And note - I wrote these posts over the course of about 10 days, so don't feel bad if you can't read it all in one sitting.

technorati tags: EMC,Symmetrix V-Max,Virtual Matrix Architecture,Auto provisioning,SRDF,SRDF/EDP,VMware Integration,SMC,Storage Management Console,ESX Clusters,VMware,vCenter

Today's announcement is chock-full with exciting news.

First, there is the breakthrough Virtual Matrix Architecture, combining the best of Scale Up and Scale Out to revolutionize enterprise storage.

And then, the Symmetrix V-Max itself, integrating the proven power and functionality of the Enginuity storage OS on a new industry standard platform to deliver cost-effective flexibility and a new definition for ease-of-use in enterprise storage.

Next up? Perhaps the most exciting (and unexpected) announcement of all.:

Fully Automated Storage Tiering (FAST)

As Chief Strategy Officer for the Symmetrix Product Group, my role in today's global Virtual Launch is to describe and answer questions about FAST in one of the half-dozen or so "break out" sessions available to all participants. Given the high level of interest in the topic, I thought I'd take a few moments and discuss FAST here as well.

So, what exactly is FAST?
 

solving the challenges of tiering storage

Put simply, EMC's FAST technology is designed to automate the allocation and relocation of data across two or more storage tiers based upon the performance requirements of the applications.

The objective of tiered storage is to minimize the cost of storage by putting the right data, on the right tier, at the right time. In a practical sense, IT shops often translate this into storage tiering standards, such as:

• Performance-sensitive applications go on mirrored 15K rpm drives
• Bulk data and backups go on large SATA drives, ideally protected by RAID 6
• Anything we aren't sure of goes on 10K rpm drives, possibly using RAID 5

At a high level, this makes empirical sense, and many customers today have deployed tiered storage to reduce their overall storage spend, especially by pushing more and more data down to the SATA tier. It is generally pretty easy to identify classes of data that really don't justify the expense of more expensive spinning disk media. Backups, photographs, documents, presentations, spreadsheets, software development, etc. are just a few examples. In fact today, you'll find development and even production database applications running on SATA drives, especially in arrays (like the DMX) that can offset the relative slow performance with intelligent caching and pre-fetch.

The introduction last year of Enterprise Flash Drives for Symmetrix DMX-4 opened up a whole new frontier for tiering.

Yes, the performance advantage of EFDs over spinning rust is huge, and indeed EMC has almost single-handedly driven the cost/GB of EFDs vs. HDDs down from about 40x a year a go to around 8x today. But because flash still costs much more than HDDs, customers tend to be very surgical in what they put on these drives, aiming to get the most possible benefit from the smallest possible capacity.

Generally speaking (as I've proposed on these pages before), the best practices for using Flash as a tier (put the busiest LUNs on flash) is more than sufficient to justify the cost. And with the ability of Symmetrix and CLARiiON Virtual LUN to non-disruptively relocate a LUN to a different tier of storage, this is exactly what many customers have done.

However, as many of my readers have pointed out over the past year, it is not always that easy to capitalize on the performance of EFDs. Finding the busiest devices can be difficult, although EMC tools, white papers and the Symmetrix Management Console offer a lot of assistance.

But even when you can find the "hot" LUN, it may not be "hot" tomorrow (or next week).

EMC's FAST technology is designed to help customers get better value out of their tiered storage, both fast EFDs and huge SATA drives, as well as their 15K and 10K rpm media. And FAST will do this by automating the allocation and utilization of available resources.

Under development for several years, EMC will be delivering FAST technology across all of its storage platforms (Symmetrix, CLARiiON and Celerra) beginning in 2009 with Symmetrix.

building the foundation of fast

FAST has not happened overnight – in fact, the foundation of FAST has been developed and delivered to customers in stages over the past several years. And that foundation is the collective of features and enhancements that EMC has delivered on Symmetrix DMX over the past 5 years.

In order to automate storage tiering effectively, you first need an array architecture that is adept at handling multiple different tiers of storage within a single array. Symmetrix was the first array to support slower drives (the 183 GB 7200rpm SCSI drive back in 2001); DMX was the first with LC-FC (aka FATA) and then native SATA drive support.

And we all know which array was first to support Enterprise Flash Drives.

But being able to plug in lots of different drive types into an array isn't enough: the array also has to be able to adjust its performance to maximize the benefit of each drive type, while not degrading the performance of important applications. And this isn't as simple an you might think. In fact, you need look no further than the excuses postured by both IBM and Hitachi over the last couple of years for why they took 1-2 years longer to support Flash and SATA in their arrays than it did EMC…they had to learn what EMC had already solved, for without proper precautions, slow SATA drives can slow everything down. And Flash drives are so fast that they can effectively starve the rest of the system – if you don't know how to protect against that like Symmetrix and CLARiiON do.

The FAST foundation doesn't stop with storage tiers, either. Dynamic Cache Partitioning and Priority Controls in Symmetrix are also key enablers to the ultimate objective of automated tiering – being able to adjust cache allocation and relative priorities for different I/O streams are crucial.

And let's not forget the 18 years of cache optimizations that have gone into Symmetrix. Automation requires analysis and prediction of what data will be required "soon," and the cache management algorithms in Enginuity are second to none. In fact, Symmetrix today can maintain very high cache hit rates, even though the servers connected to an array have more collective local cache than the entire Symmetrix! Uncanny, but true!

The final foundational technology is Virtual LUNs – the ability to non-disruptively relocate a LUN (or CKD volume, for that matter) from one tier of storage to another. Symmetrix DMX3 and DMX4 today support this feature, and customers use it widely to promote and demote data to different tiers as their needs change (CLARiiON and Hitachi USP-V also support this feature; IBM DS8K does not).

Symmetrix V-Max takes Virtual LUNs to a whole new level – beyond what any current platform offers:

First, you can change RAID protection types as you relocate LUNs and volumes;

Second, you can relocate more LUNs and volumes concurrently (up to 128 times as many as a USP-V can move simultaneously)

Third, you can relocate data faster (moving data 2.8 times faster – or more – than a USP-V).

Fourth, you can relocate data with less performance impact on the system overall and on the applications being moved

and Fifth, you can relocate LUNs and volumes without disrupting replication sessions (local or remote), maintaining compliance as you rebalance your resources.

Not a bad place to start implementing automation now, is it?

fast roll out

In a very real sense, Virtual LUNs on V-Max are the first phase of FAST – you get the "automated" quick relocation without disruption, but you have to make the decision of what to move and when yourself. Still, given the simplicity of the user interface to perform Virtual LUN relocations, it will seem almost automatic.

At least, that's the feedback EMC got from the beta sites. Most were simply amazed at how easy it was to use Virtual LUNs.

Later in 2009, EMC will deliver the first FAST add-on option for V-Max customers. Essentially this will be the automation engine for Virtual LUNs, and it will monitor performance and demands on individual LUNs to make recommendations of moves that could improve performance. As with today's Symmetrix Optimizer, we expect people to first utilize FAST in this "recommendation" mode, but over time they will probably let it do its job without any intervention. The system will just move data (and change RAID types) as workloads change, in accordance with administrator defined policy.

Following the automation of full-LUN relocation, a second version of FAST will be offered; this second FAST option will bring dynamic relocation to Symmetrix Virtual Provisioning volumes.

That's right, Martin. EMC has indeed been listening:

FAST will automatically and dynamically relocate data across multiple storage tiers, at a sub-LUN granularity, based on performance and cost requirements.

EMC's FAST for Virtual Provisioning has been designed to dynamically relocate subsets of a VP LUN to higher tiers of storage for better performance, or to lower tiers to reduce storage costs and make room for other performance-sensitive bits. And indeed, Virtual Provisioning for V-Max (in Enginuity 5874) already implements the requisite infrastructure to support FAST for VP; in fact, some of the meta-data enhancements I mentioned in an earlier post today are specifically to support utilization tracking that FAST will require with VP.

All in preparation for FAST.

and so the light dawns

With today's announcement of FAST, EMC's true intent for in-the-box storage tiering and both Flash and SATA technologies should now be much clearer.

It may surprise you to know, even, that it was partially in anticipation of the upcoming FAST option that EMC named its implementation of thin provisioning Virtual Provisioning in the first place – VP was designed from the outset to support sub-LUN level storage tiering that FAST will deliver.

And though Flash Drives were initially positioned as Tier 0, the plan all along has been that EMC's FAST will allow customers to save money through the automated utilization of tiered storage.

And now, with FAST revealed, we can envision future arrays that will employ a small amount Flash capacity to store the I/O-intensive subsets of data, plus a large amount of cost-effective SATA capacity to house infrequently accessed data. Some applications might also benefit from a sort of  middle tier of large 10/15K rpm drives for the data that isn't quite idle enough for SATA, and we might imagine adding other tiers as well (FAST won't be limited to 2 or 3 tiers, by the way).

Customers will thus require fewer 15K rpm drives than today to meet their IOPS objectives by leveraging the 10-30x or better IOPS capabilities of Flash for the busiest parts of each LUN. And they will also save storage costs by being able to more effectively leverage the low-cost capacity of SATA drives for data that is rarely touched. And combined effectively through FAST, the result will be large capacity systems that require far total fewer drives than systems built with disk drives alone.

Fewer drives, less power and cooling, smaller footprint, and better overall response times owing to the power of Enterprise Flash Drives and efficiency of SATA. The automation of FAST coupled with the management simplicity of Virtual Provisioning, Symmetrix V-Max will be even easier to operate and manage, since the system itself will

put the right CHUNK of data, on the right tier, at the right time!

This is the world for which the Virtual Matrix Architecture and Symmetrix V-Max were designed.

and for the record,

I'm excited to be able to explain EMC's real plans for leveraging Flash Drives and Tiered Storage, because I think it will truly change the nature of not only solid state, but it will revolutionize storage in general.

It is because of the FAST technology development that I haven't actually argued that Flash as a Tier 0 was the only (or best) way to leverage solid state storage technology. If you look back, you'll note that I always just pointed out that EMC was alone in delivering Flash as a tier for more than a year.

And I never denied that sub-LUN allocation of data on Flash drives was a good idea – because I've always known that sub-LUN tiering is in fact a BRILLIANT idea, since EMC engineers were hard at work on exactly that. But of course, I couldn't discuss that development until EMC announced the planned capabilities publicly. My pen was tied, so to speak.

As to using Flash as cache? Well, I think I will still have to tiptoe around that topic for a while longer, until EMC starts publicly discussing the actual FAST implementations. But one could imagine that FAST could have similar benefits as a dedicated Flash cache, only at a lower cost…so stay tuned!

Given the inevitable "me too" or "nobody needs THAT!" that we will soon be hearing from the competition, I look forward to a the future that FAST will enable…

It should be interesting, don't you think?

technorati tags: EMC,FAST,Fully Automated Storage Tieringi,Enterprise Flash Drivesr,EFD,sub-lun,Symmetrix,DMX,V-Max,Virtual Matrix Architecture,Overtake the future,Dynamic Cache Partitioning,Priority Controls,Virtual LUNs,Virtual Provisioning,SATA,fine-grained tiered storage

Taking a pause here from the launch to present an observation.

Remember how Hu Yoshida spent the first half of 2008 telling everyone who would listen that customers didn't need or want flash drives (if you missed it, I wrote about it here and there)? Hu was later silenced once Hitachi Japan announced that they actually would be selling the very same EFDs that EMC had been shipping since the beginning of 2008. (FWIW: Hitachi was supposed to ship the first of those drives in Q1'09, but I haven't been able to verify anyone receiving them).

Well, with the introduction of Symmetrix V-Max and the Virtual Matrix Architecture, it seems that EMC has once again done precisely what Hitachi's technical experts have been telling the world cannot be done. This time, however, I don't think Hitachi is going to be able to play follow the leader, since they're so mired in their backplane-limited Hitachi Universal Star Network crossbar switch architecture (formerly known simply as "Hi-Star").

Moreover, given their ever-increasing dependence on custom (expensive) ASICs, FPGAs and off-load engines, I predict it will be years before Hitachi's engineers can re-tool to leverage the price/performance curve of industry-standard components.

In support of these opinions, I offer the perspectives of none other than Hu Yoshida himself and fellow Hitachi blogger Michael Hay, in two separate and otherwise unrelated stories.

hitachi on enterprise scale-out architecture

About 3 years ago, ESG analyst Brian Garrett wrote the following in his blog on Computerworld about The storage controller of the future:

So what does the enterprise-class storage controller of the future look like? I believe it will look more like an emerging clustered based storage controller than the currently shipping enterprise-class solutions from EMC, HDS and IBM. Today clustered storage controllers communicate with each other using Gigabit Ethernet or Fiber Channel. In the future I believe we'll see 10GigE, or Infiniband, or ASI for PCI Express [he apparently overlooked RapidIO- tsa]. [snip…] Gone will be the proprietary bus architectures of legacy enterprise-class storage controllers. Gone will be the custom hardware connected to that proprietary bus. Gone will be the cost of all that custom hardware. Over time I believe that enterprise-class storage controllers built using standardized commodity components will be the [rule] rather than the [exception] as it is today.

I remember reading that back then and wondering how in the heck Brian knew about what we were doing with what we today announced as Symmetrix V-Max and the Virtual Matrix Architecture.

But no matter, I did then, and still do now, agree with his closing sentiment:

Besides storage geeks like me who've engineered storage controllers, why else would anyone care? [snip…] Vendors should care because first to market wins.

But what I found really interesting was that HDS’ CTO Hu Yoshida wasted no time in responding to Brian’s precognition with a blog post of complete and total disagreement that such an idea was basically, well, preposterous.

At the root of Hu’s disagreement was his assertion that maintaining cache coherency of inherently large-cached arrays was going to be impossible in a scale-out clustered architecture. And Brian agreed with him in his follow-up post More on the Storage Controller of the Future.

Well, sorta:

Hu got it right in his response when he stated that "a global cache is key to a scalable, enterprise storage controller." I agree wholeheartedly and believe that cache coherency is the biggest challenge when developing an enterprise-class solution…

[so do I – tsa]

Hu states that the "biggest problem with clustered solutions is that each node in a cluster has its own cache, and write data has to be replicated across the cluster of caches in order to maintain write consistency." While I agree with the second statement (cache must be mirrored), I disagree with the first statement.

In most of the emerging clustered storage architectures that I have been exposed to, the concept of a logically isolated cache per node has been scrapped. Instead a distributed service-oriented approach with a universally agreed upon, but truly distributed, addressing and lock management scheme has been implemented.

And indeed, that’s exactly what EMC has done with the Virtual Matrix Architecture and Symmetrix V-Max. In fact, distributed addressing and lock management is how Symmetrix has managed global memory since the very beginning!

I know Brian was pleasantly surprised to learn about V-Max and the Virtual Matrix.

I gotta think that the surprise this morning around Hitachi and HDS was somewhat less pleasant.

And, I suspect that Hu’s comments above speak volumes about the monolithic rat-hole that Hitachi has apparently dug themselves into with the entirely custom-built USP-V architecture.

Scale-out enterprise class storage is indeed hard, unless you already know how to do it!

hitachi on multi-core processors

For my second story, I'll turn to an almost prescient post put up by Michael Hay back on March 25th and titled To Multi-Core or Not to Multi-Core? In this post, Michael observes that Intel is finding it difficult to leverage its multi-core processors using general-purpose operating systems and applications. From there, he takes an esoteric side road to observe how the human mind "offloads" trivial tasks, often doing things without the conscious mind even realizing them.

From there, he asserts that the best architecture for storage is one built of custom hardware, proprietary ASICs and FPGAs, and whatever other expensive off-load components they can throw into the mix. And he summarizes that this is indeed the best possible approach, saying :

…Hitachi’s approach to storage uses offload techniques already today both on the block and file lines.  We have  general purpose CPUs, ASICs, and FPGAs coupled to key offloaded functions like RAID algorithms, file systems, etc. to realize our designs.  In short I think Hitachi realized long ago that a hybrid architecture with offloading to different systems is a great way to go. 

And then he closes with:

What are your thoughts on this point?

Well, Michael, my thought is that what you describe is yesterday's approach to storage.

Hitachi's archaic approach is born of the old-skool necessity to build everything custom because off-the-shelf processors weren't adept or fast enough for specialized applications Now, there are those who would point out that the USP-V us built on a particularly wimpy processor and thus needs all the help it can get (the USP-V uses MIPS-base processors, which the same CPU found in laser printers, set-top cable tv boxes and in GameBoys).

But I'll be polite and acknowledge that at least you can classify MIPS chips as "industry standard," although for an entirely different industry :-). But MIPS isn't even in the same league even as the Intel Xeon family of multi-core processors, to be sure.

And I won't even argue that most software isn't today designed to take full advantage of all the processing power in an Intel Xeon. Sure, VMware or Hyper-V can let you use those cores to run a slew of independent applications, but there's a huge void in the market of software that can actually corral all the power of multiple cores to address a single objective in a coordinated fashion.

Much less software that can today scale to use up to 32 quad-core Intel Xeons, parallelizing operations across 128 (or more) cores!

Symmetrix Enginuity does exactly this on the new Symmetrix V-Max

The fact is: Enginuity (the storage operating system for Symmetrix) has been massively parallelized since day one, back in 1991. In today's DMX4 line, Enginuity runs on up to 128 different Power processors, and in the new V-Max, Enginuity operates on up to 128 cores (on up to 32 quad-core Intel Xeons). The operating system has been designed to take advantage of multiple processors, and algorithms have been tuned for nearly 2 decades to ensure efficiency and performance with a minimum of custom support hardware.

Even though prior Symmetrix models had more custom-built components, today's Symmetrix V-Max employs but one built-for-EMC custom ASIC. This ASIC resides on the Virtual Matrix Interface controller, and is responsible for coordinating local and remote memory accesses – not so much as an off-load as it is a hardware-assisted distributed lock manager.

So you see, capitalizing on multi-core processors can indeed be difficult, unless you already know how to do it!

setting the new standard

Clearly, the Symmetrix V-Max and its Virtual Matrix Architecture are redefining storage in a big way, on multiple dimensions.

Now, obviously, both of the Hitachi perspectives I present here are taken out of context – neither of these two Hitachi representatives had any way to know specifically what EMC was up to with today's launch, and so they weren't choosing their words carefully.

But precisely because these perspectives weren't offered in response to Symmetrix V-Max, it is highly likely that that they represent the real perspective of their employer on scale-out architectures and multi-core. And given the rumors of continuing delays in the long-promised USP-V clustering, I don't think they'll be changing their spots any time soon.

Back to the launch!

technorati tags: EMC. Symmetrix V-Max,V-Max,Virtual Matrix Architecture,Overtake the future,DMX,Direct Matrix Architecture,multi-core processor,Hu Yoshida,Hitachi,HDS,USP-V,Michael Hay,industry-standard,Intel Xeon,parallel processing,scale-out,scale-up,Hi-Star,Hitachi Star Network Architecture,crossbar switch,MIPS,storage,enterprise storage,Brian Garrett,storage controller of the future.

Introducing the new Symmetrix V-Max^tm – the first enterprise storage platform to integrate the performance and efficiency of Scale-Up with the cost-effective flexibility of Scale-Out.

• Simple: Redefines enterprise storage architecture, ease-of-use and automation
• Scalable: start small, grow incrementally, supporting multiple tiers in a single array
• Cost-effective: more IOPS and more usable capacity per dollar (euro)
• Efficient: more IOPS and more usable GB per kilowatt
• Autonomic: Optimized for Fully Automated Storage Tiering across Flash, Fibre and SATA
• Compatible: Common management and SRDF interoperability with Symmetrix DMX
• Ready: Purpose-built for the Virtual Data Center

When you can list all those attributes for a single storage platform, you’ve made a statement.

Several years in development, today the new Symmetrix V-Max takes its place atop the world of external storage – right above the reigning #1 enterprise storage platform (according to IDC), the Symmetrix DMX4.

With its revolutionary scale-out Virtual Matrix Architecture, the Symmetrix V-Max literally redefines not only enterprise-class storage, but the entire storage landscape – because when an enterprise array offers the simplicity, performance, TCO, scale and flexibility previously found only in midrange offerings, you know you’ll change the world.

(By the way, BOTH Symmetrix DMX4 and Symmetrix V-Max are newer than either of IBM's and Hitachi's flagship enterprise arrays. It's like they're asleep at the wheel!)

And watch-out, you enterprise wanna-bees; V-Max has just raised the bar.

So, let’s take a look at the new king of the storage hill… 
 

specifications

No, I'm not actually going to blog the Symmetrix V-Max specifications in gory detail – the V-Max landing page is up on EMC.com if you want to see the details. And in case you're looking, you won't find any whacky Hitachi Math or claims of Millions of Meaningless IOPs (MIOPS) that Hitachi tends to throw around like candy when they introduce new products.

MIOPS: you know the ones I mean…the outrageously inflated aggregation of repeated tiny read requests served entirely from the front-end controller's local buffers, as if there were any application in the world that would never need to actually get data out of global memory or off of the disks.

So instead of MIOPS, the V-Max spec sheet shows straight specs – processor speeds, memory capacity, Virtual Matrix bandwidth. All of these are just ratings, and while the system can and will fill up the available bandwidths during workload bursts, there is no pretense that anyone will ever attain any of these maximums for any predictable length of time or for any given application.

Today's enterprise arrays are simply too complex to be represented by any single specification.

a look inside v-max

Open up the system bay of the new Symmetrix V-Max, and you'll find from 1 to 8 V-Max Engines (they're numbered 8 to 1 from top to bottom, as you saw in the video above). Each engine is actually two V-Max omni-purpose Directors, operating as independent halves of a 1+1 cluster. Each Engine can support up to 360 dual-ported fibre channel drives, and each Director is the primary initiator for half the drives, and the fall-back initiator for the other half. Drives can be any mix of FC HDDs, enterprise flash drives and SATA drives (in a special carrier that provides SATA-to-FC bridging and dual-port interfaces).

As announced today, V-Max is the first enterprise-class array built on industry standard components. Where the competition (and prior generations of Symmetrix) were based almost entirely on upon custom controller hardware, V-Max instead uses Intel® Xeon® processors and support logic, PCIe internal interconnects, and DDR2 fully-buffered ECC SDRAM DIMMs. Each director has 2 4 core Xeon processors chosen for their balance of I/O and bandwidth performance, power requirements, cost, and the supporting infrastructure.

(Although higher clock speeds and more cores do not always translate into higher I/O performance, I do expect EMC will incorporate other Intel processor versions in future V-Max systems.)

But that isn't to say that V-Max is built from off-the-shelf commodity servers – far from it. Fact is, it is today difficult (if not impossible) to find such a server that meets the reliability, availability and serviceability requirements of the storage market (enterprise –or- mid-tier, I'll argue). For example, every I/O controller, every blower and every power supply in a V-Max engine is hot-swappable. I/O controllers are built with power ramps to protect components when they are removed or inserted. Even the processor boards are hot-swappable. I don't think you'll find all of those features in any "commodity" servers.

Finally, the "secret sauce" component of Symmetrix V-Max is the Virtual Matrix Interface. Custom-designed by EMC, this controller coordinates all references to V-Max global memory, both local and remote. The distributed lock management is handled in this interface, and remote requests are forwarded across the RapidIO-based Virtual Matrix Interconnect as required.

software architecture changes

The Virtual Matrix Architecture and the Symmetrix V-Max hardware aren't all that's new –- there have been numerous enhancements to Enginuity storage OS and management interfaces for this new platform as well. Three of the most significant changes are:

1. Block CRC and memory management
2. RAID Virtual Architecture, Virtual LUNs and Mirror Positions
3. Dynamic and concurrent configuration changes – the death of the BIN file
4. Expanded configuration limits

Let's take a brief look at each:

memory management

One of the key objectives of the V-Max project was to reduce the memory footprint of the product. Or more accurately, to enable larger usable capacity configurations with less memory.

This was actually an objective for a while, but the scope of changes required to do this were not possible on the DMX3/4 series systems because of the absolute requirement that all software upgrades must be non-disruptive. And no matter how good you are, rearranging memory tables during an upgrade is a high-risk proposition. Being the conservative folk that Symmetrix developers have learned to be, memory management changes are permitted only when new hardware platforms are delivered.

See, unlike the startups wanna-bees, Symmetrix has an installed base (that doesn't tolerate disruptions).

 You know the joke, right: How did God create the world in 7 days?
He didn't have an installed base to worry about!

So, V-Max has a totally new schema for its meta-data – a design that reduces the amount of memory required per usable GB of capacity, improves memory reference performance and that is dynamically extensible to support the inevitable future meta-data expansion in a space-efficient manner.

One of the changes that impacts memory management is the introduction of block-level CRC that is now stored alongside every block of data. With V-Max, each 512-byte data block (520-byte for iSeries systems) is now digitally signed with an 8-byte data-integrity field using an algorithm based on the T10-DIF standard proposal. These additional bytes are appended to each 512-byte block on writes as the data enters the array, and verified on every read – in fact, it is verified every time the data is moved within the system!. The bytes include not only a strong CRC, but also referential information that specifies the LBA and generation of the data block. Enginuity uses this info to validate that the content of the block hasn't changed, and that it is indeed the block of data from the requested LBA (just in case the drive returned the wrong block of data). And in addition to this, Enginuity also keeps separate check bits on track tables, which are stored separately from the data and the block CRC, as a belt-and-suspenders approach to ensuring data integrity.

The net result of all of the memory work is that a V-Max with 512GB of raw memory can support close to 60% more usable capacity than a similarly configured DMX4. And with the full configuration of 1TB of global memory, a V-Max can support over 2PB of usable capacity – more than 3x what the largest DMX4 can address.

IBM or Hitachi (at al) today will require 2 or 3 complete systems to deliver that same amount of usable capacity (with the added power and cooling requirements that additional systems will require).

mirror positions

The second big change is logically based around changes in the way mirror positions are used. Long story short – because of an overhaul of the internal RAID engine (internally called the new RAID Virtual Architecture), all RAID types now require only a single mirror position on Symmetrix V-Max. And hot spare drive rebuilds no longer require a mirror position.

These "simple" changes enable several key customer benefits:

1. Drive rebuilds will no longer be delayed until replication or volume relocation is completed
2. You can relocate any LUN or CKD volume on-line, to any other drives in the system, and you can change the RAID type in the process
3. You can perform this "Virtual LUN" (or CKD volume) relocation without disrupting local AND remote replication sessions or relationships

IBM can't do #2 on the DS8K, so they can't do #3 either. And Hitachi can't do #3 on the USP-V, meaning your data is at risk (and your company is possibly out of compliance) during a LUN relocation.

bye bye bin file

OK, I lied.

The BIN File still exists. It just won't be as big of a pain anymore.

Building on work started with Enginuity 5772 (and continuing on EMC's focus on ease of use), the way most common configuration changes are effected no longer requires a BIN file edit and reload with Symmetrix V-Max. From VLUN device relocation, to hot sparing, to changing port flags – reconfigurations have been streamlined tremendously. And dynamic and concurrent provisioning enable more operations to be performed simultaneously, further streamlining storage operations.

More importantly, the impact to system performance of most operations has been reduced to a tiny fraction of what it used to be on DMX.

And common storage management operations like setting up thin provisioning volumes that takes upwards of 900 clicks and 45 minutes on a USP-V can now be accomplished with only 32 clicks and 9 minutes elapsed on a Symmetrix V-Max.

In today's economy, these time-savers can add up to some very big benefit$!

limits? we don't need no stinkin' limits!

V-Max will support twice as many hypers per drive (up to 512), and the maximum single volume size is increased 4x (from 63GB to 253GB) over Enginuity 5773. And SRDF groups have been increased from 128 to 250. The max number of volumes remains today at ~64,500 – but the internal meta-data restructuring will make it easier for EMC to extend these limits in the future, as required (and non-disruptively, of course).

v-max performance

OK, always the touchy subject. Rather than rekindle the benchmark wars, for now I'll net it out like this:

Symmetrix V-Max delivers up 2x the connectivity and 3x the usable capacity as the largest Symmetrix DMX4, while providing over 3x the application-realizable performance of the DMX4.

While EMC has lots of data to back that statement up, I find it perhaps more interesting to look at more specific comparisons – ones that will pay off immediately to customers.

For example, V-Max RAID 6 performance has been improved significantly, such that the max RAID 6 IOPS that the largest V-Max can do is approximately 3.6 times more than the DMX4-4500. And the response time improvement for writes is also formidable – so much so in fact that RAID 6 on V-Max can be as fast as RAID 1 on DMX4.

This one little comparison changes the discussion about RAID 6 and large SATA drives – makes using extremely large drives even more practical for a whole lot more applications, in fact.

Use more SATA capacity to reduce costs without sacrificing availability

Another interesting comparison is to look at the copy performance for Symmetrix V-Max Virtual LUNs vs. the LUN migration capabilities of the Hitachi USP-V. I'll look deeper at this new capability in a later post today, but for now I'll give you this: V-Max can relocate 128-times more LUNs/Volumes simultaneously than the USP-V, it can complete migrations more than 2.8 times faster than the USP-V, and it will do so with far less impact on the performance/response time of all applications on the array.

And as I mentioned above, you don't have to drop replication sessions (local or remote) while you are relocating volumes; only Symmetrix V-Max Virtual LUNs keeps you in compliance as you optimize your storage tiers.

Move more, faster, with less impact. Translated into 2009 Econo-Speak:

V-Max Saves You Money while Maximizing Asset Utilization.

and that's not all

I've only scratched the surface in these first posts on the Virtual Matrix Architecture and the new Symmetrix V-Max platform, but hopefully I've provided some insights as to the significance of the new architecture and the customer benefits of the system. In upcoming posts I'll explore some of the new features and capabilities introduced on the new system.

technorati tags: EMC,Symmetrix,V-Max,DMX,Virtual Matrix Architecture,Direct Matrix,performance,storage management,BIN file,dynamic configuration,virtual provisioning,auto provisioning,mirror positions,virtual LUN,Overtake the future,Direct Matrix Architecture,scale-out,scale-up,enterprise flash drives,enterprise storage,RapidIO

The cornerstone of today’s Overtake the future launch is of course the new EMC Virtual Matrix Architecture, the foundation upon which the virtual data center will scale and thrive henceforth.

Combining the market-proven functionality that has made Symmetrix the World’s Most Trusted storage platform with the latest in industry standard compute and I/O technologies, the Virtual Matrix Architecture liberates the power of Symmetrix from the physical barriers of backplane-based monolithic storage arrays and redefines ease-of-use for storage in today’s increasingly virtualized data centers.

But while this new architecture is inarguably revolutionary in the world of storage, the Virtual Matrix is in fact borne of a Darwin-esque evolution of the same Symmetrix architecture that launched the external storage market over 18 years ago. The result is the first storage architecture that integrates the performance and efficiency of traditional scale-up architectures with the cost-effective flexibility of scale-out, blurring the distinction between modular vs. monolithic while redefining the scope of scalable enterprise storage.

In this post I will explain the path that has led EMC to The Virtual Matrix, and along the way I’ll highlight several of the key features of this revolutionary new architecture.

global memory – the foundation of symmetrix, both old and new

The single most distinguishing feature of Symmetrix for the past 18+ years has been its global memory. In every Symmetrix, memory is a central shared resource that is accessible by every single processor and I/O stream in the system.

Over the years, the interconnect between memory and the I/O processors and the way that these processors communicate with each other both have changed, but the operational utility of global memory hasn’t. Write requests received by front-end communications ports are stored in global memory for the back-end disk directors to deliver to disk, and host read requests are fulfilled by the disk directors by placing payloads in global memory for the front-end directors to deliver back to the requestor.

Early Symmetrix systems (before the DMX-era) used a communications bus to transport data and inter-processor communications between processors and memory. The processing complexes themselves were separate and purpose built – front end SCSI, ESCON and later Fibre Channel based directors connected to hosts, while first SCSI and later FC disk directors connected to the disks. Each of these directors presented different emulations, and over the years, the directors evolved to be more like blade servers, with each “blade” supporting 2-4 independent CPU “slices”, each able to transfer data to and from central memory over the backplane.

In 2003, EMC introduced the first significant architectural change to Symmetrix since its birth – the Direct Matrix Architecture. Although still maintaining the front <—> memory <—> back implementation, the bus interconnect of the prior generations was replaced with the Direct Matrix – dedicated I/O transports between each front- and back-end director to the global memory directors. These dedicated paths eliminated the contention bottlenecks of the previous bus architecture, and they allow the Symmetrix DMX series to deliver levels of performance and capacity scalability that has still not been equaled by any other high-end storage array. Today, no other high-end array supports as many disk drives, and it has been only recently that one competitor matched the amount of global memory supported by the Symmetrix DMX-4.

As you should expect, the Virtual Matrix Architecture is also structured around the central resource of global memory – but the implementation is radically different from prior generations. But, before I explain how it differs, let’s discuss some of the reasons for changing things in the first place.

beyond the backplane

Over the past decade, Symmetrix has been portrayed by competitors as “monolithic”, especially those who sought to differentiate from the fixed-frame disk complexes that Symmetrix employed up until the introduction of the DMX3 in 2005. Pre-DMX3 all Symmetrix came in limited sizes – the 5th-generation Symmetrix 8000 series was available in a 96-drive and a 384-drive package, for example. DMX1 & DMX2 were slightly better in that there were actually 4 different sizes (DMX-850, DMX1000, DMX2000 and DMX-3000), but you still couldn’t grow from one size to the next as your needs changed.

The DMX3 and DMX4 changed all that. Customers can start with the 2-disk director DMX4-1500 and grow all the way up to the full-blown 4-disk director DMX4500, supporting up to 2400 drives. Back-end performance scaled up as you added disk directors, and the customers’ investment was protected as needs changed.

But in talking with customers even now, one would quickly come to realize that incremental growth alone wasn’t enough – customers also needed more flexibility in how they deployed their storage across the data center.

In addition, many of EMC’s largest customers had need for even larger configurations than DMX4 could offer – in fact, many of them have multiple independent DMX4’s to meet their performance SLA’s where they would prefer to have only one.

Unfortunately, the very real truth is that the laws of physics get in the way of stretching signals over a physical backplane, and the DMX4 is already at the practical limits of today’s technology. In order to scale Symmetrix even larger, it had to get beyond the limitations of a passive backplane architecture.

So the Virtual Matrix was borne out of two key customer requirements: to support ever-larger capacity storage arrays while removing the requirement that all the bays of the array must be physically adjacent.

And though switching to glass interconnects to the storage bays might solve the latter, this is an expensive alternative that unfortunately does nothing to help increase the scale limits of the backplane.

And there was one more significant motivator behind the Virtual Matrix:

those costly dis-integrated directors

In the early days of Symmetrix, it was generally necessary for every computer vendor to build their own processor complexes. Though Intel was dominating the desktop/laptop market, these were the (end of) the glory days of DEC, Prime, Wang, Data General, HP, Sun, Honeywell, Bull, Apollo, Stratus (et al), and most every one of them had their own proprietary processors. Those that didn’t used “standard” CPUs and surrounded them with custom logic designs.

Symmetrix was just a big I/O server, and so it naturally followed suit, and Symmetrix arrays through today's DMX-4 were built with a healthy helping of custom hardware – most of it centered around accelerating data movement through the system and in performing continuous error checking to ensure data integrity. This logic was purpose built for the task each director would support, and the design worked well: you could mix and match front-end directors to get the connectivity you needed, you could add memory directors to improve cache hit rates, and you could scale the back-end to deliver more cache-miss IOPS if that was what you needed.

But the downside of this purpose-built approach was that you could only put so many director blades on the backplane. No, the bigger challenge was that the design required unique and custom hardware for every generation. Every director had to be built up from the chips, every interconnect invented and implemented from the ground up. And over recent years the gap between “custom” and “industry standard” closed, leaving little leverage in custom hardware.

Back before even the first DMX came to market (the DMX1000 and DMX2000 were launched in  February 2003), Symmetrix engineers were drafting proposals to integrate the functions of the front, back and memory controllers onto a single controller form factor. That idea evolved into the notion of switching processors from the now-nichey Power 4 CPU from IBM/Motorola, with largely custom infrastructure, to the far more widely used Intel x86/IA64 processors and their lower-cost industry standard infrastructures. And by virtue of changing processor complexes, the new “unified director” could more readily integrate standard I/O cards, memory DIMMs and interconnects so as to become more cost-efficient and agile as new processors and components were introduced. The engineers just needed to figure out how to connect multiple of these new unified directors together.

Another benefit of putting the memory local to the CPU within the unified directors meant data transfers where no longer limited to the speed of the Bus or the Direct Matrix – I/O could be received and sent with far lower latency and overhead than in prior Symmetrix generations. But embedding the memory in the unified director posed a new challenge – the memory was “local” to the processors, but Symmetrix and the Enginuity storage OS were based on the tried-and-true foundation of “global” memory.

enter the virtual matrix

Summarizing the requirements, the next architectural evolution of Symmetrix had some pretty tough targets:

• memory must be globally accessible
• get the maximum performance benefits of local memory access
• maintain/extend the incremental scalability of the DMX3/DMX4 (add directors to scale)
• leverage a unified director built on industry-standard components for simplicity and flexibility,
• accommodate future scale to ever larger system images
• allow for distributing cabinets around the data center
• minimize the impact on Enginuity software
• deliver all this without compromising reliability or availability

Simple, huh?

Well the solution in fact did turn out to be rather straight-forward.

In essence, the Virtual Matrix Architecture “virtualizes” processor access to memory. That is, the code treats access to remote memory exactly the same as it does to local memory. The trick is that a small layer of software, assisted by EMC custom hardware logic (in an ASIC on the Virtual Matrix Interface), presents any location in memory as “local” to the processor complexes. If the target memory locations are indeed local, then access to memory is direct, at memory bus speeds. And if not, the request is packetized and parallelized to be sent over the Virtual Matrix Interconnect fabric to the Virtual Matrix Interface on the director that owns the specified memory target.

For the curious, the first generation of the Symmetrix V-Max uses two active-active, non-blocking, serial RapidIO v1.3-compliant private networks as the inter-node Virtual Matrix Interconnect, which supports up to 2.5GB/s full-duplex data transfer per connection – each “director” has 2, and thus each “engine” has 4 connections in the first-gen V-Max.

Why RapidIO you ask? Primarily because of its non-blocking, low latency, high bandwidth, parallelism and cost efficiency – RapidIO has been used in a broad range of embedded applications from MRI systems to military fighter jets. You can learn more about RapidIO at www.RapidIO.org.

a matrix designed for the future

In order to keep up with the exponential growth projections of both storage and virtualized server complexes, the Virtual Matrix Architecture is designed to scale well beyond the limits of the first-generation Symmetrix V-Max. In fact, the gen 1 Virtual Matrix Interconnect could easily address well beyond 256 nodes. But the Virtual Matrix Architecture doesn’t limit the fabric to being 2 RapidIOs; it could be 4 or 8 RapidIO networks running in parallel, or it could be built on a different infrastructure altogether – InfiniBand, FCoE/DCE – or whatever comes along in the coming years.

More importantly, on top of the two dimensions of memory access that the Virtual Matrix Interface implements today (direct to local memory and over the fabric to memory in peer nodes), the Architecture allows for a third dimension of interconnect – a connection between different V-Max systems. This interconnect would not necessarily expand to share memory across all the nodes in two (or more) separate V-Max arrays, but it would allow multiple V-Max arrays to perform high-speed data transfers and even redirected I/O requests between different Symmetrix V-Max “virtual partitions.” This capability of the Architecture will be leveraged in the future to “federate” different generations of V-Max arrays in order to scale to even greater capacities and performance, and will also be used to simplify technology refreshes. In the future, you’ll be able to “federate” a new V-Max with the one on your floor and non-disruptively relocate workloads, data and host I/O ports.

It ain’t magic…but it’s close.
 

technorati tags: EMC,Symmetrix,V-Max,DMX,Virtual Matrix Architecture,Direct Matrix,Overtake the future,Direct Matrix Architecture,scale-out,scale-up,enterprise flash drives,enterprise storage,RapidIO

but first, a bit of nostalgia

For over 18 years, Symmetrix has been the very definition of enterprise storage - even as that definition itself has changed over time.

The first Symmetrix array was built as an alternative to expensive IBM mainframe storage at a time when data centers were largely still centralized and mainframe-based. In the mid-1990's, EMC added Open Systems support to Symmetrix, allowing a single array to support multiple different servers and bringing the cost-benefits of storage consolidation to a broader market.

Read on…

symmetrix v-max and the virtual data center

More recently, IT has begun transitioning to the virtual server computing model, as customers strive to improve resource utilization while driving down the complexity and operating costs of the infrastructure. Symmetrix V-Max has been intentionally designed for this new market – the era of the Virtual Data Center (or the Private Cloud, if you prefer).

As the world will learn today, V-Max's integration with (and its parallels to) virtual server technologies is intentional and strategic.

And for customers not yet ready to move into the virtual world with Symmetrix V-Max, EMC will continue to offer and enhance the Symmetrix DMX4, so there's no pressure to switch to V-Max until customers are ready.

meet the newest member of the symmetrix family

The first thing you'll probably notice about the new Symmetrix V-Max is the packaging – and specifically the glossy-black panel with the blazing blue LED light bar that underscores the name Symmetrix on every door. The design team had a lot of fun blending the modern gloss-black look of today's popular personal communications devices with the image of stability and security that customers expect from Symmetrix.

I think they did very well.

In fact, the picture hardly does justice to the powerful first impression you get from seeing the packaging in real-life – especially for a full-blown 2400 drive, 11 cabinet configuration. So…I created this little bootleg video of one of the development systems in the labs so you could get a better idea for yourself:

Symmetrix V-Max in its Native Habitat
(Click here if you cannot see the video player)

Of course, this is a lab system, so you'd normally not have cables hanging out the back. Here, the rear doors have been removed and the cable management system isn't in place – in the real world, things would be much more tidy. But you get the idea – this is one big array.

I'll be back later today with more insights and perspectives on the new Symmetrix V-Max…I just thought it might be fun to start at the beginning!

surprise!

In closing, I can't resist the observation that I doubt anyone saw the significance and scope of today's announcement coming. Given all the predictions that this was going to be just the DMX 5 (or DMX V), or perhaps a new cloud offering, I actually think today’s launch will be/is a bigger surprise to the world than last year’s thoroughly unexpected introduction of Enterprise Flash Drives.

I mean, seriously, did anybody out there (other than those briefed under NDA) really expect that EMC would overhaul the Symmetrix this radically? Is there anyone who would have bet that the New Symmetrix would be built of industry standard components on top of a Scale-Out architecture – while still maintaining compatibility for the installed base? Or that EMC was actually developing software to leverage flash drives as anything more than just another tier? (More on that later today).

Heck - would anyone have believed a month ago that EMC even intended to do all this, much less that they'd been working on it for several years?

I suspect not.

In fact, after years of competitive complacency built upon their near-incessant assertions that EMC had stopped innovating, I suspect that there are more than a few competitors scrambling today to figure out just how EMC could possibly have pulled this off, right under their noses.

In short, I am sure that many competitors and wanna-bees today will realize that they have totally underestimated EMC.

Again. And again. And AGAIN…

technorati tags: EMC,Symmetrix V-Max,Virtual Matrix Architecture,Enginuity,DMX,Direct Matrix Architecture,Virtual Data Center

welcome to the symmetrix v-max virtual launch

Over the next 24 hours, EMC will be hosting virtual launch events as scheduled on www.overtakethefuture.com (you did register, didn’t you – if so, you should receive an agenda in your email this morning). Around the world, EMC Sales reps and partners will join their customers in participating in these events - Live.

To kick things off, the following press release was sent out at 12:01AM EDT:

New EMC Virtual Matrix Architecture
Delivers Massive Storage Scalability for Virtual Data Centers

Throughout today, EMC executives will be meeting with Wall Street and with Industry press to explain the new offerings and how they will change the world of storage forever. EMC employees, customers and partners around the world will receive a plethora of launch content on PowerLink, and the teaser on EMC.com has been updated with the "reveal" for today's launch. Press and analysts who were pre-briefed will be publishing articles and perspectives throughout the day.

And , of course, several of the bloggers who work at EMC will also be providing their own personal views on the launch throughout the day.

In my role as Chief Strategy Officer for the Symmetrix Product Group, I will be actively involved in today’s virtual launch events, along with many of the engineers who have worked on the products being announced today. And since today’s launch is in fact about a new Symmetrix platform and its breakthrough architecture, in-between the live virtual events, I will also be blogging additional insights into the new Virtual Matrix Architecture, the new Symmetrix V-Max storage array, and several of the key new features that this revolutionary new platform will deliver.

It is going to be one busy (and exciting) day.

overtake the future – launch day coverage

Throughout the day I will be also updating this section with links to relevant press coverage and blog posts, so bookmark this page and check back often!

So far today…

bloggers who work at emc

Chuck Hollis

• Symmetrix V-Max: Storage Architecture Redefined
• Symmetrix V-Max: A New Paradigm For Storage Virtualization?
• Symmetrix V-Max: Watching The Reaction

Dave Graham

• Welcome to the next generation: Symmetrix V-Max is here…
• Happy V-Max day!
• Symmetrix V-Max Coverage
• V-Max: What's in a Name?

Dick Sullivan – Energy Matters

• You ain't seen nothing yet.

Mark Twomey – Storagezilla

• V-Max: The Scale Out Symmetrix 
• V-Max: SRDF Evolves
• V-Max: Auto-provisioning Groups
• V-Max: Demo of Auto-provisioning Groups
• The V-Max Top Ten List

Brian Henderson – Power Windows

• EMC Symmetrix V-Max in Use at Microsoft adCenter
• EMC Symmetrix V-Max and Mixed Microsoft Workloads
• Video: Microsoft AdCenter Using V-Max

Steve Todd – Information Playground

• V-Max Quality
• Managing V-Max at Scale

Len Devanna – Confessions of an E-Biz Junkie

• A Social Launch - Introducing Symmetrix V-Max

Chad Sakac – Virtual Geek

• EMC’s VMware Storage Strategy - The 3rd Shoe Drops

Polly Pearson

• Employee Engagement Case Study in Action: Social Media to the V-Max!

Gina Minks – Adventures in Corporate Education

• Ready, Set, Go V-Max!

And the storage anarchist's posts on the event:

• 1.055: symmetrix v-max – a revolutionary evolution
• 1.056: inside the virtual matrix architecture 
• 1.057: symmetrix v-max – scale up, scale out, scale away!
• 1.058: v-max does what hi-star can't
• 1.059: fully automated storage tiering (fast)
• 1.060: the rest of the v-max launch
• 1.061: the voice of the customer
• 1.062: symmetrix v-max launch videos

other sources

Chris Evans – The Storage Architect

• Enterprise Computing: EMC Announces Next Generation V-Max Architecture

Martin Glassborow – Storagebod

• So No DMX5
• FAST and Furious
• More musings

Stephen Foskett – Gestalt IT

• EMC Symmetrix V-Max Is Neither Monolithic Nor Midrange

Chris Mellor – The Register

• Symmetrix and the death of monolithic arrays

Lucas Mearian – Computerworld

• EMC Introduces x86-based Symmetrix array for cloud storage

Vellante, Floyer & Allen – Wikibon

• Symmetrix V-Max: Virtualized Data Center of the Future Scale (not for hire opinions)

Beth Pariseau – IT Knowledge Exchange

• EMC launches Symmetrix V-Max, may add spin-down

Beth Pariseau – SearchStorage.com

• EMC clusters Symmetrix high-end arrays

Jerome M. Wendt – DCIG

• EMC Symmetrix V-Max – It's About Time

Duncan Epping – Yellow Bricks

• EMC announced the Symmetrix V-Max!

Dave Raffo – IT Knowledge Exchange

• EMC V-Max: V stands for big

Matt Davis – techmute.com

• Response to Tony's V-Max Questions

Nigel Poulton – RupteredMonkey

• Symmetrix V-Max

David Vellante – Wikibon Blogs

• V-Maxed out? Take a Deep Breath and Sharpen the Pencils.

Nick Allen – Wikibon Blogs

• Does EMC’s New Symmetrix V-Max Deserve a “V”?

Bill Mottram – Wikibon Blogs

• Some high level highlights of today’s Symmetrix V-Max announcement.

More as they come in!

technorati tags: emc,overtake the future,symmetrix,v-max,symmetrix v-max,virtual matrix architecture,emc bloggers

It is The Working Mother Experience.

And I'm not even going to try to explain it to you, pointing you instead to the blogs of the two working mothers who inspired the book in the first place: Polly Pearson and Natalie Corridan-Gregg.

Polly is EMC's VP of Employment Brand and Strategy Engagement, and Natalie is A Principle Product Manager for EMC's Symmetrix management products, six-time EMC Women's Leadership Forum president, and now the latest EMC employee who blogs.

More importantly, both are working mothers who felt compelled to collect and share their stories in hopes of inspiring other mothers to balance the challenges and joys of raising children while also pursuing a career. 

Visit both their blogs for insights into the book, and for links to purchase and/or download a (free) copy.

Hint to my fellow working fathers – get a copy, read it, and then give it to your wife. Or better yet, read it with her – we're talking MAJOR bonus points here, something us geeks always need!