Shaking up primary storage. In the decade ahead, flash memory will push hard drives out of the latency path of performance intensive applications, while SATA (slow) disk will continue to be the choice for capacity (secondary) storage. On the consumer side, this transition to all flash storage is well underway—check out our launch day end-user flash leadership event hosted by IDC, featuring Netflix, LinkedIn, eBay, and Identified, discussing in a panel how they have transformed their infrastructures with flash. The sea change from spindles to flash is upending the primary storage market—a $15B annual spend on performance (oxymoron) disk arrays that no longer makes sense. To better reflect the magnitude of the disruption, we’re rebranding Pure under the tag line  Flash for All—flash for all workloads, all budgets, and all businesses. Packaged within Pure’s architecture, flash is ten times faster, more reliable, more space and power efficient, and simpler than mechanical disk, and yet typically costs less per usable gigabyte!

As a result of Pure being the first to get the all-flash recipe right, we are now the fastest growing storage company in history. Indeed, we are currently growing about twice as fast as NetApp or Data Domain did at a comparable stage (the two fastest growers prior to Pure). Pure’s growth is coming at the expense of disk arrays, machinery we have started replacing within most industry verticals over the past three years (see below). We have now shipped hundreds of FlashArrays to customers and partners across the United States, Europe and Asia. Today, Pure has offices in London, Amsterdam, Tokyo, and Seoul, as well as around the U.S., with another wave of expansion in the works. Moreover, we have established a 7×24 support center in the U.S. that is yielding even higher customer accolades than our products, although it’s a tight race. (Disrupting the mechanical storage legacy sound like fun? Then consider joining one of the hottest pre-IPO companies in Silicon Valley. Learn more by visiting our jobs page.)

Last, but not least, Pure Storage was recently selected by In-Q-Tel for security-intensive deployments within the U.S. federal government (press release here). In-Q-Tel is the venture investing arm of the U.S. Intelligence Community. Their role is to identify best of breed technology, procure products for the agencies they represent, and collaborate with the vendor to further enhance those products to broaden applicability within that community. While we cannot talk about the specifics, our joint work with In-Q-Tel will be available to all Pure customers down the road.

Unsurprisingly, the primary competitor for Pure is EMC (NYSE:EMC) disk arrays, the market leader in the performance block segment Pure plays in. We’d argue that our initial success spurred EMC to acquire XtremIO, an all-flash array similarly being crafted from scratch for solid-state as Pure was. As we have remarked before, competition is great for our industry—it drives vendor innovation and customer value. And for Pure, nothing catalyzes a market like having the industry leader EMC, the one who has the most to lose in the transition to flash memory, validate the recipe we originated. While XtremIO is not yet generally available, we expect it to become our #1 competitor among the all flash alternatives. (No surprise that if a customer invites Pure to compete with EMC for a disk array deal, that a call is going to go out to the XtremIO team.) With the XtremIO GA expected later this year, the onus is on us to grow our 18 month technology lead (Pure’s FlashArray was GA in the Spring of ’12), and the onus on EMC to close the gap. We welcome that competition and are confident it will benefit both Pure and XtremIO.

Shaking out the all flash storage market.  We’ll save more on the differentiation between Pure and EMC for future posts. Today it makes more sense to focus on the separation we are jointly driving between the software have’s and the have not’s in flash storage. Most of the all flash vendors are demonstrably behind both Pure and XtremIO in essential flash array software features. What does it take to win in all-flash storage? Let’s recap the recipe:

• Submillisecond deduplication and compression – The aggregate across all deployed Pure Storage FlashArray’s is greater than 5:1 data reduction (with thin provisioning, that’s greater than 10:1—check out the dedupe ticker on our home page). While dedupe was quickly added to every flash vendor’s roadmap, inline data reduction has proved very difficult to reproduce in the three years since Pure first shipped the technology. When EMC makes the GA release of XtremIO’s inline dedupe (but sans compression), they will likely be only the third vendor to do so. The rest of the flash appliance vendors are finding selling without data reduction tough going, since the only way to keep their customers whole is to offer them an additional 80% discount off of their best price (90% for those still lacking thin provisioning).
• True High Availability (HA) and Non-Disruptive Upgrade (NDU) – The software to support continuous operation with no performance impact in the face of hardware components failing or being upgraded, as well as in the face of software upgrades, is extremely hard to get right. We have yet to see another all flash array deliver the resiliency and serviceability essential for mission critical storage. Don’t buy without pulling drives, cables & power cords, replacing hardware, and upgrading the software, all under load, as this is the best way to measure how the technology will perform in your data center over time.
• Metadata driven snapshots and accelerated virtual machine (VM) cloning – Snapshots have proven crucial for creating back ups of consistent system images, but they have always been expensive in space, performance or both. With a fast deduplicating array, snapshots can be taken and VMs cloned (think VMware xcopy) nearly instantaneously, solely via a metadata update, consuming near zero additional space and with near zero performance overhead. Again, software is the differentiator.

New FlashArray FA-400 Series Hardware

Almost exactly a year ago we introduced our FA-300 Series, and we’re back a year later introducing the FA-400 Series, which boasts double the performance and double the capacity of the previous generation.  The FA-400 features Intel’s Sandy Bridge processor architecture, and represents a substantial improvement in about every technical specification:

If you’d like to read all about the details, you can download the datasheet, or visit the tech specs webpage.  And for a visual tour of the new FA-400 hardware, you can watch the video below:

But ultimately, hardware’s just hardware.  It gets faster every year, and so does the FlashArray.  More importantly, this hardware refresh highlights a few critical architectural and performance advantages of Pure Storage, which I’ll dive into below.

The Power of Commodity Architecture: Moore’s Law + Rapid Iteration

Broadly speaking, there are three flavors of all-flash products on the market today:

1. Legacy disk arrays retrofit with flash
2. All-flash appliances, where raw NAND is coupled with core IP delivered in hardware, burned into custom ASICs and FPGAs
3. All-flash arrays, leveraging standard x86 processors and off-the-shelf SSDs, with their core IP in software

The first category isn’t really worth discussing, so we’ll skip it.  Most of the leading legacy vendors plus all of the newer flash-centric startups have picked a position in either category #2 or #3 – and this is where much of the competition in the industry is today.  For our part, Pure Storage’s strong belief is in model #3.  We think computing’s history is on our side, and by betting on an open, software-driven approach to innovation, we are able to ride two powerful forces in the industry:

• Moore’s Law – or more broadly the Intel x86 architecture.  Intel delivers to us a doubling of processor power every 12-18 months, the key ingredient to delivering a faster and bigger FlashArray every year.
• The commodity SSD market.  The SSD market is evolving even faster than processors are, and the result is that every 6 months or so a new batch of SSDs come to market that are bigger, faster, more reliable, and less expensive.

The Pure Storage FlashArray was architected to be able to take advantage of both processor and SSD advances quickly and incrementally, enabling us to rapidly upgrade the performance and capacity of the FlashArray.  Take a look at our 4-year history, we believe this trendline will continue year-after-year:

Those who innovate in hardware face challenges on two fronts.  First, hardware innovation is simply slower.  Early approaches leveraged FPGAs to be able to iterate a bit faster, but as flash lithography compresses the advanced ECC technologies required to manage flash now require dedicated ASICs – a slow development path.  Secondly, most of the vendors who innovate in hardware simply don’t also build the expertise to deliver rich system-level software as well, and so invariably the hardware-centric devices feel like exotic flash chassis where somewhere along the way the software was left behind.

Pure’s strategy is simple: we leave the hardware innovation to others, take advantage of the rich processor and SSD hardware marketplace, and we focus all our engineering effort and IP around being the software leaders in flash.  Over time we believe this market will follow the well-worn path of many markets before us: the hardware commoditizes, and it’s all about the software.

Incremental, Online, “Evergreen” Upgradability vs. the Flash Toaster

The second key advantage that today’s release showcases is how the FlashArray’s architecture was designed to be able to deliver incremental upgradability, or a new “evergreen” storage model, as the Pure Storage founders like to call it, vs. the closed, non-upgradable “flash toaster” appliances of our competitors.  Upgradability has long been a challenge of the storage industry – the common model is to buy a disk array, use it for 3-4 years, then throw it away.  This was already wasteful in the disk storage world, but in the flash world it is even more problematic:

• Flash evolves quickly: customers want to be able to take advantage of flash advances that come every 6-9 months, as opposed to getting stuck with the flash of 3+ years ago.
• Start small and grow: as flash is new to most customers, many want to make a small initial deployment (maybe a single database, or a VDI pilot), but would like to be able to incrementally grow and expand their flash investment over time without ripping and replacing.
• Downtime isn’t an option: flash tends to be deployed in more important, Tier 1-type applications, and for those applications, downtime for array maintenance simply isn’t an option.

So, Pure Storage delivered an architecture for the FlashArray that has separate, stateless active/active controllers, and independent storage shelves, all of which can be deployed and upgraded non-disruptively: i.e. without user downtime or performance impact.  As such, the FlashArray can grow from the smallest 5-10TB usable configuration, up to 100+TBs usable, all without wasting hardware investment or taking user downtime.

Pure Storage makes it easy to start small, dip your toe in the flash water (we promise, it’s warm!), and expand as you growth more comfortable with flash.  Want to see how upgrading works?  Check out the second post to see some live demo videos.

No More 4K IOPS Vanity Benchmarks!

You might notice that in this release we’ve chosen to start quoting FlashArray performance in 8K IOPS instead of 4K IOPS.  The previous-generation FlashArray FA-300 was spec’d at 200,000 4K IOPS, and the new FA-400 carries a spec of 400,000 8K IOPS.  We did this to start being more aggressive in our assertion that the flash industry is waaayyyy to over-focused on 4K IOPS, which very rarely exist in the real world.  In fact, the average IO size delivered from our global pool of customer FlashArrays (averaged across all the call-home data in our database) is roughly 38K, you can see the live number on this page.  38K!!!  People are often surprised by this number, as they’ve been lead by other vendors to believe that 4K IOs are commonplace, and often people confuse their application’s and OS’s block geometry vs. actual IO size.  Even if a database is set to 4K IO alignment, multiple layers in the IO stack are happy to ask for more than one 4K block at a time, leading to many larger IOs.

So…why is this important?  It highlights an important architectural difference between Pure Storage and much of our competition.  Most competitive flash products (Violin and XtremIO, to name two), are architected around a fixed 4K internal block size, which means that if they receive an 8K IO they break it into two 4K IOs, if they receive a 32K IO they break it into eight 4K IOs, etc, which leads to a linear drop-off of IOPS by block size.  Pure Storage, on the other hand, implements a variable internal geometry, meaning that we analyze IOs for data reduction down to 512-bytes in size, but we store a single IO as a single IO.  That doesn’t mean we’ll have the exact same performance for a 4K IO as we’ll have for a 32K IO, but we’ll be substantially sub-linear in our performance fall-off.  Check out the difference:

The result is that for IOs in the “zone” of most real world customer workloads, Pure Storage will substantially out-perform the 4K-optimized competition.  Try it out and you’ll see!

Better Efficiency than Brand X: It Takes 4 X-Bricks to Equal One FlashArray

Finally, let’s compare the new FlashArray to the long-announced-yet-still-not-GA Brand X.  Based on publicly-discussed information from EMC on Brand X, each x-Brick is planned to be 7TBs usable after RAID/metadata overhead, from a base of 10TBs of raw flash per brick.  Let’s assume that since both arrays are deduping/thin provisioning arrays, most customers will want to run the arrays with about 20% empty space, to allow room for safe levels of thin provisioning over-allocation and potential changes in the dedupability of data.  An important difference to understand is that Brand X does deduplication only, while Pure Storage does both deduplication and compression inline.  Analyzing the call-home data from all the FlashArrays at customer sites, the FlashArray delivers on average 5.6-to-1 data reduction, of which the compression savings portion is 2.1-to-1. This means that if we assume that both arrays will get similar deduplication results (a kind assumption, since the FlashArray can detect duplicates down to 512-bytes, 8x smaller than Brand X’s fixed 4K chunks), then whatever those dedupe results, the FlashArray will additionally reduce data by >2-to-1 with compression, resulting in less than HALF the ending space usage on Pure Storage.  Additionally, Brand X’s hardware require dedicated standby power supplies (SPSs) and dedicated redundant InfiniBand switches, all of which consume space and cost.  Here’s what the result looks like visually:

The result: a single 23TB Pure Storage FlashArray can store more data than 4 XtremIO x-bricks, and can do so in less than half the physical space (10U vs. 23U).

Ready for a Test Spin?  It’s available today.

Well, if you’ve made it this far, you are clearly interested.  Our FA-400 Series FlashArrays are available and shipping today, we’d love to give you the chance to experience on in your environment.  Click here to arrange a private demo or evaluation!

Purity 3.0 – More Resilience, More Performance, More Secure, More Interoperable, and More Supportable.

Yes, if there was a theme for this release, it was simply “more”.  Perhaps that alone is a good place to anchor this discussion – now entering our third generation of product maturity, the FlashArray delivered “the basics” of an all-flash array years ago.  This release is really about Pure Storage hitting its stride, and expanding aggressively in some important areas for customers.  If you’d like a video tour of all the new features, simply press play below, and if you’d like to go deeper, just scroll down.

So what’s new?  So much we almost couldn’t fit everything on one slide .  Here’s a visual overview of everything that’s new in Purity 3.0, we’ll hit every box on this page:

More Resilience: Non-Disruptive Everything

Perhaps for first-generation flash devices, which generally were deployed to accelerate a single database application, getting an outage window from the DBA to do an upgrade was manageable.  But for true enterprise consolidation arrays like the FlashArray, when you host 100s or 1,000s of applications, taking an outage for an upgrade just isn’t an option.  The FlashArray’s stateless controller architecture and independent storage shelves enable us to deliver non-disruptive everything:

• Capacity expansion: add additional shelves to expand the capacity of the FlashArray online.  Good luck expanding your flash appliance toaster!
• Controller upgrades: move between controller generations completely online to add performance.  Our existing FlashArray FA-300 customers can non-disruptively double the performance of their arrays by moving to FA-400 controllers.
• Software updates: the world of flash changes fast, and we update FlashArray features every 3-6 months.  Perform feature updates and maintenance updates online.
• Flash or controller replacements: occasionally, hardware fails.  The FlashArray was designed with redundant everything, so flash SSDs, NV-RAM devices, fans, power supplies, and even controllers can be replaced online.
• All without performance loss!  Non-disruptive doesn’t just mean hosts maintain access to storage, it also means no performance loss.  Since the FlashArray is significantly over-provisioned in every dimension, all the above maintenance can be performed without availability OR performance loss.

Seeing is believing?  We’ll here you go.  In these two videos Chas. Dye, database solutions architect, will demonstrate a software update under heavy load, and show the process for controller upgrades:

Read more about Non-Disruptive Everything on our site.

More Performance: Ultra-Fast ZeroSnap Snapshots

We talked a lot about performance in the last post on the new FA-400 hardware, but performance also extends deeply into the software.  In many competitive products where software is an afterthought and retrofit to the flash core, software features are optional and often come with a heavy performance penalty when enabled.  The FlashArray was designed from the ground-up with software at the core of the architecture, and that software is always-on: always deduplicating, always  compressing, always encrypting (see below), and always snapshotting.  In legacy arrays, snapshots in particular are often a performance Achilles’ heel: they impact performance of the primary volume, have all sorts of restrictions in how they can be used and are managed, and often don’t perform all that well themselves.  Pure Storage’s ZeroSnap Snapshot feature is one of the best features to demonstrate the flexible, metadata-driven nature of the Purity Software.  ZeroSnap’s name derives from that low overhead: zero performance impact, zero pre-planning required, zero restrictions in use, and zero space consumption of the snapshots once taken.

We first introduced ZeroSnap in Purity 2.5 where it was an “early adopter” feature in test with a portion of our customer base, and we’ve now GAed it for broad use across the customer base with Purity 3.0.  Seeing is believing?  Well here you go.  Watch Chas. Dye, DB Solutions Architect, demonstrate taking 500 snapshots in one minute while the FlashArray is under heavy Oracle load without impacting the primary volume performance.  Then see him mount snapshots 1, 250, and 500, and run a workload on each at even higher performance, proving that a) they are independant, b) they all perform the same, and c) they don’t impact one another.

Perfer testing your snapshots in an application environment?  We’ve got that too.  See Chas. demonstrate how easy it is to leverage ZeroSnap to clone production Oracle databases, even demonstrating the consistency groups feature, which enables multiple volumes to be snapshotted at a consistent point-in-time for application-level protection:

Read more about ZeroSnap on our site.

More Performance: ZeroSnap Accelerated VM Cloning (leveraging VMware VAAI xCopy)

Hopefully the discussion on ZeroSnap gave you a sense for the power of metadata-only snapshots.  One of the interesting nuances of the ZeroSnap implementation, is that it was not only built to snapshot any volume or group of volumes consistently, but also any part of a volume, or really any block range within the FlashArray, instantly.  One of the best ways to demonstrate the power of this capability is a new Purity 3.0 feature: ZeroSnap Accelerated VM Cloning.  What Accelerated VM Cloning does is it accepts xCopy commands from VMware vSphere via the VAAI API, and instead of interpreting them like a normal array would (i.e. copy blocks a..b to new location y..z), it simply creates a ZeroSnap snapshot of the metadata of that block range, instantly duplicating the metadata while leaving the original data alone on the SSDs.  The result is that VM cloning operations happen in a matter of just a second or two (most of this time is actually vSphere processing the command), enabling the cloning of 1,000s of VMs in about an hour.  Let’s compare the advantage with an existing disk array, even if the disk array implements xCopy as well (in this case a top-of-the-line EMC Symmetrix):

 As you can see, cloning time for a 40GB VM moves from minutes to 2 seconds, and more importantly the clones are pre-deduped, no data movement or space consumption (other than a small amount of metadata)!  ZeroSnap Accelerated VM Cloning is a potential life-saver for persistent VDI environments, or test/dev labs which make frequent use of cloning.  The same xCopy implementation also helps speed VMware Storage vMotion operations as well!  Want to see how ZeroSnap Accelerated VM Cloning works?  Check out this chalk talk:

Read more about ZeroSnap Accelerated VM Cloning on our site.

More Secure: Always on Data-at-Rest Encryption with built-in Key Management

Data security has been a growing challenge over the years.  Whether you are worried about patient data, customer credit card info or SSNs, or just want to avoid having a “front page of the paper” moment, ensuring proper security is a multi-layer approach.  Encrypting data at rest used to involve a series of painful trade-offs.  You typically had to deploy expensive encryption devices at some point in your storage IO path: appliances in the middle of the network, host HBAs on each server, or host-based file system plugins.  Each came with its own performance and management penalty….not to mention the key management pain, process, and risk.  In short, encryption required a complex solution, and the planning, management, and procurement that comes along with such a solution.

Pure Storage decided to turn the equation on its head.  What if encryption was free, effortless, had no performance impact, and was just an always-on feature of storage?  What if all the thought around encryption could be taken away, and it was always just there?  Always-on, but totally invisible to all?  Zero management, zero performance overhead, zero risk and zero worry…just effortless protection?  That’s Pure Storage’s always-on data-at-rest encryption.

Why is data-at-rest encryption important?  It protects data from physical loss, such as: a) theft of an individual or set of SSDs from the data center, b) theft or loss of a failed SSD being transported during replacement procedures, c) theft or loss of SSDs during de-commission at the ends of their useful life, or d) transport of a full array between data centers.

Let’s explore the Pure Storage encryption features:

• Always on, just like all the other software services in the FlashArray: deduplication, compression, and snapshots.  The actual encryption is performed by the ASICs inside each SSD, meaning that there’s no risk to performance.  The encryption algorithm is AES-256, and is implemented in both the SSDs and NV-RAM devices in the FlashArray (the two places data is persistently stored).
• Zero management.  So simple, you don’t even have to remember to turn it on, it’s always on.  No keys to manage or rotate, no passwords to supply, just effortless data-at-rest encryption.
• Built-in key management.  In the FlashArray, there’s never keys to worry about or manage.  They keys divided into fragments and stored in the public (non-encrypted) regions of each SSD.  When the FlashArray is booted, it looks for a “quorum” or majority of the SSDs.  As long as enough of the SSDs are present, a quorum is formed, the keys are assembled from key fragments stored on each SSD, and the entire array is “unlocked” and can begin serving data.  Conversely, if a single or small number of SSDs are stolen or found in the wild, there is no way to piece-together their keys without a quorum of the drives.

With Pure Storage, encryption is effortless – no performance penalty, no management penalty, and no additional cost.  Read more about encryption on our site.

More Interoperable: New OpenStack CINDER Driver and AIX Support

The FlashArray is already best-in-class in terms of platform support and interoperability testing.  In fact, Pure Storage has spent over $5M since inception on its state-of-the-art testing, verification, and interoperability lab, and maintains a lengthy set of interoperability partnerships. In this release we’ve completed our full implementation of the VMware VAAI APIs, including both xCopy and Thin Provisioning support.  Additionally, we’ve added full support for AIX, including ACA support to enable queue depths >1.  Finally, and perhaps most interestingly, we’ve shipped our first OpenStack CINDER provider.  OpenStack is increasingly the cloud management/automation platform of choice, and we’re seeing increasing adoption in our web and hosting customer base.

More Supportable: New CloudAssist Support

Last, but certainly not least, Pure Storage has been focused aggressively on building our support offering over the past six months. When customers first start to work with us, they often come into the relationship a bit worried that we’ll be able to support them to the same level they expect from “big storage.”  But more often than not, a few months later, they are expressing their surprise at how much better Pure Storage support is compared to the service they were used to.  We take great pride in delivering the absolute best support possible at Pure Storage, and we differentiate through a combination of best-in-class investment (focus, hiring the best people, and rigorous process) plus a unique cloud-driven technology approach….that’s CloudAssist.  I’ll be authoring a separate blog post on CloudAssist tomorrow (it deserves its own piece of the spotlight), but in the mean time if you can’t wait, watch this video:

You can also read more about CouldAssist on our site.

Purity 3.0: Come and get More

Well, that’s it….there’s enough content in this blog post to spend literally hours learning about Purity 3.0.  We hope you dig deep, and we want to hear from you.  We’ll be happy to give you a private demo, upgrade your current FlashArray to 3.0, let you try one if you don’t have one, and more than all we want to hear your feedback.  Just click “contact us” at the top of purestorage.com to get in touch.  And remember, flash storage is all about the software, and Pure Storage is dedicated to winning the software race in flash.

Posted by John Woodall, VP of Engineering, IAS 

While a Tesla may still be a little bit beyond the average person’s affordability for a car, Tesla Motors has clearly revolutionized the way we think about performance cars and how drivers power their cars.  Tesla is not just another fast car, but rather a game changing innovative technology that enables drivers to drastically reduce their day to day costs without sacrificing performance.

This is how I feel about all-flash arrays for enterprise storage, with the added bonus of affordability!

Enterprise storage leaders are turning to all-flash arrays to achieve business results never before seen with disk, all at the same cost of disk.  Over the past twenty (20) years enterprises have stretched mechanical disk technology as far as it will go, throwing more and more drives at applications to meet the performance requirements of today’s applications with shrinking budgets.  With enterprise all-flash arrays IT leaders are now building datacenters to run more efficiently, meeting user experience expectations and realizing significant cost & resource savings with flash.

What I find interesting is how quickly flash storage in general and specifically the concepts of all-flash arrays have captivated the minds of my prospects and customers.  As more and more IT leaders become aware of and educated on flash they really want to see how an all-flash array can fit in their environment and address their needs.  In some cases they really do require a high number of IOPS… however; in many cases the ability to serve data from storage with consistently sub-millisecond response times is of most interest.  When customers see how much performance they can get in such a small footprint, and the effective cost is less than what they thought, the conversation changes.  *CLICK* the light bulb just went on…

Just like driving a Tesla.

Yes, the Tesla is a car you drive… and it is nothing like anything that has come before it.  It is a ground up, clean sheet design; as is Pure Storage.  This is truly game changing and the parallels to the flash storage array are very similar.  Traditional arrays have not really changed or improved, they just gotten bigger, somewhat faster, and consume a lot of space and power.  Traditional automobiles with internal combustion engines have also not really changed in quite a while; they have effectively reached the pinnacle of balancing power-to-displacement ratios and reliability.  Along have come “hybrids” to both automobiles and storage, but this is in the end just an incremental improvement in my opinion, a good and necessary one but in the end I believe all flash arrays will become dominant in the data center.

All electric, all the time.  All flash, all the time.  That is disruptive.  Drive a Tesla and you will never be satisfied driving your old car again… deploy an all-flash array and you will never think of storage the same way again.

Delivering a properly built and designed all-flash array requires far more than just slapping SSDs into a shelf.  It requires addressing at a more granular and holistic level the very nature of flash… you must do more than rely on the SSD vendors drive firmware that handles oversubscription, garbage collection, TRIM,  wear-leveling or write amplification.  Those are table stakes but not in and of themselves sufficient.  You have to take that foundation and build on it.

Pure Storage understands how critical this is and they have engineered a software stack from the ground up for flash storage.  Storage efficiency improvements in a disk based array are about reducing the number of spindles, but in an all-flash design, deduplication, in-line compression and thin provisioning go above mere drive firmware to reduce the amount of data written which enhances what the SSD comes with.  Architectural simplicity matters a great deal.  The Purity OS is designed to be granular at a 512 Byte level… simple, no alignment issues, and the block size of the application matters far less.  No tuning or optimization; fast and simple… and because of the software engineering more reliable flash.

There are so many innovative software bits in Purity OS that once you understand them, you will shake your head, I know I did.  Not a topic of discussion for this blog per se but things that really stand out; a 100% virtualized array under the hood, RAID-3D which is amazingly adaptive in real time, the ability to rebuild vs. write just ‘cuz things work so fast… maybe the next time

So, what other parallels in the storage world have we seen that are game changing?   If you hang around any storage companies you will hear things like “tape is dead”.  My personal take is that this is not the case but it does speak to the nature of our industry; Innovation, disruptive economics and operational efficiencies will cause change.

From a data protection perspective, disk in many forms from many vendors have become a primary design requirement for any modern backup architecture at a reasonably large scale… oh and by the way, no one gets rid of their data, right?  In the same way that disk is disruptive to the backup world where tape is cost effective, much like SATA and NL-SAS disks are, flash brings disruption to the storage paradigm.  As we saw a bifurcation around cost in data protection we will see in the primary storage market the bifurcation of the market into $/IOP optimized storage (e.g. All-flash) and $/GB optimized storage (e.g. SATA/NL-SAS).

What about the cost of flash?  It is more expensive, no argument there.  But remember that Pure does in-line compression, deduplication and thin provisioning.  If you go to the Pure website, (http://www.purestorage.com/ ) you will see at the bottom of the page the “dedupe ticker” which shows at the time I was writing this a dedupe and compression benefit of 5.71:1.  What is the impact?  For every TB of useable flash on a Pure array, the average customer is getting just shy of 6 TB of capacity, and if you calculate in thin provisioning a greater than a 12:1 ratio is being seen on existing deployed systems.

So, yes flash is more expensive but with the storage efficiency technologies in the Pure FlashArray, the effective cost per GB is right down around what you would pay for 15K RPM 3 ½” SAS disks… but remember, you would need something in the range of 1,200 of them and as much as 300 RU of space to get the same performance (200K IOPS) that you get from 8 RU of Pure… Your mileage may vary but you get the idea.

Really fast, very low latency, far less space and power required and lower operating costs… If I gave you a Pure FlashArray to drive for a month, like giving you the keys to a Tesla and with a guarantee,  would you keep it?  That part is easy, the fine print:  Wait… there is no fine print!  Here is how the Pure, Love Your Storage Guarantee works:

• Try the Pure Storage Array risk-free for 30 days.
• If there is any problem, let us know.
• We’ll work with you to fix it.
• If you are not happy for any reason, return it for a full refund.

Hmmm, improved performance, dead simple to own and operate, a game changing experience with lower operating costs.  Was I really talking about Tesla?

Take it for a spin.   IAS can help you get started, contact us at info@iarchive.com.

The problem with HammerOra is that it doesn’t actually push the storage sub-system to its limits.  The bottleneck in these tests is almost always a combination of CPU resources and concurrency.  Maybe that’s why you don’t see any storage vendors sponsoring TPC benchmarks.

To demonstrate what our storage is actually capable of, we like to use the “Silly Little Oracle Benchmark”  (SLOB) tool which was developed by Kevin Closson.  Kevin was the Performance Architect in Oracle’s Exadata group, and he now works for EMC.  Unlike HammerOra (and Swingbench), SLOB is specifically designed to drive a lot of physical I/O without any application contention.  In other words, SLOB shows just how much Oracle-generated I/O your storage subsystem can do.

For example, on the read-intensive test (128 readers, 0 writers), the Pure Storage FlashArray can drive over 2GB/s of throughput at over 250,000 IOPS with sub-millisecond latency:

 Naturally latency is a concern.  As the histogram shows, we average 1ms latency for the duration of the test run.

We will have more SLOB results and discussions soon.

If you are evaluating a Flash Storage solution, send us your AWR report and we’ll schedule a free consultation with our Oracle performance experts to talk about how Pure Storage’s FlashArray can improve the performance of your Oracle environment while actually reducing the cost of your storage architecture.

]]> http://www.purestorage.com/blog/on-benchmarking-database-storage/feed/ 3 http://www.purestorage.com/blog/on-benchmarking-database-storage/ http://feedproxy.google.com/~r/purestorage/~3/4ow6C0YfNOo/ http://www.purestorage.com/blog/pure-storage-makes-the-top-ten-places-to-work-in-silicon-valley/#comments Fri, 26 Apr 2013 18:31:24 +0000 Scott Dietzen http://www.purestorage.com/blog/?p=3592 Read More...]]> Building a truly exceptional team is everything for a tech company. So we could not be more thrilled that out of more than 500 organizations, Pure was named a top ten Best Places to Work in the Bay Area for 2013!

The winner here is the whole of Team Puritan—a team we are exceeding proud of, as we are of the egalitarian, fun loving, hard working, and customer-focused culture we’ve built together. The best and brightest seek to create a world-changing vision and then successfully execute to fulfill it. When a team loves their jobs as we do, it’s because we are working well together to accomplish both.  All of our successes to date and those going forward are a result of our team’s excellence and dedication. As we said in our ten word acceptance speech: “We are proud of our work and love to work it!”

We look forward to continuing our record growth as we expand the Pure Storage family in the months and years ahead. Think you’ve got what it takes to join one of the best teams in tech? Start here! We’d love to hear from you.

Last week we introduced a new feature at purestorage.com that I’m quite excited about: the Dedupe Ticker.  The ticker is a live feed from the call-home data repository here at Pure Storage Support, and it shows the average data reduction results across all Pure Storage FlashArrays deployed at customer sites with call-home enabled.  We did this for a simple reason: to provide transparency into the dedupe results that the FlashArray delivers for customers, and to start encouraging the industry to be more honest, open, and transparent about real data reduction results.

What Are Good Data Reduction Results?

One of the challenging things about data reduction is that it is inherently variable.  We see many vendors in the market hiding behind very generalized marketing claims about the efficacy of their data reduction technologies, often claiming “5-10x” reduction as a typically achievable result.  But what does that mean?  The reality is that data reduction varies a lot by workload, and it is really important to test vendor technologies on your data.  That said, we’ve deployed enough FlashArrays here at Pure across a wide range of workloads to be able to give customers pretty good expectations before they dive into a deployment.  We also make available our PureSize tool to allow customers to analyze their own data.  Here are some example Pure Storage customers across our typical use cases:

As you can see, results vary by customer, but we’ve pretty typically seen Database-type workloads achieve 2-5x reduction, server virtualization achieve 4-8x reduction (depends on what is inside the VMs), and VDI workloads achieve 5-10x reduction (depends on persistent vs. non-persistent).  All of the results above lead to successful deployments and happy customers, but understanding the expected level of data reduction pre-deployment is critical to this.

Does Data Reduction Include Thin Provisioning?

One of the sillier areas of confusion in the marketplace right now is whether different vendor-promised data reduction rates include thin provisioning or not.  Some vendors incorporate it into their reported data reduction rates, others do not.  Savings related to thin provisioning are useful to understand, but in our minds counting them as “dedupe” is a little less than honest.  It is also bad operational advice: if thin provisioning savings are included in the dedupe rate then as the host volume fills the dedupe rate will reduce over time…but if only actual data reduction is included, then the rate will be relatively stable as the volume fills, and then can thus be used to rationally plan capacity growth.  For clarity Pure Storage provides both numbers, both are available right on the front page of the FlashArray GUI, and both are available in the new Dedupe Ticker.  Want to know EXACTLY how we calculate data reduction rates?  We’re transparent about that too.

Why Different Data Reduction Technologies are Not “All the Same”

We’ve noticed a disturbing trend in the industry, every vendor is now talking about data reduction technologies as if they are a “checkbox” item on a product spec sheet, and that all data reduction technologies are the same.  Our findings in the field are quite different: we believe that data reduction technologies differ widely by vendor, and that this is a core differentiation for Pure Storage.  Here are a few dimensions to consider:

• Data reduction type: Deduplication, Compression, Pattern Removal, or all.  Some vendors will do dedupe but not compression, some will do compression but not dedupe.  Some will do data reduction on their flash tier but not on their disk tier.  Some will reduce simple patterns like zero blocks, but not more complex patterns (i.e. real deduplication).  Pure Storage delivers always-on deduplication, compression, pattern removal, and thin provisioning.
  
• Inline vs. post process.  Some vendors will reduce data as it is coming into the array, others will first land data to disk and/or flash before processing for data reduction.  In the legacy disk world, post-process was the norm because data reduction just took too long.  In the flash world, inline deduplication becomes possible due to the performance of flash and new architectures designed for it.  It is important to understand that inline affords two key advantages in the flash world: cost reduction via increasing the effective capacity of the flash, and write avoidance (and thus flash life extension) by removing write IOs which would have otherwise hit the flash.  Pure Storage’s data reduction is an inline technology, where IOs are committed to the DRAM in the array’s controllers (and mirrored to non-volatile memory to protect against complete array power loss) so they can be acknowledged back to the host immediately, but then data reduction processing happens before the IOs ever are written to the flash, avoiding writes and improving capacity.
  
• Reduction “chunk” size, granularity, and alignment.  Data reduction technologies vary in the granularity with which they analyze data.  Granularity is a bit of a trade-off: the smaller chunks you use to analyze data the more redundancy and benefit you’ll find, but the smaller the chunks the more metadata the process creates that must be managed.  Granularity also has a direct tie to alignment…anyone who has ever dealt with mis-aligned VMs in traditional storage knows exactly what I’m talking about…if all the layers (storage, hypervisor, VM, FS, application) don’t align on the right geometry, finding duplicates is all the harder.  Most data reduction and thin provisioning technologies today operate on the 4K or larger chunk size.  Pure Storage data reduction services are unique in that they analyze data down to a 512-byte granularity (8x smaller than 4K), but they are variable chunk in nature, meaning that if we find a larger chunk we store metadata references to larger chunks to drive metadata efficiency.  512-byte granularity also has the very nice benefit of essentially being auto-aligning to all larger geometries above.
  
• Performance impact.  This one is a biggie: why hasn’t deduplication taken the primary storage world by storm like it has the backup world?  Simple, in traditional disk storage dedupe is really slow and just can’t be run in any performance environment.  In fact, we’ve seen the same challenges in poorly-designed or retrofit data reduction implmentations in the flash world as well, enabling data reduction can come with a 50%+ performance impact.  If a vendor is telling you about their data reduction, the performance impact that comes along with it should be your very first question.  Pure Storage is different – the FlashArray was designed from the ground-up for data reduction, the data reduction is always on, and all the performance specs you see from us are with data reduction turned on.
  
• Global vs. local.  Many of the existing implementations of data reduction are confined to a portion of the storage array to keep their performance and metadata manageable, such as only within a volume, LUN, FS, shelf, etc.  NetApp’s data reduction, for example, operates within the Volume.  The downside here is obvious – the more you divide data reduction into pools or zones, the more you duplicate data between these pools and reduce overall effectiveness.  Pure Storage data reduction is global across the entire array: all volumes.
  
• Purpose-built vs. retrofit.  We found at Pure Storage that to enable data reduction technologies to work at flash speed and global scale, we had to design our entire array from the ground up to be optimized for data reduction.  As other vendors have realized that data reduction is a “must have” feature to make MLC flash arrays work (both from a cost point and a write avoidance perspective), we’re seeing a wave of vendors OEM or acquire 3rd-party technologies and retrofit them to their storage arrays, often as a post-process step.  Pure Storage data reduction technologies were all developed from scratch by Pure, which is key to our differentiation in the marketplace. 

In Closing…

We’re off to an exciting start for 2013! Since my last post, we enthusiastically kicked off the new fiscal year with our Global Sales Conference, we announced our updated P3 Partner Program, and today we launched the most comprehensive CUSTOMER GUARANTEE Program in the history of the storage industry!

The Mavericks of Storage!

Our Global Sales Kick-off was innovative, informational and loads of fun. Our theme this year is “Mavericks”, after the big wave surf just north of Santa Cruz, California. For those who may not be aware, this is a 40-50 foot+ swell that occurs only when the perfect conditions align.  In order to successfully jump off the equivalent of a 4 or 5 story building and ride these waves, it requires courage, commitment, skill and passion. This is the bar we set for ourselves as Puritans, knowing that opportunities like what we have in front of us as an industry, a company, a sales team or as an individual, just don’t come around that frequently. We should be humbled by it, respect it, learn everything we can about it and then be willing to give everything we’ve got to ensure we succeed. When we do, we’re all in (our customers, partners, employees) for the ride of our lives!

Our Customers and Advisors Speak Out!

One of the highlights from the week was the Q&A with our Customer Panel, who flew in to give us feedback on what they love about working with Pure and where they want us to focus for the future. As always, our customers inspired us, especially our engineering teams, with the value they are able to unlock with the FlashArray. There is nothing more motivating for an engineer to know that his/her work and innovation has real purpose and delivers quantifiable business and personal value for those who use it! It’s clear that our customers are as invested in our success and we are in theirs and it’s this type of relationship, whether it be in the form of a Customer Panel, a NetPromoter Survey, Customer Advisory Board (CAB) or just an open dialogue, that will help us build the next great storage company. Thanks to all of those who flew in!

We were also thrilled to have Frank Slootman, the CEO of ServiceNow and former President of EMC’s BRS Division and CEO of Data Domain, share his views on winning in the marketplace and building value for customers and shareholders. His experience and perspective, in addition to his approachability and sense of humor fit in very well with our Puritan culture.  The “recipe” for Data Domain’s success was very similar to our own, except that we’re shifting from disk to flash (rather than tape to disk), doing it for production storage applications  (vs. back up) AND WE HAVE NO DESIRE TO BE ACQUIRED BY EMC!!! Currently, Frank is the CEO of another very high growth company in ServiceNow, having led them through one of the most successful IPO’s of 2012. His enthusiasm toward Pure’s opportunity to help enable this next shift from disk to flash and the significant progress we are making against our vision was infectious! Thanks, Frank!

Customer Transparency: The Dedupe Ticker

As the industry starts to realize that Pure is on to the right “recipe” (flash + inline data reduction + enterprise array services), we’ve seen more and more competitors look to fill their dedupe gap by either shipping sub-par data reduction technologies, OEMing third-party code and retrofitting it to their flash storage, or simply talking a dedupe roadmap.  These vendors are often hiding behind vague marketing claims about their data reduction and positioning that “all dedupe technologies are the same”.  We believe that our purpose-built inline data reduction technology (deduplication, compression, and thin provisioning) are key differentiators for Pure, and we’re delivering the proof to back-up our claims!  Starting today you’ll notice a brand new section on the front page of www.purestorage.com: The Dedupe Ticker.  The ticker shows LIVE data reduction results from our entire customer call-home database, showing both the average data reduction (dedupe + compression) and the average total reduction (dedupe + compression + thin provisioning) across all FlashArrays at customer sites with call-home enabled (and across all workloads ranging from databases to VMs to VDI).  Is another vendor bragging to you about their data reduction?  Ask them to show you their results!

What? We’re Offering a 100% Money Back Guarantee???

Finally, and perhaps saving “the best for last” is our announcement today of the “Love Your Storage Guarantee”.  This unconditional and 100% customer-driven money back guarantee, is something the storage industry has never seen before and we’re hoping that customers and partners embrace it with as much confidence and excitement as we have in Puritan land.  We want to help re-define the paradigm in enterprise storage and re-set the expectation that IT leaders, Storage Administrators, DBA’s and others can and should not just tolerate, but LOVE THEIR STORAGE!  Trying Pure Storage is believing, so we felt that the best way we can motivate potential customers to try is to offer this 100% guarantee.  Data reduction: Guaranteed.  Vastly improved performance: Guaranteed.  Superior resiliency: Guaranteed.  Simplified management: Guaranteed.  24×7 Support: Guaranteed.  We’re so confident in the value we deliver that we are providing this unconditional assurance: love your FlashArray, or your money back!

What I like most about this guarantee program is that it is a direct reflection of the Pure culture: if we aren’t earning your loyalty as a customer by truly delivering storage that you love, we believe that we don’t deserve to keep your money.  The simplicity of the guarantee will serve as a powerful internal motivator to our people: do right by customers and they will do right by us!

Pure Storage Partners Love the “Love Your Storage” Guarantee!

Prior to launch our team worked with several of Pure’s authorized resellers to help define the terms of the guarantee and operationalize the program.  Partners told us they love the Guarantee and that they will use it to open new customer doors, shorten sales cycles and be able to accelerate the adoption of our FlashArray by enabling their customers to evaluate this new technology with absolutely no risk!  The Love Your Storage program is available through Pure Authorized Resellers and Distributors across North America, Europe, Asia and Japan.

How Does the “Love Your Storage” Guarantee Work?

The Pure Storage Love Your Storage Guarantee is designed to be simple: every FlashArray is guaranteed for 30 days after arrival at the customer site. If you are unhappy with the FlashArray for ANY reason, simply alert Pure Storage and you can elect to return the FlashArray for a full refund.  The guarantee requires no standalone agreement, full terms of the guarantee are reflected in the standard Pure Storage license agreement and quote, provided to each customer at purchase.  For more information, visit www.purestorage.com/guarantee.html.

We hope you are as excited as we are for all that 2013 has in store. We’d love your feedback on our Love Your Storage Guarantee and the Dedupe Ticker here or reach out to us directly….We’re committed to changing the storage industry, one customer and one partner at a time!

Thanks, all. See you in the marketplace!

Hat

John "Coz" Colgrove, John Hayes, Scott Dietzen

Not surprisingly, curiosity on the topic of mergers and acquisitions (M&A) seems to go up in times of competitive product announcements, such as those from EMC (NYSE:EMC) and NetApp (NASDAQ:NTAP).  With the storage market leaders now striving to craft copycats of the Pure Storage FlashArray, why isn’t Pure looking for an exit?

The simple answer is we remain convinced that the best outcome for all of our constituencies—customers, partners, employees, and investors—is for Pure Storage to remain an independent company for the long term. Why? Customers and partners tend to be skeptical of changes in control, as the result is often a dilution in commitment to their success. And after an acquisition, Pure employees and investors would have far less control over shaping our destiny, as well as reduced upside (versus the exponential growth we are enjoying as an independent). Also, Pure is simply more valuable as an independent company in that we avoid a classic Innovator’s Dilemma—today, no one is telling the Pure team that certain customers are off limits because they are poised to buy boatloads of antiquated VMAX, VNX or OnTap.

So while it would be inappropriate to say never, I can definitively say that we have very carefully assembled the Pure Storage team for long-term independence. Pure’s Board of Directors is made up of industry veterans that have lead the creation of well more than $100 billion in public market valuation at companies like Cisco, VERITAS, Workday, Data Domain, Peoplesoft and BEA Systems. We’re not a group looking for a quick hit. Rather, we ultimately see M&A as a “safety net” if we were to come up short in our stated mission—to lead the industry transition from mechanical to solid-state storage. 

Happily the need for that safety net is looking less likely every day. Pure is now the fastest growing company in storage history (at least for companies for which we have data), and the all-flash design we have been shipping to customers for three years has now been validated by the storage industry leaders. Having an ~18-month plus time-to-market advantage with the competitors coming to our playing field is as good as you can hope for in tech.

And we’re putting our money where our mouth is: over the past year, several storage market leaders have expressed an interest in acquiring Pure, some on multiple occasions. Our response has always been “no thanks” to even having the conversation. To revisit a favorite: Steve Jobs is herein quoted by Steve Lohr at the NYTimes:

“In a conversation years ago, Jobs said he was disturbed when he heard young entrepreneurs in Silicon Valley use the term ‘exit strategy’ — a quick, lucrative sale of a start-up. It was a small ambition, Mr. Jobs said, instead of trying to build companies that last for decades, if not a century or more.”

As an entrepreneur, there’s something subtle that happens if you start to think about selling your business: you take your eye off of Job’s ultimate prize—changing the world by building a quality product and business that stand the test of time. Today’s performance storage customer isn’t getting nearly enough for their money: Moore’s Law has effectively turned hard drives into tape (from the perspective of a CPU doing virtualized I/O, today’s “fast” mechanical disks are slower than tape was fifteen years ago). By staying independent, Pure is in the best position to right this wrong—why should you have to spend roughly a 1/3 of your data center CapEx and OpEx budgets on the “new tape”?

Looking forward, we will welcome purpose-built all-flash competition from EMC XtremIO and NetApp FlashRay when it arrives.  Competition drives innovation, maximizes customer value, and grows markets. So long as we can convince potential customers and partners to give the Pure FlashArray a try, we are prepared to guarantee we can continue our winning streak. In the mean time, we’ll continue helping customers replace mechanical storage with solid state, unleashing them from the performance and complexity headaches of hard drives, and saving them money in the process!

Yesterday Pure Storage announced a new customer success story: Riverview Hospital in Indiana.  It is a great story all-around: a customer who purchased a modern tiered disk/flash storage array just last year, loaded it with a ton of flash, and yet still ran into deep storage performance challenges.  You simply can’t rely on flash caching for predictable application performance.  In this case the workload is a core electronic medical records  (EMR) management application that the productivity of an entire network of 20 hospitals and medical facilities depends upon.

But the point of this post is a different one altogether.  In addition to saving the performance of their core EMR database, Riverview experienced a side-benefit of moving to Pure Storage that we are seeing to be common in our user base.  They were able to consolidate their core application environment and their VDI environment into one array, saving a ton of money in their VDI infrastructure by not having to dedicated a separate, stand-alone array to VDI.

What you say?!??!!!??  Running production databases and VDI on the same array?  That’s heresy!!!!

Indeed, in traditional disk storage it is, but many Pure Storage customers are committing such heresy on a daily basis, and loving it.  Let’s look at another customer, the City of Davenport, Iowa (you can read their case study here).  Davenport called Pure Storage when their VDI deployment on traditional disk storage went awry, and end-users were suffering unusable performance (particularly critical in this deployment since the VDI environment runs the heart of the operations of the city, including the Police and Fire department).  They moved to a Pure Storage FlashArray and experienced an immediate turn-around of the performance of the VDI infrastructure.  But as they got more and more comfortable with the FlashArray, they experienced a great side-benefit: they began migrating their core business applications to the array as well (SQL databases, Exchange, etc.).  The result is that they were able to consolidate nearly the entire city’s operations onto a single array, with performance room to spare and have since grown their Pure Storage deployment.

Why Does it Matter?  Solving VDI Cost & Performance

Traditional disk architectures have VDI admins stuck in a catch-22 between cost and performance: VDI is already expensive, and the biggest infrastructure cost within it is typically storage.  They need to dedicate an array to VDI to get even mediocre performance, and often load it up with fast 15K disks and as much flash cache as they can.  The result is an architecture that both struggles to perform and struggles to be economic, and storage is the root of both the performance and economic issues.

Pure Storage enables VDI admins to break out of the storage catch-22:

• The FlashArray starts with all-flash performance that delivers the best VDI end-user experience possible
• The integrated data reduction (dedupe and compression) enables all-flash storage to actually cost less than disk for VDI (typically <$100/desktop)
• Users can share the array with other workloads to further reduce cost and amortize spend across multiple projects

So that’s why we call Pure Storage The Ultimate Storage for VDI.  Come see why more and more customers like Riverview Hospital and the City of Davenport are choosing Pure Storage for VDI, and please, don’t dedicate your array to VDI!