This post is a guest posting from Jerry Huang from Gladinet and discusses the differences between Microsoft’s cloud platform Azure and the Amazon S3 platform. 

 This month, Microsoft transitioned Windows Azure Platform from public preview mode to full production mode. Azure Storage is part of the Azure Platform and competes directly with Amazon S3. It is great news for cloud storage consumers but which one to choose?

This article will compare the price, performance, developer support and partner support between the two.

Round 1: Price

Microsoft has great introductory offer to celebrate the commercial launch of Azure. As shown below, everyone can get 500MB free storage and everything in January 2010 is free (check price here : http://www.microsoft.com/WindowsAzure/offers/?lang=en&country=US&offer=MS-AZR-0001P ). There are additional benefits for MSDN subscribers such as 10G free for 8 months with additional ongoing benefits (check MSDN benefits here: http://msdn.microsoft.com/en-gb/subscriptions/ee461076.aspx).

As a counter measure from Amazon S3, it offers free transfer until June 30, 2010 (check price: http://aws.amazon.com/s3/#pricing ).

Other than these free offers, the price between Azure Storage and Amazon S3 is pretty much the same with $0.150 per GB. Since Amazon is more established, it favors big customers since the price will be lower if you have more than 50TB of data. 

In round 1, Microsoft has the edge on price with free introductory offers, especially for MSDN subscribers.

Score: Microsoft Azure Storage 1  | Amazon S3 0

Round 2: Performance

For performance, we will need to compare the upload and download speed from the same location to both Azure Storage and Amazon S3. We will use a tool – Gladinet Cloud Desktop (http://www.gladinet.com ), which can map both Azure Blob Storage and Amazon S3 side by side as virtual folders in a network drive.

The test will be simple, drag a big file from local PC to Azure first for the upload, then drag it from Azure to another place on local PC for the download. Then repeat the same steps for Amazon S3. We will compare the time it takes afterwards.

File Size : 15,903,611 bytes

In round  2, Azure has better upload time while Amazon S3 has better download time.  Considering Amazon S3 may have more usage, having a shorter download time has an edge because it means  a closer data center.  However, if your use case is online backup – write once and seldom read, you may favor upload speed.

The test result may also vary from location to location so one person is close to Azure while the other person is close to Amazon data center and vice versa.  I would recommend using Gladinet Cloud Desktop to test the water for both before committing  to either one.  For important documents, you may want to consider using both.

At the end, both are fast enough for daily usage.

Score: Microsoft Azure Storage 1  | Amazon S3 1

Round 3: Tools

As it is now, doing simple Google Search on “Amazon S3″ and “Azure Blob Storage” reveals more tools for the Amazon S3 than for Azure Blob Storage.  Amazon S3 holds the edge now. However,  it is unthinkable that Microsoft will not catch up with all its software and platform power, given it time. With Azure integrated into Visual Studio,  I would expect more tools to show up for Azure Storage.

Score: Microsoft Azure Storage 0   | Amazon S3 1

Round 4: Partners

Amazon S3 has been powering several well known online storage companies, such as Dropbox, JungleDisk and etc. These vendors have locked into Amazon S3 and unlikely to switch. Microsoft Azure has its own same-company partners in Windows Live (such as Live Mesh & SkyDrive) too.

Score: Microsoft Azure Storage 1  | Amazon S3 1

Conclusion

Windows Azure Storage is an exciting cloud storage offering, competing with Amzon S3 directly.  The competition is healthy with consumers have more choices and more free options.

Reference

1. Map Drive and Backup to Azure Storage in 3 Steps (http://gladinet.blogspot.com/2010/01/map-drive-and-backup-to-windows-azure.html )
2. Manage Azure Blob Storage with Ease!  (http://gladinet.blogspot.com/2009/12/manage-azure-blob-storage-with-ease.html )

In my first review of the DroboPro, I was somewhat disappointed that I couldn’t access the ‘Pro from the Drobo Dashboard when the device is serving out iSCSI LUNs across the IP connection.  This restriction is a serious flaw, as it requires taking the DroboPro down and off the network in order to connect to a PC running the Drobo Dashboard.  However, I’ve found a workaround that allows me to run the Dashboard *and* present iSCSI LUNs at the same time.  Here’s how…

Firstly, let’s review my configuration.  I have a DroboPro that connects to my main ESXi server.  It presents eight 2TB thin provisioned iSCSI LUNs to ESXi.  These are then used as datastores or can be used for RDM devices.  It’s the use of RDM devices that proves the answer to the problem.  An RDM (Raw Device Mapping) device isn’t emulated by vSphere/ESXi as a standard vmdk would be.  ESXi passes all I/O directly to the LUN, simply retaining a pointer reference to it on an existing datastore.  I’ve used RDMs in the past to virtualise storage management servers (for example EMC’s Solutions Enabler) where the software talks directly to a LUN on the storage array using bespoke non-standard SCSI commands.  It appears that the Drobo dashboard also uses bespoke commands to connect to the DroboPro when only the iSCSI IP link is available.  Fortunately, RDM honours these too, so the DroboPro can be administered through the iSCSI LUN using it as a command device.

So, to administer your DroboPro:

1. Create a dedicated iSCSI LUN on the DroboPro using standard techniques.  The LUN can be created as a minimum of 1TB, however as its virtual, the LUN will occupy only a tiny amount of real physical space.
2. Assign the iSCSI LUN as an RDM device through ESX/ESXi to a Windows host.  Format the LUN on the host.
3. Install Drobo Dashboard on the same Windows host.  Voila!

It may well not be necessary to create a dedicated LUN, as I tested this theory using an existing server that had a DroboPro iSCSI LUN already assigned to it.  There’s one thing to be careful of though.  Remember if the virtual server you’re running Drobo Dashboard on is configured from LUNs on the DroboPro, you need to ensure you don’t affect those LUNs otherwise you compromise your management station.

Finding this workaround makes me feel happier about using the DroboPro in an ESXi/ESX environment.  I’d still recommend creating all the volumes you need on the DroboPro up-front before you start using it with ESX, however at least now there’s a way to monitor the device without relying only on the external lights.

This is a problem that’s been on my mind for some time; can the Drobo Dashboard see more than one device at the same time?  Well, it turns out the answer is yes.

At first my thought was that it wasn’t possible for the Drobo Dashboard to manage more than one device.  Currently I have a Drobo connected into a DroboShare and a shiny new DroboPro, which is serving iSCSI LUNs to my ESX server.  I’ve not been able to get Drobo Dashboard on my main Windows 7 PC to work correctly.  It seems to have issues seeing the DroboShare.  I’ve checked all the obvious stuff (like firewalls) but still can’t get it to work.  Anyway, I was testing connectivity from my MacBook to the DroboPro using USB and to my surprise, both the DroboShare (called Drobo1) and my DroboPro (called DroboNAS2) both appeared in the GUI.  Have a look at the attached graphic.  You can see the Drobos listed at the top of the screen; this can be used to toggle between a display of both devices.

Now this is the Mac version of the Dashboard.  I can’t vouch for the Windows version as mine clearly doesn’t work correctly, however I imagine it works the same way.  So the answer to the question; “Can Drobo Dashboard manage Multiple Drobos?” is clearly – Yes.

Oh, one other thing worthy of note; I can confirm that the DroboPro definitely will *not* let me connect to the USB port while it is serving iSCSI LUNs through IP.  That’s a real shame, as I can’t manage the device without an outage.  Bit of a showstopper, I think.

Thanks to everyone who’s taken the time to read my semi-focused rants over the past 12 months.  I’ve been grateful for the interaction and comments, good or bad.

You’ll see the site has a new design.  Over the coming 12 months I hope to blog more than ever and to cover technologies in more depth and breadth. 

Wherever you are in the world, thanks for your contribution and a happy new year to you all!  Let’s look forward to a tech 2010!

This is the time of year I like to do a little early spring cleaning and one group of storage devices due a clean out are my old hard drives.  After many faithful years, I’m finally saying goodbye to all of my parallel ATA devices. 

One the one hand, I hate throwing things out.  I like to use technology to the absolute limit, but eventually some things outlive their usefulness and so it has come to pass for PATA hard disks.  Unfortunately the maths and environmentals no longer stack up.

Capacity and Power

I’m disposing of twelve (12) 3.5″ drives that have a total capacity of only 344GB.  That’s a mere 28GB per HDD.  The lowest capacity drive is only 1280MB in size.  Contrast that to the latest hard drive I’ve just purchased.  It’s (another) 2TB model – the Seagate Barracuda ST32000542AS.  This drive offers five (5) times more capacity for only 1/12 of the physical space and power requirements.  With these kind of gains in efficiency, why retain old drives any further? 

RAID and Speed

Now of course, multiple drives have one or two benefits over a single hard drive; firstly they can be protected using RAID and second, multiple spindles provide improved throughput.  However the cost of 1TB and 2TB drives is now so low, they can be purchased in multiples reasonably cheaply and the RAID benefits retained.

Recycling

As a final thought, I think we should be doing more to recycle old hard drives.  After all, we recycle mundane components such as printer ink cartridges, batteries and so on.  Perhaps we could see a slot for hard disks at the local recycling centre?

As part of my work at Storage Fusion, I get to dig into the inner workings of storage arrays in a way most people don’t (either through inclination or time).  One interesting anomaly we discovered this week was the variability in capacities of CLARiiON disks.  What results from this is that the capacity you expect to see from a particular disk model may vary quite considerably.

520 Byte Sectors

It’s pretty well known that the CLARiiON array disks are low level formatted with a 520-byte sector size compared to the standard 512-bytes on most drives and on the disks you would install into your PC at home.  The additional 8 bytes are used for error checking and other additional information and are collectively known as DIBs – see Steve Todd’s informative post on the subject here.  This increased sector size effectively reduces the capacity of the drive by approximately 1.5%.

Low level formatting does, however offer another potential benefit; the drives don’t have to be reformatted to the original size specified by the manufacturer.  In fact, EMC choose standard sizes for each of their drive types, which are referenced by generic names, such as CLAR320, CLAR300 and so on.  The numeric part of the name represents the nominal size of the drive.  Having generic names enables EMC to substitute a range of drives from different manufacturers and so not be tied to taking disks from a single supplier.  It also means that the drives have been reformatted to a consistent capacity, usually lower than the manufacturer recommends – but not always.

Unlocking Free Space

Take for example the CLAR320 model.  This is typically a standard Maxtor ATA 320GB drive.  Navisphere commands indicate that these drives have a usable capacity of 304,169MB.  Based on the 520-byte sector, this represents an actual capacity of 308,922MB.  Now, a standard 320GB drive, which is quoted by the manufacturer as having decimal GB (1000×1000x1000) has a binary capacity (using 1K = 1024 bytes) of 305,176MB.  Magically the CLAR320 drive has an additional 3746MB or nearly 3.7GB of extra storage space!

Now compare to the CLAR300 drive.  A typical 300GB drive has a capacity of 286,106MB.  The CLARiiON model has a usable capacity of 274,845GB or taking into consideration the 520/512-byte blocks, 279,139MB.  This represents a shortfall of nearly 7GB on the capacity of the drive!

The obvious question is; how can there be more capacity on a drive than the manufacturer quotes?  Well, with the CLAR320 drives, that’s pretty easy to work out.  From the Seagate specification manual, the drive is configured with 2 platters and 4 heads, so 80GB per head (or recording surface).  In the same family, the 250GB model uses only 3 heads or 83.3GB per head.  Clearly the drives can achieve higher areal density and this is not fully utilised in the 320GB model.  Reformatting the drive at the low-level must unlock this reserve potential, resulting in a slightly higher capacity.

OK, in reality, although I said considerable, the potential capacity improvements aren’t huge and I’m not advocating that everyone go out and buy 320GB drives for their CLARiiONs.  However, what is interesting is how vendors will sell capacity of one size and the actual usable value can be less than you expect.  In this instance the difference between two similar drive capacities was 10GB.  Over thousands of drives, that can add up to a discrepancy which is difficult to explain to management.  At Storage Fusion, we’re working on ensuring we can help customers identify every scrap of their storage usage.

So, perhaps when you order that next EMC, HDS, IBM, HP array, you should ask exactly what the expected mileage will be…

I spent some time today looking at the release notes for Enginuity code 5874.207.166, which presumably is the one that brings the much lauded Fully Automated Storage Tiering (FAST) into general release on V-Max.  Just above the FAST paragraph I found the following:

Symmetrix Virtual Provisioning Space Reclamation reduces capacity requirements and total cost of ownership by automatically reclaiming chunks (768 KB track groups) that contain all zeros. This is most effective when used on volumes after thick-to-thin migration or replication.

So, it seems that V-Max now supports features previously only seen on 3Par InServ, HDS USP V and HP XP – that is the ability to reclaim empty “zeros” of data from LUNs – otherwise known as Zero Block Reclaim.

I don’t remember EMC mentioning this little fact as part of their big FAST announcement.  In fact, looking back over Barry B’s posts, here’s a link to a post from July in which Barry indicates (quoting again);

I cannot confirm nor deny that VP will support one or more unused space reclamation approaches in the future.

So do EMC just see ongoing space reclamation as a BAU activity, not worthy of an announcement?  I’m surprised that this would be the case.  Reclamation of “empty” storage is incredibly important when migrating from thick->thin storage environments.  Hitachi quote around 40% savings from using ZPR after a migration to thin provisioning on USP V.

Perhaps EMC don’t want us to know that migrating to V-Max can actually reduce the amount of storage in use.  After all, its not good for hardware sales, is it?

By the way, EMC, please feel free to comment on this new feature and how easy it is to use.  I’d be interested to discover how it is implemented.

This is the last in a four-part series of posts on the Sun Storage 7000 USS storage arrays.  Previous posts in this series can be found here:

Review: Sun Storage 7000 Unified Storage System – Part I

Review: Sun Storage 7000 Unified Storage System – Part II

Review: Sun Storage 7000 Unified Storage System – Part III

Previous posts have discussed the physical hardware and what you can do with it.  Sun also do a simulator version of the 7000 series array, which can be used to evaluate the technology.  The Simulator is available for both VirtualBox and VMware.  I chose the VMware version and deployed it on VMware Fusion on my MacBook.

Installation

The installation process is remarkably simple.  Download and unzip the simulator (link here) and for Fusion, simply use File -> Open to open the file “SunStorage.vmwarevm”.  The configuration process then asks for some simple details – IP address, default gateway, password and so on.  Once this is complete, the simulator starts up and can be accessed via the standard web interface on port 215.  I’ve included a few screenshots at the end of this post that highlight the configuration process.

Configuration

Once logged into the simulator, a disclaimer is presented to the user indicating that this deployment isn’t for production usage or performance measurement. This is pretty obvious; the simulator shows functionally how things work but will never provide the same performance as a dedicated device.

The simulator provides 15x 2GB drives, which although not ‘real’ are more than enough to do evaluation with.  As far as I can tell, the simulator appears to be fully functional.

Summary

Sun have provided a simulator package that appears to pretty much mirror a real USS 7000 array.  Having a fully functional system like this enables new users to gain confidence with it; although not much training is needed, making the mistakes on a simulator is much more preferable to making them on the real thing.  In addition, it’s easy for any potential purchaser to get a real feel for how easy configuring the 7000 Series can be.

The 7000 simulator is probably equal or better than the Netapp simulator, which I’ve raved (and probably ranted) about many times.  It’s a shame that Netapp don’t choose to make their simulator open to all users, but that’s another discussion entirely.  The Sun 7000 simulator simply rounds out what to me is a great product, offering storage and simplicity in a single device.

Disclaimer: Sun Microsystems provided a USS 7000 series array on loan in order for me to evaluate this technology.  The unit has since been returned.  I have not been paid by Sun to write this series of posts or received any other benefit or inducement of any kind from Sun Microsystems.

Yesterday, EMC announced Fully Automated Storage Tiering (FAST), their much hyped and much anticipated storage feature enabling the automated moving of data between tiers of storage on a policy basis.  However the most notable missing feature in the EMC announcement was the lack of support for legacy DMX-3 and DMX-4 platforms.  This to me sends a message loud and clear that despite continuing to sell it, the DMX3/4 legacy monolithic hardware is dead.  If that’s the case, why bother buying from EMC any more?

Discounting EMC in the storage array market may seem like a naive and perhaps foolish comment to make.  After all, recent IDC numbers show EMC top of the pile at nearly a quarter of all external storage arrays sold, depending on which figure you choose to use.  However, take a moment to look at the EMC briefing pages on FAST (you can find them here).  There you will see Intel co-branded with EMC, highlighting many previous messages that monolithic architectures are dead and commodity modular boxes are the way of the future.  We’ve seen that this year already with the release of Atmos.

To my knowledge, FAST is the first “innovation” of the new V-Max product line, but it isn’t unique.  In fact, I don’t think any features of V-Max are unique; the architecture is found in many other products.  There’s a whole raft of mid-range storage arrays from IBM (XIV), 3Par, Compellent, Pillar, Dell/Equallogic and HP (Lefthand) with the last two being acquisitions of successful companies.  I expect in the next 12 months we’ll see enterprise modular releases from Hitachi/HP and a revamped EVA.  Most of the products mentioned here have been designed from scratch to remove the legacy encumberances of the past that products such as V-Max still retain.

So what’s my point?  Well, simply this; EMC have legitimised the enterprise modular architecture characterised by V-Max.  This accepts that the future is commodity-based hardware with differentiation in software.  However, EMC are no longer the leaders in this field and are having to play catch up.

 There’s never been a better time to look wider than the Big 4 (EMC/Hitachi/HP/IBM) and see if the features you need can be found elsewhere.

This is the second in a series of posts looking at the DroboPro from Data Robotics Inc.  Previous post(s):

Review: DroboPro – Part I

Previously I discussed a few frustrations with connecting my new DroboPro to ESXi, the target environment for my new device.  I’ve now managed to get the ‘Pro connected and visible within ESXi.  See the attached screenshot image taken from my production ESXi server.

So far, I’ve configured eight 2TB LUNs (although the DroboPro itself only has a capacity of 5TB, 2x 2TB and a single 1TB drive) which are numbered 0-7; the LUN column on the screenshot.  Compare this to the LUNs presented from my ix4-200d, which has three LUNs all numbered ‘0′.  The difference is in the way the DroboPro is choosing to present iSCSI devices and the Runtime Name column gives a clue.  In standard Controller/Target/LUN or CxTyLz/CxTyDz methodology, the ix4 disks are using separate targets to map out each iSCSI LUN, with the LUN number always ‘0′.  The Target setting is analagous to a storage port, typically a physical port on large fibre channel arrays.  The LUNs are then numbered on that port.  The DroboPro presents all its iSCSI LUNs on the same target (in this instance number 3) and so the LUNs are numbered from 0.

This method of presentation doesn’t necessarily cause a problem, but is just a different way of presenting the LUNs.  Presumably with the Drobo Elite the multi-user functionality is achieved by enabling multiple targets per Drobo, each mapped to a separate user.  In any case, I now have visibility of my DroboPro within ESXi.  Time, to write some data.