June 19th, 2008
Apple announces ZFS on Snow Leopard
Finally, a modern file system on a consumer OS
As if Grand Central weren’t enough bad news for Microsoft, now they have ZFS to contend with. Building a reliable, high-performance file system takes years and Microsoft doesn’t have years to respond.
The formal announcement is for Snow Leopard server, which is how Apple introduces new file systems. HFS+ first arrived on a server version as well.
Who cares?
Anyone who stores data should.
Microsoft’s NTFS is 20 year old technology borrowed from DEC. Fine for small disks and puny CPUs. Not so great for today’s data intensive systems and applications.
Silent data corruption is common - only you don’t know it - because the corruption shows up as other problems, like missing DLLs.
ZFS: open source from Sun
ZFS is the first desktop file system with true end-to-end data integrity. Thanks to sophisticated tree-based checksums it detects and corrects silent data corruption anywhere in the data path: disks, cables, interfaces and more.
The checksums are stored with the parent block, so the file system always knows that the child block is both uncorrupted and the correct block. That’s just one of the errors that NTFS and most other commodity file systems - including the Mac’s HFS+ - are prone too.
Sun’s ZFS engineering team started working on ZFS 7 years ago as a clean-sheet design. It combines file system and volume management functionality. Instead of managing individual disks, you manage a pool of blocks. ZFS takes care of the details.
Turning up the heat on Microsoft
For all of Microsoft’s fine talk about innovation they don’t do squat unless someone else does it first. Remember IE 6? ZFS is a modern and innovative file system that solves some difficult data storage and integrity problems. Like these:
No more Disk Warrior
Data corruption on PCs and Macs is a sad and stupid fact of life. Power failures, flaky RAM, poor grounding, (slowly) failing hard drives, driver glitches, phantom writes and more conspire to rot your data.
ZFS eliminates that. All blocks are checksummed and the checksum is stored in a parent block. ZFS always knows if the block is correct and/or corrupt. Every block has a parent block (with one obvious exception that gets special treatment), so the entire data store is self-validating. You’ll never have to wonder if all your data is correct again. It is.
No RAID cards or controllers
ZFS implements very fast RAID that fixes the performance knock-off against software RAID. In ZFS all writes are the fastest kind: full stripe writes. And the RAID is running on the fastest processor in your system (your Mac), rather than some 3-5 year old microcontroller.
Just add drives to your system and you have a fast RAID system. With Serial Attach SCSI and SATA drives you’ll pay for the drives (cheap and getting cheaper), cables and enclosures.
No more volumes
Every time you add a disk to your Mac you see another disk icon on the desktop. If you want to RAID some disks you use Disk Utility (or something) to create the volume. Slow, error-prone, confusing.
ZFS eliminates the whole volume concept. Add a disk or five to your system and it joins your storage pool. More capacity. Not more management.
Backup made easy
ZFS does something called snapshot copy, which creates a copy of all your data at whatever point in time you want. Copy the snapshot up to a disk, tape or NAS box and you are backed up.
Create a snapshot on every write if you want, so if your database barfs you can go back to just before it choked.
But that’s not all!
For in-depth treatment of ZFS see here and here. Includes links to more technical info and benchmarks.
The Storage Bits take
It would be nice if Microsoft were driving innovation and reliability, but - like General Motors - they prefer to rest of their laurels. And like General Motors, they are facing a long and painful decline if they don’t get their act together.
GM says they are proud that 1 in 4 cars sold in America are GM - but the number used to be 3 out of 5. Microsoft is rightfully proud of their 90% market share. But that share can change - as it has for IE - and they have nowhere to go but down.
As users we benefit from the competition. Kudos to Apple for bringing the latest technology to consumers.
Comments welcome, of course. For more background on data corruption issues check out 50 ways to lose your data, How data gets lost and How Microsoft puts your data at risk.
Robin Harris has been selling and marketing data storage for over 20 years in companies large and small. See his full profile and disclosure of his industry affiliations.
| 06/20/08
