Diskeeper Blog: Disk Defrag, Computer Performance & More | Diskeeper

Experts discuss built-in defragmentation and the superior merits of Diskeeper optimization

pm.nospam@nospam.diskeeper.com (dawn) — Fri, 27 Jan 2012 09:18:00 -1300

Recently, there’s been a lot of talk about built-in defragging systems. Is Windows®7 the best option? In the latest issue of Processor Magazine, experts weigh in, making the case for Diskeeper’s optimization in the enterprise. Read the whole article here: http://www.processor.com/articles//P3402/11p02/11p02.pdf?guid

Setting the Record Straight - Windows 7 Fragmentation, SSDs, and You

pm.nospam@nospam.diskeeper.com (Howard Butler) — Sat, 21 Jan 2012 14:50:00 -1300

In today’s well connected world of electronics and instant communications I received a text from a friend asking if I had seen the recent PC World magazine (February, 2012). He said it had some tidbit of information concerning one of my favorite subjects; system performance, defragmentation, and SSDs. I located a copy here at the office and found the article. As I read the first line I realized the debate on the virtues of defragmentation especially on SSDs will be one that goes on indefinitely as no one really talks about the issue with supporting hard facts and numbers. Most articles are rehashing ideas and opinions long since debunked. They continue to surface because very few truly understand the intricacies of the Windows NTFS file system and that of the storage media, whether it is rotating magnetic hard disks or electronic solid state disks.

So let’s set the record straight… Fragmentation is exponentially more of a problem with today’s data explosion. Defragmenting once a week will still cause the user to experience slowdowns from the degradation effects and doesn’t address the issue when files are initially being written. And yes, never do a traditional defrag on SSDs.

NTFS file and free space fragmentation happens far more frequently than you might guess. It has the potential to happen as soon as you install the operating system. It can happen when you install applications or system updates, access the internet, download and save photos, create e-mail, office documents, etc… It is a normal occurrence and behavior of the computer system, but does have a negative effect on over all application and system performance. As fragmentation happens the computer system and underlying storage is performing more work than necessary. Each I/O request takes a measurable amount of time. Even in SSD environments there is no such thing as an “instant” I/O request. Any time an application requests to read or write data and that request is split into additional I/O requests it causes more work to be done. This extra work causes a delay right at that very moment in time. Whoever thought that defragmenting once a month or weekly was good enough, simply didn’t understand fragmentation.

Disk drives have gotten faster over the years, but so have CPUs. In fact, the gap between the difference in speed between hard disks and CPU has actually widened. This means that applications can get plenty of CPU cycles, but they are still starving to get the data from the storage. What’s more, the amount of data that is being stored has increased dramatically. Just think of all those digital photos taken and shared over the holidays. Each photo use to be approximately 1MB in size, now they are exceeding 15MB per photo and some go way beyond that. Video editing and rendering and storage of digital movies have also become quite popular and as a result applications are manipulating hundreds of Gigabytes of data. With typical disk cluster sizes of 4k, a 15MB size file could potentially be fragmented into nearly 4,000 extents. This means an extra 4,000 disk I/O requests are required to read or write the file. No matter what type of storage, it will simply take longer to complete the operation.

Suppose I chose to do some editing of my family videos on Tuesday evening. Even the built-in defragmentation tool in Windows 7 doesn’t do me much good because it isn’t schedule to run until Wednesday morning at 1:00am. This also means that quite a bit of fragmentation has built up since the previous week when it last ran. Maybe I’ll manually run it, but that can take quite a while and I’ve wasted time that I would have rather spent on my project. Unfortunately, the Windows built-in defragmentation utility doesn’t prevent fragmentation so even after running it manually; I still will wind up with fragmentation and slow access speed of my newly created files.

I’ve often thought about why Wednesday at 1:00am was chosen as the time to schedule defragmentation. Why isn’t it scheduled all the time? It is because there could be system resource conflicts that either interfere with getting the task done or the defragmentation process has difficulty throttling back under a variety of conditions. Regardless, this wait a week to clean up fragmentation doesn’t really help me when I need it most.

As pointed out in the article, the built-in defragmenter does not have the technology advancement to properly deal with fragmentation and SSDs. The physical placement of data on an SSD doesn’t really matter like it does on regular magnetic HDDs. With an SSD there is no rotational latency or seek time to contend with. Many experts assume that fragmentation is no longer a problem, but the application data access speed isn’t just defined in those terms. Each and every I/O request performed takes a measurable amount of time. SSD’s are fast, but they are not instantaneous. Windows NTFS file system does not behave any differently because the underlying storage is an SSD vs. HDD and therefore fragmentation still occurs. Reducing the unnecessary I/O’s by preventing and eradicating the fragmentation reduces the number of I/O requests and as a result speeds up application data response time and improve the overall lifespan of the SSD. In essence, this makes for more sequential I/O operations which is generally faster and outperforms random writes.

In addition, SSD’s require that old data be erased before new data is written over it, rather than just writing over the old information as with HDDs. This doubles the wear and tear and can cause major issues with the speed performance and lifespan of the SSD. Most SSD manufactures have very sophisticated wear-leveling technologies to help with this. The principle issue is write speed degradation due to free space fragmentation. Small free spaces scattered across the SSD causes the NTFS file system to write a file in fragmented pieces to those small available free spaces. This has the effect of causing more random I/O traffic that is slower than sequential operations.

I think I have clearly made my point…. The built-in defragmenter in Windows 7 is not a solution for neither the consumer/home user, nor the enterprise business user. Data access speeds are far more critical in the business world where time is money. In the enterprise environment there are generally many more files that are used by higher number of users that are accessing data across shared type of storage such as SAN. Even virtual platforms benefit from the same points covered. This opens the door and is the reason why robust solutions such as Diskeeper exist. More data about Diskeeper and the superior technology it offers can be found at http://www.diskeeper.com.

Storage Abstraction, and What it Means to You

pm.nospam@nospam.diskeeper.com (Damian Giannunzio) — Tue, 22 Nov 2011 04:46:00 -1300

I felt compelled to write a little bit about this subject after reading recently about some new updates to software SANs. The glamour of the virtual platform layers and Cloud have somewhat overshadowed all of the virtualization already occurring within storage, and the extra levels that are added “below decks”. It’s a topic meriting some scrutiny from any storage administrator committed to high performance.

Outside of the physical data store itself, every element of the I/O path above it is virtual. It should also be noted that at essentially each step along this I/O path, infrastructure customization and proprietary technologies can (and often do) vary or add new virtual layers to the process. All of these logical abstractions have evolved from various sources in the storage ecosystem in order to drive scalability and agile responses to disaster or growth.

Let’s consider a common hypothetical path that an I/O request takes from a Windows client VM to the physical data store in a modern infrastructure. In this example, the storage for the client VM is a virtual RAID 5 configured of LUNs from a SAN. An I/O request originating at the top OS level, Windows in this case, will go through these underlying levels before getting to the actual physical storage device. Windows > volume manager > virtual RAID > SAN LUN > physical store (with the possibility of additional abstraction levels based on storage customization).

Based upon how the storage infrastructure has been established in this scenario, there is a virtual RAID 5 implemented above the SAN LUN layer. That being the case, the volume manager directs the request to the virtual RAID beneath it. Due to Striping, I/O at this stage can end up fractured (intentionally) by the RAID, based on how the array has been provisioned. The I/O Path has now become distributed, and may be even further replicated on its way to physical storage.

The RAID sends its request to the SAN LUN below, another abstraction from the physical storage itself slicing the store into basic logical units. The SAN LUN layer completes the request directly to the physical storage. The data is then returned along the same route to the requester.

Now, numerous solutions exist for managing communication and throughput within this data pipeline. Administrators can tailor their RAID presentation, ensure partition alignment, upgrade the underlying hardware, even add new software abstraction layers intended to organize data better at lower levels. However, an interesting concept emerges after review.

None of these solutions handle the most basic, and one of the most critical vulnerabilities in the existing ecosystem’s performance: assuring that the file request is as sequential and rapid as possible at the point of origin. Whether virtualized as is so common today or installed over direct-attached storage, Windows Read and Write performance is degraded by file and free space fragmentation at this top level as it causes more I/O requests to occur. Each request through all of the abstraction layers greets its first bottleneck at the outset, in how contiguous the file arrangement is within Windows. Optimizing reads and writes at this upper level helps ensure in most cases the fastest I/O path no matter how much or how little storage abstraction has been structured beneath.

In a recent white paper, Diskeeper Corporation tested a variety of I/O metrics over SAN storage with and without file fragmentation being intelligently prevented and handled. In one such test, Iometer (an open source I/O measurement tool) displayed over a 200% improvement in IOPS (I/Os per second) after Diskeeper 2011 had handled Windows volume fragmentation. Testing was performed on a SAN connected to a Windows Server 2008 R2 virtual host:

You can read the entire white paper here: http://downloads.diskeeper.com/pdf/improve-san-performance.pdf

Space Reclamation, Above and Below

pm.nospam@nospam.diskeeper.com (Damian Giannunzio) — Mon, 07 Nov 2011 09:29:00 -1300

Thin provisioning is a fairly hot topic in the storage arena, and with good reason. Many zones within the business and enterprise see massive benefit from the scalability of thin provisioning, and it can be a cost saver besides. However, the principle of thin provisioning suffers some unique maladies at both client and storage levels.

Some storage arrays include a feature permitting thin provisioning for their LUNs. This storage layer thin provisioning occurs below the virtual platform storage stack, and essentially means scalable datastores. Horizontal scaling of data stores adds a new tier of agility to the storage ecosystem that some businesses absolutely require.

LUN thin provisioning shouldn’t be confused with Virtual Disk TP, which works at a file level (not array). Thin provisioned VMs can expand based on pre-determined use cases, adding an extra degree of flexibility to storage density. Intelligently combining TP at multiple tiers yields some pretty neat capacity results.

Datastore thin provisioning has been the source of some concern for storage administrators with regards to recovery from over-provisioning. When virtual disks are deleted or copied away from a datastore, the array itself is not led to understand that those storage blocks are now free. You can see how this can lead to needless storage consumption.

vSphere 5 from VMware introduced a solution for this issue. The new vSphere Storage APIs for Array Integration (VAAI) for TP uses the SCSI UNMAP command to tell the storage array that space previously occupied by a VM can be reclaimed. This addresses one aspect of the issue with thin VM growth.

Files are not simply being written to a virtual disk, they’re also deleted with regularity. Unfortunately, there is no associated feature within virtual platforms or Windows to inform the storage array that blocks can be recovered from a thin disk which should have contracted after deletions. Similar to the issue above, this leads to unnecessary storage waste.

With the release of V-locity 3 in 2011, we introduced a new Automatic Space Reclamation engine. This engine automatically zeroes out “dead” free space within thin virtual disks, without requiring that they be taken offline and with no impact on resource usage. So what does this mean? Thin VMs can be compacted, actually reclaiming the deleted space to the storage array for dynamic use elsewhere. The thin virtual disks themselves are kept slimmed down within datastores, giving more control back to the storage admins governing provisioning.

You can read more about VAAI for TP in vSphere 5 on the VMware blog here.

Diskeeper Corporation at Interop New York 2011

pm.nospam@nospam.diskeeper.com (Damian Giannunzio) — Mon, 10 Oct 2011 02:59:00 -1300

We’ve just returned from the Interop Expo in New York, and what a show! The recent release of V-locity® 3 was extremely well received and interest in its innovations was very high. The Diskeeper Corporation booth was constantly attended by groups of CIOs and storage administrators eager to hear about the benefits of the new virtual platform optimizer.

The lion’s share of energy and buzz at the show surrounded virtualization and cloud computing. Leading vendors across these markets as well as storage, networking, and information security exhibited for large groups of virtual admins and IT executives. Shows like Interop are critical for decision makers to stay apprised of the ever-evolving IT infrastructure landscape, and excellent opportunities to get educated about what is truly needed to grow and maintain a virtual environment that runs on all engines for them.

In addition to being asked by numerous IT analysts about the innovations underlying the incredible advantages of V-locity 3, I was interviewed by TMC (Technology Marketing Corporation) about it.

The need to meet higher Service Level Agreements and reduce Total Cost of Ownership for shared storage have reached a new plateau in virtualized networks and private clouds—what V-locity 3 does best.

If you’re reading this and you were at the event, we’d love to hear about your experiences at Interop this year.

Diskeeper Corporation will be exhibiting at the Gartner Symposium in Orlando, FL next week. If you’re planning on attending this IT Expo, stop by the booth to hear firsthand about how V-locity 3 is improving virtual systems in a whole new way.

V-locity 3.0 Full Support for vSphere 5.0, in part for Citrix XenServer, and more...

pm.nospam@nospam.diskeeper.com (Michael) — Thu, 29 Sep 2011 09:16:00 -1300

Here's what's new in V-locity 3.0.9 (update) which will be broadly available next week:

1. Full support for all V-locity features on vSphere 5.0.

2. Support for V-locity Guest installation on VMs hosted on Citrix XenServer. All optimization technology and features present in the V-locity Guest software, such as V-Aware, CogniSAN, IntelliWrite, Space Reclamation, are supported. Note: The only missing feature on Citrix XenServer is the availability of a "Host Agent" (which automatically informs the V-locity Guest component about the virtual disk types - e.g. snapshot, sparse disk, etc...). This is planned for a future release. In the interim, manual configuration may be required.

3. Fixed a rare problem with V-locity service crashing when installed on a non-boot volume or when installed to a short path.

4. Fixed a problem with connection data disappearing from Host Agent.

5. Fixed an MMC crash when remote connecting to V-locity Guest while local UI already open.

6. Fixed a warning message when opening multiple Guest UI instances.

7. Fixed some UI issues that cut-off text for English, Japanese, German and French.

8. Improved the Installation instructions to provide additional recommendations and information.

Samsung demonstrates ExpressCache (aka Fast Start) at BUILD

pm.nospam@nospam.diskeeper.com (Michael) — Wed, 21 Sep 2011 06:09:00 -1300

Samsung (the provider of the Win8 tablets that were given away to all attendees) also did a demonstration of ExpressCache at the Microsoft BUILD conference last week.

Their booth, which was at the front of the EXPO, showed many machines with ExpressCache on them and also had a demo table set up with 3 machines.

o System with a HDD only

o System with SSD and HDD ExpressCache (8GB SSD)

o System with SSD only (128GB SSD)

There was a button to push, on the counter top display, that started all of the notebooks up at the same time and then auto-launched an application. The ExpressCache system smoked the HDD notebook and was almost as fast at the large (128GB) SSD-only system. And, it offered a large capacity HDD as well.

The demo proved that, for a fraction of the price of buying a large SSD, you can get all the performance that SSDs offer with ExpressCache technology (and still store all your stuff).

10 things you can do to boost PC performance (by TechRepublic)

pm.nospam@nospam.diskeeper.com (Michael) — Tue, 20 Sep 2011 08:43:00 -1300

IT professional and author, Justin James of TechRepublic published a top 10 list of ways to speed up your PC. Number 9 was one very familar to us:

"9: Defrag. Defragging your hard drives is a great way to get some more performance. While modern Windows systems automatically defrag on a regular basis, I’ve found that the Windows defragging is fairly unaggressive. We’ve reviewed a lot of different defrag apps here at TechRepublic. I suggest that you check out your alternatives and find one that does a better job for you."

Their findings mimic what we see with many of our business customer seeking to maximize Windows 7 performance. The built in defragmenter sounds like an attractive option at first, but closer inspection and testing clearly demonstrates significant value (better ROI) in advanced third party optimization technology.

Read the whole article here: http://www.techrepublic.com/blog/10things/10-things-you-can-do-to-boost-pc-performance/2712?tag=nl.e101

The NEW Diskeeper Corporation Support Forum

pm.nospam@nospam.diskeeper.com (Michael) — Mon, 19 Sep 2011 06:01:00 -1300

The Diskeeper Corporation Web Department is happy to announce the launch of our new online support forum.

This site is located on our website at: http://www.diskeeper.com/support/forum/

Customers, trialware users and any interested parties are all welcome to come to the site and look around or even join up and submit/contribute. Its purpose is to help you get the information you need to maximize the benefits of our solutions, so we look forward to chatting with you through this new portal.

You can read the overview and rules of the fourm here: http://www.diskeeper.com/support/forum/yaf_postst12_General-Rules-and-Guidelines.aspx

New V-locity 3 Virtual Platform Disk Optimizer Released at VMworld

pm.nospam@nospam.diskeeper.com (Colleen Toumayan) — Wed, 31 Aug 2011 02:20:00 -1300

Cures Pains from I/O Bottlenecks, Slow VMs, Slow Migration, Resource Conflicts,

Wasted Storage Space and Slow Backup Speeds

Diskeeper Corporation announced at VMworld the release of new V-locity^®3 virtual platform disk optimizer for VMware^®.

V-locity 3 delivers invisible background optimization to give maximum I/O performance on virtual servers.

V-locity 3 now has full integration with VMware ESXi as well as existing support for other platforms such as ESX, and Microsoft Hyper-V™. Other developments include V-Aware™ technology which detects external resource usage from other virtual machines on the virtual platform and eliminates resource contention that would otherwise exist on the same Host Server. New CogniSAN™ technology detects external resource usage within a shared storage system, such as a SAN, and allows for transparent optimization by never competing for resources utilized by other VMs over the same storage infrastructure. CogniSAN does this without intruding in any way into SAN-layer operations. Lastly is the new automatic zeroing of free space which is a powerful engine that zeros out unused data blocks on virtual disks and makes virtual disk compaction easy.

http://www.diskeeper.com/press/releases/releases.aspx?F=2011083001.htm