StorageTrendz

More about VMware Virtual Datacenter OS

2008-09-18T22:37:00.007+05:30

Summary of the keynote given at VMWorld VMware conference sep 2008:

* Virtual Infrastructure 4.0 itself has not been announced, yet the major components have already been introduced:

* Virtual Datacenter OS [VDC OS] consists of the following:

o the hardware/interaction layer:
vCompute - platform that interacts with the hardware, with vCompute the hardware specifications have been adapted to:
# 8 virtual CPUS
# 256GB per VM
# 40 GB/s network throughput
# up to 64 nodes per cluster
# up to 4906 cores to manage
# full support for DPM
* vStorage - new features have also been announced on the level of storage. VMFS and storage vMotion will be extended with stroage thin provisioning and linked clones
* vNetwork - the network layer functionalities have been improved with virtual distributed networking and support for 3rd party solutions.

o the application layer:
# vAPP: gives applications priorities and SLA levels (hello legal disagreement with Microsoft?)
# VMware FT: high availability with 2 vMs in sync (like Marathon Technologies everrun VM)
# Security (shouldn’t that be vSecurity?): VMsafe APIs on the way

o The management layer
# Virtual Center - vCenter with chargeback, performance optimization, orchestration and many other modules
# vCenter server component will be available for Linux (as virtual appliance)

We have not yet received any confirmation on the general availability of VI4 (of which VDC-OS will be a big part), yet the solution seems to be in beta or beta 2.

Software Technology VS Hardware Technology

2008-09-18T22:37:00.006+05:30

In the Enterprise environment Hardware design & implementation was happening at never before pace. each hardware vendors (IBM, HP, Intel, AMD, SUN, Dell, EMC2, NetApp & 1000's of many more small & big enterprises) tried to come up with highly competitive technologies & they did succeeded in their achievements to a larger extent.
The Hardware innovations continue to evolve as it is happening from past few decades with even higher pace. the advancement in the multiple-core processor technologies being implemented by Intel & other vendors will continue to boast other hardware (servers, Switches, routers, storage appliances etc.) innovations in future.
But another interesting point i like to make here is about "Enterprise Software Technology". in the recent decade, Enterprise Software does not mean just Operating System (OS), it is more than just OS, we have seen enterprise Management software, Network management software, Telecom network management software, Wireless management software to name a few.
very recently, VMWARE's virtualization software success has taken the Enterprise industries by surprise & it has resulted in "Software Enterprise Technology" showdown by some of the biggest software players such as Microsoft, Xensource,SUN, IBM, HP etc. this is interesting as the enterprise software fever is catching up like wild fire, more & more competitors are born every day.
Software being easy and cost effective Technology to design & implement, there is going to be Enterprise software Revolution in the cards. one of the first sign was Virtualization software of VMware & now Microsoft windows 2008 OS.
very recently, in the just concluded VMware's VMWorld conference in Las vegas, USA. VMware announced that it is building nothing less than a "virtual datacenter OS" -- VDC OS for short -- that will eventually make traditional operating systems "all but disappear."
This is the next sign of Enterprise Software Revolution that i am talking about.

If you like to know more details of it, check out http://weblog.infoworld.com/fatalexception/archives/2008/09/vmware_proposes.html

Latest Storage market status - Q2 2008

2008-09-13T23:27:00.003+05:30

HP, IBM, Dell, SUN are all top global enterprises with broad range of products, here i have current market status for these companies in External Storage segment only.

According to analyst group IDC, Despite tough conditions in the U.S. economy, which have decreased demand for everything from new homes to semiconductors, the market for computer data storage Relevant
Products/Services systems is going strong.
EMC continued to lead the external disk storage systems market with 21.7 percent of the revenue share, followed by IBM at 13.1 percent, HP at 12.9 percent, Dell at 9.2 percent, NetApp at 8.0 percent and Hitachi at 7.6 percent. Of the top five suppliers, EMC and NetApp posted the strongest year-over-year revenue growth at 19.7 percent and 17.2 percent, respectively.
The Open SAN market grew 15.5 percent year-over-year, with EMC leading with 23.9 percent of revenue share, followed by HP at 16.7 percent. The NAS market experienced year-over-year growth of 23.5 percent, with EMC leading with 39.8 percent of revenue share, followed by NetApp at 28.5 percent. The iSCSI SAN market showed year-over-year revenue growth of 93.9 percent, with Dell leading the market (due to its acquisition of EqualLogic) with 32.8 percent of the revenue share, followed by NetApp at 14.6 percent and EMC at 14.1 percent.

**********for more information check out http://www.echannelline.com/usa/story.cfm?item=23687 *******************

IBM's Storage product Buzz:

2008-09-13T12:56:00.003+05:30

In Sep 2008, IBM announces 30 new products and services, saying that it took an investment of $2 billion, three years, and 2,500 engineers and other specialists from nine countries to complete the project.
IBM said that all the new products and services fall under the auspices of its New Enterprise Data Center strategy, which utilizes intellectual property IBM has acquired in the last two years.

Few of them are listed below & you can get more info from http://www-03.ibm.com/systems/storage/product/showcase/index.html
Newly-announced storage products
IBM XIV Storage System
IBM TS7650G ProtecTIER De-duplication Gateway
IBM System Storage TS2900 Tape Autoloader
IBM System Storage DS5000 series
IBM System Storage DS8000 R4.0/4.1
IBM System Storage TS3500 high-density frames
IBM N Series N6000
IBM N Series SnapManager for Virtual Infrastructure
IBM N Series OSSV SnapVault

Latest information about IBM's DS5000 product here http://storagetrendz.blogspot.com/2008/09/ibm-new-ds5000-storage-product-released.html

IBM New DS5000 Storage Product released:

2008-09-12T22:40:00.002+05:30

Disclaimer: The opinions expressed by the Blogger and those providing comments are theirs alone, and we are not responsible for the accuracy of any of the information supplied by the Bloggers.

Most of the Netizens beleive IBM DS5000 storage product came from the acquisition of XIV storage company by IBM [check http://www.xivstorage.com/].
but i differ with this wide perception. the DS5000 which is released by IBM in sep, 2008 looks to me as an extension of IBM's old products in the mid-range (ex:DS4000 series).
yet again i like to differ myself saying it is not extension, it is whole new enterprise storage architecture and silent features are given below:
* offers improved performance.
* Scalability of FC/SATA disks upto 448.
* Boost up with 8G FC & 10G iSCSI connectivity.
* PCI-Express (PCI-e) bus architecture & so on.
* Supports RAID levels RAID 0, 1, 3, 5, 6 and 10 (RAID 1+0).
* Supports up to 2048 volumes & 2048 host logins.
* Premium features FlashCopy (up to 16 copies per volume), VolumeCopy, Remote Volume Mirroring (up to 128 pairs).
* Cache memory of 8GB to 16GB cache & USB Cache backup memory for Power failures.
* IBM DS5000 series offers two models
DS5100
DS5300
Obviously for EMC & HP bloggers, DS5000 series looks no threat to their Clariion & EVA products respectively but there are specific indications
that DS5000 series future offerings will be definitely going to cut the EMC & HP business in this segment (Keep watch on this).
the beauty of this product is that it has the potential to bridge the gap between IBM's Mid-range & high range (DS8000) product offerings.

The pictures associated with this post gives you more performance & scalability offering of DS5000

IBM Storage Products

2008-09-06T21:03:00.004+05:30

NOTE: IBM has comprehensive range of storage hardware & software products such as servers, SAN, NAS, Switch, Tape libraries, switches & SAN arrays, NAS devices etc. so here am listing only few storage specific products.

IBM Disk storage subsystems
Enterprise systems - DS6000 and DS8000 family
DS6000 series
DS8000 series
DCS9550

Mid-range systems - DS4000 family
DS5000
DS4800
DS4700 Express
DS4200

Entry level systems - DS3000 family
DS3200
DS3300
DS3400

Enterprise SAN directors [High end switches]
For highest availability and scalability enterprise solutions.
IBM System Storage SAN768B
IBM TotalStorage SAN256B
IBM TotalStorage SAN140M
IBM TotalStorage SAN256M

Mid-range SAN switches
For scalable, affordable SMB and enterprise solutions.
IBM System Storage SAN80B-4
IBM System Storage SAN64B-2
IBM System Storage SAN40B-4
IBM System Storage SAN32B-3

Entry SAN switches
For simple, affordable small and medium business (SMB) solutions.
IBM System Storage SAN24B-4 Express
IBM TotalStorage SAN16B-2

IBM Express Model SAN switches
For affordable, simple-to-use small and medium business (SMB) solutions.
IBM System Storage SAN24B-4
IBM TotalStorage SAN16B-2

Tivoli Storage Manager
Manage risk through complete storage management with enhanced performance and access for Sequential Disk. Manage complexity with encryption key management. used by more than 21,000 customers.

OEM Storage Management softwares
Basically OEM storage product software marketed with IBM logo.

IBM TotalStorage Productivity Center
Identify, evaluate, control and predict the growth of data through its lifecycle to meet storage service levels.

IBM System Storage Virtualization
Designed to help simplify storage infrastructure, optimize storage utilization, and enable business to adapt quickly and dynamically to variable environments

HP Storage Products

2008-09-05T21:21:00.001+05:30

NOTE: HP has comprehensive range of stroage hardware & software products such as servers, SAN, NAS, Switch & HBA software, HBA's, switches & SAN arrays, NAS devices etc. so here am listing only few storage specific products.

Disk Storage Systems [SAN]

XP Disk Array - High Level
[HP StorageWorks XP24000 Disk Array, XP20000 Disk Array, XP12000 Disk Array, XP10000 Disk Array]
• Bulletproof storage for 24x7 business continuity demands
• Massive consolidation for greater efficiency
• Virtualization platform for internal and external data

EVA Disk Arrays - Mid-level
[HP StorageWorks Enterprise Virtual Array 8100, EVA6100, EVA4400, EVA4100 & EVA Software]
• Powerfully simple enterprise class storage
• Affordable and virtualized storage with a low entry price and low total cost of ownership
• Reliable and available

MSA Disk Arrays - Entry level
[HP StorageWorks Modular Smart Array 2000sas, MSA2000fc, MSA2000i & Storage Resource Management Software (SRM)]
• Flexibility to start small and migrate drives & enclosures into larger configurations
• Increase server capacity
• Modular design enables expansion

NAS Systems
[HP All-in-One AiO1200r SATA, AiO1200r SAS, AiO600, AiO400t, AiO400r,AiOSB600c]
• Unbelievably simple: No storage expertise required
• Uniquely affordable: A complete storage solution
• Unquestioningly reliable: Integrated data protection and HP reliability

Scalable NAS

• Scalable, Available Enterprise NAS
• Windows and Linux File Serving
• Oracle over NFS, Microsoft SQL Server Consolidation

Switches
B-series [Fabric switches, Blade switches, Director switches]
• B-series switches and directors are used by small departments and global organizations to access and share data in a consolidated, manageable, and a scalable manner.
C-Series [Fabric switches, Blade switches, Director switches]
• C-series family of multilayer directors and fabric switches is a comprehensive line of SAN switches that provides an effective way to access, manage, and protect growing information resources.
• C-series switches are basically CISCO switches marketed by HP.

HP Storage Software

• Storage Resource Management Software (SRM)

• HP StorageWorks EVA Device Management Software
• HP StorageWorks EVA Replication Software

• HP StorageWorks XP Replication Software
• HP StorageWorks XP Device Management Software

• HP Storage Data Protection and Recovery Software
• HP StorageWorks Storage Replication Software
• NAS Device Management Software
• B-series Switch Device Management Software
• C-series Switch Device Management Software

StorageTrendz: EMC Storage Products lineage part1:

2008-08-31T00:50:00.000+05:30

StorageTrendz: EMC Storage Products lineage part1:

EMC Storage Products lineage part1:

2008-08-11T22:32:00.000+05:30

Hardware

Celerra - NAS family
NAS product line of EMC. the products include NAS gateways, NAS storage devices & integrated devices.
CLARiiON - SAN family
Clariion series is the SAN products for entry-midrange-entrylarge level models to tap the high selling market segment.
Prodocuts includes FC & iSCSI san architectures.
Connectrix - Switches or Fabrics family
Move your organization's vital information where it needs to go—quickly, easily, and reliably. Advanced directors and switches make it happen. Get best-in-class availability and easy management.
Centera - CAS family
Store and manage your "fixed content"—unchanging digital assets—and keep them available online and accessible. All with EMC Centera® content-addressed storage (CAS) systems. Be ready for growth with petabyte scalability.
Symmetrix - SAN infrasture family
Make high-end networked storage part of your information infrastructure with systems that take performance, availability, and security to new heights. Manage and protect your information today—and expand in the future.

Software

ControlCenter[Multi-vendor management software]
Simplify and automate key tasks—such as discovery, monitoring, reporting, planning, and provisioning—for even the largest, most complex storage environments.
Celerra Manager[NAS management Spftware]
Easily configure, administer, and monitor your EMC Celerra IP storage environment from a single web-based interface—ensuring high availability.
Navisphere Management Suite [SAN management software]
Get simple, secure, web-based management from any location. Discover, monitor, configure, and report on multiple CLARiiON arrays from your browser. Use these insights to get more value from your investment.
PowerPath
Enable EMC host-based solutions including multipathing, data migration, and host-based encryption.
Symmetrix Dynamic Cache Partitioning
Enable the consolidation of multiple workloads in a single Symmetrix while maintaining predictable performance.
Symmetrix Management Console [Symmetrix management software]
Simplify day-to-day management of your EMC Symmetrix storage with this powerful, easy-to-use browser-based management tool.
Symmetrix Virtual Provisioning
Increase storage capacity utilization, improve ease of use, and enhance performance for certain applications and workloads.

Host Bus Adapter [HBA]

2008-05-13T11:51:00.000+05:30

* HBA is an important component of Storage networks, especially in SAN architecture.
* HBA is usually host-side component & can be connected to PCI or PCI-X or PCI-E slots on the host or servers. it provides the crucial connectivity between Hosts-StorageSubsystems in Direct attached setup, provides connectivity between Host-FabricSwitches-StorageSubsystems in SAN environment.
* HBA vendors are very few, some vendors provide HBA management software & drivers along with HBA (ex: Qlogic, Emulex, Brocade etc).
* HBAs available in market may be of Fibre Channel (FC), SCSI, SAS and iSCSI interfaces.
* HBA comes in different priceband and the difference in price band is largely driven by the number of ports. HBAs are available in the market with single port, dual(2) Ports, Quad(4) ports, octa(8) ports etc. Depending on the number of Ports & the type of interfaces, the HBA cost varies from 500$ to 5000$ or even more.

Direct Attached Storage -DAS

2007-11-16T18:59:00.000+05:30

For expert consultation contact Mail ID: storagetrendz@gmail.com

DAS is Direct Attached Storage, as the name suggests, the Storage devices such as Hard drive,Tape drive, Tape Library, JBOD/Enclosure, Storage arrays are directly connected to the Server or Host systems.The simplest example of DAS is the internal hard drive of a server computer.
DAS environment can have one or multiple servers and associated Storage devices. hence Storage resources are dedicated to specific servers and cannot be shared.
in DAS, the main interfaces are Drive-side interfaces & Host-side interfaces and underlying interface technologies are ATA/IDE, SATA, SCSI, SAS and Fiber Channel.
Advantages:
Simple to design & Implement and good option for very small to medium business companies.

Disadvantages:
Direct-attached storage can be more expensive to manage because you cannot redeploy unused capacity, which results in underutilization.
• Having storage distributed throughout the organization makes it difficult to get a consolidated view of storage across the organization.
• Disaster recovery scenarios are limited because a disaster will cause both server and storage outages.
• For data backup and recovery, you need to choose whether to attach local backup devices to each server, install dual network adapters in each server and back up the data over a separate LAN, or back up the server over the corporate LAN.
Large organizations have found that placing stand-alone tape drives in individual servers can quickly become expensive and difficult to manage, especially when the number of servers in the organization grows into the hundreds.
In this situation, it is often best to back up servers over a network to a storage library, which offers backup consolidation and eases management.
For expert consultation contact Mail ID: storagetrendz@gmail.com

Latest Storage Industry News - Dell to buy EqualLogic for $1.4 bn

2007-11-07T17:29:00.000+05:30

From REUTERS,
NEW YORK: Dell Inc said on Monday it would buy data storage network company EqualLogic for $1.4 billion, its largest acquisition ever, as it seeks to boost its fastest-growing business segment.
The world's second-largest computer maker said it expects the acquisition to reduce its earnings per share, excluding the amortization of intangibles, by 2 cents to 5 cents in aggregate for the fiscal years 2009 and 2010.
The deal has been approved by the boards of both companies but still must pass regulatory muster. It is expected to close late in the fourth quarter of Dell's fiscal year 2008 or early in the first quarter of its fiscal 2009.

My Opinion: This is definitely one of the biggest news in the Storage Business domain in the recent past.
'VIRTUALIZATION' is the latest buzz across the storage industries, numerous Startup companies trying to come up with their own software & some of Big names are VMWARE (acquired by EMC2), XenSource & so on. i can predict more aquisitions or mergers of Virtualization based companies in the future.
For DELL, this is the right move. DELL is trying to be competative with HP, IBM, SUN & others in the Storage business and after the IPO success of VMWARE (EMC2) certainly DELL's move to acquire EqualLogic is value addition.
The deal will give Dell a presence in iSCSI & Virtualization storage technology.

Storage Business Pointers - Cost, Performance, Capacity, Availability, Scalability & Thin Provisioning.

2007-10-29T15:02:00.000+05:30

Before reading my upcoming DAS, SAN & NAS related technical Posts, it's important for we all to know the basic Storage Business terminologies such as COST, AVAILABILITY, CAPACITY, SCALABILITY, PERFORMANCE & THIN PROVISIONING.
Because, the Storage domain comprising different technologies, Topologies, Protocols, Interfaces, Platforms, Architectures, Implementations etc. are driving the storage Business in terms of cost, Performance, Capacity, Scalability, Availability & Thin Provisioning.

Capacity: The trend of Data Storage Capacity growth rate has not only followed Moore's Law for decades but also, going to surpass it in the future.
(Moore's Law: Data storage growth rate is exponential & doubles every 18 months). the requirement of data storage space is increasing and hence Capacity is the generic term yet very important factor to consider for all the storage design & implementations.

Scalability: Scalability is the ability of storage solution (design & implementation) to satisfy the Current storage requirements & also, to provide easy & effective mechanism for expansion of storage resources for the ever increasing future storage capacity demands.
Most of the storage solutions provided by different vendors are based on different technologies, Protocols, Interfaces, Platforms, Architectures & hence while making expansion of storage resources for future needs, it is important to consider the Scalability factor.

Performance: the bandwidth for delivering storage content is growing to match the increased speed of computer processing power, the speed of data communication networks,
and the speed requirement of emerging applications such as multimedia applications. for example, Fibre Channel is innovated over period of time from 1Gbits/Sec to 4Gbits/sec or more.

Availability: it is the factor by which we can emphasis on the need for making Storage solutions more reliable and provide 24/7 availability for the demanding applications (ex: Banking, Financial Applications etc.).
Mission critical storage networks are required to achieve 99.999 availability and the capability to recover from catastrophic disasters via mirroring and backup techniques that protect the content through geographic diversity.

Cost: Cost is very important factor to consider because anything & everything on the earth is available to anybody with associated cost. Storage solutions are available to customer with associated cost that includes Hardware system & Software costs, design, implementation, Maintenance & management costs.
the business emphasis is to maintain low or affordable cost to customers & to provide more reliable, scalable, available storage solutions.

Thin Provisioning: The major factor that affected traditional Storage solutions was inefficient 'Storage provisioning'. Provisioning is the method of effective allocation & utilization of storage resources.
The most obvious issue of traditional Storage solutions is the amount of allocated storage that becomes unused and therefore increases the total cost of ownership. Additionally, since this allocated but unused storage capacity cannot be reclaimed for other applications, customers have to buy more storage capacity as their environments grow, increasing cost even further.
At some point, customers may actually be required to buy a completely new storage system in addition to the one they have in place.
Eventually, Thin Provisioning has become the buzz word of storage business. Thin Provisioning eliminates the 'Over-allocation or over-subscription' problem of traditional provisioning & implements 'Just-in-Time allocation of storage resources using which Administrators can efficiently & effectively optimize the existing storage capacity.

Storage Interfaces SATA, SCSI, SAS, FC - part2

2007-10-19T11:25:00.000+05:30

ATA (Advanced Technology Attachment):
ATA interface introduced in the mid 1980’s, is the primary internal storage interface for the PC, connecting the host system to peripherals such as hard drives, optical drives, CD-ROMs.
ATA is also called as PATA because it is Parallel Interface.
The latest version of the PATA is Ultra ATA & supports up to maxium of 100Mbytes/sec data transfer rates using double edge clocking.
ATA supports upto 2 interface connections, that means only two devices can be connected.
ATA is being replaced by SerialATA.

SATA (Serial ATA):
The SATA interface is an evolution of the ATA interface from parallel bus to serial bus architecture. The serial bus architecture overcomes the difficult electrical constraints hindering continued speed enhancement of the parallel ATA bus.
The first generation SATA-I technology is designed to be a direct serial transport mechanism for ATA protocol data at 150Mbyte/sec that is fully compliant with the ATA protocol at the software level.
SATA II is Structured and prominent protocol enhancements to further speed increases to 300 Mbyte/sec (3Gbit/s) and to improve the protocol efficiency in a multitasking environment.
With these enhancements, the SATA would approach the performance of SCSI/FC-based disk drives.

SCSI (Small Computer System Interface):
SCSI is also Parallel interface technology and SCSI has gone through a long evolution (from 1980s), with SCSI-I, SCSI-2, and SCSI-3 over different types of cabling, and including Wide SCSI and Narrow SCSI.
The primary benefits of SCSI include: cross platform interoperability; support for a large number of devices; easy expandability; long cabling distances for external device connectivity;
very good support for multitasking disk accesses that allow for interleaving of multiple concurrent transfers; tag queue and out-of-order data delivery.
SCSI interface supports up to seven devices with Narrow SCSI, and 15 devices with Wide SCSI.
SCSI products tend to cost quite a bit more than the ATA/SATA disk drives for the same disk capacity.
SCSI evolution (SCSI1/FastSCSI/WideSCSI/UltraSCSI) has also improved its data transfer speed from 5MB/s to 320 MB/s.

SAS (Serial Attached SCSI):
the Serial Attached SCSI (SAS) is being defined to replace the physical layer of SCSI i.e parallel to serial bus/interface technology.
The goal is to achieve higher data transfer rates and yet maintain the protocol compatibility of SCSI at the command set level.
Currently SAS supports upto 300 MB/s and work is in-progress for development of 600MB/s & higher speed.
SAS uses a serial, point-to-point topology and hence can connect 2 devices in direct connection and also using SAS Expander(hub) can connect upto 128 devices.
SAS & SATA interfaces looks very similar and with Expanders, SAS & SATA devices can be connected to similar ports using a device called 'Interposer card'. thus making huge advantage of different interfaces co-exist in same Storage box (devices like SAS Enclosure).

Note: SAS interface is primarily Direct Drive interface like SCSI & SATA but using SAS expanders(hub) & SAS Switches can be Networked as 'SAS Domain' and hence can also used as Interconnection topology.

Firbe Channel:
FC is High speed Serial Interface, prominent interface since its existence from 1990s.
Like SCSI, FC is also has well defined Protocol structure and promising capabilities compared to other interfaces.
Like S.A.S, FC interface has Point-point topology, Loop topology(FC-AL), Fabric-Switched topology. P-p supports 2 devices, Loop supports 126 devices & Switched fabric supports upto 16 million devices.
FC started with data transfer speed of 100MB/s (1Gbit/s), 300MB/s,400MB/s & plans for 600MB/s to 1000MB/s and hence most of the Networked Storage environment (SAN) uses FC technology.

Note:
* Interfaces like ATA/SATA/SCSI/SAS are majorly used as drive-side interfaces in DAS & SAN environments.
* FC is not only used as Drive-side interface in DAS but also, pre-dominantly used Drive-side & Host-side interface in SAN Environment.
* For more information you can access www.SNIA.org, www.t10.org (for SCSI interface) & www.t11.org (for FC interface).
Your Queries are my inspiration & Hence Mail to Storagetrendz@gmail.com

Storage Interfaces in brief - part1

2007-10-16T18:38:00.000+05:30

In order to understand the storage systems or Devices, it is important to understand the disk drives, the most common building block of system systems are disk drive interface technologies.
Looking at the disk drive industry today, the interface type divides the market into 3 broad categories:
° The low-end PC market, with relatively low performance and reliability but at very economical price, dominated by IDE,ATA & SATA interfaces;
° The mid-range enterprise market, with higher performance and reliability, serviced by SATA, SCSI & SAS interfaces;
° The high-end enterprise application market, with the highest performance, reliability and scalability, provided by Fibre Channel, SATA, SAS Drives interfaces.
It is important to keep in mind that these disk drive interfaces have gone through several generations of evolution (IDE/ATA/EIDE/UDMA/Ultra-ATA, Narrow/Wide SCSI-1/2/3) and are still evolving rather quickly. Serial ATA (SATA) is emerging to replace ATA for higher performance. Serial Attached SCSI (SAS) was invented to be the next generation SCSI interface.
Fibre Channel is evolving from 1Gbit/s to 2Gbit/s to 4Gbit/s and 10Gbit/s to satisfy the ever increasing demand for higher bandwidth.
At present, the Most imprtant drive interfaces being implemented are SATA, SCSI, SAS and Firbe Channel.

Type of Devices designed using Interfaces:
Hard Disks (IDE/ATA/SATA/SCSI/SAS/FC).
Internal RAID Controllers (IDE/ATA/SATA/SCSI/SAS/FC) Such as Adaptec, Promise technologies, LSIlogic (& so on)RAID controllers mainly used in DAS environment
Host Bus Adapters (SCSI/SAS/FC).
JBODs & Enclosures (SATA/SCSI/SAS/FC).
External RAID Controllers (SATA/SCSI/SAS/FC) such as EMC2, HP, IBM, LSIlogic, NetApp (& so on) SAN & NAS raid boxes majorly used in SAN & NAS environment.
Cables & Connectors (IDE/SATA/SCSI/SAS/FC).
Hubs & Switches (SAS/FC).

RAID Storage Concept unleashed - Part 4:

2007-10-09T11:17:00.000+05:30

Software RAID Implementations
Software RAID can be implemented in 2 ways: A] as a pure software based RAID solution, or B] as a hybrid solution that includes some hardware designed to increase performance and reduce system CPU overhead (utilization).

1) Pure Software Model – Operating System Software RAID
In this case, the RAID implementation is an application running on the host without any additional hardware. This type of software RAID uses hard disk drives which are attached to the computer system via a built-in I/O interface or a processor-less host bus adapter (HBA).
The RAID becomes active as soon as the operating system has loaded the RAID driver software. Such pure software RAID solutions often come integrated into the server OS and usually are free of additional cost for the user.
2) Hybrid Model – Hardware-based Software RAID
While this is still software RAID, the hardware helps to overcome some of the weaknesses of pure software RAID. Such solutions usually come with additional hardware (e.g. an HBA with a RAID BIOS or just a RAID BIOS integrated onto the motherboard).
The additional BIOS makes the RAID functionality available when the system is switched on, providing redundancy during boot that reduces the impact of medium errors on RAID which otherwise could lead to data corruption
or an inoperable system. In addition, most of these solutions provide a BIOS setup software which is available at system boot.
This allows for easy setup and maintenance of the RAID array without the need to install or boot an OS from hard disk or CDHost

Hardware RAID Implementations
Hardware RAID can be implemented in a variety of ways: A] as a Independent (external) RAID Controller Card, or B] as integrated AISC chip based on RAID-on-Chip technology.
1) A Independent RAID Controller Card
This is a plug-in expansion card that usually has a built-in RAID processor (I/O processor) and its own interfaces to the drives (I/O controllers). It usually plugs into PCI-X or PCIe slots of the computer system’s motherboard.
Such plug-in cards are usually the most costly, but also the most flexible and best performing RAID solutions.Here, the RAID functionality is completely independent of the host (computer system).
2) Integrated (Embedded) Hardware RAID solutions based on RAID-on-Chip (ROC) technology
In ROC solutions, the RAID processor, memory controller, host interface, I/O interfaces for hard disk drive connection, and sometimes even the memory, are all integrated into one single chip.
This chip can be integrated into the motherboard and offers hardware RAIDcapabilities with reduced cost (as just one,highly integrated ASIC, is needed).

Note: Hardware RAID controllers can be connected to PCI or PCI-X or PCI-E slots on the servers depending on the controller designs.

RAID Storage Concept unleashed - Part 3:

2007-10-05T11:32:00.001+05:30

RAID 4: Block-level Striped Disk Array with Dedicated Parity (using XOR)
• Data is Block-level striped, except that one drive is reserved to hold parity data.
• The parity is the result of a XOR operation performed on the Blocks of the striped data.
• The minimum number of drive required for RAID4 creation is 3 and maximum is no limit.
• The capacity of a RAID 4 group is (n-1)*s, where n is the number of drives, and s is the size of the smallest drive in the group.
• RAID 4 can withstand upto one drive failure and if more than one drive fails, the data in RAID4 array will be lost.
• Despite good features RAID4 has many drawbacks such as very low write performance, Difficult and inefficient data rebuild in the event of disk failure & so on.

RAID 5: Block-level Striped Disk Array with Distributed Parity (using XOR)
• Both Data & Parity are Block-level striped accross all the drives of array.
• The parity is the result of a XOR operation performed on the Blocks of the striped data.
• The minimum number of drive required for RAID5 creation is 3 and maximum is no limit.
• The capacity of a RAID 5 group is (n-1)*s, where n is the number of drives, and s is the size of the smallest drive in the group.
• RAID 5 can withstand upto one drive failure and if more than one drive fails, the data in RAID5 array will be lost.
• The performance of RAID 5 is very good, only slightly slower than RAID 0 on both writes and reads.

RAID 6: Block-level Striped Disk Array with Independent Distributed Dual Parity (using XOR & Reed-Solomon ECC)
• Like RAID5, Data & Parity is striped on a block level across a set of drives, and a second set of parity is calculated and written across all the drives;
• The actual dat storage capacity of a RAID 6 group is (n-2)*s, where n is the number of drives, and s is the size of the smallest drive in the group.
• The minimum number of drive required for RAID6 creation is 4.
• RAID 6 array can withstand upto Two drive failure & Two different algorithms are employed to achieve this purpose.

Note: Apart from basic RAID levels, there are extended raid levels such as RAID01 (0+1), RAID10 (1+0), RAID50 (5+0) and so on.
these are mainly results of combination of two or more basic raid levels to achive higer performance & Redundancy.
Extended RAID levels are also extensively implemented and used in the existing RAID products.
Detailed information about the same can be found in http://en.wikipedia.org/wiki/Nested_RAID_levels.

RAID Storage Concept unleashed - Part 1

2007-09-26T13:04:00.000+05:30

Basic Form of storage devices are Hard disk, Tapedrive, Flash memory,CD,DVD,Pen drives, Tape Libraries, JBOD, Enclosurs, Storage RAID Arrays & so on and the major Players of this business are IBM, HP, Hitachi, Seagate, Maxtor, Fujitsu, StorageTek, Toshiba, Intel, Sun, EMC2, Netapp etc...
Innovation continued with sense of grouping mupltiple Storage devices together in order to increase total available space for stoage and hence JBOD (Just Bunch of Disks) boxes are designed. (JBOD is a storage box containing mulptiple Hard disks and when connected to server or host, server assumes it as single disk and stores data).
JBOD's usage is limited by its design because aprat from combining multiple drives it does not give any further inteligence (dumb box). now emphasis was on adding inteligence to the JBODs and hence Enclosures came into existence with RAID (redundant array of independent disks) implementations.
With the Design & implementation of RAID concepts in the form of different RAID levels (RAID0,RAID1,RAID2,RAID3,RAID4,RAID5,RAID6,RAID0,RAID10,RAID01,RAID50 and so on), storage technology started exploring new dimensions of growth. Storage technology together with Server technology & Networking technology heralded the Design and implementation of Direct Attacted storage, Storage Area Network and Network attached Storage.
Storage devices (Hardware) have seen major changes in the form of innovations and also software in the form of management softwares, Configuration softwares, Backup softwares, Volume manager softwares, Virtualization softwares and so on have equally contibuted to implement complex form of solutions and at present existing Storage technologies (Hardware & Software) are driving the Data management business.

The term RAID was first defined in paper published by David A. Patterson, Garth A. Gibson and Randy Katz at the University of California, Berkeley in 1987. RAID (Redundant Array of Independent Disks) is a technology for managing how data is stored on the physical or hard disks that reside in your system or are attached to it.
To understand RAID, you must understand fundamental terms such as physical drive, physical array, Logical Drive or Volume and logical Arrays.
Physical Drive: Physical drives are typically indivisual Hard disks that are connected to Server (Host)
Physical Array: One or more physical drives are collected together to form a physical array (JBOD, Enclosures, Storage Arrays).
Logical Arrays: Logical arrays are formed by splitting or combining physical arrays. Typically, one logical array corresponds to one physical array. However, it is possible to set up a logical array that includes multiple physical arrays (typically used to allow multiple RAID levels). It is also possible to set up two entirely different logical arrays from a single physical.
Logical Drives: One or more logical drives are formed from one logical array (much the same way they would normally be formed from one physical drive in a non-RAID system). These appear to the operating system as if they were regular disk volumes, and are treated accordingly, with the RAID controller managing the array(s) that underlie them.

To Implement RAID, you must uderstand fundamental concepts of RAID such as Striping, Mirroring, Parity & Spanning.
Mirroring: Mirroring is one of the two data redundancy techniques used in RAID (the other being parity). In a RAID system using mirroring, all data in the system is written simultaneously to two hard disks instead of one; thus the "mirror" concept. The principle behind mirroring is that this 100% data redundancy provides full protection against the failure of either of the disks containing the duplicated data. Mirroring setups always require an even number of drives for obvious reasons.
The chief advantage of mirroring is that it provides not only complete redundancy of data, but also reasonably fast recovery from a disk failure. Since all the data is on the second drive, it is ready to use if the first one fails. Mirroring also improves some forms of read performance (though it actually hurts write performance.)
The chief disadvantage of RAID 1 is expense: that data duplication means half the space in the RAID is "wasted" so you must buy twice the capacity that you want to end up with in the array. Performance is also not as good as some RAID levels.
Mirroring is used in RAID 1, as well as multiple-level RAID involving RAID 1 (RAID 01 or RAID 10).

Striping: Striping is a technique of fragmenting the huge file (or collection of files) into indivisual Pieces (Stripes) of standard size (stripesize) and then storing the each stripes parallelly to all the Physical drives connected in RAID array.
For example, if we need to read a large file, instead of pulling it all from a single hard disk, it is much faster to chop it up into pieces, store some of the pieces on each of the drives in an array, and then use all the disks to read back the file when needed.
Striping can be done at the byte level, or in blocks. Byte-level striping means that the file is broken into "byte-sized pieces" The first byte of the file is sent to the first drive, then the second to the second drive, and so on.
Sometimes byte-level striping is done as a sector of 512 bytes. Block-level striping (group of Bytes) means that each file is split into blocks of a certain size and those are distributed to the various drives. The size of the blocks used is also called the stripe size (or block size, or several other names).
RAID 0 uses block-level striping without parity; RAID 3 and RAID 7 use byte-level striping with parity; and RAID 4, RAID 5 and RAID 6 use block-level striping with parity.
RAID 2 Uses Bit-level striping with ECC pattern but RAID2 is no more implemented anywhere because of disadvantages associated with it.

Parity: Parity technique is used as alternative technique for Mirroring because of high overhead cost disadvantage.
the term "parity" before, used in the context of system memory error detection; in fact, the parity used in RAID is very similar in concept to parity RAM.
The principle behind parity is simple: take "N" pieces of data, and from them, compute an extra piece of data. Take the "N+1" pieces of data and store them on "N+1" drives. If you lose any one of the "N+1" pieces of data, you can recreate it from the "N" that remain, regardless of which piece is lost. Parity protection is used with striping,
and the "N" pieces of data are typically the blocks or bytes distributed (Striped) across the drives in the array. The parity information can either be stored on a separate, dedicated drive, or be mixed with the data across all the drives in the array.
The parity calculation is typically performed using a logical operation called "exclusive OR" or "XOR". As you may know, the "OR" logical operator is "true" (1) if either of its operands is true, and false (0) if neither is true. The exclusive OR operator is "true" if and only if one of its operands is true; it differs from "OR" in that if both operands are true, "XOR" is false.

Concatenation or Spanning: Spanning (meaning 'Merge' drives) is a technique that allows to only group all the capacity of the multiple drives in an array. In this configuration disks storage space will be placed sequential where the Server operating system sees just one huge disk. Unfortunately, there are no other benefits except capacity with the spanning solution.
This mode is sometimes called JBOD, or "Just a Bunch Of Disks". Spanning is RAID controller (Hardware) specific implimentation and not specific to RAID implementation.

Storage- Introduction

2007-09-12T17:00:00.000+05:30

Store is the term generally used to preserve everything that can be preserved & can be retrived at any point of time. to store perticular thing we all use perticular methods, ways and technologies.
ex: to preserve money we use Banks, lockers and so on. to preserve livelyhood we use different methods.
In technical business espicially information technology, the most important thing to preserve is Information (digital data) and the methods or technologies that facilitate you to preserve information is 'Storage Technology'.
Storage technology evolved over a period of time and very primitive storage devices are Puntch cards, tapes, Magnetic tapes, drum & so on and at present Storage technology not only remained technology but become huge business and lot of well known & top companies are into this business, to name a few IBM, HP, Seagate, Hitachi, Maxstor, Dell, SUN, EMC2 & so on.
various modren Hardware Storage technologies that are currently deployed includes TApe drive/Library for Backup, Direct Attached storage (DAS), Network attached Storage (NAS), Storage Area Networks (SAN) and the software technologies include RAID, SNMP/CIM based Storage management SW, Virtualization SW, SAN Management SW, Volume Management softwares and so on.
with such diverse technologies we can store and retrive Information in organised way and also safegaurd it with many security features for many many years.

Need for having multiple Storage technologies:
* The information (digital data) generated by all the companies which uses information technologies is different and the way information needed to be stored, mantained, safegaurded and retrived is different.
for example: For Financial companies such as Banking, Accounting companies along with information storage & retrival demand more security, more availability and scalability. similarly if its just an IT company (ex: Google) then requirement will be different, along with Availability, scalability, security, performance, cost & many more factors has to be considered.
* The evolution of Storage technology should also be blamed because newer technologies developed mostly focused on immidate needs and took very long time to focus on futuristic needs and hence diverse storage technologies co-exist.
* Storage technology is must for all the companies who uses Information technology but the information (digital data) generated by these companies are different (ex: normal data, Audio, video,imaging and so on). so newer storage technologies developed also depended on type of information to be stored and in future as long as newer type of Information gets generated there might be more specific storage technologies developed to deal with them.

Storage History - the back bone for information technology

2007-09-10T14:42:00.000+05:30

Revolution of information technology not only triggered world business growth in almost all sectors but also laid the path for newer technologies and storage technolgy is one of the prominent and promising technology. Storage technology mainly deals with the storage of digitial information and reliable presentation of the same to the end users.
here is the brief history of evolution of storage.
When What
1940s In the 1940s, data was mostly stored on punched cards and punched paper tape. The punched cards and tapes are the distant relatives of the modern CD and DVD disks, which use small punched pits to store information.
late 1940s The first magnetic memory was introduces in the form of an array of magnetic cores, with each core storing one "bit" of data - the smallest unit of data storage. The interconnections between the cores allowed for random access to the data during read and write operations. This memory was non-volatile, reliable, and fast, but, unfortunately, the data was erased every time it was read, requiring an immediate rewrite. This technology later evolved into three-dimensional arrays of doughnut-shaped ferrite cores with large capacities of a few Kilobits (Kb).
1951 UNIVAC 1 (universal automatic computer), which is also the first computer to use magnetic tape for storage.
1956 In 1956 IBM introduced the RAMAC (random access method of accounting and control) - the first commercial hard disk drive.
1961 The first hard disk drive with the air-bearing slider was introduced, advancing hard disk drive technology towards much higher recording densities and reliability.
1962 The laser diode was invented (also in IBM) becoming the fundamental technology for read-write optical storage devices.
1963 IBM introduced the first storage unit with removable disks (IBM 1311), effectively ending the era of the punched-cards.
1967 The next major achievement came in 1967 the form of digital audio demonstrated by NHK (Japan Broadcasting Corporation). The medium used was a 1-inch tape for a helical-scan video tape recorder (VTR). The encoding used for recording was called PCM (Pulse Code Modulation). This first digital sound system is a close relative of the modern computer magnetic tape, the floppy disk, and the hard disk drive. From this point in time, high-quality audio technology became digital. A mix of sound and picture eventually gave birth to a new - non-mechanical (like the gramophone) and non-magnetic (like the tape) technology - the compact disk, or CD.
1967 IBM decided to discontinue the development of magnetic core memory in favor of volatile monolithic semiconductor memory chips with much faster data access and lower cost. This completed a logical separation of computer memory and storage.
1970 The portable storage was born with the invention of the floppy disk.
1977 Three Japanese companies - Sony, Mitsubishi, and Hitachi, demonstrated their optical digital audio disk (DAD) systems which used large disk, about 30 cm in diameter (like the LP records). By 1978, Philips developed a much smaller version of DAD. It used disks with a diameter of only 11.5 cm. After some negotiations, a compromise was reached and the modern CD was born with a diameter of 12 cm and with 74 minutes of play-back time. This is the approximate length of a Beethovens Ninth Symphony.
1978 The first patent for RAID (Redundant Arrays of Independent Disks) technology was filed.
1981 The history of SCSI is started when Shugart Associates (presently, Seagate Technology Corporation) joined NCR Corporation in their efforts to develop an intelligent interface for disk drives. This new interface was called Shugart Associates Systems Interface or SASI interface, a predecessor to SCSI interface.
1981 IBM introduced its first personal computer, the IBM PC, which rapidly became a standard in microcomputing.
1982 SCSI was born on base of SASI (see 1981).

1984 The history of the IDE inerface started by Compaq with the idea to embed the hard disk drive controller on a drive. Compaq, together with Western Digital Co., produced the ST506 controller that could be mounted on the hard disk drive and connected to the PC using a 40-pin cable.
1985 A company called Imprimis built the first IDE drive by integrating ST506 controller in the hard disk drive.
1987 The initial definition of RAID levels is introduced at Berkeley.
1986 SCSI (pronounced scuzzy) specification is defined in an ANSI standard X3.131-1986. SCSI is the acronym for Small Computer System Interface, a high performance parallel peripheral interface that can independently distribute data among peripherals attached to the PC.

1987 A 1-gigabit-per-square-inch magneto-optical recording with a blue-wavelength gas laser was demonstrated. A few years later, in , the same recording density barrier was broken for magnetic recording with the help of the first magnetoresistive (MR) head.

1989 A 1-gigabit-per-square-inch recording density barrier was broken for magnetic recording with the help of the first magnetoresistive (MR) head.
1994 SCSI-2 became a ANSI standard X3.131-1994. By incorporatin synchronous data transfer mode, SCS-2 increased DTR to 10 MB/s.
1994 the IDE standard was approved by the ANSI under the name ANSI X3.221-1994.
1995 The first commercial products implementing Firewire technology were Sony's DCR-VX700 and DCR-VX1000 digital video camcorders.

1996

The ATA-2 interface that complies with the ANSI X3.279-1996 standard and is the AT Attachment Interface with Extensions. Offers higher DTR and some new commands.
1996

The ATA-3 interface that complies with the ANSI X3.279-1996 standard and is the AT Attachement-3 Interface. ATA-3 doesnt offer higher speed but adds new commands and more precisely defined procedures.

1998 The first DVD-ROM drives became available for the computer users.
1998 IBM demonstrated the ability to write 100 GB of data on a single LTO (linear tape open) tape cartridge, the highest tape cartridge capacity in the industry at the time.
1998 The ATA/ATAPI-4 interface that complies with the ANSI NCITS 317-1998 and is the AT Attachment Interface with Packet Interface Extension. Supports the maximum DTR of 33 MB/sec (in burst mode).
1998 The Digital Millennium Copyright Act (DMCA) was signed into law by President Clinton on October 28, 1998. The legislation implements two 1996 World Intellectual Property Organization (WIPO) treaties: the WIPO Copyright Treaty and the WIPO Performances and Phonograms Treaty. The DMCA has significant impact on how the data is distributed, copied, and stored.
1999 After the motion picture industry spent years negotiating the encryption standard for digital video discs (DVD), a small group of Norwegian hackers released a program, called DeCSS, that can break the encryption on almost any DVD disk.
1999 Usbyte.com is started to cover the data storage technology.
2000 The ATA/ATAPI-5 interface that complies with the ANSI NCITS 340-2000 and is the AT Attachment Interface with Packet Interface-5. One of the newest and fastest IDE interfaces. Doubles the DTR of ATA-33 by supporting the maximum DTR of 66 MB/sec (in burst mode).
2000 The Serial ATA 1.0 Working Group was established to specify Serial ATA for desktop applications.

2000 IBM introduced the 1-GB microdrive, which was smaller than a matchbook and weighed only 16 grams.
2000 The first ATA-100 hard disk drive was announced in June/2000 by the Quantum Corporation
2003 Hitachi buys IBM Data Storage Division. New company - Hitachi Global Storage Technology - is formed. IBM's involvement in the disk drive technology is ended.