Networks and Servers

How to use the Virtualization Lab (II)

2012-01-11T15:22:00.001-08:00

Picking up from where I left, it was now time to change the setup into something very different. The first step was the creation of another VM inside Hyper-V to be used as an alternative source for iSCSI storage. I achieved this by installing the Microsoft iSCSI Target 3.3 on a new Server 2008 R2 x64 VM. I created this machine with two vhd files; one for the OS and the other one for the iSCSI storage.

I will now show you the steps taken to create three new iSCSI virtual disks:

Creation of the iSCSI target:

Read more »

How to use the Virtualization Lab (I)

2012-01-03T04:04:00.001-08:00

I finished last post on this series with a fully working cluster installed between two Hyper-V virtual machines (VM) using a virtual iSCSI solution installed on a Virtual Box VM as depicted in the next picture:

Before moving on in the process of adding complexity to the lab scenario, don't forget to safeguard your work; although this just a lab, it doesn't reduce the nuisance of having to reinstall everything in the event of any failure. So, create VM snapshots:

Read more »

High Availability with Failover Clusters

2011-12-28T13:48:00.001-08:00

Before moving on to the next chapter on my virtualization lab series, I think this might be a good opportunity to review some of the clustering options available today. I will use Windows Server Failover Clustering with Hyper-V because in today's world the trend is to combine Virtualization with High Availability (HA).

There are many ways to implement these solutions and the basic design concepts presented here can be adjusted to other virtualization platforms. Some of them will actually not guarantee a fault-tolerant solution, but most of them can be used in specific scenarios (even if only for demonstration purposes).

Two virtual machines on one physical server

In this scenario an HA cluster is built between two (or more) virtual machines on a single physical machine. Here we have a single physical server running Hyper-V and two child partitions where you run Failover Clustering. This setup does not protect against hardware failures because when the physical server fails, both (virtual) cluster nodes will fail. Therefore, the physical machine itself is a single point of failure (SPOF).

(Click to enlarge)

How to Setup a Virtualization Lab (III)

2011-12-12T16:08:00.001-08:00

Failover Cluster Networking

The first step in the setup of a failover cluster is the creation of an AD domain because all the cluster nodes have to belong to the same domain. But before doing so, I changed the networks settings again in order to adjust them for this purpose.

LAB-DC:IP: 192.168.1.10
Gateway: 192.168.1.1 (Physical Router)
DNS: 127.0.0.1
Alternate DNS: 192.168.1.1

LAB-NODE1:
IP: 192.168.1.11
Gateway: 192.168.1.1
DNS: 192.168.1.10 (DC)
Alternate DNS: 192.168.1.1 (Physical Router)

LAB-NODE2:IP: 192.168.1.12
Gateway: 192.168.1.1
DNS: 192.168.1.10
Alternate DNS: 192.168.1.1

LAB-NODE3:
IP: 192.168.1.13
Gateway: 192.168.1.1
DNS: 192.168.1.10
Alternate DNS: 192.168.1.1

LAB-STORAGE:IP: 192.168.1.14
Gateway: 192.168.1.1
DNS: 192.168.1.10
Alternate DNS: 192.168.1.1

Therefore, I created a domain comprised of 5 machines; a DC and two member servers as Hyper-V VMs, a member server as a VMware VM and another member server as a VirtualBox VM.

So far I have demonstrated the possibility of integrating in the same logical infrastructure virtualized servers running on different platforms using different virtualization techniques; in this case we have VMs running in a Type 1 hypervisor (Hyper-V) and in two distinct Type 2 hypervisors (VMware Workstation and VirtualBox).

The option to create a network with static IP addresses is as valid as the alternative of using DHCP. Later on I plan to explore the several options provided by the cluster networking in Windows 2008 but for the time being I kept my network in a simple and basic configuration in order to proceed with the lab installation.

Read more »

How to Setup a Virtualization Lab (II)

2011-12-02T17:04:00.001-08:00

As mentioned at the end of my previous article, the installation of my lab continued with the creation of virtual machines on the desktop computer. But this time I used VMware and VirtualBox to explore the possibility of using a set of virtualized servers across different and competing virtualization technologies.

I insisted on the network configuration details because that is the basis of all the work ahead; a single virtual machine may be important but I want to show how they can work together and therefore the correct network configuration of paramount importance.

Import a Virtual Machine into VMware

I started by installing a VM on VMware Workstation. Better yet, I took advantage of what was previously done and used the generalized .vhd file I left behind! Since VMware does not directly support the use of .vhd files, I had to convert the file from the format used by Hyper-V (Virtual Hard Disk, i.e., .vhd) to the format used by VMware (Virtual Machine Disk, i.e., .vmdk).

The VMware vCenter Converter Standalone utility is a free application which can be obtained directly from VMware’s official site but doesn’t solve the problem as it doesn’t support this type of conversion, although it can convert from other formats and even directly from servers running Hyper-V. But what interested me was to use the work already done and so I resorted to the WinImage tool.

The process was very simple:

I selected the appropriate option from the Disk menu and select the proper source file;

Read more »

How to Setup a Virtualization Lab (I)

2011-11-29T23:19:00.001-08:00

Now that I have concluded a general overview of most of the theory related to High Availability and Virtualization it is time to start testing some of those concepts and see them in action.

My goal for the next posts is to produce a series of tutorials showing how anyone can easily install a handful of virtual machines and be able to explore the wonderful possibilities provided by this technology. I will be using an old laptop powered by a Turion 64 X2 CPU with a 250 Gb SSD HD and 4 Gb of RAM combined with a desktop running Windows 7 Ultimate on a Athlon 64 X2 4800+ with 4 Gb of RAM and lots a free disk space scattered through 3 SATA hard drives.

Virtual Machines Creation

I will not go through the details of OS installation because I am assuming the ones reading these tutorials are way passed that.

I started by installing a fresh copy of Windows Server 2008 R2 SP1 Standard on a secondary partition in my laptop. Once I was done with the installation of all the available updates from Windows Update and with OS activation, I was ready to add the Hyper-V role in order to be able to install the virtual machines. To do this I just went into Server Manager/Roles, started the Add Roles Wizard, selected Hyper-V and followed the procedures. Nothing special so far, right?

Note: All the pictures are clickable and will open a larger version in a separate window.

Read more »

Scientists replicate brain using a chip

2011-11-24T16:52:00.001-08:00

Scientists are getting closer to the dream of creating computer systems that can replicate the brain. Researchers at the Massachusetts Institute of Technology (MIT) have designed a computer chip that mimics how the brain's neurons adapt in response to new information. Such chips could eventually enable communication between artificially created body parts and the brain and it could also pave the way for artificial intelligence devices.

There are about 100 billion neurons in the brain, each of which forms synapses - the connections between neurons that allow information to flow - with many other neurons. This process is known as plasticity and is believed to underpin many brain functions, such as learning and memory.

Read more »

Bacteria Inspire Robotics

2011-11-24T08:51:00.001-08:00

Researchers at Tel Aviv University have developed a computational model that better explains how bacteria move in a swarm -- and this model can be applied to human-made technologies, including computers, artificial intelligence, and robotics. The team of scientists has discovered how bacteria collectively gather information about their environment and find an optimal path to growth, even in the most complex terrains.

Studying the principles of bacteria navigation will allow researchers to design a new generation of smart robots that can form intelligent swarms, aid in the development of medical micro-robots used to diagnose or distribute medications in the body, or "de-code" systems used in social networks and throughout the Internet to gather information on consumer behaviors.

Simulated interacting agents collectively navigate towards a target (credit: American Friends of Tel Aviv University)

Hardware-Assisted Virtualization Explained

2011-11-22T12:30:00.001-08:00

Hardware-assisted virtualization was first introduced on the IBM System/370 in 1972, for use with VM/370, the first virtual machine operating system. Virtualization was forgotten in the late 1970s but the proliferation of x86 servers rekindled interest in virtualization driven for the need for server consolidation; virtualization allowed a single server to replace multiple underutilized dedicated servers.

However, the x86 architecture did not meet the Popek and Goldberg Criteria to achieve the so called “classical virtualization″. To compensate for these limitations, virtualization of the x86 architecture has been accomplished through two methods: full virtualization or paravirtualization. Both create the illusion of physical hardware to achieve the goal of operating system independence from the hardware but present some trade-offs in performance and complexity.

Thus, Intel and AMD have introduced their new virtualization technologies, a handful of new instructions and — crucially — a new privilege level. The hypervisor can now run at "Ring -1"; so the guest operating systems can run in Ring 0.

Hardware virtualization leverages virtualization features built into the latest generations of CPUs from both Intel and AMD. These technologies, known as Intel VT and AMD-V respectively, provide extensions necessary to run unmodified virtual machines without the overheads inherent in full virtualization CPU emulation. In very simplistic terms these new processors provide an additional privilege mode below ring 0 in which the hypervisor can operate essentially leaving ring 0 available for unmodified guest operating systems.

Read more »

A new quantum state of matter?

2011-11-21T14:49:00.001-08:00

Researchers at the University of Pittsburgh have made advances in better understanding correlated quantum matter by studying topological states in order to advance quantum computing, a method that harnesses the power of atoms and molecules for computational tasks.

Through his research, W. Vincent Liu and his team have been studying orbital degrees of freedom and nano-Kelvin cold atoms in optical lattices (a set of standing wave lasers) to better understand new quantum states of matter. From that research, a surprising topological semimetal has emerged.

Since the discovery of the quantum Hall effect by Klaus Van Klitzing in 1985, researchers like Liu have been particularly interested in studying topological states of matter, that is, properties of space unchanged under continuous deformations or distortions such as bending and stretching. The quantum Hall effect proved that when a magnetic field is applied perpendicular to the direction a current is flowing through a metal, a voltage is developed in the third perpendicular direction. Liu's work has yielded similar yet remarkably different results.

"We never expected a result like this based on previous studies," said Liu. "We were surprised to find that such a simple system could reveal itself as a new type of topological state -- an insulator that shares the same properties as a quantum Hall state in solid materials."
"This new quantum state is very reminiscent of quantum Hall edge states," said Liu. "It shares the same surface appearance, but the mechanism is entirely different: This Hall-like state is driven by interaction, not by an applied magnetic field."

Liu says this liquid matter could potentially lead toward topological quantum computers and new quantum devices for topological quantum telecommunication. Next, he and his team plan to measure quantities for a cold-atom system to check these predicted quantum-like properties.

Operating System-Level Virtualization Explained

2011-11-21T03:37:00.001-08:00

This kind of server virtualization is a technique where the kernel of an operating system allows for multiple isolated user-space instances. These instances run on top of an existing host operating system and provide a set of libraries that applications interact with, giving them the illusion that they are running on a machine dedicated to its use. The instances are known as Containers, Virtual Private Servers or Virtual Environments.

Operating system level virtualization is achieved by the host system running a single OS kernel and through its control of guest operating system functionality. Under this shared kernel virtualization the virtual guest systems each have their own root file system but share the kernel of the host operating system.
Read more »

Paravirtualization Explained

2011-11-15T14:55:00.001-08:00

“Para“ is an English affix of Greek origin that means "beside," "with," or "alongside.” Paravirtualization is another approach to server virtualization where, rather than emulate a complete hardware environment, paravirtualization acts as a thin layer, which ensures that all of the guest operating systems share the system resources and work well together.

Under paravirtualization, the kernel of the guest operating system is modified specifically to run on the hypervisor. This typically involves replacing any privileged operations that will only run in ring 0 of the CPU, with calls to the hypervisor (known as hypercalls). The hypervisor in turn performs the task on behalf of the guest kernel and also provides hypercall interfaces for other critical kernel operations such as memory management, interrupt handling and time keeping.

Read more »

Full Virtualization Explained

2011-11-13T15:32:00.001-08:00

This is probably the most common and most easily explained kind of server virtualization. When IT departments were struggling to get results with machines at full capacity, it made sense to assign one physical server to every IT function taking advantage of cheap hardware A typical enterprise would have one box for SQL, one for the Apache server and another physical box for the Exchange server. Now, each of those machines could be using only 5% of its full processing potential. This is where hardware emulators come into play in an effort to consolidate those servers.

A hardware emulator presents a simulated hardware interface to guest operating systems. In hardware emulation, the virtualization software (usually referred to as a hypervisor) actually creates an artificial hardware device with everything it needs to run an operating system and presents an emulated hardware environment that guest operating systems operate upon. This emulated hardware environment is typically referred to as a Virtual Machine Monitor or VMM.

Hardware emulation supports actual guest operating systems; the applications running in each guest operating system are running in truly isolated operating environments. This way, we can have multiple servers running on a single box, each completely independent of the other. The VMM provides the guest OS with a complete emulation of the underlying hardware and for this reason, this kind of virtualization is also referred to as Full Virtualization.

Read more »

Quantum cryptography breached?

2011-11-11T11:34:00.001-08:00

Quantum cryptography has been pushed onto the market as a way to provide absolute security for communications and, as far as we know, no current quantum cryptographic system has been compromised in the field. It is already used in Swiss elections to ensure that electronic vote data is securely transmitted to central locations.

Quantum cryptography relies on the concept of entanglement. With entanglement, some statistical correlations are measured to be larger than those found in experiments based purely on classical physics. Cryptographic security works by using the correlations between entangled photons pairs to generate a common secret key. If an eavesdropper intercepts the quantum part of the signal, the statistics change, revealing the presence of an interloper.

The Swiss general approach can be summed up as follows: if you can fool a detector into thinking a classical light pulse is actually a quantum light pulse, then you might just be able to defeat a quantum cryptographic system. But even then the attack should fail, because quantum entangled states have statistics that cannot be achieved with classical light sources—by comparing statistics, you could unmask the deception.

But there's a catch here. I can make a classical signal that is perfectly correlated to any signal at all, provided I have time to measure said signal and replicate it appropriately. In other words, these statistical arguments only apply when there is no causal connection between the two measurements.

You might think that this makes intercepting the quantum goodness of a cryptographic system easy. But you would be wrong. When Eve intercepts the photons from the transmitting station run by Alice, she also destroys the photons. And even though she gets a result from her measurement, she cannot know the photons' full state. Thus, she cannot recreate, at the single photon level, a state that will ensure that Bob, at the receiving station, will observe identical measurements.

That is the theory anyway. But this is where the second loophole comes into play. We often assume that the detectors are actually detecting what we think they are detecting. In practice, there is no such thing as a single photon, single polarization detector. Instead, what we use is a filter that only allows a particular polarization of light to pass and an intensity detector to look for light. The filter doesn't care how many photons pass through, while the detector plays lots of games to try and be single photon sensitive when, ultimately, it is not. It's this gap between theory and practice that allows a carefully manipulated classical light beam to fool a detector into reporting single photon clicks.

Since Eve has measured the polarization state of the photon, she knows what polarization state to set on her classical light pulse in order to fake Bob into recording the same measurement result. When Bob and Alice compare notes, they get the right answers and assume everything is on the up and up.
The researchers demonstrated that this attack succeeds with standard (but not commercial) quantum cryptography equipment under a range of different circumstances. In fact, they could make the setup outperform the quantum implementation for some particular settings.

(Adapted from ArsTechnica)

Software to Prevent Child Abuse

2011-11-09T03:11:00.000-08:00

Investigators estimate that there are currently more than 15 million photographs and videos of child abuse victims circulating on the Internet, or in the Darknet. By the time this material has been tracked down and deleted, pedophiles have long since downloaded it to their computers. Seeking and tracking hundreds of thousands of illegal media files in the suspect’s computer was tedious and extremely time-consuming process for investigators, until now.

Researchers from Fraunhofer Institute come up with an automated assistance system, called “desCRY”, that can detect child-pornographic images and video, from among even large volumes of data.

desCRY search results

The desCRY software uses novel pattern-recognition processes to navigate through digital photos and videos in search of illegal content, no matter how well-hidden it may be. The heart of the software consists of intelligent pattern-recognition algorithms that automatically analyze and classify images and video sequences combining technologies such as facial and skin-tone recognition with contextual and scene analyses to identify suspicious content.

The software searches all of the files in a computer, e-mail attachments and archives included and has many types of filtering allowing for a wide variety of search options. It can perform content-based data sorting and filtering, for instance. This way, investigators can sort files by person, object or location, for example.
The algorithms use up to several thousand characteristics that describe properties such as color, texture and contours in order to analyze whether an image depicts child abuse. If the system is run on a standard PC, it classifies up to ten images per second, drastically accelerating the investigation works.

Quantum Cloning Advances

2011-11-08T10:15:00.000-08:00

Quantum cloning is the process that takes an arbitrary, unknown quantum state and makes an exact copy without altering the original state in any way. Quantum cloning is forbidden by the laws of quantum mechanics as shown by the no cloning theorem. Though perfect quantum cloning is not possible, it is possible to perform imperfect cloning, where the copies have a non-unit fidelity with the state being cloned.

The quantum cloning operation is the best way to make copies of quantum information therefore cloning is an important task in quantum information processing, especially in the context of quantum cryptography. Researchers are seeking ways to build quantum cloning machines, which work at the so called quantum limit. Quantum cloning is difficult because quantum mechanics laws only allow for an approximate copy—not an exact copy—of an original quantum state to be made, as measuring such a state prior to its cloning would alter it. The first cloning machine relied on stimulated emission to copy quantum information encoded into single photons.

Scientists in China have now produced a theory for a quantum cloning machine able to produce several copies of the state of a particle at atomic or sub-atomic scale, or quantum state. A team from Henan Universities in China, in collaboration with another team at the Institute of Physics of the Chinese Academy of Sciences, have produced a theory for a quantum cloning machine able to produce several copies of the state of a particle at atomic or sub-atomic scale, or quantum state. The advance could have implications for quantum information processing methods used, for example, in message encryption systems.

In this study, researchers have demonstrated that it is theoretically possible to create four approximate copies of an initial quantum state, in a process called asymmetric cloning. The authors have extended previous work that was limited to quantum cloning providing only two or three copies of the original state. One key challenge was that the quality of the approximate copy decreases as the number of copies increases.

The authors were able to optimize the quality of the cloned copies, thus yielding four good approximations of the initial quantum state. They have also demonstrated that their quantum cloning machine has the advantage of being universal and therefore is able to work with any quantum state, ranging from a photon to an atom. Asymmetric quantum cloning has applications in analyzing the security of messages encryption systems, based on shared secret quantum keys.

Server Virtualization Explained

2011-11-02T16:31:00.001-07:00

You have probably heard about lots of distinct types of server virtualization; full, bare metal, para-virtualization, guest OS, OS assisted, hardware assisted, hosted, OS level, kernel level, shared kernel, hardware emulation, hardware virtualization, hypervisor based, containers or native virtualization. Confusing, right?

Fear not my faithful readers; the whole purpose of this blog is exactly to explain these things so that everyone can have a clear view over issues usually restricted to a bunch of geeks. But keep in mind that some of these terms are popularized by certain vendors and do not have a common industry-wide acceptance. Plus, many of the terms are used rather loosely and interchangeably (which is why they are so confusing).

Although others classify the current virtualization techniques in a different way, I will use the following criteria:

On the following exciting chapters I will explain these techniques, one by one, but before that I believe it would be useful to give you a quick introduction to some underlying concepts.

Read more »

Virtualization (III)

2011-10-24T14:57:00.001-07:00

Server Virtualization

Out of all three of the different types of virtualization discussed in this blog, I believe that server virtualization is the type of virtualization everybody is most familiar with. When people say "virtualization", they are usually referring to server virtualization because this is the main area of virtualization, whereby a number of “virtual machines” are created on one server meaning that multiple tasks can then be assigned to the one server, saving on processing power, cost and space.

Server virtualization inserts a layer of abstraction between the physical server hardware and the software that runs on the server allowing us to run multiple guest computers on a single host computer with those guest computers believing they are running on their own hardware. The physical machine is translated into one or more virtual machines (VMs). Each VM runs its own operating system and applications, and each utilizes some allocated portion of the server's processing resources such as CPU, memory, network access and storage I/O. This means that any network tasks that are happening on the server still appear to be on a separate space, so that any errors can be diagnosed and fixed quickly.

By doing this, we gain all the benefits of any type of virtualization: portability of guest virtual machines, reduced operating costs, reduced administrative overhead, server consolidation, testing & training, disaster recovery benefits, and more.

Read more »

Virtualization (II)

2011-10-19T15:22:00.001-07:00

Network Virtualization

When we think of network virtualization, we always think of VLANs but there is much more to network virtualization than just VLANs. Network virtualization is when all of the separate resources of a network are combined, allowing the administrator to share them out amongst the users of the network. Thus, it is a method of combining the available resources in a network by splitting up the available bandwidth into channels, each of which is independent from the others, and each of which can be assigned (or reassigned) to a particular server or device in real time. This allows each user to access all of the network resources from their computer either they are files and folders on the computer, printers or hard drives etc.

The theory behind network virtualization is to take many of the traditional client/server based services and put them "on the network". Certain vendors advertise virtualization and networking as a vehicle for additional services and not just as a way to aggregate and allocate network resources. For example, it's common practice for routers and switches to support security, storage, voice over IP (VoIP), mobility and application delivery.

One network vendor actually has a working card that is inserted into a router. On that card is a fully-functioning Linux server that has a connection to the backbone of the router. On that Linux server, you can install applications like packet sniffers, VoIP, security applications, and many more.
Network virtualization provides an abstraction layer that decouples physical network devices from operating systems, applications and services delivered over the network allowing them to run on a single server or for desktops to run as virtual machines in secure data centers, creating a more agile and efficient infrastructure. This streamlined approach makes the life of the network administrator much easier, and it makes the system seem much less complicated to the human eye than it really is.

Network virtualization is a versatile technology. It allows you to combine multiple networks into a single logical network, parcel a single network into multiple logical networks and even create software-only networks between virtual machines (VMs) on a physical server. Virtual networking typically starts with virtual network software, which is placed outside a virtual server (external) or inside a virtual server, depending on the size and type of the virtualization platform.

Read more »

Virtualization (I)

2011-10-16T11:20:00.001-07:00

Virtualization is a massively growing aspect of computing and IT creating "virtually" an unlimited number of possibilities for system administrators. Virtualization has been around for many years in some form or the other, the trouble with it, is being able to understand the different types of virtualization, what they offer, and how they can help us.

Virtualization has been defined as the abstraction of computer resources or as a technique for hiding the physical characteristics of computing resources from the way in which other systems, applications, or end users interact with those resources. In fact, virtualization always means abstraction. We make something transparent by adding layers that handle translations, causing previously important aspects of a system to become moot.

Storage Virtualization

The amount of data organizations are creating and storing is rapidly increasing due to the shift of business processes to Web-based digital applications and this huge amount of data is causing problems for many of them. First, many applications generate more data than can be stored physically on a single server. Second, many applications, particularly Internet-based ones, have multiple machines that need to access the same data. Having all of the data sitting on one machine can create a bottleneck, not to mention presenting risk from the situation where many machines might be made inoperable if a single machine containing all the application’s data crashes. Finally, the increase in the number of machines causes backup problems because trying to create safe copies of data is a tremendous task when there are hundreds or even thousands of machines that need data backup.

For these reasons, data has moved into virtualization. Companies use centralized storage (virtualized storage) as a way of avoiding data access problems. Furthermore, moving to centralized data storage can help IT organizations reduce costs and improve data management efficiency. The basic premise of storage virtualization solutions is not new. Disk storage has long relied on partitioning to organize physical disk tracks and sectors into clusters, and then abstract clusters into logical drive partitions (e.g., the C: drive). This allows the operating system to read and write data to the local disks without regard to the physical location of individual bytes on the disk platters.
Storage virtualization creates a layer of abstraction between the operating system and the physical disks used for data storage. The virtualized storage is then location-independent, which can enable more efficient use and better storage management. For example, the storage virtualization software or device creates a logical space, and then manages metadata that establishes a map between the logical space and the physical disk space. The creation of logical space allows a virtualization platform to present storage volumes that can be created and changed with little regard for the underlying disks.

The storage virtualization layer is where the resources of many different storage devices are pooled so that it looks like they are all one big container of storage. This is then managed by a central system that makes it all look much simpler to the network administrators. This is also a great way to monitor resources, as you can then see exactly how much you have left at a given time giving much less hassle when it comes to backups etc.

In most data centers, only a small percentage of storage is used because, even with a SAN, one has to allocate a full disk logical unit number (LUN) to the server (or servers) attaching to that LUN. Imagine that a LUN fills up but there is disk space available on another LUN. It is very difficult to take disk space away from one LUN and give it to another LUN. Plus, it is very difficult to mix and match storage and make it appear all as one.

Storage virtualization works great for mirroring traffic across a WAN and for migrating LUNs from one disk array to another without downtime. With some types of storage virtualization, for instance, you can forget about where you have allocated data, because migrating it somewhere else is much simpler. In fact, many systems will migrate data based on utilization to optimize performance.

Read more »

High Availability Storage (II)

2011-09-27T15:09:00.001-07:00

Storage Area Network

A Storage Area Network (SAN) is a dedicated high-performance subnet that provides access to consolidated, block level data storage and is primarily used to transfer data between computer systems and storage elements and among multiple storage elements, making storage devices, such as disk arrays, tape libraries, and optical jukeboxes, accessible to servers so that the devices appear like locally attached devices to the operating system.

A SAN typically has its own communication infrastructure that is generally not accessible through the local area network by other devices. A SAN moves data among various storage devices, allowing for the sharing data between different servers, and provides a fast connection medium for backing up, restoring, archiving, and retrieving data. SAN devices are usually installed closely in a single room, but they can also be connected over long distances, making it very useful to large companies.

SAN Benefits

The primary benefits of a SAN are:

High Availability: One copy of every piece of data is always accessible to any and all hosts via multiple paths;
Reliability: Dependable data transportation ensures a low error rate, and fault tolerance capabilities;
Scalability: Servers and storage devices may be added independently of one another and from any proprietary systems;
Performance: Fibre Channel (the standard method for SAN interconnectivity) has now over than 2000MB/sec bandwidth and low overhead, and it separates storage and network I/O;

High Availability Storage (I)

2011-09-24T05:23:00.001-07:00

RAID Concepts

The acronym RAID stands for Redundant Array of Inexpensive Disks and is a technology that provides increased storage functions and reliability through redundancy. It was developed using a large number of low cost hard drives linked together to form a single large capacity storage device that offered superior performance, storage capacity and reliability over older storage systems. This was achieved by combining multiple disk drive components into a logical unit, where data was distributed across the drives in one of several ways called "RAID levels".
This concept of storage virtualization and was first defined as Redundant Arrays of Inexpensive Disks but the term later evolved into Redundant Array of Independent Disks as a means of dissociating a low-cost expectation from RAID technology.
There are two primary reasons that RAID was implemented:

Redundancy: This is the most important factor in the development of RAID for server environments. A typical RAID system will assure some level of fault tolerance by providing real time data recovery with uninterrupted access when hard drive fails;

Increased Performance: The increased performance is only found when specific versions of the RAID are used. Performance will also be dependent upon the number of drives used in the array and the controller;

Hardware-based RAID

When using hardware RAID controllers, all algorithms are generated on the RAID controller board, thus freeing the server CPU. On a desktop system, a hardware RAID controller may be a PCI or PCIe expansion card or a component integrated into the motherboard. These are more robust and fault tolerant than software RAID but require a dedicated RAID controller to work.

Hardware implementations provide guaranteed performance, add no computational overhead to the host computer, and can support many operating systems; the controller simply presents the RAID array as another logical drive

Software-based RAID

Many operating systems provide functionality for implementing software based RAID systems where the OS generate the RAID algorithms using the server CPU. In fact the burden of RAID processing is borne by a host computer's central processing unit rather than the RAID controller itself which can severely limit the RAID performance.

Although cheap to implement it does not guarantee any kind of fault tolerance; should a server fail the whole RAID system is lost.

Hot spare drive

Both hardware and software RAIDs with redundancy may support the use of hot spare drives, a drive
physically installed in the array which is inactive until an active drive fails. The system then automatically replaces the failed drive with the spare, rebuilding the array with the spare drive included. This reduces the mean time to recovery (MTTR), but does not completely eliminate it. Subsequent additional failure(s) in the same RAID redundancy group before the array is fully rebuilt can result in data loss. Rebuilding can take several hours, especially on busy systems.

Read more »

Failover Clustering (IV)

2011-09-20T15:35:00.001-07:00

Cluster Node Configurations

The most common size for an high availability cluster is a two-node cluster, since that's the minimum required to assure redundancy, but many clusters consist of many more, sometimes dozens of nodes and such configurations can be categorized into one of the following models:

Active/Passive Cluster

In an Active/Passive (or asymmetric) configuration, applications run on a primary, or master, server. A dedicated redundant server is present to take over on any failure but apart from that it is not configured to perform any other functions. Thus, at any time, one of the nodes is active and the other is passive. This configuration provides a fully redundant instance of each node, which is only brought online when its associated primary node fails.
The active/passive cluster generally contains two identical nodes. Database applications single instances are installed on both nodes, but the database is located on shared storage. During normal operation, the database instance runs only on the active node. In the event of a failure of the currently active primary system, clustering software will transfer control of the disk subsystem to the secondary system. As part of the failover process, the database instance on the secondary node is started, thereby resuming the service.

This configuration is the simplest and most reliable but typically requires the most extra hardware.

Read more »

Failover Clustering (III)

2011-09-16T14:49:00.001-07:00

Cluster Quorum Configurations

In simple terms, the quorum for a cluster is the number of elements that must be online for that cluster to continue running. Server clusters require a quorum resource to function and this, like any other resource, is a resource which can only be owned by one server at a time, and for which servers can negotiate for ownership. In effect, each element can cast one “vote” to determine whether the cluster continues running. The voting elements are nodes or, in some cases, a disk witness or file share witness. The quorum resource is used to store the definitive copy of the cluster configuration so that regardless of any sequence of failures, the cluster configuration will always remain consistent. Each voting element (with the exception of a file share witness) contains a copy of the cluster configuration, and the Cluster service works to keep all copies synchronized at all times.

When network problems occur, they can interfere with communication between cluster nodes. A small set of nodes might be able to communicate together across a functioning part of a network but not be able to communicate with a different set of nodes in another part of the network. This can cause serious issues. In this "split" situation, at least one of the sets of nodes must stop running as a cluster.
Negotiating for the quorum resource allows Server clusters to avoid "split-brain" situations where the servers are active and think the other servers are down.
To prevent the issues that are caused by a split in the cluster, the cluster software requires that any set of nodes running as a cluster must use a voting algorithm to determine whether, at a given time, that set has quorum. Because a given cluster has a specific set of nodes and a specific quorum configuration, the cluster will know how many "votes" constitutes a majority (that is, a quorum). If the number drops below the majority, the cluster stops running. Nodes will still listen for the presence of other nodes, in case another node appears again on the network, but the nodes will not begin to function as a cluster until the quorum exists again.

Read more »

Failover Clustering (II)

2011-09-12T15:54:00.001-07:00

The failover mechanism

In the cluster, shared resources can be seen from all computers, or nodes. Each node automatically senses if another node in the cluster has failed, and processes running on the failed node continue to run on an operational one. To the user, this failover is usually transparent. Depicted in the next figure is a two-node cluster; both nodes have local disks and shared resources available to both computers. The heartbeat connection allows the two computers to communicate and detects when one fails; if Node 1 fails, the clustering software will seamlessly transfer all services to Node 2.

The Windows service associated with the failover cluster named Cluster Service has a few components:

Event processor;
Database manager;
Node manager;
Global update manager
Communication manager;
Resource (or failover) manager.

The resource manager communicates directly with a resource monitor that talks to a specific application DLL that makes an application cluster-aware. The communication manager talks directly to the Windows Winsock layer (a component of the Windows networking layer).
The failover process is as follows:
Read more »