Mike's Technology and Finance Blog

Simple Sorting in .Net With Comparison and IComparer

2013-01-25T13:38:00.001-08:00

Sorting is an important piece for just about any programmer's job. The ability to effectively search and sort often has a significant impact on the execution time of a batch job and on the user's measure of "responsiveness" in a traditional GUI or web based application. There are a number of people that go through and write their own implementations of common sorting algorithms (insertion sort, bubble sort, merge sort, etc.), but I prefer to use built in library functions whenever possible.

Many of the collection types in System.Collections implement a Sort() method that can be used to sort data in a (mostly) arbitrary way. I say mostly in this case because an inconsistent comparison may lead to an ArgumentException.

There are a few basic rules to remember. Given a Comparison or IComparer that compares objects x (the first argument) and y (the second argument), the following may happen:

If the comparison returns -1 (or any value less than 0), object x will be placed before y.
If the comparison returns 0, object x and y will retain their positions.
If the comparison returns 1 (or any value greater than 0), object y will be placed before object x.

To begin, let's look at a very simple example of an ascending (low -> high) sort and a descending (high -> low) sort (note that the examples given here are given as Visual Studio Unit Tests):

[TestMethod]

public void AscendingSort()

{

List<int> unsorted = new List<int>(new int[] { 2, 1, 3 });

unsorted.Sort((x, y) =>

{

int retval;

if (x < y) { retval = -1; }

else if (x == y) { retval = 0; }

else { retval = 1; }

return retval;

});

Assert.AreEqual(1, unsorted[0]);

Assert.AreEqual(2, unsorted[1]);

Assert.AreEqual(3, unsorted[2]);

}

[TestMethod]

public void DescendingSort()

{

List<int> unsorted = new List<int>(new int[] { 2, 1, 3 });

unsorted.Sort((x, y) =>

{

int retval;

if (x < y) { retval = 1; }

else if (x == y) { retval = 0; }

else { retval = -1; }

return retval;

});

Assert.AreEqual(3, unsorted[0]);

Assert.AreEqual(2, unsorted[1]);

Assert.AreEqual(1, unsorted[2]);

}

Since the integer types implement the IComparable interface, these examples can be written more compactly as:

[TestMethod]

public void AscendingSort()

{

List<int> unsorted = new List<int>(new int [] { 2, 1, 3});

unsorted.Sort((x, y) =>

{

return x.CompareTo(y);

});

Assert.AreEqual(1, unsorted[0]);

Assert.AreEqual(2, unsorted[1]);

Assert.AreEqual(3, unsorted[2]);

}

[TestMethod]

public void DescendingSort()

{

List<int> unsorted = new List<int>(new int[] { 2, 1, 3 });

unsorted.Sort((x, y) =>

{

//could also be written as y.CompareTo(x)
return -1 * x.CompareTo(y);

});

Assert.AreEqual(3, unsorted[0]);

Assert.AreEqual(2, unsorted[1]);

Assert.AreEqual(1, unsorted[2]);

}

It should be noted that if a collection contains objects that do not implement IComparable, the Sort() method will throw an InvalidOperationException unless a custom Comparison delegate or implementation of IComparer is used. People often run into this with List<T> and ArrayList<T>, among other specialized types.

Up until now, I've demonstrated using a System.Comparison delegate with the Sort() method. Here is an example of a class that implements IComparer and its use in a corresponding Sort() operation:

public class AscendingComparison : IComparer<int>

{

public int Compare(int x, int y)

{

return x < y ? -1 : x == y ? 0 : 1;

}

[TestMethod]

public void AscendingSort()

{

List<int> unsorted = new List<int>(new int [] { 2, 1, 3});

unsorted.Sort(new AscendingComparison());

Assert.AreEqual(1, unsorted[0]);

Assert.AreEqual(2, unsorted[1]);

Assert.AreEqual(3, unsorted[2]);

}

As a general pattern, the delegate form makes the most sense in simple code that is not expected to be reused (potentially with the transaction script pattern) because it is potentially difficult to modify if it becomes spread across multiple locations. More complex code (think object composition and more complex design patterns) or code that requires the ability to define a sort at runtime should implement IComparer in a different class that can be reused and centrally modified as necessary.

In the next set of examples, I'll look at more complex sorting scenarios.

Strange Amazon Ads

2013-01-22T07:11:00.001-08:00

I don't usually pay too much attention to ads on the Internet, but a recent Amazon ad caught my eye and has caused me some confusion. I've been studying a bit on facial expression, facial action (based off of the work of Dr. Paul Ekman), microexpressions, and lie detection and there are a number of emotions that have been found to be universal. Before I go too much further, look at the following ad and interpret the emotion shown in the top picture.

Before we go into what facial expression this is, let's start by asking ourselves "If I had been propositioned for a cheap date, what emotion would I feel?" In most circumstances, I'd probably say that either an anger or disgust expression (or perhaps a blend of the two) would be shown, but neither of those are present. Thinking through the universal emotions (surprise, fear, anger, disgust, happiness, and sadness), the 2 emotions that hit the bottom of the list of the ones that I'd expect to see are happiness and fear (because someone asked on a cheap date could simply say no).

The expression has most of the signs of a fear expression, though the brow is less raised (AU1 and AU2) than you might expect (you would typically expect to see horizontal wrinkles on the forehead that may or may not stretch all of the way across).

The takeaway here is that the marketers at Amazon need to figure out their own emotions, since the ones that they are showing in their ads make no sense...

Unit Testing an Artificial Neural Network

2012-10-30T13:34:00.000-07:00

I am working on a Big Data experiment where I am using artificial neural networks to classify different data sets into conforming and non-conforming patterns and I needed a good artificial neural network library in C#.Net.

I surveyed a number of implementations and was dismayed that I could not find a good one. I was even more dismayed by the number of blog posts and code articles that had deeply flawed implementations, where someone couldn't expect to get these implementations to work in test, much less production.

So... because I couldn't find a good implementation, I built my own library into my Big Data application. I am going to play the implementation details close to the chest, but I wanted to share one of my unit tests that I developed for the Backpropagation algorithm since it was time consuming to work out an example in Microsoft Excel. I could not find any other worked examples online, so I built an example by hand in Microsoft Excel for the following network:

I use both weights on the adjacencies and a threshold/bias factor for each node and I used the hyperbolic tangent function for the activation function. Below is the Visual Studio Unit Test showing the worked example. For simplicity, I round everything to 6 decimal points for test purposes.

[TestMethod]

public void TestBackpropagationMultilayerUntrained()

{

//Arrange

NeuralNetwork n = new NeuralNetwork("BackProp3LayerTest", "Back Propagation Test for 2 Processing Layers", new HyperbolicTangentActivationFunction(), 3, 3, 2, 3);

n.layers[0].Adjacencies[0][0].Weight = .75D;

n.layers[0].Adjacencies[1][0].Weight = -.75D;

n.layers[0].Adjacencies[2][0].Weight = .75D;

n.layers[0].Adjacencies[0][1].Weight = -.8D;

n.layers[0].Adjacencies[1][1].Weight = .8D;

n.layers[0].Adjacencies[2][1].Weight = -.9D;

n.layers[1].Nodes[0].Threshold = .5D;

n.layers[1].Nodes[1].Threshold = -.5D;

n.layers[1].Adjacencies[0][0].Weight = .4D;

n.layers[1].Adjacencies[0][1].Weight = -.5D;

n.layers[1].Adjacencies[0][2].Weight = .6D;

n.layers[1].Adjacencies[1][0].Weight = -.4D;

n.layers[1].Adjacencies[1][1].Weight = .5D;

n.layers[1].Adjacencies[1][2].Weight = -.6D;

n.layers[2].Nodes[0].Threshold = .1D;

n.layers[2].Nodes[1].Threshold = -.1D;

n.layers[2].Nodes[2].Threshold = 0D;

BackPropagationLearning l = new BackPropagationLearning(n);

double[] input = { .5D, -.5D, .75D };

double[] output = { 1D, -1D, 1D };

double error;

//Act

error = l.Step(new List<double>(input), new List<double>(output));

//Assert

//Total error E

Assert.AreEqual(.085699D, Math.Round(error, 6));

///////////////////////Verify Deltas///////////////////////////

//Input layer (0)

Assert.AreEqual( .018949D, Math.Round(l.deltas[0][0],6));

Assert.AreEqual(-.018949D, Math.Round(l.deltas[0][1], 6));

Assert.AreEqual( .019989D, Math.Round(l.deltas[0][2], 6));

//Hidden layer (1)

Assert.AreEqual( .014179D, Math.Round(l.deltas[1][0], 6));

Assert.AreEqual(-.010394D, Math.Round(l.deltas[1][1], 6));

//Output layer (2)

Assert.AreEqual( .154514D, Math.Round(l.deltas[2][0], 6));

Assert.AreEqual(-.083378D, Math.Round(l.deltas[2][1], 6));

Assert.AreEqual( .061175D, Math.Round(l.deltas[2][2], 6));

///////////////////////Verify Threshold Updates/////////////////

//No threshold updates for input layer

Assert.AreEqual(.0D, l.ThresholdUpdates[0][0]);

Assert.AreEqual(.0D, l.ThresholdUpdates[0][1]);

Assert.AreEqual(.0D, l.ThresholdUpdates[0][2]);

//Hidden Layer (1)

Assert.AreEqual( .001418D, Math.Round(l.ThresholdUpdates[1][0], 6));

Assert.AreEqual(-.001039D, Math.Round(l.ThresholdUpdates[1][1], 6));

//Output Layer (2)

Assert.AreEqual(.015451D, Math.Round(l.ThresholdUpdates[2][0], 6));

Assert.AreEqual(-.008338D, Math.Round(l.ThresholdUpdates[2][1], 6));

Assert.AreEqual(.006117D, Math.Round(l.ThresholdUpdates[2][2], 6));

///////////////////////Verify Weight Updates/////////////////

//Input layer (0) - Hidden Layer (1)

Assert.AreEqual(.000709D, Math.Round(l.WeightUpdates[0][0][0], 6));

Assert.AreEqual(-.000520D, Math.Round(l.WeightUpdates[0][0][1], 6));

Assert.AreEqual(-.000709D, Math.Round(l.WeightUpdates[0][1][0], 6));

Assert.AreEqual(.000520, Math.Round(l.WeightUpdates[0][1][1], 6));

Assert.AreEqual(.001063D, Math.Round(l.WeightUpdates[0][2][0], 6));

Assert.AreEqual(-.000780D, Math.Round(l.WeightUpdates[0][2][1], 6));

//Hidden Layer (1) - Output Layer (2)

Assert.AreEqual( .014649D, Math.Round(l.WeightUpdates[1][0][0], 6));

Assert.AreEqual(-.007905D, Math.Round(l.WeightUpdates[1][0][1], 6));

Assert.AreEqual( .005800D, Math.Round(l.WeightUpdates[1][0][2], 6));

Assert.AreEqual(-.014868D, Math.Round(l.WeightUpdates[1][1][0], 6));

Assert.AreEqual( .008023D, Math.Round(l.WeightUpdates[1][1][1], 6));

Assert.AreEqual(-.005886D, Math.Round(l.WeightUpdates[1][1][2], 6));

///////////////////////Verify Final Weights and Thresholds/////////////////

//Input layer (0) - Hidden Layer (1)

Assert.AreEqual(0.750709D, Math.Round(n.layers[0].Adjacencies[0][0].Weight, 6));

Assert.AreEqual(-0.800520D, Math.Round(n.layers[0].Adjacencies[0][1].Weight, 6));

Assert.AreEqual(-0.750709D, Math.Round(n.layers[0].Adjacencies[1][0].Weight, 6));

Assert.AreEqual(0.800520D, Math.Round(n.layers[0].Adjacencies[1][1].Weight, 6));

Assert.AreEqual(0.751063D, Math.Round(n.layers[0].Adjacencies[2][0].Weight, 6));

Assert.AreEqual(-0.900780D, Math.Round(n.layers[0].Adjacencies[2][1].Weight, 6));

//Hidden Layer (1)

Assert.AreEqual(0.501418D, Math.Round(n.layers[1].Nodes[0].Threshold, 6));

Assert.AreEqual(-0.501039D, Math.Round(n.layers[1].Nodes[1].Threshold, 6));

//Hidden Layer (1) - Output layer (2)

Assert.AreEqual(0.414649D, Math.Round(n.layers[1].Adjacencies[0][0].Weight, 6));

Assert.AreEqual(-0.507905D, Math.Round(n.layers[1].Adjacencies[0][1].Weight, 6));

Assert.AreEqual(0.605800D, Math.Round(n.layers[1].Adjacencies[0][2].Weight, 6));

Assert.AreEqual(-0.414868D, Math.Round(n.layers[1].Adjacencies[1][0].Weight, 6));

Assert.AreEqual(0.508023D, Math.Round(n.layers[1].Adjacencies[1][1].Weight, 6));

Assert.AreEqual(-0.605886D, Math.Round(n.layers[1].Adjacencies[1][2].Weight, 6));

//Output Layer (2)

Assert.AreEqual(0.115451D, Math.Round(n.layers[2].Nodes[0].Threshold, 6));

Assert.AreEqual(-0.108338D, Math.Round(n.layers[2].Nodes[1].Threshold, 6));

Assert.AreEqual(0.006117D, Math.Round(n.layers[2].Nodes[2].Threshold, 6));

///////////////////////Verify Consistent Movement/////////////////

double lasterr = error;

for (int i = 0; i < 10; i++)

{

lasterr = error;

error = l.Step(new List<double>(input), new List<double>(output));

Assert.IsTrue(lasterr >= error);

}

-->

Windows Symbolic Links and File Sharing

2012-10-05T14:25:00.000-07:00

Symbolic Links, Hard Links, and Junction Points are useful tools for an experienced administrator to manipulate the file system to bring things in different locations under a single logical tree. Symbolic Links have been posssible to create since Windows 2000 (using the linkd.exe, mountvol.exe, and delrp.exe utilities from the Windows 2000 Resource kit), but only in the most recent versions of Windows (Windows Vista, Windows 7, Windows 8, Windows Server 2008, Windows Server 2008 R2, and Windows Server 2012) has the mklink utility been provided with the OS to create symbolic links. The original functionality in Windows 2000 was called the NTFS reparse point, which is basically just a file system object with extended information. This functionality isn't particularly well known, but it is useful in a wide variety of scenarios.

By default, users in the local administrators group have the ability to create symbolic links, but this permission can be granted by modifying the "Create Symbolic Links" user right:

There are a lot of potential uses for symbolic links, but some of the ones that I've dealt with are:

A static content directory in IIS needs to be available at multiple points in the site/application tree so that pages do not need to use absolute or relative references that point outside of the current logical directory or a particular piece of content needs to appear at multiple points in the site.
A user wants to double click a folder in the "My Documents" folder and be able to access the root of the C: drive (note that this is not a shortcut)
A developer needs to create a separate tree that includes files used for unit tests, but wants changes to the "primary" source tree to be instantaneously propagated to the tree used for unit testing (I've used this one myself during the development of complex Latex/pdf documents that need to be generated with ASP.Net)
Scripts need to be developed that access other parts of the filesystem from a single folder.
The same file needs to be accessed by multiple names

Now that we understand what they are and some of the use cases, let's look at how to create/use them. The basic syntax of the command is

c:\symlinktest>mklink

Creates a symbolic link.

MKLINK [[/D] | [/H] | [/J]] Link Target

        /D      Creates a directory symbolic link.  Default is a file

                symbolic link.

        /H      Creates a hard link instead of a symbolic link.

        /J      Creates a Directory Junction.

        Link    specifies the new symbolic link name.

        Target  specifies the path (relative or absolute) that the new link

                refers to.

Directory Hard Link

Long story short, you can't make a directory hard link. The mklink utility doesn't give a particular error, just "access is denied:"

c:\symlinktest>mklink /H test2 test
Access is denied.

c:\symlinktest>mklink /D /H test2 test
Access is denied.

Why can't a directory hard link be created? Because each directory has a pointer to its parent directory. In the case of a hard link, there would be ambiguity as to what the parent directory actually is because the hard link is actually another entry in the master file table (MFT). Directory hard links would also create the possibility of infinite recursion through the filesystem.

File Hard Link

Hard links create an additional pointer in the master file table (MFT) for a particular file, but with a different name. Hard links can be deleted, but the file will not be deleted until all of the hard links that reference the file are gone. When a hard link is created, the target has to exist (because it points to an existing file). Additionally, the file and the hard link have to exist on the same volume (this is the same as the Linux/UNIX world since Windows is POSIX compliant).

In this example, I created a file (important.txt) that I want to edit by another name (important2.txt),

c:\symlinktest>mklink /H important2.txt important.txt
Hardlink created for important2.txt <<===>> important.txt

Looking at the directory, the files look the same (and they are the same). Permissions are set on the file object, so changing the permissions on one hard link changes it on both of them.

Directory of c:\symlinktest

10/05/2012 02:17 PM    <DIR>          .
10/05/2012 02:17 PM    <DIR>          ..
10/05/2012 02:10 PM                 0 important.txt
10/05/2012 02:10 PM                 0 important2.txt
10/05/2012 02:02 PM    <SYMLINKD>     system32 [c:\windows\system32]
10/05/2012 01:57 PM    <DIR>          test
10/05/2012 01:58 PM    <SYMLINKD>     test2 [test]
               2 File(s)              0 bytes
               5 Dir(s) 694,212,771,840 bytes free

From a file sharing perspective, a hard link can exist to a file outside of the root of the share (but on the same storage volume on the server) and be accessible to users who have share/NTFS permissions to the file.

File Symbolic Link

NTFS file symbolic links are filesystem objects that are different from shortcuts (.lnk files), but point to either a relative or absolute path of a file. These links are created without the /H option with mklink,

c:\symlinktest>mklink important2.txt ..\symlinktest\important.txt
symbolic link created for important2.txt <<===>> ..\symlinktest\important.txt

c:\symlinktest>mklink important3.txt c:\symlinktest\important.txt
symbolic link created for important3.txt <<===>> c:\symlinktest\important.txt

Looking at the links,
...
10/05/2012 02:10 PM                 0 important.txt
10/05/2012 02:33 PM    <SYMLINK>      important2.txt [..\symlinktest\important.txt]
10/05/2012 02:33 PM    <SYMLINK>      important3.txt [c:\symlinktest\important.txt]
...

In Windows Explorer, these look like shortcuts, but aren't.

The properties are greyed out and cannot be edited. Symbolic links do not have to point to a target that exists and can point between drive letters and volumes. From a file sharing perspective, a symbolic link that points outside of the shared directory structure is inaccessible (Errors like 'Operation not supported' and 'the network path was not found' are common). Relative path symbolic links that point to files within the same share structure are accessible, as are symbolic links that point to UNC shares, but this has a tendency to create a lot of complexity in a file sharing environment. Additionally, remote-remote and remote-local symlinks are disabled on clients by default. See below for more details.

Directory Symbolic Link

The Directory symbolic link is essentially the same as the file symbolic link, but the target is a directory instead. Here is a brief example:

c:\symlinktest>mklink /D test2 test
symbolic link created for test2 <<===>> test

Directory of c:\symlinktest

10/05/2012 02:17 PM    <DIR>          .
10/05/2012 02:17 PM    <DIR>          ..
10/05/2012 02:10 PM                 0 important.txt
10/05/2012 02:10 PM                 0 important2.txt
10/05/2012 01:57 PM    <DIR>          test
10/05/2012 01:58 PM    <SYMLINKD>     test2 [test]
               2 File(s)              0 bytes
               4 Dir(s) 694,212,771,840 bytes free

Directory Junction Point

Junction points act more like hard links than symbolic links and are specifically for directories. They are created with the /J option of mklink and they can be traversed just like a directory. Remote-Remote symbolic links (see below) do not need to be enabled because it is not technically a symbolic link, but rather a file object that allows a directory to be linked at multiple points in the filesystem.

As an example, we will create a ZZZ folder that traverses to c:\windows:

c:\symlinktest>mklink /J ZZZ c:\windows
Junction created for ZZZ <<===>> c:\windows

c:\symlinktest>dir
...
10/05/2012 03:10 PM    <JUNCTION>     ZZZ [c:\windows]
               4 File(s)     19,577,576 bytes
               8 Dir(s) 694,211,403,776 bytes free

Accessing this junction from a share allows the server's c:\windows directory to be traversed, even though only the c:\symlinktest folder is shared. If the user has administrative permissions, then they can change files remotely.

If you make an error and create one for a file, it'll create, but won't work:

c:\symlinktest>mklink /J test.exe FEPInstall.exe
Junction created for test.exe <<===>> FEPInstall.exe

Accessing the test.exe folder will fail with "The directory name is invalid"

Deleting a Symbolic Link

You can delete a symbolic link or junction point the same way that any other file can be deleted (Remote-Item in PowerShell, del in cmd, or right click -> delete in Windows explorer).

Dealing with "The symbolic link cannot be followed because its type is disabled"

As a security measure, symbolic links on file shares are not followed. You can see what the current behavior is by running fsutil behavior query SymlinkEvaluation:

Here is the default behavior for symlinks:

c:\symlinktest>fsutil behavior query SymlinkEvaluation
Local to local symbolic links are enabled.
Local to remote symbolic links are enabled.
Remote to local symbolic links are disabled.
Remote to remote symbolic links are disabled.

To enable remote-remote symlinks, run fsutil behavior set SymlinkEvaluation R2R:1

c:\symlinktest>fsutil behavior set SymlinkEvaluation R2R:1

c:\symlinktest>fsutil behavior query SymlinkEvaluation
Local to local symbolic links are enabled.
Local to remote symbolic links are enabled.
Remote to local symbolic links are disabled.
Remote to remote symbolic links are enabled.

I wouldn't recommend enabling remote to local symlinks, but this is also possible.

Dealing with "Location is Not Available"

"Location is not available: %s refers to a location that is unavailable. It could be on a hard drive on this computer, or on a network. Check to make sure that the disk is properly inserted, or that you are connected to the Internet or your network, and then try again. If it still cannot be located, the information might have been moved to a different location"

If you get this error, your link isn't pointing to the right place, or the user has a different view (ex. mounting a share farther down in the tree than he/she should be) than the link was put in place to support. Look at the link and make sure it points to the right place. Access the share as the user and see if the path that the link references actually exists from the end user's perspective.

How to Mount a DFS Share in Linux

2012-10-03T11:57:00.000-07:00

Microsoft Distributed File System (DFS) is a technology that allows multiple servers to host a single file share (providing fault tolerance and performance enhancement for multi-site Active Directory topologies). It is typically used in conjunction with a replication technology such as DFS-R (R is for Replication). Though these features have similar names, you don't have to have DFS to use DFS-R and vice versa (A typical use case might be the desire to replicate folders between servers, but not share them, which is accomplished via DFS-R without using a DFS namespace).

Windows clients (Windows XP SP3, Windows Vista, Windows 7, and Windows 8) and servers (Windows Server 2003 SP2, Windows Server 2008, Windows Server 2008 R2, and Windows Server 2012) mount shares in a DFS namespace seamlessly, but the same isn't true for Mac OS (earlier than Lion). Linux systems have the capability to mount DFS, but they require some additional configuration above and beyond issuing a 'mount' command. In the modern versions of Linux, Windows file shares are mounted using the kernel Common Internet File System (CIFS) driver.

In order to mount a DFS volume, the following packages need to be available (these may vary per platform, but are consistent on RedHat/CentOS and SUSE):

cifs-utils (specifically mount.cifs and cifs.upcall)
keyutils

Based on testing in RedHat Enterprise Linux, samba is NOT required to mount a CIFS share hosted in a DFS namespace. A small piece of configuration is necessary to mount a DFS share on Linux. The configuration needs to be added to the keyutils configuration file (bolded below in the list of files included with the package),

/bin/keyctl
/etc/request-key.conf
/etc/request-key.d
/sbin/request-key
/usr/share/doc/keyutils-1.4
/usr/share/doc/keyutils-1.4/LICENCE.GPL
/usr/share/doc/keyutils-1.4/README
/usr/share/keyutils
/usr/share/keyutils/request-key-debug.sh
/usr/share/man/man1/keyctl.1.gz
/usr/share/man/man5/request-key.conf.5.gz
/usr/share/man/man8/request-key.8.gz

The following lines need to be added to the end of the /etc/request-key.conf file:

create cifs.spnego * * /usr/sbin/cifs.upcall -c %k
create dns_resolver * * /usr/sbin/cifs.upcall %k

Note that if you have a newer version of cifs-utils, the -c above is not necessary (and is actually a deprecated option). After adding the lines to the request-key.conf file, you should now be able to mount the share:

# mount -t cifs //mydfsdomain.lan/namespaceroot/sharedfolder /mnt -o username=windowsuser

If all goes well, this asks for a password and mounts. Note that mydfsdomain.lan should be your Active Directory name namespaceroot should be the root that you create using DFS Management in MMC. Note that all read/write/modify operations will be performed as the "windowsuser" user in the example above.

How to Dual Boot Windows 8 and Windows 7

2012-09-27T22:43:00.000-07:00

Generally when a new operating system version is released, people are timid about making the leap to upgrade to the new version. Many people want to take some time to try the new version while retaining access to their old files and folders on the older release.

This post will cover how to dual boot Windows 8 RTM and Windows 7. Before trying this procedure, be sure to back up your data in case something goes wrong.

To start out, I have a Windows 7 installation on a 120 GB disk. My Windows 7 files and data are only taking 20 GB or so and I want to create a partition to install Windows 8 using 50 GB (Note that the minimum recommended size is 20 GB, but I usually prefer 40-50GB). First, access the disk manager using mmc or by typing the following

Windows key + R to access the run prompt
Enter "diskmgmt.msc" and press enter

You can also use the search bar in the start menu,

Once you are in disk manager, right click the partition that belongs to Windows 7 and click "Shrink"

After you click shrink, specify 50 GB or whatever size you actually want (20 GB is the recommended minimum). Note that 50 GB = 51200 MB.

Click Shrink to start the shrinkoperation on the volume

Now that we have space for a 50 GB volume, restart and boot off of the Windows 8 DVD or Bootable USB Media.

We go through the installation procedure as follows. Select your regional settings and click next,

Then click "Install now"

Accept the EULA and start a Custom Installation,

Select the 50 GB of unallocated space that we created and click next

Wait for the install to complete (or grab a cup of coffee or a donut)...

After the reboot, the boot loader should show the older Windows release and the new Windows 8 option that we just installed.

You might also like
Navigating the Windows 8 User Interface
How to Create USB Windows 8 Installation Media (also works for Windows Server 2012, Windows 7, Windows Server 2008 R2)

How to Create Bootable USB Installation Media for Windows 8 and Windows Server 2012

2012-09-27T22:24:00.000-07:00

There are a number of reasons to install Windows off of USB Media instead of using a DVD. Some of them include

Installing from USB Media is usually faster than an optical drive
Some newer PCs (like an Ultrabook that I just bought) do not have optical drives of any kind
Perhaps you ran out of blank DVDs and the store is closed...

Since this procedure involves formatting a flash drive, the reader is cautioned to back up all of the data on the system (you'll see why later) and to make sure that anything that needs to be kept is backed up off of the flash drive. To perform this process, you will need access to Windows 8 media (either in ISO format or DVD) and you will also need a flash drive (> 6 GB).

.

To start the process, open a command prompt with an administrative authentication token,

If your command prompt is not pinned to the start menu, simply type cmd into the search bar, then right click the command prompt that appears and select "run as administrator." Then type diskpart and press enter. You should be here at this point,

Enter "list disk" and hit enter. It should be pretty clear form the sizes which disk is the flash drive. For me, it is Disk 3. Be sure to select the right disk, or you will destroy all of the data on one of the other system disks (this would be very bad for most people). For me, it is Disk 3 because I am using an 8 GB flash drive

Next, type "select disk 3" while substituting 3 for the right disk number on your system.

Now type "Clean" and hit enter. At this point all of the data will be gone from the disk (though it could still be recovered by a forensic analyst/data recovery company)

If you selected the wrong disk, you have a big problem now... If you selected the right disk, you can create a new partition by typing "create partition primary" and hit enter

This should be partition 1, but type "list partition" to be sure.

Now select partition 1 and format the partition as a FAT partition.

Mark the partition as active and we are ready to copy files over,

Mount the Windows 8 or Windows Server 2012 ISO and copy all of the files on the ISO to the newly created/formatted volume.

At this point, you now have bootable Windows 8 or Windows Server 2012 USB media.

Adding Multiple Mail Accounts in Microsoft Mail for Windows 8

2012-09-26T11:57:00.001-07:00

The mail application in the release preview versions of Windows did not support adding mail accounts for IMAP/POP accounts. In the Windows 8 RTM release, the Mail App supports adding e-mail accounts for Hotmail, Live, MSN, Exchange, Office365, and Outlook.com, Google/Gmail, Yahoo, and a custom account type that allows IMAP/POP accounts to be added. To add an account, access the settings by pointing your mouse or tapping the top right or bottom right corners of the App

Note that you have to be in the Mail app to see the right settings. Then click Accounts

From here, you can select the right type of account to add

If you are adding an IMAP/POP account to Windows Mail in Windows 8, you can either enter your e-mail/password and have the app try to find the right settings, or you can view more details and specify server names/ports and SSL/TLS settings.

That's all there is to it.

You might also like:
Navigating the Windows 8 User Interface

How To Set Up A New Active Directory 2012 Domain

2012-09-23T16:14:00.002-07:00

Establishing a new Active Directory domain in Windows Server 2012 involves planning along the following areas:

Choice of forest/root domain names
Number, type, location, and sizing of domain controllers
Special administrative issues that may require multiple domains/forests
Administration model/OU structure
Planning sites/subnets/replication

These are all issues that needed to be planned in previous Active Directory versions, but new with Windows Server 2012 is the ability to install the Active Directory Domain Services role without installing the DNS Server role. This is useful because some organizations use a different platform for DNS (such as ISC's Bind). There is also a possibility that there are more need for Active Directory/LDAP processing than DNS services and having the DNS server role installed/running takes significant system resources that might be better used for other processing.

Like Windows Server 2008 and Windows Server 2008 R2, installation is a two step process:

Install the Active Directory Domain Services (ADDS) Role
Promote the first DC in the forest

The first step in the process is determining what we are doing. In this case, we want to create a new forest and a new root domain (as opposed to adding a DC to another domain or adding a new subdomain).

Next determine the domain and forest functional level for the new Active Directory Forest/Domain. A rule of thumb is to not choose a level that is greater than the oldest OS version expected to be used for the domain controller (i.e. don;t pick 2008 or 2008 R2 if you plan to promote a 2003 domain controller). Note that this setting has no implications for the types of clients/non DC servers that can be joined to the domain. This step also allows you to decide whether to install the DNS role and create AD-integrated zones for the new forest/domain. Most native Windows environments will need at least 1 DNS server (and preferably more to not create a single point of failure). The first DC promoted in the forest has to be a global catalog server and cannot be a read-only domain controller (RODC). Finally, the directory services restore mode password should be documented in the event that the directory needs to be restored from backup.

If you choose to install the DNS role, you may have the option to create a delegation for the server to host the piece of the DNS namespace that is covered by the forest/domain. Most companies that invent a top-level domain will not have the ability to do this. Organizations that use a subdomain of their existing DNS namespace or use a real registered domain may have the ability to create the delegation.

Next, choose a NetBIOS domain name to correspond with the new forest/domain. This is a single label that is typically the first component of the domain name (ex ad.microsoft.com -> AD, mikesblog.lan -> MIKESBLOG).

Finally, configure the storage location for the Active Directory database and transaction logs. For domains expecting heavy DC use, it may be advantageous to place the database and log files on separate high speed volumes. The SYSVOL folder should be placed in a location with adequate space to store group policies, logon scripts, and anything else that the domain admins want to replicate to all of the DCs. This may need to be moved to external storage if the organization heavily utilizes group policy software installation.

Finally, you can review the options and generate a script for deploying DCs with the same configuration.

Below is an example of the script generated to create a new forest using the Install-ADDSForest cmdlet.

After that, your new domain is set up, but the domain controller that you just created is a single point of failure. You should probably install a second one.

Navigating the Windows 8 User Interface

2012-09-18T09:18:00.000-07:00

During the beta/release candidate phases of Windows 8 development, I couldn't help thinking "Is Microsoft insane? The new user interface totally sucks..." Since I installed the Windows 8 Enterprise RTM and have used it for a few weeks, I've reversed my position on how I felt about the user interface.

I dare say that Microsoft has created one of the cleanest and easy to use interfaces I've seen in any version of Windows (or any other operating system, including Apple's iOS). Using the Windows 8 interface comes down to managing the learning curve because there are a number of radical changes from the traditional Windows OS. I'll cover a few of the biggest changes and how to adjust to the new user interface.

Where did my start menu go?

The start menu is radically different in Windows 8, but you still access the start menu in almost the same way that it is accessed in Windows 7. Instead of having a "start orb", you simply move the mouse to the lower left corner of the screen:

The start menu looks a lot different too, pinned tiles are placed on the start menu for commonly used applications ("apps" nowadays).

Should you want to see everything on the start menu, right click and select "All Apps."

I've found that I almost never do this anymore. Instead, I use the new "Search" feature of the menu that pops up when you move your mouse to either the top-right or bottom-right corner.

With the search feature, a few characters of the application's name (or the full name of anything within the %PATH% variable) can be found.

What about metro apps?

Metro style apps are the new paradigm for Windows development. They are different from traditional Windowed applications because they provide a full-screen, always-on, immersive, user experience. Windows Phone users will see a striking similarity between Windows Phone 7 and Windows 8.

Despite the advertising hype, these apps do continue to consume resources in the background (though their resources are minimized using a few tricks with the Windows kernel) and the biggest pain point that I had was learning how to close a "metro" style app. The easiest way is to move the mouse to the top middle of the screen (so that the cursor becomes a hand), then drag the top of the app to the bottom of the screen.

Open Apps can be switched using alt+tab or by moving the mouse to the top-left corner.

How do I shut down/restart/put my computer to sleep?

It is easy to hut down, restart, or sleep the computer by accessing the settings using the top-right or bottom-right corners of the screen,

The bottom line...

Like it or hate it, the new user interface is here to stay. It takes some getting used to, but it is a significant improvement on the old start menu paradigm and the new user interface provides a decent experience on both the PC and tablet.

You might also like:
Adding Multiple Mail Accounts in Microsoft Mail for Windows 8

Get HyperTerminal in Windows 8 and Windows Server 2012

2012-09-14T11:20:00.000-07:00

Who needs HyperTerminal nowadays?

If you run any of the myriad of devices that use any form of serial connection (ex. RS-232) or an application that is built against the HyperTerminal library, you likely need a console emulator like HyperTerminal or an alternative. A lot of devices that still use serial cables include networking devices (routers, switches, load balancers, etc), CRAC (Computer Room Air Coonditioner) units, Power distribution panels, bank vaults, and many others.

HyperTerminal is the classic serial console communication tool that was provided with Windows up until Windows XP and Windows Server 2003. In windows Vista and Windows 7, Hyperterminal still works, but you have to copy it from a Windows XP or Windows Server 2003 machine (it is easy to just build a VM for this purpose and destroy it later). These two files need to be taken from the Windows XP/Server 2003 installation:

c:\Program Files\Windows NT\Hypertrm.exe
c:\Windows\System32\hypertrm.dll

As long as the hypertrm.exe executable can find the hypertrm.dll library, you can install this anywhere on a new Windows 8 system, though it might be logical to place the files in c:\program files (x86)\HyperTerminal\ and create shortcuts or place them in c:\Windows\System32 (this is part of the default system path, so you can simply run hypertrm from the run command or the search menu).

If you don't have access to a Windows XP or Server 2003 installation, don't worry, there are a number of alternative serial emulators for Windows 8 and Windows Server 2012.

Mounting an ISO Image in Windows 8

2012-09-14T10:49:00.000-07:00

There are a number of things that have been relatively standard in the open source world and have left Windows users to ask the "why can't windows do this yet?" Even through the service packs for Windows 7 and Windows Server 2008 R2, it was impossible to mount/extract an ISO image without a tool like Virtual CloneDrive or Daemon Tools. In Windows Server 2008 R2, you could mount an ISO for a Hyper-V guest using the Hyper-V management tools, but not for the host OS.

Microsoft made this feature really easy in Windows 8 and Windows Server 2012. Simply navigate to the folder with the downloaded ISO image, right click the image, and click "Mount"

Windows automatically assigns a drive letter and drops you into the drive root in Explorer:

If you are done with the ISO and are ready to unmount it, simply eject the drive by right clicking the appropriate drive letter and selecting "Eject."

That's all there is to it. Microsoft did a good job in Windows 8 making this a clean process... It's about time...

awk Script To Combine Lines Of A File or Output

2012-09-10T12:28:00.000-07:00

I ran into a unique problem where I needed to combine 3 lines of numbers into a single line separated by spaces (ironically again, to use awk to generate C# code that does something interesting). I found posts that gave me an idea for how to accomplish the task; however the examples that I found don't cover what to do if there are no indications of the next "record" to parse (i.e. some sort of starting delimiter):

BEGIN {
   i=0
   l=""
}
{
   if (i == 3) {
      i=1
      print l
      l=$0
   }
   else {
      if (i == 0) {
         l=$0
      }
      else {
         l=l" "$0
      }
      i=i+1
   }
}
END {print l}

This script can be generalized to combine N lines of output by replacing the line:

if (i == 3) {

with

if (i == N) {

where N is a variable or the desired number of lines to combine. This can eliminate a tedious job like deleting newlines from a file (particularly when copying some PDF tables into a text editor like notepad, emacs, or vi) and make a task part of a larger and more automated process. Replacing $0 with $1..$n will only concatenate the nth field. This is easy to run on UNIX/Linux (since awk is included with most distributions), but on Windows it requires Cygwin (which I use) or a native port of the GNU awk utility (check out the GNUWin32 project on SourceForge).

How to Determine Installed Sharepoint 2010 SKU

2012-09-10T11:58:00.001-07:00

I had an issue arise where I had to determine whether a SharePoint 2010 server was installed with SharePoint 2010 Foundation or one of the SharePoint Server products (Standard/Enterprise). After a lot of searching, I found an MSDN article that shows how to perform the task in .Net, but this seemed fairly useless because I didn't want to have to compile an application in C# just to determine the version. Below is a PowerShell port of the application demonstrated on MSDN:

# SharePoint 2010 SKU List Script
# Developed/Copyright September 2012 by Mike Burr
# Provided AS-IS With No warranty
# Based loosely on http://msdn.microsoft.com/en-us/library/ff721969.aspx
#

$featureindex = @{}
$featureindex["BEED1F75-C398-4447-AEF1-E66E1F0DF91E"] = "SharePoint Foundation 2010"
$featureindex["1328E89E-7EC8-4F7E-809E-7E945796E511"] = "Search Server Express 2010"

$featureindex["B2C0B444-3914-4ACB-A0B8-7CF50A8F7AA0"] = "SharePoint Server 2010 Standard Trial"
$featureindex["3FDFBCC8-B3E4-4482-91FA-122C6432805C"] = "SharePoint Server 2010 Standard"
$featureindex["88BED06D-8C6B-4E62-AB01-546D6005FE97"] = "SharePoint Server 2010 Enterprise Trial"
$featureindex["D5595F62-449B-4061-B0B2-0CBAD410BB51"] = "SharePoint Server 2010 Enterprise"

$featureindex["BC4C1C97-9013-4033-A0DD-9DC9E6D6C887"] = "Search Server 2010 Trial"
$featureindex["08460AA2-A176-442C-BDCA-26928704D80B"] = "Search Server 2010"

$featureindex["84902853-59F6-4B20-BC7C-DE4F419FEFAD"] = "Project Server 2010 Trial"
$featureindex["ED21638F-97FF-4A65-AD9B-6889B93065E2"] = "Project Server 2010"

$featureindex["926E4E17-087B-47D1-8BD7-91A394BC6196"] = "Office Web Companions 2010"

#Clean method for getting all of the registry value names in a particular key
$path = "HKLM:\SOFTWARE\microsoft\Shared Tools\Web Server Extensions\14.0\WSS\InstalledProducts"

$regvalues = Get-Item $path | select -ExpandProperty property

# Cycle through all of the registry values

foreach ($i in $regvalues) {
   Write-Debug ("i: " + $i)
   $value = (Get-ItemProperty -Path $path).$i
   Write-Debug ("Value: " + $value)

   #For each value that we look at, are any of the corresponding SharePoint SKU GUIDs represented?
   foreach ($feature in $featureindex.Keys) {

      #If these are equal, then we have a particular SKU installed
      write-Debug ($value + " == " + $feature + " ?")
      if ($value -eq $feature) {
         write-output ($feature + " --> " + $featureindex[$feature] + " is installed")
      }
   }
}

There are third interesting pieces to the script. One is the easy way to iterate through the registry values contained in a particular key. The second is how to get the actual value (once you have the name). The third is that you can determine which SKUs are installed by using the InstalledProducts registry key created when SharePoint 2010 is installed.

To run the script, all that is needed is read access to the appropriate registry key and its values.

Facebook is a Bad Investment

2012-05-22T20:35:00.002-07:00

Nasdaq is receiving a lot of bad press regarding the Facebook IPO and problems with their trading platform. Some of this is justified because the IPO was interrupted and trades were not executed or cancelled successfully, but ultimately it is not the cause of Facebook's meltdown in the open market. The bottom line is that Facebook is a bad investment and it was never worth the initial valuation of $100 billion. At the time of this writing, FB has lost more than 20% of its value since the IPO started.

There are a lot of people in the world that like Facebook as a social media platform. It has an immense customer base including individuals, governments, and organizations. Many companies regularly use Facebook in their marketing and work to socially connect their companies with customers. Facebook also develops a lot of ad impressions and has taken steps to build a platform that developers can use to make money. Many pages have a "like"button or functionality to share something on Facebook. For all that it provides, Facebook fails the test of both long term and short term ability to serve shareholders (both from a growth and income standpoint).

Facebook launched its IPO in both a declining profit and declining revenue scenario. Facebook, as a product, is in its decline stage (and it reached a peak a couple of years ago). Although people will continue to use it, it will become easier for competitors to chip away at Facebook's regular user base. Coupled with Facebook's horrific ad performance and their lack of partnership with other organizations with regard to its demographic data, this spells a long term underperforming stock and an almost permanent sell (or short sell) rating.

When you look at Facebook compared to other technology giants (like Apple, Google, and Microsoft), ask "What is Facebook actually selling?" Right now, they are selling underperforming ad space and a couple of digital products with limited marketability. They have no real long term product offerings and their sources of revenue are difficult to imagine without Facebook compromising customer privacy.

Ultimately, individuals and organizations need to communicate with their financial professionals to determine the suitability of any investment for their portfolio. My opinion of Facebook is that they are a strong sell (and a never buy), but we'll see if the market proves me wrong.

Frame Relay Switching Lab Using Multiple Switches (Tunnels)

2012-05-16T14:33:00.003-07:00

Frame Relay is an important exam topic in the CCNA (mostly ICND2 exam), CCNP, and CCIE certifications for the routing and switching track. This post builds on the frame relay switching post that showed configuration of partially meshed and fully meshed permanent virtual circuits (PVCs). There are a lot of examples that show how to configure a single router as a frame-relay switch, but very few that show how to configure multiple provider devices to enable an end-to-end frame relay network. This lab, built in GNS3/Dynamips, demonstrates 2 customers being connected using a provider network with an IP backbone and frame-relay switching on the edge devices. Configurations shown are Cisco 3725 routers running Cisco IOS 12.4(15)T7.

The topology below shows the entire network as it has been configured. PE1 and PE2 perform frame-relay switching for Customer 1 (Cust1) and Customer 2 (Cust2). The connections between PE1, P, and PE2 utilize HDLC (Cisco's default for serial interfaces) and IP. The internal provider (P) router has no knowledge of any of the frame relay configuration on the provider edge routers (PE1 and PE2). The provider network runs OSPF to maintain internal IP connectivity. A GRE-IP tunnel is configured between PE1 and PE2 to encapsulate the frame-relay packets as they cross the frame relay cloud.

The main interesting points of configuration exist on the provider network routers: PE1, P, and PE2. The simplest configuration is on the P router where the IP addresses and routing protocol (OSPF in this case) are configured:

interface Serial0/0
ip address 172.16.1.2 255.255.255.252
!
interface Serial0/1
ip address 172.16.2.2 255.255.255.252
!
router ospf 1
log-adjacency-changes
network 172.16.0.0 0.0.255.255 area 0
!

On router PE1, we perform some frame-relay configuration for the customer interfaces and specify that the frame-relay encapsulated frames should be tunneled. Customer 1 and 2 have a single point-to-point PVC defined between the customer edge routers connecting to PE1 and PE2:

!
frame-relay switching
!
interface Serial0/1
no ip address
encapsulation frame-relay
clock rate 2000000
frame-relay intf-type dce
frame-relay route 100 interface Tunnel0 200
!
interface Serial0/2
no ip address
encapsulation frame-relay
clock rate 2000000
frame-relay intf-type dce
frame-relay route 150 interface Tunnel0 300
!

We also configure one side of the GRE-IP tunnel:

interface Tunnel0
no ip address
tunnel source 172.16.1.1
tunnel destination 172.16.2.1
!

Note that since GRE-IP is the default tunnel mode, the tunnel mode gre ip command is not necessary. Finally, the internal provider connection and the routing protocol are configured. If multiple paths existed between PE1 and PE2, a tunnel source using a loopback interface would be appropriate, but this case does not require loopback interfaces. Additionally, with the different tunnel choices, GRE over IP or GRE over IPv6 are the main tunnel types that will work for encapsulating frame relay over IP.

PE1(config-if)#tunnel mode ?
aurp    AURP TunnelTalk AppleTalk encapsulation
cayman Cayman TunnelTalk AppleTalk encapsulation
dvmrp   DVMRP multicast tunnel
eon     EON compatible CLNS tunnel
gre     generic route encapsulation protocol
ipip    IP over IP encapsulation
ipsec   IPSec tunnel encapsulation
iptalk Apple IPTalk encapsulation
ipv6    Generic packet tunneling in IPv6
ipv6ip IPv6 over IP encapsulation
mpls    MPLS encapsulations
nos     IP over IP encapsulation (KA9Q/NOS compatible)
rbscp   RBSCP in IP tunnel

Here we can finish the discussion of PE1's configuration with its connection into the provider cloud and its routing protocol configuration:

!
interface Serial0/0
ip address 172.16.1.1 255.255.255.252
!
router ospf 1
log-adjacency-changes
network 172.16.0.0 0.0.255.255 area 0
!

The configuration is very similar on PE2. From the customer's perspective we are using global DLCI addressing:

!
frame-relay switching
!
interface Tunnel0
no ip address
tunnel source 172.16.2.1
tunnel destination 172.16.1.1
!
interface Serial0/0
ip address 172.16.2.1 255.255.255.252
!
interface Serial0/1
no ip address
encapsulation frame-relay
clock rate 2000000
frame-relay intf-type dce
frame-relay route 150 interface Tunnel0 300
!
interface Serial0/2
no ip address
encapsulation frame-relay
clock rate 2000000
frame-relay intf-type dce
frame-relay route 100 interface Tunnel0 200
!
router ospf 1
log-adjacency-changes
network 172.16.0.0 0.0.255.255 area 0
!

Verification of the setup from the customer side is easy, if traffic flows, then the PVC is working:

Cust1_CE1(config-subif)#do ping 192.168.0.2

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.0.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 4/37/112 ms

We can also see that the PVC is active and assigned:

Cust1_CE1(config-subif)#do show frame-relay pvc

PVC Statistics for interface Serial0/0 (Frame Relay DTE)

              Active     Inactive      Deleted       Static
Local          1            0            0            0
Switched       0            0            0            0
Unused         0            0            0            0

DLCI = 100, DLCI USAGE = LOCAL, PVC STATUS = ACTIVE, INTERFACE = Serial0/0.100

input pkts 104           output pkts 106          in bytes 31197
out bytes 31863          dropped pkts 0           in pkts dropped 0
out pkts dropped 0                out bytes dropped 0
in FECN pkts 0           in BECN pkts 0           out FECN pkts 0
out BECN pkts 0          in DE pkts 0             out DE pkts 0
out bcast pkts 91        out bcast bytes 30303
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
pvc create time 01:32:24, last time pvc status changed 01:31:54

On the provider routers, verification involves examining the status of the tunnel to make sure it is up/up (using the show interface tunnel 0 command) and the PVC (show frame-relay pvc). If there is a problem, the PVC will show as Inactive (where a PVC should show Active if it is working correctly).

See Also,
The Road to the CCIE

Setting Up a CCIE Switching Lab

2012-05-12T17:42:00.001-07:00

Layer 2 Ethernet switching is a very important part of a network engineer's job and is a critical component to successfully passing the CCIE written and practical exams. Setting up a switching lab is different from setting up a routing lab because switches cannot be effectively emulated in GNS3/Dynamips. In order to work with a switch IOS, the physical hardware is required because of the specialized ASICs involved. Some switch features can be configured on the Ethernet switching modules available on specific platforms.

The CCIE Routing and Switching exam uses Cisco 3560 switches running a 12.2 IP Services IOS release. In lieu of having access to one or more 3560s, Catalyst 3750 switches are a reasonable substitute and support many of the same features in a similar way. The Cisco 2960 platform is unsuitable for CCIE preparation because the universal image that is used does not support routing. Chassis based switches (4500 and 6500) can also be used, but in many cases the features supported are different from the standalone models.

So, now that we know more about the models that are suitable for preparation, two questions arise: "How many?" and "How should I connect them?" The answer lies in one of the subtleties of the CCIE exam. The goal of the exam is to develop an "internetwork model" according to a set of specifications. The goal of an internetwork model is to develop a scaled-down representation of a complex enterprise network. This gives rise to a lot of the patterns presented in Cisco's Service Oriented Network Architecture (SONA) and the traditional "core-distribution-access" approach. When dealing with these models, there are a couple of useful reference topologies, what I call the "3-switch topology," "core-distribution topology," and "distribution-access topology."

The simple 3-switch topology is a good starting pattern for learning spanning-tree protocol (STP) concepts and how the different forms of STP (STP, Multiple STP [MSTP], and Rapid STP [RSTP]) work together. Generalizing from a 3 switch model to a more complex topology is extremely simple once an individual understands the port roles and transitions that can occur in each type of STP. Utilizing multiple connections between the switches can also demonstrate port aggregation and link-level fault tolerance concepts.

The distribution-access topology provides a fault tolerant layout for many access scenarios. Each of the access layer switches (located at the bottom of the topology) have redundant connections to the distribution layer switches (the top two), a lot of scenarios involving access-layer routing and first-hop redundancy protocols (FHRP) can be demonstrated. Effective demonstrations of the PortFast and UplinkFast features are also possible using this topology.

The distribution-core topology can be used to work with a 4 switch core or redundant connections between the distribution and core layers of an area of the network. This topology can be configured to effectively demonstrate the UplinkFast and BackboneFast features of the switch IOS.

Additional topologies can be created to show activity in specific scenarios, but variations on the above topologies are all that is needed to demonstrate the configuration and behavior of most of the IOS switch features tested on the Cisco certification exams (CCNA, CCNP, and CCIE).

See Also,
The Road to the CCIE

Why Should a CCIE Read Standards?

2012-05-04T12:39:00.002-07:00

The stud/maintenance process for the CCIE exams is a long and difficult process. It takes an incredible amount of determination and self discipline to successfully understand everything that needs to be understood by a CCIE to effectively manage, evolve, and troubleshoot a complex Enterprise network. Cisco develops exam objectives loosely using Bloom's taxonomy. In the case of the CCIE Routing and Switching exam, all of the topics start with the word "Implement," "Evaluate," "Determine," and "Suggest."

There is probably a large potential debate about what the word "Implement" actually means, but I'll give my definition of what I think it means:

In the CCIE exam objectives, the combination of terms above around a given technology likely result int he following skills:
- Understand all of the concepts with a given technology and be able to unambiguously explain key terms and recall key facts
- Explain differences between the standard and the vendor-specific implementation
- Determine where a technology fits in a given network architecture
- Successfully design and optimize using the given feature/technology
- Evaluate a proposed design and develop positive and negative aspects and counter the proposal with design improvements
- Understand and code all of the related technology parameters and configuration options
- Explain how a technology works with technologies in higher, equal, and lower layers of the OSI model
- Evaluate the steps from start to finish to implement a change
- Explain the business impacts of making a specific change

Basically, in a one sentence summary: Effective operation at the CCIE level requires critical thinking, evaluation, and implementation using any feature/technology that exists in the networking industry regardless of whether Cisco implements it in a specific product or line of products.

This provides an insight into the title for this post. The level of knowledge required for the CCIE cannot be attained through memorization alone (in fact, memorization without context will likely hurt the CCIE candidate's chances for passing the exams on the first or second try). Simply, too much needs to be understood to be memorized. So why read standards?

Standards offer the CCIE candidate a complete, unabridged coverage of a technology or topic. Whether it is a set of RFCs released by the Internet Engineering Task Force or the authoritative electrical and mechanical specifications published by the IEEE, a topic can be comprehensively understood and applied by understanding the standards. Starting fromt he beginning, a CCIE can understand how a technology evolved and predict the future direction of a feature/technology.

Reading anything besides an authoritative standard is a paraphrase/summary of the standard. Depending on the source, a summary might be incorrect or provide an incomplete/misleading picture of what a technology/feature achieves. If a candidate uses the Cisco Press books to achieve the CCNA/CCNP, then they have gained a simplified understanding of the main technologies used in enterprise networks and may know the main features of technologies like Ethernet, Frame Relay, IP, TCP/UDP, but their level is likely at the "memorizing" level of understanding. This is not to say that the CCNA and CCNP aren't important, rather they help build some of the summary understanding required to successfully read/interpret a difficult IEEE standard or RFC.

Reading standards takes a candidate beyond memorization to a comprehensive end-to-end understanding of a protocol or technology. To achieve "Expert" status, this is really what is required. It's not easy, takes a lot of time and patience, but ultimately ends up in a level of knowledge where memorization is neither helpful nor required. Instead of memorizing the fields of an Ethernet frame, a candidate can break down why each field is present, where it originated, and can build an Ethernet frame without actually memorizing any fields. I structure my post on Ethernet basics in this way, by taking a bottom up approach from the 802.3 CSMA/CD fields to 802.2 LLC fields to RFC 1042 SNAP fields. By having read the majority of each of the standards, I know everything that needs to be known to create a layer 2 frame using Ethernet.

Would-be CCIEs who don't want to read standards and build a complete understanding are probably better off pursuing a different certification or staying at the CCNA/CCNP level. It simply takes too much time and too much effort for most people to gain the required level of understanding. Those who manage to magically pass the CCIE based on memorization alone give every other CCIE a bad reputation because they lack the knowledge that is required to be effective in an advanced network engineering role. This is why Cisco has made every effort to make the CCIE differentiate the top 1% of the networking world.

For those studying for the CCIE, understand that it is a long and difficult process. Most people can't complete the required study in 6 months and many can't complete the required study within a year. For many, it takes years to develop the required mastery of networking. If you measure the CCIE in pages of reading, you are easily into the tens of thousands...

Best of luck to those of you who are working towards this certification.

Below is a list of standards that I've found helpful in understanding the topics required for the CCIE (though, not a complete reading list and also not complete at this point):

Institute for Electrical and Electronics Engineers:
IEEE 802-2001: IEEE Standard for Local and Metropolitan Area Networks: Overview and Architecture
IEEE 802.1d-2004: Media Access Control (MAC) Bridges
IEEE 802.1q-2011: Media Access Control (MAC) Bridges and Virtual Bridged Local Area Networks
IEEE 802.3-2008: Carrier sense multiple access with Collision Detection (CSMA/CD) Access Method and Physical Layer Specifications
IEEE 802.2-1985: Logical Link Control

Internet Engineering Task Force (IETF):
RFC 1042: A Standard for the Transmission of IP Datagrams over IEEE 802 Networks

The Broadband Forum (formerly the Frame Relay Forum)

See Also,
The Road to the CCIE

Notes on Cisco IOS Interface Order of Operations

2012-04-25T09:26:00.003-07:00

Part of being a good network engineer is having the ability to effectively troubleshoot problems and answer questions without ambiguity. For the entry level networking technician (CCENT/CCNA), the implementation and troubleshooting tasks may not be involved enough to require detailed knowledge of how a router handles packets from ingress to egress. For intermediate network engineers (those at the CCNP level), this knowledge is important and as an individual transitions from the intermediate level to the advanced level (CCIE), this type of knowledge is extremely important to quickly isolate, identify, and solve issues in an enterprise network.

There are a lot of different manipulations that can happen with a router, including manipulations like tunneling/IPSec encryption, network address translation (NAT), policy routing, and Quality of Service (QoS) manipulation. Understanding the order that these happen is important for designing and troubleshooting different flows through a network. Cisco has published a couple of Technology Notes on the order of operations involving NAT and QoS and these same steps are in the CCNP TSHOOT study guides.

Here are the order of operations for interfaces involved with NAT

Inside to Outside	Outside to Inside
Input Access Control List (ACL) for Encrypted Packet	same
IPSec Decryption	same
Input ACL for Decrypted Packet	same
Input QoS	same
Input Accounting	same
Web Cache Redirection	same
Policy Routing	NAT inside global to inside local translation
Routing	Policy Routing
NAT inside local to inside global translation	Routing
Crypto Map (mark for encryption)	same
Output ACL	same
IP Inspection (a.k.a Context Based Access Control [CBAC], Stateful IOS firewall)	same
TCP Intercept	same
IPSec Encryption	same
Output QoS and Queuing	same

The logic is fairly simple for NAT, it is impossible to make a forwarding decision without performing an inside global to inside local address translation. Without NAT, the NAT step above simply disappears and the order of operations are the same. Additionally, any features that are not configured drop out of the steps above. In the simplest case, the only action is routing.

I was working through a CCIE lab exam study guide and it posed an interesting question that got me thinking more about order of operations. The question asked to set different QoS markings (DSCP values) based on packet length. With regular class based marking this is not possible, but it is possible to match length using a route map. I was curious to see which would take precedence, DSCP set using a route map or a DSCP set using an egress policy map. I set up a really simple lab below in Dynamips/GNS3 using Cisco c3725 routers:

The idea is that I want to change the DSCP for ICMP traffic that is being NAT'd from 10.0.0.1 to 192.168.0.3. I configured a policy map on the NAT router that set the DSCP to AF21 on egress via fa0/0. I set up a route-map that set the appropriate IP precedence (3/immediate) and TOS (2/max-reliability) to support a combined DSCP of AF31. Important configuration pieces on the NAT router are below (MQC commands are in green and policy routing related commands are in blue):

!
class-map match-all match-echo
match protocol icmp
!
!
policy-map set-icmp
class match-echo
set ip dscp af21
!
!
interface FastEthernet0/0
ip address 192.168.0.2 255.255.255.0
ip nat outside
ip virtual-reassembly
ip policy route-map set-icmp-dscp
duplex auto
speed auto
service-policy output set-icmp
!
!
access-list 101 permit icmp any any
!
!
!
route-map set-icmp-dscp permit 10
match ip address 101
set ip precedence immediate
set ip tos max-reliability
!

The final question that we really have to look at on the outside router is what DSCP values are received for the ICMP traffic (ping in this case)? This can be found by setting an extended access list that matches one or more DSCP values and the correct source/destination addresses:

access-list 101 permit ip any host 192.168.0.1 dscp af21

And then we watch for packets that match that ACL:

debup ip packet 101

Performing a ping to the outside router indicates that the traffic is received with DSCP AF21. This indicates that the egress marking takes precedence over marking that happens due to policy routing.

Outside(config)#do debug ip packet 101
IP packet debugging is on for access list 101
*Mar 1 00:25:06.879: IP: tableid=0, s=192.168.0.3 (FastEthernet0/0), d=192.168.0.1 (FastEthernet0/0), routed via RIB
*Mar 1 00:25:06.883: IP: s=192.168.0.3 (FastEthernet0/0), d=192.168.0.1 (FastEthernet0/0), len 100, rcvd 3
*Mar 1 00:25:06.959: IP: tableid=0, s=192.168.0.3 (FastEthernet0/0), d=192.168.0.1 (FastEthernet0/0), routed via RIB
*Mar 1 00:25:06.959: IP: s=192.168.0.3 (FastEthernet0/0), d=192.168.0.1 (FastEthernet0/0), len 100, rcvd 3
*Mar 1 00:25:07.003: IP: tableid=0, s=192.168.0.3 (FastEthernet0/0), d=192.168.0.1 (FastEthernet0/0), routed via RIB
*Mar 1 00:25:07.003: IP: s=192.168.0.3 (FastEthernet0/0), d=192.168.0.1 (FastEthernet0/0), len 100, rcvd 3
*Mar 1 00:25:07.043: IP: tableid=0, s=192.168.0.3 (FastEthernet0/0), d=192.168.0.1 (FastEthernet0/0), routed via RIB
*Mar 1 00:25:07.043: IP: s=192.168.0.3 (FastEthernet0/0), d=192.168.0.1 (FastEthernet0/0), len 100, rcvd 3
*Mar 1 00:25:07.087: IP: tableid=0, s=192.168.0.3 (FastEthernet0/0), d=192.168.0.1 (FastEthernet0/0), routed via RIB
*Mar 1 00:25:07.087: IP: s=192.168.0.3 (FastEthernet0/0), d=192.168.0.1 (FastEthernet0/0), len 100, rcvd 3

If there was no marking occurring on egress, then the policy routing marking would take precedence.

See Also,
The Road to the CCIE

Configuring Cisco On-Demand Routing (ODR)

2012-04-16T11:05:00.000-07:00

One of the goals at the CCIE level is to gain a mastery of the configuration of most of the features on the Cisco IOS platform. On Demand Routing (ODR) is not often used and not often tested and covered superficially on lower level certifications like the CCNP and CCNA.

ODR is primarily applicable in a hub and spoke network where CDP can be used and it is not desirable to add the overhead of a routing protocol like RIP, EIGRP, or OSPF. The main idea is that the prefixes can be exchanged from the spoke routers (also known as stub routers) to the hub router and the hub router can advertise a default route to the spoke routers. Prefixes learned via ODR can be redistributed into another routing protocol to enure full end-to-end connectivity with other parts f the network.

ODR relies on Cisco Discovery Protocol (CDP), so only networks that support CDP can be part of an ODR configuration. This immediately eliminates Frame Relay with point-to-multipoint subinterfaces interfaces and other non-broadcast topologies.

The following topologies show demonstrations of ODR using IPv4. IPv6 is not currently supported in ODR. This lab is built on Dynamips/GNS3 using Cisco c7200 routers running the Advanced Enterprise Services release of 15.0(1)M4.

ODR is only enabled on the hub router with the router odr command. On ASBR_HUB, we configure ODR and redistribution into OSPF:

router odr
!
router ospf 1
log-adjacency-changes
redistribute odr subnets
network 10.0.0.0 0.0.0.255 area 0
!

After the network converges and routes are communicated via CDP, we can see the default route advertised via ODR on any spoke router:

Stub2#show ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route, + - replicated route

Gateway of last resort is 10.0.2.1 to network 0.0.0.0

o*    0.0.0.0/0 [160/1] via 10.0.2.1, 00:00:56, FastEthernet0/0
      10.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
C        10.0.2.0/30 is directly connected, FastEthernet0/0
L        10.0.2.2/32 is directly connected, FastEthernet0/0
      192.168.1.0/32 is subnetted, 1 subnets
C        192.168.1.1 is directly connected, Loopback0

On the hub router, we see the networks connected to the spoke routers advertised via CDP and included in the routing table as ODR routes:

ASBR_HUB#show ip route odr
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route, + - replicated route

Gateway of last resort is not set

      192.168.0.0/32 is subnetted, 1 subnets
o        192.168.0.1 [160/1] via 10.0.1.2, 00:00:59, FastEthernet0/1
      192.168.1.0/32 is subnetted, 1 subnets
o        192.168.1.1 [160/1] via 10.0.2.2, 00:00:39, FastEthernet1/0
      192.168.2.0/32 is subnetted, 1 subnets
o        192.168.2.1 [160/1] via 10.0.3.2, 00:00:59, FastEthernet1/1

With the redistribution into OSPF, we can confirm end-to-end connectivity from the OSPF router and from one of the spoke routers:

Area0#ping 192.168.1.1 source lo0

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.1.1, timeout is 2 seconds:
Packet sent with a source address of 172.16.1.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 60/74/92 ms

Stub2#ping 172.16.1.1 source lo0

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 172.16.1.1, timeout is 2 seconds:
Packet sent with a source address of 192.168.1.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 48/57/76 ms

Troubleshooting ODR isn't very complicated compared to other protocols because there is not a lot of complexity. 90% of troubleshooting ODR is making sure CDP is functioning properly and ensuring that it is not disabled globally or on the interfaces that are propagating routes.

Most often, people don;t know which router(s) to configure the router odr command on (since it is only enabled on the hub router). Using the debug cdp ip command helps to identify possible issues if the routing tables are not being populated.

ASBR_HUB#debug cdp ip
CDP IP info debugging is on

*Apr 16 11:54:29.915: CDP-IP: IP TLV length (4) invalid for IP prefixes.
                      Expecting IP prefix from stub router

The message above indicates that more than one hub has been configured. Removing the router odr command from the misconfigured spoke router should resolve this issue. Proper operation shows the hub router writing default routes to the spoke routers and updating routes from spoke-connected networks:

*Apr 16 11:57:29.703: CDP-IP: Writing default route 10.0.1.1 for FastEthernet0/1
*Apr 16 11:57:29.703: CDP-IP: Writing default route 10.0.2.1 for FastEthernet1/0
*Apr 16 11:57:29.707: CDP-IP: Writing default route 10.0.3.1 for FastEthernet1/1
ASBR_HUB#
*Apr 16 11:57:30.651: CDP-IP: Writing default route 10.0.0.2 for FastEthernet0/0
ASBR_HUB#
*Apr 16 11:57:34.951: CDP-IP: Aging routes
ASBR_HUB#
*Apr 16 11:57:38.511: CDP-IP: Reading prefix 192.168.1.1/32 source 10.0.2.2 via FastEthernet1/0
*Apr 16 11:57:38.511: CDP-IP: Updating prefix 192.168.1.1/32 in routing table

Routing updates can be controlled using distribute lists and timers can be configured using the timers router command. By default, ODR sends updates every 60 seconds. If updates are not received in 180 seconds, routes are marked as invalid and flushed after 240 seconds.

The Road to the CCIE

OSPF Virtual Links

2012-04-11T11:40:00.001-07:00

The Open Shortest Path First routing protocol is a critical piece of knowledge for any networking professional working in an enterprise environment. Most major networking certifications covering routing and switching including the Cisco Certified Network Associate (CCNA), Cisco Certified Network Professional (CCNP), and Cisco Certified Internetwork Expert (CCIE) extensively test OSPF knowledge and skills. OSPF is the most popular dynamic routing protocol used in complex enterprise networks. This example will demonstrate the concepts and configuration involved with setting up OSPF virtual links to support a non-contiguous backbone area. This lab is built using Dynamips/GNS3 utilizing Cisco c3725 routers.

Virtual links are designed to allow the backbone area (area 0) to be noncontiguous. They can also be used for non-backbone areas to protect against loss of end-to-end connectivity if an area is partitioned. In this example, the parts of the backbone area have no direct connection, OSPF brings up a virtual interface to allow the two separated routers in area 0 to share a common link and transmit LSAs. The following topology shows how virtual links are applied:

In order to ensure continuity in area 0, type 1 and type 2 LSAs are transmitted between routers 2.2.2.2 and 4.4.4.4. to allow all of the routers in area 0 to have a complete topology view. Type 3 (network summary) LSAs are transferred into area 0 at the two ABRs.

Virtual links can only be created over regular areas, when an area containing a virtual link is changed to a stub area or NSSA, the change fails with the following:

ABR_1_0(config-router)#area 1 nssa
% OSPF: Area cannot be nssa as it contains a virtual link

ABR_1_0(config-router)#area 1 stub
% OSPF: Area cannot be a stub as it contains a virtual link

Attempting to create a virtual link through a stub area fails

ABR_0_1(config-router)#area 1 virtual-link 4.4.4.4
% OSPF: Area 1 is a stub or nssa so virtual links are not allowed

Configuration is fairly straightforward. The area virtual-link router-id is used on the ABRs. The transit area specified is the area that divides the area involved with the virtual link (typically area 0, but other areas are also possible) and the router ID is the router ID of the ABR on the other side of the virtual link. In this example the transit area is area 1 and the virtual links are between routers 4.4.4.4 and 2.2.2.2 On the ABRs:

On ABR_0_1:

router ospf 1
router-id 2.2.2.2
log-adjacency-changes
area 1 virtual-link 4.4.4.4
network 10.0.0.0 0.0.0.3 area 0
network 10.0.1.0 0.0.0.3 area 1
!

On ABR_1_0:

router ospf 1
router-id 4.4.4.4
log-adjacency-changes
area 1 virtual-link 2.2.2.2
network 10.0.2.0 0.0.0.3 area 1
network 10.0.3.0 0.0.0.3 area 0
!

OSPF hello packets are not sent periodically over virtual links and all of the received LSAs are marked with the DNA (do not age) bit. In a stable network, this is not a problem, but LSAs marked with the do not age bit are not refreshed every 30 minutes and they are not automatically timed out and deleted. Here is an example of the OSPF database for area 0 on ABR_1_0:

ABR_1_0#show ip ospf database

            OSPF Router with ID (4.4.4.4) (Process ID 1)

                Router Link States (Area 0)

Link ID         ADV Router      Age         Seq#       Checksum Link count
1.1.1.1         1.1.1.1         27    (DNA) 0x8000001F 0x00D21B 1
2.2.2.2         2.2.2.2         1     (DNA) 0x8000002A 0x00C9CD 2
4.4.4.4         4.4.4.4         7           0x80000025 0x00CBC0 2
5.5.5.5         5.5.5.5         634         0x8000001E 0x00FDC8 1

                Net Link States (Area 0)

Link ID         ADV Router      Age         Seq#       Checksum
10.0.0.1        1.1.1.1         27    (DNA) 0x80000001 0x0069B8
10.0.3.2        5.5.5.5         634         0x8000001C 0x007C5E

                Summary Net Link States (Area 0)

Link ID         ADV Router      Age         Seq#       Checksum
10.0.1.0        2.2.2.2         39    (DNA) 0x80000003 0x000E13
10.0.1.0        4.4.4.4         372         0x8000001A 0x0008EF
10.0.2.0        2.2.2.2         1     (DNA) 0x80000001 0x006BAC
10.0.2.0        4.4.4.4         119         0x8000001C 0x00946A

...

None of the area 0 type-1 or type2 LSAs show up in Area 1. Instead, the database shows type-3 network summary LSAs originating from area 0:

Area1#show ip ospf database

            OSPF Router with ID (10.0.2.1) (Process ID 1)

                Router Link States (Area 1)

Link ID         ADV Router      Age         Seq#       Checksum Link count
2.2.2.2         2.2.2.2         184         0x80000022 0x00AF2B 1
4.4.4.4         4.4.4.4         184         0x8000002B 0x001FA0 1
10.0.2.1        10.0.2.1        199         0x80000029 0x00DBBE 2

                Net Link States (Area 1)

Link ID         ADV Router      Age         Seq#       Checksum
10.0.1.2        10.0.2.1        194         0x80000003 0x0040CB
10.0.2.1        10.0.2.1        1133        0x8000001A 0x007577

                Summary Net Link States (Area 1)

Link ID         ADV Router      Age         Seq#       Checksum
10.0.0.0        2.2.2.2         200         0x8000001B 0x00E821
10.0.3.0        4.4.4.4         550         0x8000001E 0x008576

If the ABR attached to the virtual link fails, it's LSAs will continue to exist on the other area routers; however the routes will eventually disappear from the routing table. Looking that the database, OSPF picks up that the advertising router is not reachable and removes the route from the routing table, but the LSA remains in the database, Here is an example of a type-1 (router) LSA in the unreachable part of the backbone area after the virtual link goes down:

ABR_1_0#show ip ospf database router 1.1.1.1

            OSPF Router with ID (4.4.4.4) (Process ID 1)

                Router Link States (Area 0)

Adv Router is not-reachable
LS age: 27 (DoNotAge)
Options: (No TOS-capability, DC)
LS Type: Router Links
Link State ID: 1.1.1.1
Advertising Router: 1.1.1.1
LS Seq Number: 8000001F
Checksum: 0xD21B
Length: 36
Number of Links: 1

    Link connected to: a Transit Network
     (Link ID) Designated Router address: 10.0.0.1
     (Link Data) Router Interface address: 10.0.0.1
      Number of TOS metrics: 0
       TOS 0 Metrics: 10

Virtual links shouldn't exist in well designed OSPF networks. The main applications for using virtual links are to prevent connectivity loss for partitioned areas and utilizing a physically separated backbone area. For using a physically separated backbone area, this most often occurs during a merger/acquisition and the decision has been made to use a single OSPF routing domain instead of redistribution.

See Also:
The Road to the CCIE

The OSPF Totally Not-So-Stubby-Area (NSSA)

2012-04-10T07:35:00.000-07:00

Regular (non-stub) area
Stub area
Totally stubby area
Not-so-stubby area (NSSA)
Totally stub not-so-stubby area (totally NSSA)

Similar to the totally stubby area, the totally NSSA is not a standard area defined in the RFC specifications for OSPF. Vendors including Microsoft, Cisco, Juniper, and others have implemented these areas as their own custom extensions to the OSPF protocol.

The design principles around totally NSSAs are similar to totally stubby areas, with the exception that the underlying assumptions are relaxed to indicate that the routers that are internal to the totally NSSAs need to have no knowledge of subnets outside of the area (with the exception of the routes injected by the NSSA ASBRs).

The following topology provides everything necessary to examine the mechanics of the totally NSSA:

Here we have E1 and E2 routes created by the ASBR in area 0 and N1 and N2 routes created by the ASBR in area 1 (the totally NSSA). The configuration of the ASBR and NSSA_ASBR routers follows a similar configuration to the NSSA example.
On ASBR:

router ospf 1
log-adjacency-changes
redistribute connected subnets route-map redist-lo-into-ospf
network 10.0.0.0 0.0.0.255 area 0
!
ip prefix-list lo-into-ospf-e1 seq 5 permit 172.16.1.0/24
!
ip prefix-list lo-into-ospf-e2 seq 5 permit 172.16.2.0/24
!
! **Make 172.16.1.0/24 an E1 route**
!
route-map redist-lo-into-ospf permit 10
match ip address prefix-list lo-into-ospf-e1
set metric-type type-1
!
! **Make 172.16.2.0/24 an E2 route**
!
route-map redist-lo-into-ospf permit 20
match ip address prefix-list lo-into-ospf-e2
set metric-type type-2
!

On NSSA_ASBR:

router ospf 1
log-adjacency-changes
area 1 nssa no-summary
redistribute connected subnets route-map lo-to-ospf
network 192.168.1.0 0.0.0.255 area 1
!
ip prefix-list lo-to-ospf-t1 seq 5 permit 192.168.100.0/24
!
ip prefix-list lo-to-ospf-t2 seq 5 permit 192.168.200.0/24
!
route-map lo-to-ospf permit 10
match ip address prefix-list lo-to-ospf-t1
set metric-type type-1
!
route-map lo-to-ospf permit 20
match ip address prefix-list lo-to-ospf-t2
set metric-type type-2
!

For ABRs that have interfaces in area 1, the area nssa no-summary command is required to identify the area as a totally NSSA. The no-summary option instructs the ABRs not to propagate type-3 LSAs into the totally NSSAs. The no-summary option is also only required on ABRs, as there are no other sources of type 3 LSAs. Type-5 LSAs are not propagated into stub or NSSA areas.

The area 1 ABRs still create type-3 summary LSAs and type 5 external LSAs (from the type-7 NSSA external LSAs) and propagate these into the backbone area and other type-5 capable areas.

Let's examine the routing table and OSPF database on the internal area 1 router and the internal area 0 router. Since area 1 is totally NSSA, we would only expect to find networks advertised in area 1 and the external routes from NSSA_ASBR:

Area1#show ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route

Gateway of last resort is 192.168.0.1 to network 0.0.0.0

O N2 192.168.200.0/24 [110/20] via 192.168.1.2, 20:17:53, Serial0/1
C    192.168.0.0/24 is directly connected, Serial0/0
C    192.168.1.0/24 is directly connected, Serial0/1
O N1 192.168.100.0/24 [110/84] via 192.168.1.2, 20:17:53, Serial0/1
O*IA 0.0.0.0/0 [110/65] via 192.168.0.1, 10:06:39, Serial0/0

Similar to the stub and totally stubby areas, the default route is propagated as a type-3 LSA originating from the ABR and there are no other LSAs advertising routes/networks from outside of area 1:

Area1#show ip ospf database

            OSPF Router with ID (192.168.1.1) (Process ID 1)

                Router Link States (Area 1)

Link ID         ADV Router      Age         Seq#       Checksum Link count
192.168.0.1     192.168.0.1     1434        0x8000002B 0x002A33 2
192.168.1.1     192.168.1.1     1046        0x80000029 0x009A1E 4
192.168.200.1   192.168.200.1   656         0x80000027 0x00418C 2

                Summary Net Link States (Area 1)

Link ID         ADV Router      Age         Seq#       Checksum
0.0.0.0         192.168.0.1     661         0x80000013 0x0003B6

                Type-7 AS External Link States (Area 1)

Link ID         ADV Router      Age         Seq#       Checksum Tag
192.168.100.0   192.168.200.1   656         0x80000025 0x004F36 0
192.168.200.0   192.168.200.1   656         0x80000025 0x00821E 0

In the backbone area, the routing table shows all of the networks that are advertised into OSPF (either through redistribution or OSPF configuration):

Area0#show ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route

Gateway of last resort is not set

     172.16.0.0/24 is subnetted, 2 subnets
O E1    172.16.1.0 [110/84] via 10.0.0.1, 16:28:13, Serial0/0
O E2    172.16.2.0 [110/20] via 10.0.0.1, 16:28:13, Serial0/0
O E2 192.168.200.0/24 [110/20] via 10.0.1.2, 16:28:33, Serial0/1
     10.0.0.0/24 is subnetted, 2 subnets
C       10.0.0.0 is directly connected, Serial0/0
C       10.0.1.0 is directly connected, Serial0/1
O IA 192.168.0.0/24 [110/128] via 10.0.1.2, 16:32:30, Serial0/1
O IA 192.168.1.0/24 [110/192] via 10.0.1.2, 16:32:30, Serial0/1
O E1 192.168.100.0/24 [110/212] via 10.0.1.2, 16:28:34, Serial0/1

Looking at Area 0, we see type 5 LSAs for the external routes advertised by NSSA_ASBR and we see type 3 LSAs for the networks that are internal to area 1:

Area0#show ip ospf database

            OSPF Router with ID (10.0.1.1) (Process ID 1)

                Router Link States (Area 0)

Link ID         ADV Router      Age         Seq#       Checksum Link count
10.0.1.1        10.0.1.1        933         0x8000004F 0x0074D3 4
172.16.2.1      172.16.2.1      1063        0x8000004E 0x00F9BA 2
192.168.0.1     192.168.0.1     1748        0x8000002A 0x00F38B 2

                Summary Net Link States (Area 0)

Link ID         ADV Router      Age         Seq#       Checksum
192.168.0.0     192.168.0.1     1748        0x80000025 0x00FF05
192.168.1.0     192.168.0.1     1748        0x80000025 0x00774C

                Type-5 AS External Link States

Link ID         ADV Router      Age         Seq#       Checksum Tag
172.16.1.0      172.16.2.1      1305        0x8000004A 0x009ABF 0
172.16.2.0      172.16.2.1      1316        0x8000004A 0x0013C5 0
192.168.100.0   192.168.0.1     1759        0x80000025 0x0061F6 0
192.168.200.0   192.168.0.1     1759        0x80000025 0x0094DE 0

See Also,
The Road to the CCIE

Are Cloud Services Highly Available?

2012-04-10T07:12:00.000-07:00

Businesses, governments, an non-profit organizations are relying more and more on the use of externally hosted cloud services for email, web hosting, applications, and more. From a business unit perspective, these services require little more than an internet connection and a client software installation (in some cases). Some companies are even reducing or eliminating their datacenter space in favor of emerging cloud technologies like virtual private server hosting (VPS). Unfortunately, the correct pieces of the IT department might not be involved until after a contract is signed and critical damage is done to the organization's high availability strategy.

Cloud services are hosted by all kinds of people and organizations nowadays. Some organizations offer very highly available systems that could withstand the failure of an entire part of the country while others are offered out of a person's basement (really... an old boss of mine hosted a "cloud" service out of his basement). Some providers even cater to niche markets like the eco-friendly market by offering green cloud services, green virtual private server hosting, and green web hosting. Although good for marketing, niche factors don't determine what a "good" cloud provider really offers.

Utilizing cloud services is a form of outsourcing, so the primary business drivers are reducing total cost of ownership and not negatively impacting the quality of the IT service delivered. For the most part, organizations are most successful with outsourcing business support processes while keeping the IT infrastructure and applications for core business services maintained internally.

When evaluating competing cloud services, it is important to look at what the hosting organization provides in terms of the service offered and the availability/performance guarantee. If the hosting company stores your organization's data in a single datacenter at a single site, then this creates a risk of loss in the event of a catastrophic failure or natural disaster at the service provider's datacenter. Perhaps the provider has multiple sites, but relies on tape backup and restore in the event of a failure at the primary site.

Other factors around availability, performance, and overall service quality need to be considered before signing a contract with a cloud provider. When you trust your data to a cloud provider, you are only as highly available as they are. Furthermore, in the event of a disaster affecting a service provider, their internal DR plan will probably be to restore the biggest and highest paying company's data first, so smaller organizations could be at a bigger disadvantage.

The business case for cloud services exists because an organization can manage their IT costs as operational expenses rather than capital expenses and the total cost of ownership (TCO) goes down in some instances (though not as often as you might think). Unfortunately, the use of some cloud technologies can cause an organization's IT service to be less highly available than if it were offered internally by the organization's IT staff.

See Also,
High Availability, Business Continuity Planning, and Disaster Recovery

The OSPF Not-So-Stubby Area (NSSA)

2012-04-09T14:58:00.001-07:00

Regular (non-stub) area
Stub area
Totally stubby area
Not-so-stubby area (NSSA)
Totally stub not-so-stubby area (totally NSSA)

The NSSA Option for OSPF areas is defined in RFC 1587. The NSSA option partially undoes a portion of what the stub area is designed to do. The OSPF stub area eliminates the propagation of external routes in an area by creating a default route that is propagated into the NSSA by the ABR. The NSSA standard creates a new type of LSA, the type 7 (NSSA External) LSA, and floods that through the NSSA. ABRs for the NSSA translate type 7 LSAs and propagate them to the backbone area and other type-5-LSA capable areas as type 5 (external) LSAs. ASBRs in the NSSA are not actually advertised into the backbone area as type 4 (ASBR summary) LSAs.

From a network design standpoint, NSSAs have a limited application to large enterprise networks and service providers using OSPF as an interior gateway protocol. The idea centers on the assumption that there are non-transit networks that are candidates to be connected to a stub area, but are not part of the OSPF routing domain. In a way, these external networks and the OSPF area that they are connected to can be considered a leaf in the larger spanning tree developed by the other OSPF areas.

The following topology shows everything that we will need to explore the not-so-stubby area in Cisco's OSPF implementation.

The ASBRs create both E1 (N1) and E2 (N2) routes. Remember that paths to E2/N2 routes are evaluated only on the external cost. E1/N1 costs are determined by a combination of the external and internal (to the OSPF routing domain) costs to reach a subnet. E2/N2 networks are considered higher cost than E1/N1 networks, regardless of whether the E2/N2 metric is actually lower than the E1/N1 metric.

The configuration of the ASBR and NSSA_ASBR redistribution is most easily accomplished using a route map that matches prefix lists for the routes:

On ASBR:

router ospf 1
log-adjacency-changes
redistribute connected subnets route-map redist-lo-into-ospf
network 10.0.0.0 0.0.0.255 area 0
!
ip prefix-list lo-into-ospf-e1 seq 5 permit 172.16.1.0/24
!
ip prefix-list lo-into-ospf-e2 seq 5 permit 172.16.2.0/24
!
! **Make 172.16.1.0/24 an E1 route**
!
route-map redist-lo-into-ospf permit 10
match ip address prefix-list lo-into-ospf-e1
set metric-type type-1
!
! **Make 172.16.2.0/24 an E2 route**
!
route-map redist-lo-into-ospf permit 20
match ip address prefix-list lo-into-ospf-e2
set metric-type type-2
!

On NSSA_ASBR:

router ospf 1
log-adjacency-changes
area 1 nssa default-information-originate
redistribute connected subnets route-map lo-to-ospf
network 192.168.1.0 0.0.0.255 area 1
!
ip prefix-list lo-to-ospf-t1 seq 5 permit 192.168.100.0/24
!
ip prefix-list lo-to-ospf-t2 seq 5 permit 192.168.200.0/24
!
route-map lo-to-ospf permit 10
match ip address prefix-list lo-to-ospf-t1
set metric-type type-1
!
route-map lo-to-ospf permit 20
match ip address prefix-list lo-to-ospf-t2
set metric-type type-2
!

For routers attached to the NSSA, the area nssa router configuration command is required to set the correct flag in the OSPF hello packet. All routers in the area have to agree that the particular area is an NSSA area, otherwise they will not be able to form adjacencies. A special issue for NSSAs is also that the ABR does not generate a default route unless the default-information-originate option is used.

We know from previous examples that the ABRs have full databases for all of the areas that they are connected to, so let's look at the routing tables and OSPF databases on Area0 and Area1. Area1 is an internal router for area 1, meaning that all of it's OSPF-enabled interfaces are in area 1, the NSSA. In this case, we see the two external routes for the networks advertised by NSSA_ASBR and the default route advertised by the ABR (unlike a stub area, this is originated by the ABR as a type 7 LSA):

Area1#show ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route

Gateway of last resort is 192.168.0.1 to network 0.0.0.0

O N2 192.168.200.0/24 [110/20] via 192.168.1.2, 03:24:47, Serial0/1
     10.0.0.0/24 is subnetted, 2 subnets
O IA    10.0.0.0 [110/192] via 192.168.0.1, 03:28:20, Serial0/0
O IA    10.0.1.0 [110/128] via 192.168.0.1, 03:28:20, Serial0/0
C    192.168.0.0/24 is directly connected, Serial0/0
C    192.168.1.0/24 is directly connected, Serial0/1
O N1 192.168.100.0/24 [110/84] via 192.168.1.2, 03:24:47, Serial0/1
O*N2 0.0.0.0/0 [110/1] via 192.168.0.1, 00:02:47, Serial0/0

The OSPF database from Area1 is shown below, LSAs for networks originating from area 0 are green and LSAs for networks originating in area 1 are blue. The special type-7 LSA for the default route is purple.:

Area1#show ip ospf database

            OSPF Router with ID (192.168.1.1) (Process ID 1)

                Router Link States (Area 1)

Link ID         ADV Router      Age         Seq#       Checksum Link count
192.168.0.1     192.168.0.1     785         0x8000000D 0x006615 2
192.168.1.1     192.168.1.1     589         0x8000000B 0x00D6FF 4
192.168.200.1   192.168.200.1   390         0x80000009 0x007D6E 2

                Summary Net Link States (Area 1)

Link ID         ADV Router      Age         Seq#       Checksum
10.0.0.0        192.168.0.1     785         0x80000007 0x0093A8
10.0.1.0        192.168.0.1     785         0x80000007 0x000675

                Type-7 AS External Link States (Area 1)

Link ID         ADV Router      Age         Seq#       Checksum Tag
0.0.0.0         192.168.0.1     359         0x80000001 0x00FA4C 0
192.168.100.0   192.168.200.1   390         0x80000007 0x008B18 0
192.168.200.0   192.168.200.1   390         0x80000007 0x00BEFF 0

Area0 is an internal router for the backbone area. In this case, the LSAs originating in area 1 for the external 192.168.100.0 and 192.168.200.0 networks are converted to type-5 LSAs for any other areas in the OSPF routing domain that are type-5 capable. The routing table shows that the N1 routes have been converted to E1 and the N2 routes have nbeen converted to E2.

Area0#show ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route

Gateway of last resort is not set

     172.16.0.0/24 is subnetted, 2 subnets
O E1    172.16.1.0 [110/84] via 10.0.0.1, 1d00h, Serial0/0
O E2    172.16.2.0 [110/20] via 10.0.0.1, 1d00h, Serial0/0
O E2 192.168.200.0/24 [110/20] via 10.0.1.2, 03:37:32, Serial0/1
     10.0.0.0/24 is subnetted, 2 subnets
C       10.0.0.0 is directly connected, Serial0/0
C       10.0.1.0 is directly connected, Serial0/1
O IA 192.168.0.0/24 [110/128] via 10.0.1.2, 03:41:43, Serial0/1
O IA 192.168.1.0/24 [110/192] via 10.0.1.2, 03:40:23, Serial0/1
O E1 192.168.100.0/24 [110/212] via 10.0.1.2, 03:37:34, Serial0/1

In the OSPF database, we also see that there are no type 7 LSAs and the LSAs advertising the 192.168.100 and 192.168.200 external networks are now type 5 LSAs. The same coloring scheme is used here as the example above,

Area0#show ip ospf database

            OSPF Router with ID (10.0.1.1) (Process ID 1)

                Router Link States (Area 0)

Link ID         ADV Router      Age         Seq#       Checksum Link count
10.0.1.1        10.0.1.1        1748        0x8000002E 0x009ECA 4
172.16.2.1      172.16.2.1      898         0x8000002F 0x00389B 2
192.168.0.1     192.168.0.1     1573        0x8000000C 0x00306D 2

                Summary Net Link States (Area 0)

Link ID         ADV Router      Age         Seq#       Checksum
192.168.0.0     192.168.0.1     1573        0x80000007 0x003CE6
192.168.1.0     192.168.0.1     1330        0x80000007 0x00B32E

                Type-5 AS External Link States

Link ID         ADV Router      Age         Seq#       Checksum Tag
172.16.1.0      172.16.2.1      898         0x8000002C 0x00D6A1 0
172.16.2.0      172.16.2.1      898         0x8000002C 0x004FA7 0
192.168.100.0   192.168.0.1     1330        0x80000007 0x009DD8 0
192.168.200.0   192.168.0.1     1330        0x80000007 0x00D0C0 0

This post showed basic configuration and mechanics of a NSSA in Cisco's OSPF implementation.

See Also,
The Road to the CCIE

Overclocking is a Bad Idea

2012-04-05T22:45:00.000-07:00

I see a lot of forums issues related to problems that are ultimately traced back to overclocking on systems. Overclocking is probably the single worst thing you can do for the lifespan of your PC and the instability caused by overclocking far outweighs any possible performance gains. Professional systems engineers don't overclock systems, and home users shouldn't either for a vast number of reasons. I'll go through a few of them in this post.

What is overclocking?

Overclocking refers to modifying a computer system component in a way that it is operating at a higher frequency/clock rate, voltage, or other measure of performance. Most overclocking is done by people who are conducting experiments with hardware to determine the maximum stress point that a motherboard, processor (CPU), system bus, graphics processing unit (GPU), system memory (RAM), or hard drive can handle before reaching a failure point.

Some misguided individuals overclock their systems and use the overclocked state for their standard configuration. These individuals typically feel the pain shortly after overclocking and they tend to go running to the various hardware forums on the Internet to say that their system is acting up.

Why is overclocking good?

To show the other side of the story, overclocked systems typically have a higher level of performance than their non-overclocked counterparts. These systems typically process numeric calculations, graphics operations, and other system functions at higher rates and appear more responsive to the end user. This comes at a major cost to the system's stability and

Why is overclocking bad?

It immediately voids your warranty for most components. Hardware vendors don't support running system components above their default thresholds. They set these limits after doing numerous tests to determine the right mix between component longevity, performance, and failure rate. Most vendors can detect if a component was overclocked when it failed, and if it happens, it typically voids an RMA request or a warranty repair and the component has to be repurchased by the end user.

Think about it in the context of how a car operates. Overclocking runs a component at or above the "redline" RPM level for an extended period of time. Even with appropriate cooling, this substantially increases component wear and exponentially increases the probability of component failure. For a car running above the "redline" level, the engine will seize after a short period of time, even with appropriate cooling and lubrication.

Overclocking throws off the system timing, increasing the probability of memory, hard drive, and processor cache/instruction corruption. The system is designed to have the processor and bus communicate with the other components at a specified rate, increasing the rate of operation can potentially overrun other system components and cause corruptions in calculations performed by the system and sent to/from other components.

Some components cannot be overclocked effectively. An example is a graphics processing unit. Although AMD (ATI) and Nvidia have developed experimental applications for overclocking a GPU, this causes increased stress on a system that does not have adequate dedicated cooling to begin with. Additionally, GPUs have a poor track record of enhancing system stability and adding nonstandard timings/voltage creates another variable that can contribute to increased timeout detection and recovery (TDR) activity/failures in Windows and general GPU related hangs and system failures in Linux/UNIX.

The bottom line is simple... don't overclock your system. The risks outweigh the benefits and overclocking often creates more problems than it solves. If you do overclock and break something, then don't go crying to anyone else because you ultimately did something stupid and received a fair reward. Don't ask the manufacturer for help, because they will find out that you overclocked and that your warranty/support agreement is void.

For end users... don't buy an overclocked system. This is a dangerous scenario if you are storing important documents and potentially irreplaceable data like family pictures on the system. Some unscrupulous computer people will sell these "gaming" systems to you, but if you want faster hardware, then buy faster hardware instead of overclocking cheaper and error-prone hardware. The cost difference typically pays itself back when it is compared to the downtime that overclocking typically causes.

Real computing professionals don't overclock systems. They know the risks and know that the risks outweigh the rewards. Those who do overclock typically have a spare system that they use for experiments, but don't store any real data on because of the increased risk of failure. Overclocked systems are mainly used to answer "what if" type questions and if you really need to have an extra system and willingness to lose this system (or parts of it) to answer these questions.

See Also,
How to Rescue Files From a Damaged Account

Windows Crash Dump Analysis
0x124 WHEA_UNCORRECTABLE_ERROR
0x9C MACHINE_CHECK_EXCEPTION.
Troubleshooting Memory Errors
How to Detect a Failing Hard Drive
Stress Testing a CPU to Detect Hardware Failure
Stress Testing a Video Card