I am a firm believer in voting for who you love so head over and take the quick couple minutes to choice your favorites. If something I have done on Nickapedia.com has been cool, useful, or inspiring then showing me through a vote would be awesome.

Here are some of the most popular posts I have done in the last year:

• Celerra VSA UBER 3.2
• UBERAlign – VMware Alignment
• Virtual Selection
• Fistful of Cloud (vSphere + Kinect)
• UBERShell (Powershell for Celerra)
• UBER Network Fuser (Mac OSX tool for VMware Fusion)

But, even if I don’t make your list. Vote anyways – this is a great way to provide feedback to bloggers.

.nick

Turns out there was a method being used on a Cocoa control that required changes made in the 10.7 SDK. I rewrote the function to not need the method and have released a new version (version 1 build 701). All download links are updated but you can download the new version here:

UBER Network Fuser 1.701 Download

Lion users don’t need to update.

Thanks,

.nick

Because of this switch I have come to learn the things that the Mac (and Lion) do really well with some things while Windows does better with others. For almost all of my UBER projects I use my beefy Windows workstation running VMware Workstation to setup my lab/dev environments.

But, recently I started trying to move over to my Mac and using VMware Fusion to build lab virtual machines for developing. And to my surprise I found that things in the Fusion world aren’t quite the same as for Workstation. Namely, the networking options are rather limited. By default you only get the choices of three networks 1) DHCP w/ NAT  2) DHCP and 3) Bridging to a physical interface. For me that meant that I couldn’t get a network without DHCP (important if you are testing it) or create multiple isolated networks like I could on Workstation.

So I went digging and found that at one point someone has written some slick scripts to allow for custom multi-network setup but it stopped being updated after Fusion 2. I also found that VMware has some KB articles on how to hack your way to adding networks. But neither was very easy to do or dealt with modifying the VM’s well.

So I decided to fill this gap myself. Over the Christmas holiday I worked furiously to make this and now I am proud to present the next UBER release and my first project in the Manic Innovation Challenge: UBER Network Fuser (UNF)

UNF is a native Mac OSX application supported on Snow Leopard and Lion that allows you to add additional networks to VMware Fusion, customize their settings, and easily change network selection for any Fusion VM’s. I designed it to be simple to use and be similiar to how VMware Workstation network editor works.

Here is the full feature list:

1. Allows up to 10 additional custom networks (total of 12)
2. You can enable/disable DHCP, NAT, virtual host adaptor on any network
3. Protects and provides rollback of default Fusion settings
4. Allows dynamic changes of network membership with Fusion VM’s
5. You may alias networks with custom names (even Workstation doesn’t do this)
6. Tested and confirmed that VLAN tagging works as expected within private networks
7. Saves configuration per user (names, paths, etc)
8. Integrates with Apple’s Security framework for elevated privileges when needed

I created a video to demonstrate how it works below:

And to download (free as always) use the link below:

Download UBER Network Fuser 1.0 – DMG for Snow Leopard & Lion (Updated link to new 1.701 version)

This is my first release in the Manic Innovation Challenge. It is written in Objective C & C and uses the Cocoa and Security frameworks. Challenge-wise it was definitely an experience learning and writing an app in Objective C in  few weeks. But, it was crazy fun and I have all kinds of cool ideas for the Mac now.

Also, this marks post #100 for me in the 2.3 years I have been running Nickapedia.com. Being that it is also my birthday I am considering today a good day.

As always please test and play with it and let me know with some comments below.

.nick

And honestly I don’t know where I will end up in 2012. But I have made myself a goal for self development that I am hoping will both challenge myself and encourage others to challenge themselves.

I am going to be taking on (what I call) the Manic Innovation Challenge in 2012. In simple terms and with simple rules to follow I will attempt to create something unique and creative using as wide a variety of frameworks, codebases, and presentation methods as possible. My goal is to release one example each month with a stretch goal of one example every two weeks.

All this work will be in my spare time unless it is also an EMC supported example. I have a couple crazy ideas kicking around in my head and one midway to completion.

I will also be starting a new series of commentary posts focused on what I see are cool trends in modern information technology at a very geek-centric level. My goal on these is brevity and something to hook you into exploring more on a topic.

Ultimately this blog has been a massive outlet for my personal development and I want to extend that in 2012 even further. Thanks for reading this so far and hopefully I can find a way to keep your interest this coming year.

.nick

 Top 7 Things I Learned in 2011

1. I must be doing something right

This is the obligatory stats part of my list. Comparing 2011 to 2010 I doubled my visits, unique visitors, and pageviews. Time on site went up about 33% per visit which may just be because I cleaned up the look and made it easier to find stuff. The most interesting thing is my traffic has been much higher in the last half of this year. It is yet to be seen if I can keep this up in 2012.

2. Rental cars need love too

I learned a couple good lessons about rental cars this year. First, if you attend a conference and park your rental car right under the awning of a luxury golf resort hotel. And because you travel all the time you *think* you remember that you had it valeted when you didn’t. You may have an awkward moment 22 hours later when the very large head of security finds you by having the local police track your license plate. And an even more awkward walk through the gauntlet of freshly pressed security guard suits while you are holding the keys in your hand to go move it.

Also, a nice tip to remember is that push button cars require you to push the button to turn them off. And 4 hours of idling in a parking lot will make a 2011 Dodge Charger a refrigerator with foggy windows if you happen to have left the AC on MAX…

3. Vegas can get old

Seriously, not trying to make those that didn’t get to go to Vegas tons of times feel bad. But, after calling Vegas hotels home for more conferences that I can remember in 2011 I am ready for some more Boston, San Francisco, or Orlando in 2012.

4. What being a zombie is like

As part of the prep for making the EMCWorld vLabs work this year both myself and my team lost a LOT of sleep. In fact that was the first time I saw a human being work for 40 hours straight without sleeping and still be lucid (Chris “The Rock” Horn). There are gaps in my memory that I will never fill. But when the fog lifted and the labs rocked I don’t think I have ever had quite such an awesome feeling. I think I can still walk from a Venetian hotel room through the casino and into the convention center by sound and smell alone.

5. If you build it they will come

I have had a massive amount of downloads from both the older VSA’s and newer tools (UBERAlign for one). YTD comes out to over 30,000+ downloads with the majority being over 2GB in size. Nothing makes me happier than some crazy idea in my head making someone’s life easier.

6. I know jack about Cloud

This is a tough one to admit. But, 2011 was the year where I ended it questioning whether I had a decent grasp of cloud computing like I thought I did. I believe the key to development is knowing you need to know more. And I really want to dive into the guts of the evolving cloud paradigm from more than just a virtualization or storage perspective.

7. People matter

From my incredible teammates, my managers, my close friends, my family, and my community friends I have been supported in so many ways. Leaving my old vSpecialist team was very hard. And I miss their infectious interactions. I am looking forward to working closely with more people in 2012, meeting new rockstars I can learn from, and hopefully positively affecting others.

2011 was a good year. I am looking forward to what this next year will be like. Thanks for visiting Nickapedia.com in 2011.

.nick

At first I was a little taken back. I think @Beaker thought I didn’t get the point (and I don’t more than I do).

So I *cheated* and just turned on Web Sharing with the good tips from my other twitter buddies. But the Node.js idea bugged me because technically Chris is right. It should work fine.

So I did it. I installed the Node.js package for OSX (been using the Linux version) and fired up TextEdit and wrote this simple bit of code:

var http = require('http');
var fs = require('fs');
var path = require('path');
var rootpath = "/Users/lynxbat/Webroot";
 
console.log('*Starting');
http.createServer(function (request, response) {
 console.log('Server started');
 var pathToFile = rootpath + request.url;
 console.log ('File was requested: ' + pathToFile);
 path.exists(pathToFile, function(exists) {
 if (exists) {
  fs.readFile(pathToFile, function(error, content) {
   if (error) {
    response.writeHead(500);
    response.end();
   } else {
    response.writeHead(200, { 'Content-Type': 'application/x-gzip' });
    response.end(content);
   }});
   } else {
     response.writeHead(404);
     response.end();
   }});
}).listen(8010);
console.log('Server running at http://127.0.0.1:8010/');

I fired up the server:

node webserver.js

And opened my Safari browser to request a file I had dropped in my root directory.

Which works perfectly. Output on cli:

*Starting
Server running at http://127.0.0.1:8010/
Server started
File was requested: /Users/lynxbat/Webroot/node-v0.6.6.pkg

Now in my case I am using the Webserver to hand out packages for installing as a local mirror for some VM’s I am building (big demo for Jan). So I hardcoded the MIME type and left response encoding unset. If you were looking to properly serve you should perhaps add some logic for switching types and responses based on the file extension. Kind of like a proper web server works.

So Chris was right. This works. Though I think proving him right probably isn’t a good habit to pick up.

.nick

What is interesting about the intra-IT delineation of suspect proficiency is that the ever-shifting world of technology and its self-evolving nature lead to new roles, terms, and sudden expertise in a constantly changing landscape. While the COBOL programmers were a hot item back when I was in diapers, most of those folks evolved to fit other roles as time marched on (but not all).

This evolution of new boxes to put people in can be driven by market dominance demanding new silos of expertise. Being an expert in Microsoft, VMware, Oracle, Cisco, EMC (obligatory plug) and many more can be directly linked to the success in the product’s dominance in some facet of the market. And to that extent we all should admit that the further you develop and professionalize yourself for a silo product, the more invested you are in their success.

The scope of this can also be driven by the degree of depth needed. Disciplines driven by deep foundations (such as the Cisco CCIE program) or some of the requirements for being an expert in intimidating tech (like Linux Kernel development) lead to highly paid fur-lined boxes for specific needs. This is based more on a technology skill being a bottleneck, or maybe we should say enabler for the needs of an organization. It can be argued either way whether this is more prevalent in software development or infrastructure circles.

Though the finely tuned and specific box for the Exchange guru, or the Oracle database master, and the C# event-driven wizard is easy to understand, well-defined silos are not the only type of stereotype to apply. Many individuals out there claim the *generalist* title and claim it well. They possess broad abilities to understand software stacks from multiple vendors as well as (and maybe even more importantly) how to layer and integrate vendor components together. I see the root of this type being based on critical troubleshooting and analysis skill sets. Those with the ability to understand and consume a tech quickly and then offer immediate value eventually learn that this has demand. These titles usually fall under more generic names like Network Administrator or Senior Systems Engineer. And when these generalists grow up they can develop into nicely titled generic boxes like Infrastructure, Software, or even Cloud Architect(my new favorite). These boxes are a loose analogue to hold this individual to a semblance of definition for their broad ability to connect and understand dots.

The real question is, how the paradigm changes for our somewhat newly evolving cloud constructs will affect these definitions? How will we slice ourselves into containers and how will existing skill sets migrate to them? In my mind I see the effects being completely different depending on where you start.

Let us start with an example of a powerful Infrastructure Architect. In their current world, value for them comes from an innate or trained ability to assemble layers of hardware and software stacks into consumable services. Whether as an internal role or as a service sold to a business as a consultant, they can take the complex and varied and produce a result. They do not need the ability to debug a kernel or write a endpoint for a service bus to accomplish their job. What is interesting is how the new infrastructure or platform as a service models(IaaS, PaaS) affect them. Suddenly organizations are being presented the option to purchase stacks that come pre-built for internal use or offerings that provide the required services and frameworks immediately as an instantiated third-party platform.

Now we have an issue for our generalist. While the adoption of these new *aaS models will take time, steadily and to some degree his or her value proposition will become diminished. Why does a business need a smart and savvy architect to align the stars to make our services work if we can buy them for purpose and on demand? Maybe they might have a role in evaluating new service platforms; but in itself this might be limited and a conflict of interest. With IaaS (public/private) for hosting of vertical applications like Exchange, SAP, Oracle, and more becoming solution-based offerings on a menu. And with PaaS services now offering enterprise architects and developers the ability to demand and develop on their own, where does our infrastructure expert move to next?

I think the answer is two-fold. In reality the demand for a person with broad skills and maybe even more importantly, possessing a broad-mindset, does not actually go away. Rather, it just relocates to be more aligned with the newer model. If the consumer of platform and infrastructure services no longer has to assume the responsibility to build and create them, then the demand moves to whomever now provides that service. As time moves on and business start expecting an IaaS or PaaS service to meet their needs, the demand for these providers will grow. This new galaxy of platform and infrastructure suppliers will need highly skilled and broad-minded generalists to help them build a better mousetrap.

And while we are on the subject of mousetraps, this becomes an important driver also. As adoption of IaaS and PaaS develops, demand for different levels and specializations of an *aaS will surface. Some services will spring up that need to meet specific demands by verticals such as healthcare, government, or even really narrow ones like insurance providers. Now providers we are in a position where expertise in these fields in building broad platforms while understanding the basic tenant needs will be valuable. And now, our infrastructure architect with 10 years experience in the healthcare industry suddenly is valuable again. She or he is now a critical asset in helping shape an *aaS that meets demands. They become an important cog in building the better mousetrap for our business consumer. The same skills in understanding vendors, layers, and models are just as important for a provider as they were as part of a business consumer.

And I would venture that this creates an even more specialized role for a broad skilled individual. In a previous life, they may have had to deal with business demands that led to environments with nice clean lines of separation between infrastructure and development zones of control to make things work. But in an *aaS world, things may naturally be more focused.  Providing a well-defined and measured service is critical to the success of a cloud service. The greater the ability of our architect to have in-depth understanding of selected codebases and deep integration levels, where maybe they were not so deep before; the better equipped they are for their business goals. Now we have an interesting situation where broad skills could possibly evolve into more focused experience in delivering value. This also makes an interesting case for why something like DevOps is becoming a hot topic in the convergence of operations and development silos.

Perhaps our generalist does not necessarily lose value after all with cloud constructs. Their value migrates in the new model. And maybe in the end the broad abilities become more narrow and specific in a quest to deliver that better mousetrap.

Now let us switch back to our focused expert, our development guru in .Net or our data analytic expert. It is much tougher to predict how cloud models will affect an individual with highly specialized skills. Really, the same rules apply above in many ways. You would expect some roles in networking, database or development silos to remain in demand and just slowly merge across service-consumer and service-provider lines. But in the end, some may always remain within medium to larger orgs. It is likely that some will also phase out as technologies begin to be replaced completely by cloud offerings. This of course means people in those roles may have to migrate to a newer way of doing the same thing; which is something that has happened many times over in the past.

But what is more interesting at a choice level is how things like PaaS enable more options around platform choices. A good example is recent announcements around .Net enablement of formerly primarily open-system focused language PaaS platforms like Cloud Foundry (see: Iron Foundry and Uhuru). Which opens up interesting thoughts on whether that changes *how* developers are specialized.

Lets say I am a developer that has used and abused .Net since C# was barely a twinkle in Anders Hejlsberg’s eye. I work for a banking institution that has moved from building applications in-house and hosting them on infrastructure managed by an outsourcing firm, to using a newly minted PaaS provider with a focus on banking vertical. In my previous life I would find a way to solve a problem using my core codebase. But now, my new PaaS provider not only has great options for using the .Net framework- they also offer options for Ruby, Node.js, Scala, and other frameworks. Instead of limitations because of integration and build hurdles by the outsourcing firm(expense/time) and even my own feelings of safety by remaining on my core knowledge-base; I am now presented with easy avenues to utilize codebases I never had before. And because the path is easier, my excuses for not using the best-of-breed development solution for a given service model or core component become fewer. It presents an interesting question whether the new multi-codebase PaaS cloud service eventually causes developers to become more broad in the way they look at attacking problems. If I can use the same development toolkit to deploy .Net, C++, Ruby, or  Scala to the same PaaS platform without having to worry about obstacles normally inherit in internal enterprise solutions, how does that change my personal development?

It presents an interesting idea of the multi-disciplined developer who uses any given framework available via his PaaS provider(s) depending on the core need. They could approach module building differently than the old paradigm by using Scala for building resilient APIs, C# for data presentation, Node.js for Map/Reduce streaming, and Ruby for user portal creation/extension or any other permutation. More than this, exposed constructs around normally infrastructure-based terminology can be used by developers in modern IaaS/PaaS designs for cool orchestration, isolation, and metrics in a new way (one barely out of the gate example).

As the languages, models and more become consumable by internal development teams and the walls dividing languages and frameworks break down- we are left with an interesting scenario for our silo-trained experts. And while it is impossible to predict outcomes, I predict many individuals branching out into new tech and new ways of doing things. No longer is their proficiency domain sacred in this new world. I offer no other proof than the simple example of how Microsoft themselves both helped the development(iisnode) and released Node.js on their Azure platform recently. I think this speaks to the wisdom of looking at languages and frameworks as tools and the viability of multiple codebase options with PaaS as a model.

And what we are left with is interesting if we step back. The narrow-focused expert may end up broadening how they apply their efforts in building value. It may no longer be about how to be the best with a hammer since it is the only tool you can bet on. It may be about knowing the difference between the hammer and the nail-gun. And even more importantly, knowing when and where to use each.

The key to either change is that responsibilities may move but in the end people are still required. What I personally find fascinating is how valuable individuals with curiosity for knowledge and passion for communication will have the opportunity to excel. A Infrastructure Architect who spends his/her time really understanding the needs and constructs of a vertical-focused PaaS could be a boon to a new startup. There are opportunities in these new worlds for ideas to provide major innovation and market value. Especially around stuff like DevOps integration(process & toolchain), infrastructure automation/modeling for IaaS, and fast-to-market development frameworks for PaaS.

Another interesting thought is how this may affect vendor-focused specialties. Someday being specifically an expert in a single vendor may be less useful than being an expert in the application of the common model used. I think advanced training and hands-on experience with a leading vendor’s technology will always be valuable, especially ones that stay current with cloud adoption and model their own products accordingly. And as always, this is usually controlled more by the market dominance and success of the vendor than anything. But, this is definitely something to watch as time goes by for whether it morphs into something different.

Having said all that let me get to the point of this blog post and say this directly to you. If you are a person that fits in a box like I described above and you are wondering where you fit in the crazy world of change- take heart that this is more of an opportunity for development than a dead end. Nothing changes overnight, and much has yet to be proven and produced. Because of this, there is going to be a massive demand for individuals that are capable of evolving into new roles for cloud models. There will be a need a combination of strong foundations in technology and forward-thinking mindset to attack the impossible in a new way. Or in a more concise and less dramatic way of saying it: technology will always need good people.

I believe that critical mass is reached where several of the IT containers we have placed ourselves in start to converge and evolve through the need for communication, common understanding, and the enabling technologies like IaaS and PaaS. Some see cloud as a threat to their career development paths. Some see it as marketing labels and hot air. I see it as the next evolution of how IT defines itself. A reordering of roles and ideas. Nothing new, just the next evolution of technology. And I for one, welcome our new cloud overlords.

.nick

As part of this I have started using KVM as a hypervisor on an Ubuntu 11.10 platform in a portion of my home lab. I have learned quickly that getting something simple done in vSphere can be a bit of a chore in the KVM world. But on the flipside, KVM has been a fun learning experience in understanding virtualization in a more raw format.

One of these challenges I have decided to share is a simple one. I was wanting to play with the Dell-created DevOps deployment tool: Crowbar. Crowbar is an wrapper for OpsCode Chef Server. While a pretty slick little utility to research in the cloud deployment and automation space; one glaring problem is it is designed to run on Dell PowerEdge servers. Since I don’t have PowerEdge servers lying around anymore I needed to run this in a virtual machine. This in itself isn’t a huge problem as a virtual machine can pretty much match most of the logical hardware pieces needed. But, the one problem I ran into was that Crowbar out of the box likes to have a couple interfaces with the ability to tag VLANs itself. In a bare-metal world connecting 802.1q trunk ports to a server is pretty common. And even in the VMware vSphere world you can create trunk portgroups for a guest VM using documented methods(See VGT). But the problem I ran into is how do I pass a trunk port through in KVM/QEMU when using the default bridge-utils setup (not using ovswitch in this case)?

It turns out that quite a lot of articles out there on KVM explicitly state that this isn’t possible (link, link). Seems like the default behavior strips tagging between bridge connected interfaces which defeats any attempt to trunk to a guest VM. So it seemed it was not possible.  And at one point after testing myself I was convinced it was not possible too. But, thanks to a teammate who somehow got it working in his lab I decided to press on further and see if there was a way to configure bridge-utils in Ubuntu 11.10 to pass VLAN tags frames in a way that behaves like a trunk port to its guest VM.

And it turns out that not only is it possible but it is also not that hard after all.

My baseline Ubuntu is the 11.10 Server edition with KVM and bridge-utils installed. Here the list of packages installed via apt-get after a clean OS install:

• openssh-server (SSH)
• ubuntu-desktop (for using virt-manager)
• gtk2-engines-pixbuf (see above)
• vlan (need vlan ability on host)
• bridge-utils (base bridge for KVM guests)
• qemu-kvm (the KVM/QEMU packages for Ubuntu 11.10)
• kvm-pxe (needed for PXE and to stop annoying error messages)
• libvirt-bin (Libvirt API for managing my KVM)
• virtinst (Handy python tool for creating guests)
• virt-manager (Cool GUI interface for managing KVM guests)
• virt-top (‘top’ like tool for guests)

And for those that want to past a nice clean line into a shell window

apt-get install openssh-server ubuntu-desktop gtk2-engines-pixbuf vlan bridge-utils qemu-kvm kvm-pxe libvirt-bin virtinst virt-manager virt-top

After installing packages above we need to do a couple steps to configure our host system bridging and networking.

First let’s activate our VLAN module now:

modprobe 8021q

And we need to drop it into our modules file for loading next boot:

echo “8021q” >> /etc/modules

By default our Libvirt will come configured with a default network that will create a virtual bridge called “virbr0” which is setup for NAT to internal guests. In this case we want to use our own custom bridge and remove this virtual one. You can skip this if you want both.

Stop the network if active:

virsh net-destroy default

Undefine (remove) the network from the libvirt config:

virsh net-undefine default

Restart our libvirt service:

service libvirt-bin restart

By default Ubuntu 11.10 will come with some rules setup to allow iptables/ebtables to do filtering on packets passed on bridge interfaces. You have two choices with how to deal with this. You can either setup nice clean rules in iptables/ebtables that allow traffic (including VLAN tagged) through or you can stop them from filtering in the first place. The latter is much easier to do and what I am showing below. If you want to do the former then hit Google up for articles on how to enable filtering with iptables/ebtables and bridge-utils.

First let’s add our bridge module in earlier in the boot process:

echo “bridge” >> /etc/modules

Now open /etc/sysctl.conf up in VIM or Nano and add the following lines to the end:

net.bridge.bridge-nf-call-ip6tables = 0
net.bridge.bridge-nf-call-iptables = 0
net.bridge.bridge-nf-call-arptables = 0
net.bridge.bridge-nf-filter-pppoe-tagged = 0
net.bridge.bridge-nf-filter-vlan-tagged = 0

These lines disable any filtering of frames on the bridge devices.

Let’s enable the new settings with this command:

sysctl –p

And we add this command to make sure these settings are applied last when our host server boots. Add the following to /etc/rc.local above the ‘exit 0’ line:

/sbin/sysctl -p /etc/sysctl.conf

Now in my setup I am using a host with two network interfaces both of which are presented trunk ports from the attached switch. The first interface (Eth0) will be used for Guest VM traffic and the second (Eth1) will be used for host server management (think: vSphere Service Console). While we will manually create the networking config in a second; we need to have a startup config present so we don’t have to do this every time we boot up. Here are the contents of my /etc/network/interfaces:

# Loopback
auto lo
iface lo inet loopback
 
# eth0 used by br0 so we put it in manual mode
# and add ‘up/down’ commands to enable it
auto eth0
iface eth0 inet manual
up /sbin/ifconfig eth0 up || /bin/true
down /sbin/ifconfig eth0 down || /bin/true
 
# eth1 is used by host for management so we leave dhcp
# enabled. This can be replaced with static config also
auto eth1
iface eth1 inet dhcp
 
# We need to make sure our eth0 interface is 8021q enabled
# We do this by adding an arbitrary VLAN 0 subinterface to it
# In manual mode so it should never actually enable by init
# vlan-raw-device is required to map when in manual mode
auto eth0.0
iface eth0.0 inet manual
vlan-raw-device eth0
 
# We create our bridge interface here
# Since eth0 can’t be used as a host interface I am using br0
# as one itself. By enabling dhcp it will pull an IP and behave
# just like eth0 for the host itself. You can also put into manual
# mode but must uncomment the ‘up’ line to make sure it comes up
auto br0
iface br0 inet dhcp # ‘dhcp’ can be ‘manual’ for no IP or ‘static’ for a static IP
bridge_ports eth0 # Add our host VLAN-enabled interface to bridge
bridge_stp off # disable STP
pre-up /sbin/ifconfig eth0 up || /bin/true # make sure eth0 is started
#up /sbin/ifconfig br0 up || /bin/true # uncomment if br0 in manual mode
 
# You can also have a host interface on the bridge for a specific VLAN.
# This is optional and needed if using single interface host. This is not
# required to make VLAN trunking work.
# My host will get a VLAN 100 network DHCP address for a br0.100 subinterface
auto br0.100
iface br0.100 inet dhcp
vlan-raw-device br0
# Nicholas Weaver
# last mod: 11/28/2011

Once you customize the file above and update your interfaces file you can actually just run ‘/etc/init.d/networking restart’ to apply the changes. But let’s walk through the manual config also to see the steps:

# Clear any IP address on eth0 as it won’t work
# and routing will be hosed once the bridge owns the port
ip address flush eth0
# We need to enable VLANs on our source interface (eth0)
vconfig add eth0 0
# Create our bridge
brctl addbr br0
# Add our interface to our bridge
brctl addif br0 eth0
# Get an IP address for eth0 on the br0 (optional, see above)
dhclient br0
# Create a VLAN subinterface for host VLAN traffic on eth0 (optional, just an example)
vconfig add br0 100
# Get DHCP address for br0.100 from the VLAN 100 DHCP server
dhclient br0.100

The commands above result in a single bridge (br0) attached to an interface (eth0) that is enabled for VLAN tagging. At this point we can create a guest VM and test. I recommend using the ‘ubuntu-vm-builder’ package available via apt-get or ‘virt-install’ which we already installed. What is important is that with our current config we are no longer using the default network that comes with Librvirt (virbr0). So some changes need to be configured in the guest domain XML in Libvirt to bind virtual interfaces in the guest to our newly created bridge. Here is an example of how to modify your guest XML:

    <interface type='bridge'>
      <mac address='52:54:00:ba:ba:bf'/>
      <source bridge='br0'/>
      <target dev='vnet0'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </interface>

As you can see above the interface type is changed to ‘bridge’ and the source bridge is pointed at ‘br0’. This means when Libvirt starts our KVM Guest in QEMU it will place the Vnet0 interface from our guest on the Br0 bridge. We can then create subinterfaces inside the guest for Eth0 (which is Vnet0 on the host) and tag traffic for transmission our our VLAN-enabled Br0 & Eth0. In my case I created the default Eth0 and an Eth0.100 and was able to communicate on both the native VLAN and VLAN 100 from within the guest.

Now one last important note for those running some distros underneath KVM. I had issues with VLAN tagging being stripped by the guest VM before it was submitted to the underlying bridge even though VLAN tagging was enabled in the guest. Turns our that the config above automatically chooses Virtio as the interface type used. And in my case my Crowbar VM was build for Ubuntu 10.10 and was not correctly installing the driver for the Virtio network device it was presented. My solution was to switch to the more compatible E1000 device type (also slower) and tagging started working like a charm. Here is an example config with ‘model type’ specified:

    <interface type='bridge'>
      <mac address='52:54:00:ba:ba:bf'/>
      <source bridge='br0'/>
      <model type='e1000'/>
      <target dev='vnet0'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </interface>

So if you are doing as I am and testing the Crowbar VM inside KVM you will need to use E1000 to get the tagging working.

Couple of last cool notes. The entire config above was actually done inside VMware Workstation 8. It does work with VLAN tagging between VMs on the same Host Network. So in my lab I am using VMW –> KVM –> Guest VMs for some great Inception-style networking.

Also, it is possible to configure the all the above as a Libvirt defined network in newer versions of Libvirt. But by default the libvirt-bin package installed via apt-get for Ubuntu 11.10 is 0.9.2 which is missing some of the options we need.

I have more posts coming on some of the stuff I am playing with soon. So stay tuned.

Comments/corrections/free-food welcome.

.nick

1. The CLI

Each vAligner is a Ubuntu Linux VM. On the VM is a set of binary files that do all work. One is a startup file for initializing, one is a daemon for accepting new jobs via the REST API, and the final is the actual magic behind the scenes. From the beginning UBERAlign was designed to be run from the command line. In fact back months ago the vSpecialist actually got a copy of this to try out and help me test. So for those that do not want to use the Console here are some reasons and instructions on how to run alignment, reclaim, and alignment+reclaim jobs manually.

Some of the reasons you may want to do this:

1. Hate MS Windows – Since the console is a .Net WPF app some Mac users (@mcowger) have already asked how to skip using a Windows VM.
2. VM size is too big – If the VM is more than 50% of the size of the datastore then a snapshot of it can potentially cause an out of space issue if it grows to full size (which and align can do). So if you have a good backup or are using array-based snapshots why use VMware snapshots? The CLI allows you to skip having to use a VMware snapshot which is required for using the Console.
3. Aligns disks not attached to a VM -  The manually method just needs a valid VMDK file. Whether from a backup or anywhere. The Console requires a VM. Technically you can use tricks with snap and copy similar to the way VMware Converter works with online VM’s if you want.

So here are the steps:

1. You must have the vAligner running and it must be on a vSphere host than can reach the datastore your VMDK is on.
2. Make sure that any secondary disks are removed from the vAligner from a previous job.
3. Make sure no UBERAlign Consoles are using this vAligner. Easy way is to disconnect the NIC and use the console if you are worried. If you console is showing weird FD0 garbage, grab a new copy of the vAligner OVA as this was patched.
4. Add a Hard Disk to the vAligner (while it is running) to SCSI 0:1. Make sure and choose an existing disk and point it to the VMDK you want to run against.
5. Login to the vAligner with root/UBERAlign.
6. Make sure at least 15-20 seconds have passed since step 4. From the command line run: ‘fdisk –lu’ and make sure /dev/sdb is showing. Confirm the partition layout is as expected.
7. Change directory to the UBERAlign binaries with ‘cd /opt/uberalign/bin’

Now that you have everything attached we are running to run a job. If you run ‘./uberalign’ you will see a printout of the command syntax like what follows:

./uberalign MODE  DEVICE  [OFFSET]  [ID]  [NAME]

MODE = The mode you want to run in and is formatted. Syntax:  -r:[a|r|s|c]

• [a] = Align & Reclaim
• [r] = Align only
• [s] =  Simulate
• [c] = Check alignment only
• [z] = Reclaim only

DEVICE = The hardware device your VMDK is mounted on(normally /dev/sdb): Syntax: /dev/sdb

OFFSET = The target offset you want to align to. This defaults to 2048 sectors if not specified.

ID / NAME = Only useful for the API leave these off or use for logging purposes.

Example:

./uberalign –r:a /dev/sdb 2048

Once you run this command the uberalign program will process and log to the screen it’s progress while it works. When it completes, disconnect your VMDK from the vAligner and check your VMDK within a VM. It is as simple as that.

2. The API

The UBERAlign vAligner comes with a REST API that the UBERConsole uses for managing. While the state machine, orchestration for vSphere, and GUI are all a part of the console; you can still use the vAligner API to integrate into any existing processes you may have. Here is a short guide to the REST API. All data is return in JSON format. The first section explains the API get/put and the second explains the object structure in a pseudo format.

Methods

Root Path: http://<valigner IP>/uberalign/api/

Get Current State (GET)
Path: /uberalign/api/state
Parameter: n/a
Return: UAState
Description: Returns the current state of the vAligner

Get Current Job (GET)
Path: /uberalign/api/job/current
Parameter: n/a
Return: UAJob
Description: Returns the current job of the vAligner. On first boot this is an rather blank object.

Get All Completed(Historical) Jobs (GET)
Path: /uberalign/api/job
Parameter: n/a
Return: UAJob[]
Description: Returns an array with all completed jobs. This list will include all jobs still the vAligner was first deployed. Can be useful for historical purposes.

Get Session (GET)
Path: /uberalign/api/session
Parameter: n/a
Return: UASessionLock
Description: Returns the session lock information. This object contains information on the current lock status of the vAligner.

Lock vAligner (PUT)
Path: /uberalign/api/session/lock
Parameter: UASessionLock
Return: string (“true” | “false”)
Description: Used to lock a vAligner for 30 seconds. A locked vAligner will not accept new jobs from another source. Once a job is started it will not accept a new job until idle again. Use this method to maintain a lock on a vAligner while using it. Must specify a unique GUID that must match the GUID in the job ticket. GUID is a string and should be unique and not change for the duration of the session.

Submit New Job Ticket (PUT)
Path: /uberalign/api/job/new
Parameter: UAJobTicket
Return: string (“true” | “false”)
Description: Used to submit a new job ticket to the vAligner. A job ticket goes into a queue and is picked up by the daemon <15 seconds. The submitted UAJobTicket must have the same GUID as the lock and a lock must exist. Also the vAligner state must either be in Idle(0) or IdleWithError(9). You can use both the Get Current State and Get Current Job to watch for a job starting. The Current Job returns the UAJob object which will have a Name and ID that will match the  UAJobTicket Name and GUID fields you submitted.

Object Definitions:

Class UAState
{
string ip;
string errmsg;
string mac;
States state;
}

Class UAJob
{
string id;
string name;
string disk;
UADiskDetails diskdetails;
int offset;
States type;
string errmsg;
string currentstep;
string laststep;
double duration;
UAPartition[] partitions;
string msg;
string timestamp;
string completetimestamp;
}

Class UAPartition
{
string _system;
string _aligned;
string _id;
string _device;
string _offsetdiff;
string _boot;
string _start;
string _blocks;
string _end;
}

Class UADiskDetails
{
string _cyclinders;
string _size;
string _sectorsize;
string _totalsectors;
string _sectoratrack;
string _heads;
string _diskserial;
}

Class UASessionLock
{
string guid;
string ip;
int timestamp;
bool locked;
int secondsleft;
}

Class UAJobTicket
{
States type;
double offset;
string id;
string name;
string guid;
}

Enum States
{
Idle = 0,
AlignNoZero = 1,
AlignWithZero = 2,
Simulate = 3,
CheckOnly = 4,
ZeroOnly = 5,
GrowOnly = 6,
IdleWithError = 9
}

The objects above are simplified from my C# class structure.

This workflow for a job submission would look like this:

1. Check vAligner state (is idle?)
2. Lock vAligner (and continue to re-lock <30s until done with it)
3. Mount Disk using vSphere API/Manually
4. Create and submit new Job Ticket
5. Watch vAligner State and Current Job to watch status
6. After seeing the job completes (state=Idle or IdleWithError) un-mount disk

I know what some of you are going to say now: “But, Nick – how can I use the API now?”. The answer is provided by the vSpecialist rock-star Clint Kitson in his EMC Community release today. Clint built an awesome example of using Powershell to integrate and control UBERAlign via the REST API. This is so cool because if you already have scripting/code toolsets you use for automation, you can integrate UBERAlign right in. Check out more on this in Clint’ post here.

This is a basic overview and I am sure will probably just lead to more questions . Please feel free to play around with both the CLI and API and post questions/comments below.

Thanks,

.nick

You know how in cartoons they show a small snowball rolling down a hill until it grows into a massive beast of a snow boulder?

Well, that is kind of how my most recent UBER project has gone. I know it is a been a little while since I have released a tool for the community and I am hopefully making that up with my newest creation: UBERAlign.

The idea of creating this came from the lack of a decent free alignment tool out there for VMware admins. Most every other one at there was either something you had to purchase or you had to be a customer of the vender to get access to it. And even after getting access these tools were either (in my opinion) limited in what they did, how they did it, or had become obsolete in a console-less vSphere 5 architecture.

For those they don’t know, alignment with Virtual Machine disks on top of Storage Arrays has been a performance issue for a long time. I won’t go into long detail explaining the problem or the benefits to alignment. There are great posts by Duncan (http://www.yellow-bricks.com/2010/04/08/aligning-your-vms-virtual-harddisks/) and Kevin(http://www.blueshiftblog.com/?p=300) on what the issues are and some of the tools available.

So my goal in creating UBERAlign was to once and for all create a free and powerful tool resolve this issue for everyone in the VMware community. Along the way I ended up probably overdoing the way I went about it. Turns out there are issues with aligning Windows boot disks, drive letter mappings getting lost on changes, current tools requiring copying of the whole disk, and tools frying NTFS metadata. I ended up deep in the weeds learning more than I have wanted to about NTFS, MBR/GPT, and disk logical formats. I also ended spending days writing data movement algorithms on my whiteboard over and over and I tried to solve specific problems with aligning a VM. And along the way I realized I had an opportunity to solve another issue that did not have a good universal answer.

So let me cut the background story down and get right to the details.

Presenting UBERAlign, a tool for VMware Virtual Machine alignment and Space Reclamation

Features:

• Allows for fast alignment checking of virtual machines with detailed logging.
• Can perform alignment to any offset you want. Even the crazy ones that you shouldn’t choose.
• Works with both Windows 2000/XP/2003/2008 (NTFS) and Linux Distros (EXT2/EXT3/EXT4).
• Is able to work on NTFS boot drives perfectly. It does this by rewriting NTFS Metadata (the right way).
• Auto detects Windows 2008 and Windows 7 native installs (alignment not needed). Will not touch a System Reserved Partition (important for Windows 2008).
• Preserves all Windows drive mapping (AFAIK only one to do so). This means no having to remap drive letters and complete support for non “C:\”  system drives with some Windows builds (some Citrix stuff).
• Doesn’t trash the NTFS and Boot mirrors like other tools.
• Handles Primary and Extended partitions like it is no big deal on both Windows and Linux.
• Has the ability to handle multiple disks for a VM.
• Multiple disks + Multiple Partitions + Multiple types (primary, logical) + Multiple file systems (NTFS, EXT#) =  no problem
• Also allows for optional Space Reclamation on both NTFS and Ext! That’s right: you can choose to do space reclamation at the same time as an alignment or as a option to itself. This means you can retrieve space no longer used on Thin VM’s using UBERAlign.
• Operational model allows for completely CONCURRENCY with processing VMs. You can run up to 6 simultaneous jobs per Console and as many Consoles as your VCenter can handle. This was designed to allow people with big environments to process through a large set of VM’s.
• Options to check, align, or reclaim any choice of disks in a VM.
• Powerful very simple to use graphic console and easy to deploy OVA’s.
• Orchestration for batch operations allowing you to process groups of VM’s with just a couple clicks.
• Getting started is simple with just entering VCenter credentials/IP and pointing at a vAligner.
• Space Reclamation should also help with possibly speeding up defragmentation of some NTFS file systems after. Your mileage may vary.
• Space Reclamation can help you turn a thick VM into a thin one and actually get the space back!
• Does all operations IN-PLACE! My first big goal was this. No more having to copy disks using the ESX command line(especially since ESX is going away). This will process a VM’s disks in-place.
• Automatically makes a snapshot before running for failback. If you turn on your VM to check it and see anything you don’t like you can simply revert to the UBERAlign snapshot and be right back. (You should always have a backup and test also, see prereqs)
• Automatically rolls the snapshots back if it sees an error. UBERAlign has the ability to do health check throughout the jobs and if it sees something wrong it will roll back it’s own snapshots for you.
• Automatically enables CHKDSK scanning on each NTFS volume on the next boot.
• Completely Storage Array agnostic. That’s right: if it connects to vSphere and host storage UBERAlign will work with it. This includes local disks (see prereqs below) and arrays other than EMC. Don’t say that the EMC vSpecialists don’t love all VMware users.
• Completed tested against vSphere 4.1 / 5 environments.

So as you can see UBERAlign got to be a bit of a beast along the way.

UBERAlign comes in two pieces. The UBERAlign Console which is a graphical interface meant to be run on a Windows XP/7/Server system with .Net 4.0 or greater. And the UBERAlign vAligner vAppliance which is deployed from OVA into a vSphere environment. The console connect to the vSphere via SOAP and to up to 6 vAligners via REST interface. The way it works is: each vAligner can process VM’s on the storage the vSphere Host it lives on has access to. So you should spread vAligners across clusters and make sure one is on any vSphere Host that has local storage you want to access.

These features are meant to make the life of a VMware admin much easier when taking on this kind of task. But, let me cover some prereqs and how-to information.

Prerequisites/Tips/Caveats:

• All VM’s must be turned off to run any operation on. I won’t go into detail on why in-place + on is risky but suffice to say I wasn’t going to give it to you
• Console has been tested on Windows 7 SP1 and Windows 2008 R2. You need the very latest .Net 4.0 updates also.
• As said above: only VM’s powered off and residing on storage visible to an attached vAligner will be selectable. You don’t have to restart the console but it may take up to 60 seconds for you to see a newly turned off VM.
• Only vAligners managed by the VCenter you connected will be useable. One VCenter per console.
• Concurrency is based on the VM level. Which means each vAligner can handle and process through different VMs. But, a single VM with multiple disks will not be split across multiple vAligners. They will process in a linear fashion one at a time.
• If UBERAlign detects an error on a VM disk when a previous disk was processed it will revert the previous jobs also. This is because snapshots are handled at a VM-level.
• When doing multi-disk Windows VM’s it is recommended to align them all and to use the same offset. When the System disk is processed it assumes all the disks will be done and at the same offset. You don’t HAVE to do this but you may have to remap drive letters otherwise (not a huge deal, just annoying).
• In order for Space Reclamation to work you must boot VM (check that it is healthy), delete all snapshots, svMotion to a different datastore while specifying that you want it thin (important). Another option is to clone the existing VM from a power off state to a new VM on another datastore while specifying Thin for the disk format.
• By default *Natively* installed Windows 2008 and Windows 7 installs don’t need alignment. Upgrades from Windows 2003 do. But Space Reclamation works on all of them.
• Space Reclamation does make alignment jobs take about 15-20% longer. It all depends on the speed of the storage underneath.
• You need to have at least 20% free space on any NTFS volume to safely align/reclaim. This can be less on a very large volume but is a safe rule to follow. If you have to, expand a drive to make a little extra room. The alignment check reports information you can use to check.
• vAligners currently pick up a DHCP address. You can view what the address has become by looking at the info pane in VCenter. Or you can set an IP manually. The vAligners are running Ubuntu and the login is root/UBERAlign.
• I have tested the console over WAN (Texas –> North Carolina) and it works very well.
• LVM’s are NOT supported. Long story but #1 the test cases for this are crazy and difficult to QA and #2 technically LVM gives you its own options.
• UBERAlign is aware of how much storage space is used/available within a datastore. It keeps track and will not start jobs if there is a possibility it could cause an out of space error.

*DISCLAIMER*
Ok, before I go any further I want to mention one important thing. UBERAlign is an experimental tool and carries no support from EMC Corporation or myself (Nicholas Weaver). It is being released in beta state and while it does have functions that allow for failback you should only perform operations on Virtual Machines for which you have a solid backup. Also, I recommend you test in your lab thoroughly to make sure you understand it fully. You accept full responsibility when you use this tool.
*DISCLAIMER*

Ok, now that that is out of the way here are a few videos showing off the console and some of the cool things you can do (these are some quick/rough cuts):

And now for the downloads:
(New links are HTTP, no more FTP issues)

UBERAlign Console 1.1 – FTP

UBERAlign vAligner OVA (~500 MB) – FTP

This was a long process for me. It was something I started in June and had to delay because of VMworld 2011 and side projects. It is my last tool as a vSpecialist (more on my move here) and I hope it helps the VMware community somehow. The good news is this was designed from the beginning to be hypervisor agnostic so watch for a Hyper-V and Xen version in the future.

I also want to thank Larry Whitlock (EMC vSpec) who was the primary tester during the harder part of this. Without his help I would not have gotten anywhere.

Questions, comments, and critisism are all welcomed.

.nick