DoubleCloud

Ruby to Manage and Automate VMware vSphere?

Steve Jin — Mon, 30 Aug 2010 05:01:26 +0000

As I mentioned in a previous blog, vSphere(VI) Java API can be used in any JVM languages/frameworks. We have samples in Jython, Groovy, Grail. This weekend I got a sample in JRuby shared by our community member Martin Jackson in the API forum. Thanks Martin!

I think it would be fun to share it with you. If you can write Ruby code, you can take advantage of VI Java API for managing and automating vSphere as well. If you have samples leveraging the API to share, I am happy to hear about it.

Now, let us take a look at Martin’s sample code ported from a VI Java API sample.

#  VI Java API Hello using JRuby
require 'java'
require 'dom4j-1.6.1.jar'
require 'vijava2120100824.jar'

# Only using a single class from this package so I'll just import it
import java.net.URL
# Creating ViJava Module so I don't have to import each com.vmware.vim25.mo class individually.
module VIJava
  include_package "com.vmware.vim25.mo"
end

si = VIJava::ServiceInstance.new(URL.new("https://192.168.1.9/sdk"), "root", "password", true)
rootFolder = si.getRootFolder() vms = VIJava::InventoryNavigator.new(rootFolder).searchManagedEntities("VirtualMachine")
puts "Hello " + vms[0].getName()

si.getServerConnection().logout()

Author: Steve Jin is the author of VMware VI and vSphere SDK (Prentice Hall), creator of VMware vSphere Java API. For future articles, please subscribe to Email or RSS, and follow on Twitter.

Eclipse Day at GooglePlex: A Brief Trip Report

Steve Jin — Fri, 27 Aug 2010 05:48:54 +0000

I attended Eclipse Day at Google headquarter today. It’s a great event packed with several great talks.

Mobile application is for sure a hot topic these days. There were several talks on mobile application development like Android Tools for Eclipse by Xavier Ducrohet from Google, Eclipse Sequoyah for Android App Developers by Eric Cloninger from Motorola, Tools for Mobile Web by Paul Beusterien from Symbian Foundation, and EMF for GWT by Ed Merks from Cloudsmith.

While sitting in the talk on Instantiations Eclipse Tools, I learned that Google bought the company 3 weeks ago. I expect Google will soon make WindowsBuilder, a famous WYSIWYG GUI authoring tool, free for GWT users, therefore further drive the adoption of GWT in competition with Flex, SilverLight.

Hopefully Google will also make it free to all others like SWT, SWING, RAP, RCP users. Big companies spend millions and billions of dollars on advertising. Google does not advertise by itself, and should spend money instead on free products that are useful to developers. In long term, I think community goodwill is more important than pure advertisement for a company’s brand recognition.

Another interesting talk is Git and Eclipse by Chris Aniszczyk from Red Hat and Shawn Pearce from Google. It’s mainly about EGIT plugin which allows you to use Git in Eclipse as CVS and Subversion. Two related projects are JGit, a Java implementation of original C based Git, and Gerrit which is “a web based code review system, facilitating online code reviews for projects using the Git version control system.” Even you don’t use Eclipse, you can benefit from JGit and Gerrit.

Author: Steve Jin is the author of VMware VI and vSphere SDK (Prentice Hall), creator of VMware vSphere Java API. For future articles, please subscribe to Email or RSS, and follow on Twitter.

vSphere Java API 3.0 Kicks off, code name “Crescendo”

Steve Jin — Thu, 26 Aug 2010 07:13:43 +0000

WIth 2.1 GAed yesterday, I am happy to announce the 3.0 project kicks off officially. For more fun, I picked up a code name for 3.0 release: Crescendo. For folks know music, crescendo means music gets louder and louder. That is where I want to bring the project to. It’s been a huge success for this VMware sponsored open source project. We’ve had 9,000+ downloads, plus 1000+ SVN code sync, after its first debut in 2008.

So where are we going next?

Before answering the question, let’s take a look at the themes of previous releases. The theme of 1.0 was ease of use with full object model and getter methods hiding property collector. The theme of 2.0 was Just Enough High Performance Web Service Engine resulted in not only performance boost, but also clean license with pure BSD, and much smaller footprint and zero memory leak.

Now it’s time to re-visit ease of use again, but from a different perspective. As I discussed early, the learning curve of vSphere API comes from two folds: lack of object model, and complexity of data objects. The 1.0 release solved the problem nicely. Now it’s time to tackle the second one.

There are two possible ways to take care of this. First, provide more samples with better documentation as starting point. Secondly, come up with a higher level of APIs that capture the most common use cases so that you have a giant shoulder to stand on. Crescendo will take the second approach.

Let me give a quick example here. As today if you want to add a hard disk to a virtual machine, you have to use the reconfigVM_Task() method with tons of possible parameters while you actually need just a few. What is the point to read deeply into the parameter data objects? These parameters can easily get wrong. You can, for instance, mistakingly assign a unit number that has been taken by other devices under the same controller. Many more details exposed than needed. Therefore I see a need for a higher level APIs that address this use case with far simpler method.

This is just one sample of many common use cases you may find in real projects. To make sure it’s something you find useful, I would love to hear from you on how you use vijava today and what you wish to have in the future. If you have some code to contribute, that would be even better. I can use them as the base to abstract high level of APIs. I will give you credits whenever it’s due.

Architecture wise, the new layer won’t be mixed up with existing layer, meaning you have choice to skip this layer. But if not, you will enjoy a much higher productivity than using the current abstraction.

Another important thing is that crescendo will empower system administrators on writing automation scripts. The current vijava API is mainly targeted toward software developer even though we have nice jython, Groovy tutorials. With Crescendo, the scripts will be easier and shorter.

BTW, I will be at VMworld next week presenting vSphere API best practices, helping at VMware booth in exhibition hall, and book signing at bookstore (Tuesday 12:30~1PM). If you happen to be there, please feel free to stop by.

Author: Steve Jin is the author of VMware VI and vSphere SDK (Prentice Hall), creator of VMware vSphere Java API. For future articles, please subscribe to Email or RSS, and follow on Twitter.

Announcing vSphere Java API 2.1 GA Release

Steve Jin — Thu, 26 Aug 2010 06:20:21 +0000

Right after vSphere 4.1 released, VI Java API 2.1 beta supporting vSphere 4.1 was released on July 15. After 40 days, I am pleased to announce GA of the 2.1 release. Many thanks to all vijava community memembers who helped to try 2.1 beta and give feedbacks.

The 2.1 beta is pretty good in terms of quality. I got several emails reporting greeen. I did get several bugs, some of which are carry-overs that should have been fixed in previous releases. Check the end of this blog for a list of bug fixes.

During the beta period, I started a poweredby page which now features 10 companies/products which use vijava API. If you would like your organizations/products included, please let me know.

Enough being said, are you ready to give 2.1 a try? Please feel to download it here. Even you are new to this API, 5 minutes is good enough to have your first HelloVSphere running with this tutorial.

Bug Fixes
3049871 Missing SetPublicKey in ExtensionManager
3040909 getProfile in HostProfilesManager throws a classcastexceptio
3042149 match version in SessionManager.cloneSession() method
3042167 xml characters not escaped (partial fix with password which is more likely to contain special chars)

Author: Steve Jin is the author of VMware VI and vSphere SDK (Prentice Hall), creator of VMware vSphere Java API. For future articles, please subscribe to Email or RSS, and follow on Twitter.

Tipping points: the Social Aspect of Cloud Computing

Steve Jin — Wed, 25 Aug 2010 05:01:24 +0000

Many people already know the book “The Tipping Point: How Little Things Can Make a Big Difference.” According to the author Malcolm Gladwell, tipping points are “the levels at which the momentum for change becomes unstoppable.” He defines the term as sociological and uses it to explain sociological epidemics.

Three Rules of Epidemics

In his book, Gladwell laid out the “three rules of epidemics” as follows:

1) The Law of the Few.
“The success of any kind of social epidemic is heavily dependent on the involvement of people with a particular and rare set of social gifts.” The author categorized people into Connectors who link us up with the world; Mavens who are “people we rely upon to connect us with new information;” and Salesmen who are charismatic persuaders.

2) The Stickiness Factor
The specific content of a message that renders its impact memorable.

3) The Power of Context
Human behavior is sensitive to and strongly influenced by its environment.

Although the research comes from sociology, I think it applies to technology as well. After all, technology is social. Just think about social networks like Facebook, and the recent success of Apple’s iPad.

If you want your technology to be a huge success, you cannot ignore its social side story. In the end, it is human beings who make decisions regarding any technology adoption or product purchase.

Two Tipping Points

Cloud computing has become a buzzword. It has also climbed the CIO’s priority list from near the bottom to second place in a recent survey. While we’ve made huge progress in terms of marketing, we haven’t yet seen the tipping point of cloud computing in reality.

Unlike the Hush Puppies shoes discussed in the book, technologies like cloud computing are far more expensive to adopt and and they require much more expertise than buying a pair of shoes.

This uniqueness of cloud computing results in an interesting phenomenon – two tipping points. The first tipping point is with marketing. As an industry, we are pretty good at creating buzz around new technologies. No other industry is as eager as high tech to embrace the “next big thing.”

The second tipping point is with adoption in production. It may or may not come after the first tipping point. After the first tipping point, people may give cloud computing a try with pilot projects. At this tipping point, it becomes absolutely critical that the technology brings real vale to businesses or adoption dies.

There are examples in technology where the second tipping point arrived and then reversed due to the absence of sufficient value to businesses. One of the best examples in my opinion was the EJB (Enterprise Java Bean). It was widely adopted but then faded away. It failed to deliver sufficient value.

Tipping points for Cloud Computing

With so many people talking about cloud computing, we obviously have achieved the first tipping point from marketing perspective. We haven’t yet seen massive adoption through a second tipping point however.

Where will businesses find sufficient value in cloud computing? Cost saving is definitely one, but for sure not the only one. As hardware cost keeps dropping, capital spending on servers may be less a concern for most businesses in the future. They would look for flexibility, control, and growth from cloud computing. These kinds of value will be driven by people if we look at the three rules of social epidemics from a technology perspective. At the second tipping point for cloud computing, breakthrough adoption is now up to the connectors, mavens, and sales people.

With many companies and organizations in the cloud movement, we will for sure have many connectors, mavens, and salespersons who may have different understandings of cloud computing, and different agendas due to company interests. We need to collaborate effectively with each other for the bigger goal: showing real world values to the customers. Only then we will see the second tipping point that we all look forward to.

Author: Steve Jin is the author of VMware VI and vSphere SDK (Prentice Hall), creator of VMware vSphere Java API. For future articles, please subscribe to Email or RSS, and follow on Twitter.

Secret of vApp Template in vSphere

Steve Jin — Tue, 24 Aug 2010 05:01:37 +0000

My colleagues and I had a discussion regarding the vApp template. After virtual machine template for virtual machine, you would expect vApp template for vApp and manage it in a similar way from the vSphere Client. But you cannot.

Most of us know that from vSphere Client, you have context menu item allowing you to convert a virtual machine to a template easily with a click. However you cannot find a similar menu item with a vApp. You can choose to convert a virtual machine inside a vApp, but then the converted template will jump out of the vApp container.

Can we have vApp template? The answer is we can, but in a different way.

To fully understand what’s happening, let’s look into vSphere API which will shed some insights about this. To most people’s surprise, vApp (VirtualApp) is a sub type of ResourcePool, not VirtualMachine although it behaves somewhat like a virtual machine in that you can power on and off a vApp as would a virtual machine. This is a good example in which knowing API gives you insights on how the system is constructed under the hook. That is why system administrators want to learn about API, especially object model, even though not developing code.

The difference of a virtual machine and a virtual machine template is that a virtual machine is not associated with a ResourcePool object. That is why you need to specify a resource pool when turning a template to a virtual machine, as shown in the following method:

public void markAsVirtualMachine(ResourcePool pool, HostSystem host)

Note that the first pool parameter is a required parameter and cannot be null as the second one. Given a vApp itself is a ResourcePool, you cannot remove its association with a ResourcePool object, or destroy itself.

Now, how can we have a template for vApp? In fact, it’s pretty easy. You would use OVF as the template.

From vSphere Client, you can get to the “Export OVF Template” dialog box, shown below, from File->Export->Export OVF Template. It allows you to export a vApp to your local machine in OVF.

It’s also easy to deploy an OVF package to vApp with “Deploy OVF Template” dialog box as follows from File->Deploy OVF Template menu path. You can specify either from a local file or from a URL pointing to an OVF package. Unlike the other direction, you will have to go through several more steps here including accepting end user license agreement.

You can of course export and deploy (import) OVF packages with vSphere API. Check out this blog for more details and samples.

Author: Steve Jin is the author of VMware VI and vSphere SDK (Prentice Hall), creator of VMware vSphere Java API. For future articles, please subscribe to Email or RSS, and follow on Twitter.

Restriction with User Name in ESX 4.1

Steve Jin — Mon, 23 Aug 2010 06:02:10 +0000

According to a recent post by Duncan, there is an issue with password in ESX(i) 4.1. Only the first 8 characters of a password are taken and validated. A VMware KB article offered solutions to this issue.

Interestingly enough, vSphere Java API community reported a bug with vSphere API’s HostLocalAccountManager managed object: a piece of code creating a new user account used to work with ESX 4.0, but no longer with ESX 4.1 without any change. My first thought was to check the API reference to see if there is any change with parameters, especially those data objects embedded in the parameters. Then I suspected it might be associated with policies of password. I found another KB article related to password while searching for answers.

After several rounds of trials, it turned out the issue was related to the user name. As reported, the problem was caused by “.” (dot character) which is no longer allowed in user name of ESX 4.1. Removing the dot solved the problem.

I think this issue may not be as common as the password because not many people use “.” in user name. But if you get this problem, it’s hard to root the cause. That is why I write this so that you can easily avoid the issue before it happens.

Author: Steve Jin is the author of VMware VI and vSphere SDK (Prentice Hall), creator of VMware vSphere Java API. For future articles, please subscribe to Email or RSS, and follow on Twitter.

How to Get Event Type with vSphere API?

Steve Jin — Thu, 19 Aug 2010 05:01:31 +0000

There is a recent question asking how to get the type of event from vSphere API in my previous blog. On one hand, you can clearly see the types of events on a vSphere Client, for instance “info”, “warning”, “error”, and “user.” On the other hand, you cannot find any information about the type from a given event itself using vSphere API.

Strictly speaking an event just indicates something has happened. That is it. You can categorize it differently depending on your goal. The Event type itself in vSphere API models an event as what it is, not about how you look at it. This is a right design philosophy, but turns out to be a little tricky for you to figure out the type of an event.

How does vSphere Client do the trick?

The trick is with the EventManager, which has a property called description typed as EventDescription[]. The EventDescription data object includes category sub-property which holds information about the possible categories/types of all events, namely “info”, “warning”, “error”, and “user.” This is exactly you find from vSphere Client.

Another sub-property of EventDescription is called eventInfo typed as EventDescriptionEventDetail[]. Each EventDescriptionEventDetail object has a key, a category, and several format strings. So the mapping is here! The following is a screen shot showing the “HostAddFailedEvent” as “error.” The reason I used Eclipse is because MOB does not show eventInfo correctly. You can optionally use the VI Java Browser.

The solution is now clear. All you need to do is to retrieve the eventInfo sub-property and establish the mapping locally for lookup. In Java, you can load them into a HashMap object for fast look up.

Note that the mapping is just what VMware looks at these events. You can change the mapping for your applications when appropriate. That is why I said current design follows right philosopy.

One important performance tip is that you should NOT retrieve it twice in a session because eventInfo doesn’t change with a particular server. I wouldn’t recommend you to persist it locally either because it’s that expensive to get it.

Following is the Java code if you want to inspect the eventInfo property.

import java.net.URL;

import com.vmware.vim25.EventDescription;
import com.vmware.vim25.TaskInfoState;
import com.vmware.vim25.VirtualDevice;
import com.vmware.vim25.VirtualDisk;
import com.vmware.vim25.mo.EventManager;
import com.vmware.vim25.mo.InventoryNavigator;
import com.vmware.vim25.mo.ServiceInstance;
import com.vmware.vim25.mo.Task;
import com.vmware.vim25.mo.VirtualMachine;

/**
* @author Steve Jin
*/

public class FindEventType
{
public static void main(String[] args) throws Exception
{
ServiceInstance si = new ServiceInstance(new URL("https://server/sdk"), "root", "password", true);
EventManager em = si.getEventManager();

EventDescription ed = em.getDescription();
// set a break point here to inspect the ed variable for more info

si.getServerConnection().logout();
   }
}

Author: Steve Jin is the author of VMware VI and vSphere SDK (Prentice Hall), creator of VMware vSphere Java API. For future articles, please subscribe to Email or RSS, and follow on Twitter.

InterCloud vs. Internet: What’s Missing in Cloud Computing?

Steve Jin — Wed, 18 Aug 2010 05:01:19 +0000

As more and more clouds go live, it’s time to think about how they will need to interconnect and interact. InterCloud is a new terminology coined for cloud computing after Internet for networking.

Vint Cerf, the “father” of the Internet, said recently that the cloud is much like networking in 1973 when computer networks couldn’t connect or interact. He called for open standards for cloud computing so that InterCloud can become a reality.

It’s hard to design standards when people are still trying to reach a consensus on defining what a cloud is in the first place! The good news is that as an industry we went through a similar process for the Internet. So we can learn from that experience.

The idea is simple: look at basic building blocks we have for the Internet and think about their equivalent for the InterCloud. Believe it or not, InterCloud and Internet share many common characteristics. The following table summarizes some of these.

	Networking	Cloud Computing
Content	Data	Computing workload
Cornerstone	Ethernet	Virtualization
Format	ASN.1	OVF + ?
End point	Host	Hypervisor
Protocols	OSI, TCP/IP, etc.	?
Directory service	DNS	?
ID	IP address/host name	?
Resource locator	URI/URL	?
Interconnectivity	Internet	InterCloud
Killer use cases	Email, Web	System provisioning, Dynamic scaling

Let me emphasize the key difference between the Internet and the Intercloud —the Internet moves data and the InterCloud moves computing workloads. With virtualization and other high level virtual machine technology, computing workload is essentially data. Viewed in this fashion, that would mean the InterCloud can leverage the Internet as a high level application.

As you see from the table above, there remain many question marks where we do not yet have InterCloud equivalents to the Internet. That is something we need to think about and work on moving forward.

Let me know what you think. Do you have suggestions for equivalents?

Author: Steve Jin is the author of VMware VI and vSphere SDK (Prentice Hall), creator of VMware vSphere Java API. For future articles, please subscribe to Email or RSS, and follow on Twitter.

Manage Lockdown Mode with New API in vSphere 4.1

Steve Jin — Tue, 17 Aug 2010 05:01:10 +0000

As a feature, lockdown mode has been added to vSphere 4.0 . Enabling it disables all remote root access to an ESXi machine. Any local changes to the host must be using:

DCUI (Direct Console User Interface).
vSphere Client or vCLI connecting to vCenter.
vSphere Client or vCLI connecting to ESXi with a local user account on the host.

My colleague Duncan Epping has summarized a table showing whether you can change ESXi with different access methods in two modes.

As a general practice for better security, it’s recommended to enable lockdown mode. However the lockdown mode could be breached by adding root user to local groups, according to vSphere Hardening Guide ( see HCN03 on page 48). You may want to read it carefully.

You can manage lockdown mode through vSphere Client or DCUI, but not from public API until vSphere 4.1. As you would expect, there are two methods defined with the HostSystem type in vSphere Java API:

public void enterLockdownMode() throws HostConfigFault, RuntimeFault, RemoteException;
public void exitLockdownMode() throws HostConfigFault, RuntimeFault, RemoteException;

These two methods are pretty straightforward. To call these methods, the user must connect to vCenter and have “Host.Config.Settings” privilege though.

host.enterLockdownMode();

or,

host.exitLockdownMode();

How to tell whether an ESXi host is in lockdown mode? You can easily tell from the “config. adminDisabled” sub-property as shown in the following code:

boolean lockeddown = host.getConfig().getAdminDisabled();

Please note this code works *ONLY* while you connect to vCenter directly, the same requirement for calling the two methods.

Author: Steve Jin is the author of VMware VI and vSphere SDK (Prentice Hall), creator of VMware vSphere Java API. For future articles, please subscribe to Email or RSS, and follow on Twitter.

DoubleCloud

Ruby to Manage and Automate VMware vSphere?

Related Posts

Eclipse Day at GooglePlex: A Brief Trip Report

Related Posts

vSphere Java API 3.0 Kicks off, code name “Crescendo”

Related Posts

Announcing vSphere Java API 2.1 GA Release

Related Posts

Tipping points: the Social Aspect of Cloud Computing

Related Posts

Secret of vApp Template in vSphere

Related Posts

Restriction with User Name in ESX 4.1

Related Posts

How to Get Event Type with vSphere API?

Related Posts

InterCloud vs. Internet: What’s Missing in Cloud Computing?

Related Posts

Manage Lockdown Mode with New API in vSphere 4.1

Related Posts