Event Log Blog

How to dynamically toggle your Wireless adapter with EventSentry

Sun, 22 Jan 2012 16:05:41 -0600

Most of the time I work on a Lenovo laptop running Windows 7, and I'm overall quite happy with the laptop (especially after the mainboard was replaced and it stopped randomly rebooting). However, a minor nuance had been bugging me for a while: If I plugged my computer into a LAN (I have a docking station at work and at home) while a wireless signal was also available (and configured on the laptop), Windows 7 would keep both connections active.

1. The Problem

So I'd have my laptop in the docking station, connected to a 1Gb Ethernet network, and yet the laptop would also be connected to a WiFi network. While not a big deal per se, it does have a few advantages to automatically disable the WiFi connection when already connected to Ethernet:

Avoid potential connectivity issues
Increase security by not transmitting data via Wifi when not necessary
Increase battery life when connected to a LAN
Because you can!

Now, Lenovo equips most (if not all) laptops with a software called "Access Connections", a pretty nifty and free tool! One of the things it can do, is disable a Wireless adapter when the computer is connected to Ethernet. However, it never re-enables it when you disconnect from the wired network (at least I haven't found a way), and besides not everybody has Lenovo laptop.

So how could I disable the WiFi connection automatically when I connected the laptop to an Ethernet, yet automatically re-enable it when there is no Ethernet connection?

2. The Research

After some intense brainstorming, I remembered two things:

Most Ethernet NIC drivers log event to the System event log when a network port is connected/disconnected.
A while back, I used the netsh command to configure DNS servers from the command line. Maybe one could toggle the state of network adapters with this tool as well?

3. Evidence Gathering

The first one was easy, a quick look at the system event log revealed the following event:

A similar event is logged when the "network link" has connected. The event shown here is specific to the driver of my laptop's network card (an Intel(R) 82577LM adapter), but most newer drivers will log events when a cable is disconnected or the link is otherwise lost. If you are already running EventSentry with its hardware inventory feature enabled, then you can obtain the name of the network adapter from any monitored host on the network through the hardware inventory page, an example is shown below.

Coming up with a way to enable and disable a particular network connection with netsh.exe was a bit more challenging, but I eventually cracked the cryptic command line parameters of netsh.exe.

Enable a connection
netsh interface set interface "Wireless Network Connection" ENABLED

Disable a connection
netsh interface set interface "Wireless Network Connection" DISABLED

And yes, you do need to specify the word "interface" twice. If you do find yourself wanting to automate network adapter settings with scripts and/or the command line frequently, then you should check out this link.

4. The Solution

So now that we have all the ingredients, let's take a look at the recipe. In order to accomplish the automatic interface toggling, we need to create:

an embedded script called wifi_enable.cmd, using the command line from above
an embedded script called wifi_disable.cmd, again using the command line from above
a process action "Wifi Enable", referencing the above wifi_enable.cmd embedded script
a process action "Wifi Disable", referencing the above wifi_disable.cmd embedded script
an event log filter for event source "e1kexpress" and event id 27, triggering the "Wifi Enable" action
an event log filter for event source "e1kexpress" and event id 32, triggering the "Wifi Disable" action

A couple of clarifications: First, you do not need to use embedded scripts, you can create the scripts in the file system too and then point the process action to those files. I prefer embedded scripts since I don't have to worry about maintaining the script, as it gets distributed to remote hosts automatically when needed. Second, the event source and event id will depend on the network card you have installed on your network, the above example will only work with Lenovo T410 laptops.

So what happens is pretty straightforward: When I connect my laptop to a LAN, the Intel NIC driver writes event id 32 with the event source e1kexpress to the system event log. EventSentry intercepts the event and triggers the Wifi Disable action, which in turns runs the netsh.exe process, disabling the WiFi connection.

5. Setting it up in the management console

Embedded Scripts
You can manage embedded scripts via Tools -> Embedded Scripts. Click "New", specify a descriptive name (e.g. wifi_enable.cmd) and paste the command line netsh interface set interface "Wireless Network Connection" ENABLED into the script content window. Then, do the same for the wifi_disable.cmd script, but this time use the netsh interface set interface "Wireless Network Connection" DISABLED command line. You can leave the interpreter empty as long as you give the filename the .cmd extension.

Actions

Create two process actions, one pointing to wifi_enable.cmd, and one pointing to wifi_disable.cmd. You can access these embedded scripts by clicking the pull-down - you should see the embedded script(s) you created in step one - each prefixed with the @ symbol. The resulting dialog should look like this:

I recommend enabling both "Event Log Options", as this will help with troubleshooting. Now we just need the event log filters, and we are all set.

Like I mentioned earlier, you can also reference any external process or .cmd file with the process action, if you'd rather not use embedded scripts.

Event Log Filters
Since we'll need two filter, we'll create a new event log package called "Toggle Wifi" by right-clicking the "Event Log Packages" container and selecting "Add Package". Inside the package we can then add the two filters: One to trigger the "Wifi Enable" action when the NIC drivers logs its event indicating that the network cable was unplugged, and one that will trigger the "Wifi Disable" action when the NIC drivers logs that the network cable was plugged in. The filter will look similar to this, but note that the event source as well as event id will depend on the network card and driver.

That's pretty much it. If you enabled the event log options in the process action earlier, then you can see the output from the process action in the event log, as shown below:

Here are some links to the official EventSentry documentation regarding the features used:

Why complex passwords may be less secure than you think

Sat, 20 Aug 2011 22:03:51 -0600

Passwords are everywhere. You use them to log on to your network, login to business applications and Facebook, check your personal email, and more.

I'll be rethinking passwords in this blog, and what you can do to make authenticating with passwords more secure.

As it turns out, the British comedian Nick Helm won an award in Edinburgh for the funniest joke, just one day before I posted this article. He won for the joke: "I needed a password eight characters long so I picked Snow White and the Seven Dwarves."

Of course, passwords have been around for a while, even though more advanced ways to authenticate like fingerprint readers and biometric scans exist today. Still, passwords prevail as the primary method to authenticate for the majority of networks and computer systems. One-time pads like RSA's SecurID are another secure alternative, but any system can be exploited as recent events have shown. Even fingerprint readers can be fooled: either with brainpower or through more "traditional" methods.

Password Cracking goes Mainstream
By 1999, Windows NT 4.0 started gaining a lot of traction in networks across the globe. Of course we all know that with popularity comes quantity, and with quantity comes increased exposure. As more and more networks were using Windows NT (to authenticate among other things), a new piece of software called l0phtcrack (it had a GUI!) was gaining popularity. What l0phtcrack could do - and quite easily I might add - was download all password hashes from the Windows NT user database, and then run both brute-force and dictionary attacks on those hashes.

If a user chose a password that was in the English dictionary, l0phtcrack could often crack it within seconds. If the password was a bit more complicated, it would take a couple of days. Due to the way Windows NT stored password hashes (in the name of compatibility with LanManager), passwords with 7 or fewer characters were particularly easy to crack. And, as CPUs grew stronger and faster, the time required to run those brute-force attacks kept getting shorter. Of course this general mechanism is and was not restricted to Windows NT and l0phtcrack; you could do the same thing with any password hash. For example, I used a Perl script (utilizing a dictionary text file) back in 2001 against hashes obtained from our NIS system to show the UNIX admins that the NIS installation was, politely speaking, insecure.

So choosing a password that is in any dictionary is clearly not a good idea (and really shouldn't be allowed when setting the password) since a dictionary attack can be fast. An easy way to prevent against a simple dictionary attack is to require users to choose an additional non-letter character. Since words in dictionaries usually don't contain characters other than letters, this is certainly a step in the right direction.

Secure Password for Dummies
Technically, adding a single non-letter character to an English word would indeed prevent a dictionary attack. Yes, a bad one! A persistent and motivated attacker (and most attackers are persistent and motivated) could modify their dictionary attack, and automatically prepend and append numbers to dictionary words, so that a password like "house7" or even "house1!" could still be found. This may sound like a lot of work, after all this would increase the time a dictionary attack takes around 20-fold "0house, 1house ... house0, house1, ....house9". True, but dictionary attacks are so fast that this technique would still be preferable to a brute-force attack. An attacker would still prefer a 4-hour dictionary attack over a 2-month brute-force attack (I made those numbers up, but the idea is that dictionary attacks are a lot faster). It also turns out that users tend to use the same numbers / special characters in their passwords, e.g. "1!", "99", "123" and so forth. Even worse, there appear to be a set of "favorite" passwords: http://www.schneier.com/blog/archives/2006/12/realworld_passw.html.

The obvious way to protect against a dictionary attack is to not use words from a dictionary in the first place. Indeed, many authentication systems require the use of letters with uppercase/lowercase, numbers as well as special characters. A requirement like this will surely protect us against even the most sophisticated dictionary attacks. Mission accomplished. Easy!

Not so fast. Attackers still have a few more options at their disposal. The attacker can:

Look for software vulnerabilities so that they can inject their own malware that would give them access to the server/workstation/network. They would then simply create a new user, reset the password of an existing user, or - if possible - just download whatever data they need. Event Log Monitoring can help here, since you can get notified when a new user is created/deleted or a password is changed (you could setup a filter to email you when a user password is changed between 11pm and 6am for example).
Employ social engineering techniques to get access to the password, either through physical access, a phone call or something similar. A combination of (1) & (2) is most common, as an attacker will send a malformed PDF (or similar) to the target, which will then implant some Trojan horse.
Use a brute-force attack to guess the password, either against an offline database (if the attacker was lucky enough to obtain one), or run the attack directly against the login system (a web site, a Windows domain, etc.). Owasp has a good article about brute-force attacks against web sites, which can be very susceptible to these types of attacks.

Brute-force attacks generally only work in two cases:

One has access to an encrypted password database.
The system one is trying log on to does not employ an account-lockout technique, so that a brute-force attack can be aimed directly at a logon portal.

Most network systems do employ an account lockout mechanism, and I highly recommend you enable this on systems which support it. Many systems, in particular web sites, do not support this functionality, however, so brute-force attacks are still a real risk.

When enabling account lockout, it's important to keep your end users in mind. Your users will ultimately need to log on to a network in order to do their work, and if the system locks them out every time they type in a wrong password twice, then your support team will spend a lot of time unlocking user accounts, and your users (depending on how calm they are) will be more or less annoyed. A log monitoring solution like EventSentry can email you when an account lockout occurs on a system (e.g. on Windows through the event log, on other devices through Syslog).

A patient system administrator on the phone troubleshooting a logon problem
So, when we brute-force a password, we try every combination regardless of dictionary and such. We start at "a" and make our way to, say, "ZZZZZZZZZZZ". Consequently, when the required password length is short (say 7 characters), a brute-force attack will be faster than when the required length is large (say 15 characters).

All this begs the ultimate question: Is it better to use a short complex password like C0mP1eX! (8 characters), or a long more simple password like ClimbingUpATree (15 characters)? Time to bring out the calculator. In order to come up with a conclusion, we'll create three password policies: One that requires complex but shorter passwords, and two with longer but less complex passwords.

Password Policy 1: "Complex Is Best"

Minimum length: 8 characters
Required character groups: One lowercase letter, one uppercase letter, one number, one special character out of: !@#$%^&*()_+[]{}
Possible passwords: 1,370,114,370,683,140 (yes, that's one quadrillion)

That's mighty complex, but a password like C0mp1ex! would be valid.

Password Policy 2: "A little long is enough"

Minimum length: 10 characters
Required character groups: One lowercase letter, one uppercase letter, one number. Special characters are allowed but not required.
Possible passwords: 839,299,365,868,340,000 (that's 839 quadrillion and a little bit)

Of course there would be even more possible passwords if a user decides to include a special character in their password (after all the policy only specifies the minimum requirement, and we wouldn't dare prohibit additional complex characters now, would we?). This policy is 612 times more complex than the previous policy, even though it only requires two more characters. It's flaw, however, is that a user could potentially use insecure passwords like Gardenhose1 which could be guessed with a sophisticated dictionary attack.

Password Policy 3: "The longer, the better"

Minimum length: 15 characters
Required character groups: One lowercase letter, one uppercase letter
Possible passwords: 54,960,434,128,018,700,000,000,000 (that's 54 septillions, 960 sextillions, 434 quintillions - you get the idea)

Phew, you need a lot of GPUs and a time machine to brute-force a password from that selection - and that's without even requiring a user to include a number! This policy is 40 billion (40,117,105,202 to be exact) times more complex than the first policy. And long passwords are not hard to come up with - just use a simple sentence like "Idontlikepasswords55" is a pretty long password (20 characters) and not that hard to remember at all.

A compromise?
As you can see, length trumps complexity in most cases, but as is often the case in computer security, things aren't always as simple as they seem. The numbers are correct, but a longer password without complex requirements might, as mentioned before, encourage a user to choose a password that could be guessed with a sophisticated dictionary attack.

For example, "Gardenhose1" would match the 2^nd policy's requirements but not be very secure. Users also tend to use family names, user names and the like in their passwords. A smart attacker could leverage this and adapt their dictionary attack accordingly. So if "Jean Reno" was to use "JeanReno1948" as his password, then this would still not be as secure as assumed - despite the 12-character length.

We can see that every additional character in the length of a password increases the possible combinations exponentially, more so than a few additional special characters. Still, not requiring special characters at all might allow the end user to pick passwords that are in a dictionary. Requiring extremely complicated passwords, on the other hand, will make it difficult for many users to remember them, and your end users might resort to writing their passwords down on a post-it, the bottom of their keyboard, or come up with "secure" passwords like ASDFasdf1!. Yes, annoyed users can be very creative. Put yourself in the shoes of somebody who is not familiar with security and needs to choose a password, would you voluntarily choose something like T3a#fE@8 ?

You do not want your users doing this
Password expiration policies, while certainly important, can also result in a backlash from your users. Users can get very annoyed with overly ambitious expiration policies, and fight back with number schemes or passwords stored on post-it notes (see http://open.salon.com/blog/unemployedmarx/2011/02/17/changing_passwords if you don't believe me). The result is the opposite: weaker passwords and less security. How about a "secure" 9-letter password that even exceeds our first policy?

Cats2010!
Cats2011!

Q1cats2011!
Q2cats2011!

This is heaven for an attacker: as long as the user sticks to the same pattern, the attacker will always know the password - even if the user changes it every 3 months. See "Changing Passwords" (http://www.schneier.com/blog/archives/2010/11/changing_passwo.html) for a more thorough discussion on this topic.

So what's the solution? As often, probably a little bit of everything. Dictionary passwords need to be avoided like the plague, so we'll never get around requiring some complexity. Complexity alone can be misleading though, so a minimum length of 12 characters seems like a good baseline. In addition, enable account lock-out techniques and set a (reasonable) maximum password age. A pretty good password policy would look like this:

18 characters minimum
Lowercase, uppercase & numbers
180-day password age
No part of first name, last name, username, etc. allowed in password

Also, don't forget to educate your users, so that people know why and how. Tell them that corporate espionage is a real threat, and suggest the use of a sentence for a password. Of course there will always be naysayers, but the majority of your user base should understand this.

Abuse

I have seen web sites (e.g. banking) require me to use a complex password, yet require that it shall be no longer than 8 characters! Whatever the reason behind something like that, it's far from secure and counterproductive. Even if I'd want to choose a strong 14-letter password I couldn't, and I would have to settle for something less secure.

Password Reuse
Another often overlooked risk is the reuse of passwords. Nowadays, people are required to use passwords at a multitude of web sites and systems. Some of those web sites store confidential information (SSN, credit card), but many don't.

The more often one uses the same password, the higher the risk that it is compromised. As such, your password - if used at more than one place - is only as strong as its weakest link. Don't use the same password that you use for your banking web site on your photo-sharing site!

I personally don't care too much if some cracker hacks the photo-sharing site I use, and downloads (and cracks) all the passwords. But I do care if my password to my banking web site is compromised. An attacker may not be able to easily guess a password at bankofamerica.com, but if I use the same password as my photo-sharing site, then I'm just asking for trouble. Recycling and reuse are a good thing - but not with passwords.

I hope this longer than expected article inspired you to review your corporate password policy, and maybe even your personal password habits. If you made it this far then I have included some relevant links regarding ... well ... passwords!

Nick Helm's password joke:
http://www.bbc.co.uk/news/uk-scotland-14646532

Interesting Statistics:
http://www.passwordresearch.com/stats/statindex.html

A Strong Password Isn't the Strongest Security:
http://www.nytimes.com/2010/09/05/business/05digi.html

Graphical Passwords: A Survey:
http://www.acsac.org/2005/papers/89.pdf

So long,
Ingmar.

EventSentry v2.92 + iPhone App

Thu, 23 Jun 2011 11:26:19 -0600

EventSentry v2.92
The main new features in version 2.92 are:

Network Services component
Embedded performance / environment charts
HTTP action
Hardware Inventory & Monitoring

Please see the release history or the documentation for a complete list of all bug fixes and new features.

Network Services
The network services component is the main new feature in EventSentry, and adds powerful functionality - the ability to receive SNMP traps - to our monitoring solution. Any network device that supports SNMP v1, v2c or v3 can be configured to send traps to EventSentry, which can then either forward the trap details via email, or log them to the database for archival and searching purposes.

Unlike other products, configuring the snmp trap daemon is easy and uncomplicated. Simply specify any MIBs you want to load, configure any SNMP communities and/or v3 users, and (optionally) setup text filters to include/exclude specific traps based on their content.

The network services component now also includes the syslog daemon, which was previously embedded in the EventSentry agent. This change includes higher availability, since the network services will cache incoming (snmp and/or syslog) packets if the main database is temporarily unavailable.

Another new "feature" is the ability to run the network services daemon on Linux and OS X operating systems. We developed all code in the network services to run on multiple platforms, and will continue to do so with all upcoming new components in EventSentry. Cross-platform functionality is still experimental, and as such we don't currently have an installer for non-Windows operating systems. If you are interested in running the network services daemon on Linux or OS X, then please send a request to our support team.

Visualizing performance and environment data with embedded charts
Performance monitoring and alerts have always been a popular and useful feature in EventSentry, but a numerical alert can only convey so much information. If an alert shows that the CPU usage has been exceeded, say 80% over a time period of 10 minutes, then it is often helpful to see the performance data in a chart. This would show whether the CPU usage spiked during the beginning of the 10-minute interval and is back in an acceptable range, or whether the CPU usage has been mostly constant. In the past, you would need to review the performance history through the web-based reporting (which is still an option and desirable for long-term analysis), but starting with v2.92 email alerts can now include an embedded chart which shows exact performance information from the monitored time interval.

For example, if your performance alert is configured to generate an alert when the CPU usage exceeds 90% for more than 30 minutes, then you can receive an attachment in addition to the text alert, showing performance data from the last 30 minutes.

Embedded charts are also included with temperature and/or humidity environment alerts, an example chart is shown below:

HTTP Action
To better interact with web-based solutions like iService, EventSentry now includes a new notification type: HTTP. With the new HTTP action, you can submit event data to any HTTP or HTTPS-based form, making it extremely easy and reliable to interact with web-based systems (e.g. ticketing systems). Setting up a HTTP action is easy: Simply provide the URL, authentication and specify any event fields that you want to submit.

Hardware Inventory & Monitoring
Also new are network link speed monitoring, which can alert you when the NIC speed changes. This can be useful if a host is accidentally connected to a different switch, or simply to inventory all network connections.

EventSentry also monitors connected USB storage devices and can alert you when a drive is added or removed. The S.M.A.R.T. status of drives is also monitored, and junction points are now taken into consideration when evaluating the disk space of a logical drive. Also new is a "physical disks" section in the web reports.

iPhone App
Why yes, there is an app for that! How convenient would it be to review the health status of all your monitored hosts from your iPhone - including health statistics, hardware details and more?

Dream no more, with the EventSentry iPhone app you can review the status of all your servers from your phone (so long as it has access to your web reports of course).

Upon launch the app shows a searchable list of all monitored hosts and their respective health status. Tapping on a host will show detailed information, such as performance data, hardware information and more. Please take a look at the official iTunes page for the EventSentry app for more information and screenshots.

If you are an Android user then there is no reason to worry, an Android version of the EventSentry Mobile app is planned later this year.

Again, please see the version history for a complete list of features and fixes in v2.92.

Do not trust thee RAID alone

Sat, 19 Feb 2011 16:08:24 -0600

I'm assuming that most readers are familiar with what RAID, the "Redundant Array of Inexpensive Disks", is. Using RAID for disk redundancy has been around for a long time, apparently first mentioned in 1987 at the University of California, Berkeley (see also: The Story So Far: The History of RAID). I'm honestly not sure why they chose the term "inexpensive" back in 1987 (I suppose "RAD" isn't as catchy of a name), but regardless of the wording, a RAID is a fairly easy way to protect yourself against hard drive failure. Presumably, any production server will have a RAID these days, especially with hard drives being as inexpensive as they are today (unless you purchase them list price from major hardware vendors, that is). Another reason why RAID is popular, is of course the fact that hard drives are probably the most common component to break in a computer. You can't really blame them either, they do have to spin an awful lot.

Source: ServerBurn4 by Topato

Lesson #1: Don't neglect your backups because you are using RAID arrays
That being said, we recently had an unpleasant and unexpected issue in our office with a self-built server. While it is a production server, it is not a very critical one, and as such a down-time of 1-2 days with a machine like that is acceptable (albeit not necessarily desired). Unlike the majority of our "brand-name" servers, which are under active support contracts, this machine was using standard PC components (it's one of our older machines), including an onboard RAID that we utilized for both the OS drive as well as the data drive (it has four disks, both in a RAID 1 mirror). Naturally, the machine is monitored through EventSentry.

Well, one gray night it happened - one of the hard drives failed and a bunch of events (see myeventlog.com for an example) were logged to the event log, and immediately emailed to us. After disappointingly reviewing the emails, the anticipated procedure was straightforward:

1) Obtain replacement hard drive
2) Shut down server
3) Replace failed hard drive
4) Boot server
5) Watch RAID rebuilding while sipping caffeinated beverage

The first 2 steps went smoothly, but that's unfortunately how far our IT team got. The first challenge was to identify the failed hard drive. Since they weren't in a hot-swappable enclosure, and the events didn't indicate which drive had failed, we chose to go the safe route and test each one of them with the vendors supplied hard drive test utility. I say safe, because it's possible that a failed hard drive might work again for a short period of time after a reboot, so without testing the drives you could potentially hook the wrong drive up. So, it's usually a good idea to spend a little bit of extra time in that case, to determine which one the culprit is.

Eventually, the failed hard drive was identified, replaced with the new (exact and identical) drive, connected, and booted again. Now normally, when connecting an empty hard drive, the raid controller initiates a rebuild, and all is well. In this case however, the built-in NVidia RAID controller would not recognize the RAID array anymore. Instead, it congratulates us on having installed two new disks. Ugh. Apparently, the RAID was no more - it was gone - pretty much any IT guys nightmare.

No matter what we tried, including different combinations, re-creating the original setup with the failed disks, trying the mirrored drive by itself, the RAID was simply a goner. I can't retell all the things that were tried, but we ultimately had to re-create the RAID (resulting in an empty drive), and restore from backup.

We never did find out why the RAID 1 mirror that was originally setup was not recognized anymore, and we suspect that a bug in the controller firmware caused the RAID configuration to be lost. But regardless of what was ultimately the cause, it shows that even entire RAID arrays may fail. Don't relax your backup policy just because you have a RAID configured on a server.

Lesson #2: Use highly reliable RAID levels, or configure a hot spare
Now I'll admit, the majority of you are running your production servers on brand-name machines, probably with a RAID1 or RAID5, presumably under maintenance contracts that ship replacement drives within 24 hours or less. And while that does sound good and give you comfort, it might actually not be enough for critical machines.

Once a drive in a RAID5 or RAID1 fails, the RAID array is in a degraded state and you're starting to walk on very thin ice. At this point, of course, any further disk failure will require a restore from backup. And that's usually not something you want.

So how could a RAID 5 not be sufficiently safe? Please, please: Let me explain.

Remember that the RAID array won't be fully fault tolerant until the RAID array is rebuilt - which might be many hours AFTER you plug in the repaired disk depending on the size, speed and so forth. And it is during the rebuild period that the functional disks will have to work harder than usual, since the parity or mirror will have to be re-created from scratch, based on the existing data.

Is a subsequent disk failure really likely though? It's already pretty unlikely a disk fails in the first place - I mean disks don't usually fail every other week. It is however much more likely than you'd think, somewhat depending on whether the disks are related to each other. What I mean with related, is whether they come from the same batch. If there was a problem in the production process - resulting in a faulty batch - then it's actually quite likely that another bites the dust sooner rather than later. It happened to a lot of people - trust me.

But even if the disks are not related, they probably still have the same age and wear and, as such, are likely to fail in a similar time frame. And, like mentioned before, the RAID array rebuild process will put a lot of strain on the existing disks. If any disk is already on its last leg, then a failure will be that much more likely during the RAID array rebuild process.

RAID 6, if supported by your controller, is usually preferable to a RAID5, as it includes two parity blocks, allowing up to two drives to fail. RAID 10 is also a better option with potentially better performances, as it too continues to operate even when two disks fail (as long as it's not the disks that are mirrored). You can also add a hot spare disk, which is a stand-by disk that will replace the failed disk immediately.

If you're not 100% familiar with the difference between RAID 0, 1, 5, 6, 10 etc. then you should check out this Wikipedia article: It outlines all RAID levels pretty well.

Of course, a RAID level that provides higher availability is usually less efficient in regards to storage. As such, a common counterargument against using a more reliable RAID level is the additional cost associated with it. But when designing your next RAID, ask yourself whether the savings of an additional hard drive is worth the additional risk, and the potential of having to restore from a backup. I'm pretty sure that in most cases, it's not.

Lesson #3: Ensure you receive notifications when a RAID array is degraded
Being in the monitoring business, I need to bring up another extremely important point: Do you know when a drive has failed? It doesn't help much to have a RAID when you don't know when one or more drives have failed.

Most server management software can notify you via email, SNMP and such - assuming it's configured. Since critical events like this almost always trigger event log alerts as well though, a monitoring solution like EventSentry can simplify the notification process. Since EventSentry monitors event logs, syslog as well as SNMP traps, you can take a uniform approach to notifications. EventSentry can notify you of RAID failures regardless of the hardware vendor you use - you just need to make sure the controller logs the error to the event log.

Lesson #4+5: Test Backups, and store backups off-site
Of course one can't discuss reliability and backups without preaching the usual. Test your backups, and store (at least the most critical ones) off-site.

Yes, testing backups is a pain, and quite often it's difficult as well and requires a substantial time commitment. Is testing backups overkill, something only pessimistic paranoids do? I'm not sure. But we learned our lessen the hard way when all of our 2008 backups were essentially incomplete, due to a missing command-line switch that recorded (or in our case did not) the system state. We discovered this after, well, we could NOT restore a server from a backup. Trust me: Having to restore a failed server and having only an incomplete, out-of-date or broken backup, is not a situation you want to find yourself in.

My last recommendation is off-site storage. Yes, you have a sprinkler system, building security and feel comfortably safe. But look at the picture on top. Are you prepared for that? If not, then you should probably look into off-site backups.

So, let me recap:

1. Don't neglect your backups because you are using RAID arrays.
2. Use highly reliable RAID levels, or configure a hot spare.
3. Ensure you receive notifications when a RAID array is degraded
4. Test your backups regularly, but at the very least test them once to ensure they work.
5. Store your backups, or at least the most critical, off-site.

Stay redundant,
Ingmar.

Creating your very own event message DLL

Thu, 04 Nov 2010 11:11:03 -0600

If you've ever wrote code to log to the Windows event log before (e.g. through Perl, Python, ...), then you might have run into a similar problem that I described in an earlier post: Either the events don't look correctly in the event log, you are restricted to a small range of event ids (as is the case with eventcreate.exe) or you cannot utilize insertion strings.

In this blog post I'll be showing you how to build a custom event message DLL, and we'll go about from the beginning to the end. We'll start with creating the DLL using Visual Studio (Express) and finish up with some example scripts, including Perl of course, to utilize the DLL and log elegantly to the event log.

Let's say you are running custom scripts on a regular basis in your network - maybe with Perl, Python, Ruby etc. Your tasks, binary as they are, usually do one of two things: They run successfully, or they fail. To make troubleshooting easier, you want to log any results to the event log - in a clean manner. Maybe you even have sysadmins in other countries and want to give them the ability to translate standard error messages. Logging to the event log has a number of benefits: It gives you a centralized record of your tasks, allows for translation, and gives you the ability to respond to errors immediately (well, I'm of course assuming you are using an event log monitoring solution such as EventSentry). Sounds interesting? Read on!

Yes, you can do all this, and impress your peers, by creating your own event message file. And what's even better, is that you can do so using all free tools. Once you have your very own event message file, you can utilize it from any application that logs to the event log, be it a perl/python/... script or a C/C++/C#/... application.

To create an event message file, you need two applications:

Microsoft Platform SDK
Microsoft Visual C++ 2010 Express. Earlier version will do as well, but since it's free we'll be utilizing the latest version, 2010.

The reason you need the platform SDK, is because Visual Studio Express does not ship with the Message Compiler, mc.exe, for some reason. The message file compiler is essential, as without it there will be no event message file unfortunately. When installing the platform SDK, you can deselect all options except for "Developer Tools -> Windows Development Tools -> Win32 Development Tools" if you want to conserve space. This is the only essential component of the SDK that's needed.

An event message file is essentially a specific type of resource that can be embedded in either a DLL file or executable. In EventSentry, we originally embedded the message file resources in a separate DLL, but eventually moved it into the executable, mostly for cleaner and easier deployment. We'll probably go back to a separate message DLL again in the future, mostly because processes (e.g. the Windows Event Viewer) can lock the event message file (the executable in our case), making it difficult to update the file.

Since embedding an event message file in a DLL is more flexible and significantly easier to accomplish, I'll be covering this scenario here. The DLL won't actually contain any executable code, it will simply serve as a container for the event definitions that will be stored inside the .dll file. While it may sound a little bit involved to build a DLL just for the purpose of having an event message file (especially to non-developers), you will see that it is actually surprisingly easy. There is absolutely no C/C++ coding required, and I also made a sample project available for download, which has everything setup and ready to go.

In a nutshell, the basic steps of creating an event message file are as follows:

1. Create a message file (e.g. messagefile.mc)
2. Convert the message file into a DLL, using mc.exe, rc.exe and link.exe

Once we have the message file, we will also need to register the event message file in the registry, and associate it with an event source. Keep in mind that the event source is not hard-coded into the message file itself, and in theory a single event message file could be associated with multiple event sources (as is the case with many event sources from Windows).

So let's start by creating a working folder for the project, and I will call it "myapp_msgfile". Inside that directory we'll create the message file, let's call it myapp_msgfile.mc. This file is a simple text file, and you can edit it with your favorite text editor (such as Ultraedit, Notepad2 or Notepad++).

The file with the .mc extension is the main message file that we'll be editing - here we define our event ids, categories and so forth. Below is an example, based on the scenario from before. Explanations are shown inline.

MessageIdTypedef=WORD

LanguageNames=(
English=0x409:MSG00409
German=0x407:MSG00407
)

Here we define which languages we support, and by which files these languages will be backed. You will have to look up the language id for other languages if you plan on supporting more, and you can remove German if you only plan on supporting English.

MessageId=1
SymbolicName=MYTOOL_CATEGORY_GENERAL
Language=English
Tasks
.
Language=German
Jobs
.

Our first event id, #1, will be used for categories. Categories work in the exact same way as event ids. When we log an event to the event log and want to include a category, then we only log the number - 1 in this case.

MessageId=100
SymbolicName=TASK_OK
Language=English
Task %1 (%2) completed successfully.
.
Language=German
Job %1 (%2) war erfolgreich.
.

This is the first event description. The "MessageId" field specifies the event id, and the symbolic name is a descriptive and unique name for the event. The language specifies one of the supported languages, followed by the event message text. You end the event description with a single period - that period has to be the only character per line.

MessageId=101
SymbolicName=TASK_ERROR
Language=English
Task %1 (%2) failed to complete due to error "%3".
.
Language=German
Job %1 (%2) konnte wegen Fehler "%3" nicht abgeschlossen werden.
.
MessageId=102
SymbolicName=TASK_INFO
Language=English
Task Information: %1
.
Language=German
Job Information: %1
.

Since we're trying to create events for "custom task engine", we need both success and failure events here. And voila, our event message file now has events 100 - 102, plus an id for a category.

So now that we have our events defined, we need to convert that into a DLL. The first step now is to use the message compiler, mc.exe, to create a .rc file as well as the .bin files. The message compiler will create a .bin file for every language that is defined in the mc file. Open the "Visual Studio Command Prompt (2010)" in order for the following commands to work:

mc.exe myapp_msgfile.mc

will create (for the .mc file depicted above):

myapp_msgfile.rc
msg00407.bin
msg00409.bin

With those files created, we can now create a .res (resource) file with the resource compiler rc.exe:

rc.exe /r myapp_msgfile.rc

which will create the

myapp_msgfile.res

file. The "/r" option instructs the resource compile to emit a .res file. Now we're almost done, we're going to let the linker do the rest of the work for us:

link -dll -noentry -out:myapp_msgfile.dll myapp_msgfile.res

The myapp_msgfile.res is the only input file to the linker, normally one would supply object (.obj) files to the linker to create a binary file. The "-noentry" option tells the linker that the DLL does not have an entry point, meaning that we do not need to supply a DllMain() function - thus the linker is satisfied even without any object files. This is of course desired, since we're not looking to create a DLL that has any code or logic in it.

After running link.exe, we'll end up with the long awaited myapp_msgfile.dll file.

The end. Well, almost. Our message file is at this point just a lone accumulation of zeros and ones, so we need to tell Windows that this is actually a message file for a particular event log and source. That's done through the registry, as follows:

Open the registry editor regedit.exe. Be extremely careful here, the registry editor is a powerful tool, and needs to be used responsibly :-).

All event message files are registered under the following key:

HKLM\System\CurrentControlSet\Services\eventlog

Under this key, you will find a key for every event log as well as subkeys for every registered event source. So in essence, the path to an event source looks like this:

HKLM\System\CurrentControlSet\Services\eventlog\EVENTLOG\EVENTSOURCE

I'm going to assume here that we are going to be logging to the application event log, so we'd need to create the following key:

HKLM\System\CurrentControlSet\Services\eventlog\Application\MyApp

In this key, we need to following values:

TypesSupported (REG_DWORD)
EventMessageFile (REG_EXPAND_SZ)

TypesSupported is usually 7, indicating that the application will log either Information, Warning or Error events (you get 7 if you OR 1[error], 2[warning] and 4[information] together).

EventMessageFile is the path to your message DLL. Since the type is REG_EXPAND_SZ, the path may contain environment variables.

If you plan on utilizing categories as well, which I highly recommend (and for which our message file is already setup), then you need two additional values:

CategoryCount (REG_DWORD)
CategoryMessageFile (REG_EXPAND_SZ)

CategoryCount simply contains the total number of categories in your message file (1, in our case), and the CategoryMessageFile points to our message DLL. Make sure that your message file does not contain any sequence gaps, so if your CategoryCount is set to 10, then you need to have an entry for every id from 1 to 10 in the message file.

We could create separate message files for messages and categories, but that would be overkill for a small project like this.

Now that we have that fancy message DLL ready to go, we need to start logging. Below are some examples of how you can log to the event log with a scripting language. I'll be covering Perl, Kix, and Python. Me being an old Perl fand and veteran, I'll cover that first.

PERL
The nice thing about Perl, is that you can take full advantage of insertion strings, so it supports event definitions containing more than one insertion string.


use strict;
use Win32::EventLog;



# Here we define our event source and category, which we consider static throughout 
# the application. You can change this if the category is different


my %eventDetails = (Source      => "MyApp",
                    Category    => "1");    

# this is id from the message file
                                            # which was set aside for the category
                                                    

# Call this function to log an event


sub logMessage
{
    my ($eventID, $eventType, @eventDetails) = @_;
    
    my $evtHandle = Win32::EventLog->new("Application");

    $eventDetails{EventID}       = $eventID;    
    $eventDetails{EventType}     = $eventType;
    $eventDetails{Strings}       = join("\0", @eventDetails);

    $evtHandle->Report(\%eventDetails);
    
    $evtHandle->Close;
}



# This is what you would use in your scripts to log to the event log. The insertion strings 
# are passed as an array, so even if you only have one string, you would need to pass it
# within brackets ("This is my message") as the last parameter



logMessage(100, EVENTLOG_INFORMATION_TYPE, ("Database Backup", "Monitoring Database"));
logMessage(102, EVENTLOG_INFORMATION_TYPE, ("Step 1/3 Complete"));

PYTHON
Python supports event logging very well too, including multiple insertion strings. See the sample code below:


import win32evtlogutil
import win32evtlog



# Here we define our event source and category, which we consider static throughout
# the application. You can change this if the category is different



eventDetails = {'Source': 'MyApp',    # this is id from the message file
                'Category': 1}        # which was set aside for the category




# Call this function to log an event


def logMessage(eventID, eventType, message, eventDetails):
    if type(message) == type(str()):
        message = (message,)
    win32evtlogutil.ReportEvent(eventDetails['Source'], eventID, eventDetails['Category'], eventType, tuple(message))


logMessage(100, win32evtlog.EVENTLOG_INFORMATION_TYPE, ("Database Backup", "Monitoring Database"), eventDetails)
logMessage(102, win32evtlog.EVENTLOG_INFORMATION_TYPE, ("Step 1/3 complete"), eventDetails)

KIXTART
The pro: Logging to the event log using KiXtart is so easy it's almost scary. The con: It only supports message files that use one insertion string.


LOGEVENT(4, 102, "Database Backup", "", "MyApp")

Curiosity Kills the Cat

Tue, 20 Jul 2010 19:02:17 -0600

25 years ago, on July 24th 1985, the Amiga 1000 was introduced in New York City (check out the ad). Coincidentally, the Amiga 500 was my first computer and I loved playing games on the Rock Lobster - despite the 7.15909 MHz processor. Well, those were the good old days, the days before mainstream email, the days before spam. Or were they? Believe it or not, in 1985 it had already been 7 years since the first spam email was sent by Gary Thuerk over the ARPAnet.

Source: Wikimedia Commons, Amiga 1000

I don't know about you, but 32 years later I still get spam delivered to my inbox on a daily basis, and that's despite having 2-3 spam filters in place. What's more, I still get legitimate email caught by the spam filter, mostly to the dismay of the sender.

Now, of course WE all know not to open spam - or to even look at it - as it will potentially confirm receipt (if you display images from non-trusted sources) and could also trigger malware (again depending on your email reader's configuration).

But, we've all seen spam emails and I can't help but wonder who actually reads these emails (for purposes other than to get a chuckle), much less opens them! Let's not even think about who opens attachments or clicks links (yikes!) from spam emails.

Source: Wikimedia Commons, Spam.jpg

The Facts

So WHO are those people opening, clicking spam? Well, turns out that the MAWWG, the Messaging Anti-Abuse Working Group determines exactly that (and presumably other things too) - every year. Better yet, they publish that information for our enjoyment.

It's been a few months since the latest findings were published, but I'd consider them relevant today nevertheless (and a year from now for that matter).

In a nutshell, the group surveyed the behavior of consumers both in North America and Europe, and published key findings in regards to awareness, consumer confidence and so forth.

Before I give the link to the full PDF (see the Resources section below); here are what I think are some of the most interesting facts:

Half of all users in North America and Europe have "confessed" to opening or accessing spam. 46% of those who opened spam, did so intentionally to unsubscribe or out of some untameable sense of curiosity. Some were even interested in the products "advertised" to them!

Bottom Line: 1 out of 4 people open spam emails because they want to know more, or want to unsubscribe.
In more detail, 19% of all users surveyed either clicked on a link from an email (11%) or opened an attachment from an email (8%) that they themselves suspected to be spam. I found that to be one of the most revealing numbers in the report.
Young users (under 35) consider themselves more experienced, yet at the same time engage in more risky behavior than other age groups. In Germany, 33% of all users consider themselves to be experts. Compare that to France, where only 8% of all users think they are pros.
Less than half of users think that stopping spam or viruses is their responsibility. Instead, they feel that the responsibility lies mainly with the ISP and A/V companies. 48% of all respondents do realize that it is their responsibility. The report doesn't state whether this particular question, which lists 10 choices, was a multiple choice question.
When asked about bots, 84% of users were familiar with the possibility that software, say a virus, can control their computer. At the same time, only 47% were familiar with the terms "bot" or "botnet".
On the upside, 94% of all users are running A/V software that is up-to-date, which is a comforting fact. I can only imagine that the remaining 6%, given Apple's market share, account for most of the rest.

My opinion: OS X users are probably still oblivious and don't see the need to install A/V or any other type of security software on their computers. Still, some PC users apparently still don't install AntiVirus/AntiMalware on their computers, despite many free options being available today.

Wow, that's a lot of bad news to digest. So if I may summarize - the reason why we keep getting spam in our inboxes, is because every 5th person with a computer clicks on links or opens attachments (ah!) from spam emails, and because 6% of all users with a computer don't run security software. Given the amount of people that dwell in the western hemisphere, that amounts to a lot of people.

Well, at least I know now why I keep getting those nuisance emails in my inbox. But somehow I don't feel any better about them.

Training Day

I think what this report shows us the importance of user education. While people are apparently aware of spam, it doesn't look like the average Joe is aware of the implications that a simple click in an email can have.

If you are reading this email, then you are probably a network professional working in an organization. With that, you have a unique opportunity to organize a simple workshop with your employees to educate them about the potential threats, and remind them that it's not a good idea to do anything with suspect emails.

Source: Wikimedia Commons, Botnet for Beginners

There is a wealth of information available on the web about educating users on spam and general computer security. We all know that software can only do so much - it's a constant cat & mouse game between the researchers and the bad guys. It's simply not possible, at least not today, to make the computers we use on a daily basis 100% secure.

While securing computers in a corporation is possible to some extent using whitelisting, content filters and such, doing the same thing for home computers is much more difficult. And it's those computers that are most likely to be part of a botnet.

I can only imagine that the average user does not know that botnets can span thousands, if not millions, of computers. The Conficker botnet alone infected around 10 million computers and has the capacity to send 10 billion emails per day.

Let's face it, the situation will not improve as long as people will click links in emails and open attachments from suspicious senders.

I encourage you to organize a training session with your users on a regular basis. If your organization is large, then you might want to start with the key employees first, and maybe create a tiered training structure.

Our Network is Safe

You might think that your network is safe. You have AntiVirus, white listing, AntiMalware, firewalls in every corner, web content filters and more. Scheduling a training sessions to tell your users on not to do the obvious, is probably the last thing on your mind.

But read on.

Risky behavior by your end users will not only affect global spam rates, but your organization as well. Corporate espionage is growing, and spies (whether they are from a foreign government or corporation) often use email to initially get access to an individuals computers. See SANS Corporate Espionage 201 (PDF) for some techniques being employed.

For example, pretty much every organization has people working from home. If a malicious attacker can compromise a home computer that is used to access a corporate network (even if it's just used to access emails) and install a key logger, then they will most likely have gotten access to your corporate network. Once they have their foot in the door, it's only a matter of time.

There are plenty of resources available on the net on how to educate users on security, spam and so forth. A short training session of 20 minutes is probably enough. The message to convey is simple, and if you keep a few points in mind the session can even be fun. Consider the following for the training session:

Be sure to interact with your users. Start off by asking them if they use A/V software or AntiMalware software at home.
Tell them about botnets, and if they would be happy knowing that their computer is part of a 10 million botnet controlled by people in the Ukraine.
Be sure to explain that a single users actions can compromise their corporate network.
Explain that technology cannot provide 100% security against intruders.

Of course, user education alone is not the answer to solving security problems like viruses, phishing and the like. Encryption, digital signatures (especially for corporate emails), white-listing all should be employed regardless of user education.

Resources

2010 MAAWG Consumer Survey Key Findings Report (6 pages)
2010 MAAWG Consumer Survey Full Report (87 pages)

Using Cartoons to Teach Internet Security
Get IT Done: IT pros offer tips for teaching users

UNICODE - ONE code to rule them all

Tue, 20 Apr 2010 17:10:19 -0600

If you live in an English-speaking country like the United States, United Kingdom or Australia, then you are in the lucky position where every character in your language can be represented by the ASCII table. Many other languages aren't as lucky unfortunately, and it is no surprise given the fact that over 1000 written languages exist. Most of these languages cannot be interpreted by ASCII, most notably Asian and Arabic languages.

Take the text below for example, ASCII would be struggling with this a bit (to say the least):

النمسا

Understanding UNICODE is no easy feat however - just the mere abbreviations out there can be mind-boggling: UTF-7, 8, 16, 32, UCS-2, BOM, BMP, code points, Big-Endian, Little-Endian and so forth. UNICODE support is particularly interesting when dealing with different platforms, such as Windows, Unix and OS X.

It's not all that bad though, and once the dust settles it can all make sense. No, really. As such, the purpose of this article is to give you a basic understanding of UNICODE, enough so that the mention of the word UNICODE doesn't give you cold shivers down your back.

Unicode is essentially one large character set that includes all characters of written languages, including special characters like symbols and so forth. The goal - and this goal is reality today - is to have one character set for all languages.

Back in 1963, when the first draft of ASCII was published, Internationalization was probably not on the top of the committee member's minds. Understandable, considering that not too many people were using computers back then. Things have changed since then, as computers are turning up in pretty much every electrical device (maybe with the exception of stoves and blenders).

The easiest way to start is, of course, with ASCII (American Standard Code for Information Interchange). Gosh were things simple back in the 60s. If you want to represent a character digitally, you would simply map it to a number between 1 and 127. Voila, all set. Time to drive home in your Chevrolet, and listen to a Bob Dylan, Beach Boys or Beatles record. I won't go in to the details now, but for the sake of completeness I will include the ASCII representation of the word "Bob Dylan":

String:      B    o    b         D    y    l    a    n
Decimal:     66   111 98   32   68   121 108 111 110
Hexadecimal: 0x42 0x6F 0x62 0x20 0x44 0x79 0x6C 0x6F 0x6E
Binary:      01000010 01101111 01100010 00010100
             01000100 01111001 01101100 01101111 01101110

Computers, plain and simple as they are, store everything as numbers of course, and as such we need a way to map numbers to letters, and vice versa. This is of course the purpose of the ASCII table, which tells our computers to display a "B" instead of 66.

Since the 7-bit ASCII table has a maximum of 127 characters, any ASCII character can be represented using 7 bits (though they usually consume 8 bits now). This makes calculating, how long a string is for example, quite easy. In C programs for example, ASCII characters are represented using chars, which use 1 byte (=8 bits) of storage. Here is an example in C:

char author[] = "The Beatles";
int authorLen = strlen(author);        // authorLen = 11
size_t authorSize = sizeof(author);    // authorSize = 12

The only reason the two variables are different, is because C automatically appends a 0x0 character at the end of a string (to indicate where it terminates), and as such the size will always one char(acter) longer than the length.

So, this is all fine and well if we only deal with "simple" languages like English. Once we try to represent a more complex language, Japanese for example, things start to get more challenging. The biggest problem is the sheer number of characters - there are simply more than 127 characters in the world's written languages. ASCII was extended to 8-bit (primarily to accommodate European languages), but this still only scratches the surface when you consider Asian and Arabic languages.

Hence, a big problem with ASCII is that is essentially a fixed-length, 8-bit encoding, which makes it impossible to represent complex languages. This is where the Unicode standard comes in: It gives each character a unique code point (number), and includes variable-length encodings as well as 2-byte (or more) encodings.

But before we go to deep into Unicode, we'll just blatantly pretend that Unicode doesn't exist and think of a different way to store Japanese text. Yes! Let us enter a world where every language uses a different encoding! No matter what they want to make you believe - having countless encodings around is fun and exciting. Well, actually it's not, but let's take a look here why.

The ASCII characters end at 127, leaving another 127 characters for other languages. Even though I'm not a linguist, I know that there are more than 127 characters in the rest of the world. Additionally, many Asian languages have significantly more characters than 255 characters, making a multi-byte encoding (since you cannot represent every character with one byte) necessary.

This is where encodings come in (or better, "came" in before Unicode was established), which are basically like stencils. Let's use Japanese for our code page example. I don't speak Japanese unfortunately, but let's take a look at this word, which means "Farewell" in Japanese (you are probably familiar with pronunciation - "sayōnara"):

さようなら

The ASCII table obviously has no representation for these characters, so we would need a new table. As it turns out, there are two main encodings for Japanese: Shift-JIS and EUC-JP. Yes, as if it's not bad enough to have one encoding per language!

So code pages serve the same purpose as the ASCII table, they map numbers to letters. The problem with code pages - opposed to Unicode - is that both the author and the reader need to view the text in the same code page. Otherwise, the text will just be garbled. This is what "sayōunara" looks like in the aforementioned encodings:

EUC-JP
0xA4 B5 A4 E8 A4 A6 A4 CA A4 E9

Shift_JIS
0x82 B3 82 E6 82 A4 82 C8 82 E7

Their numerical representation between EUC-JP and Shift_JIS is, as is to be expected, completely different - so knowing the encoding is vital. If the encodings don't match, then the text will be meaningless. And meaningless text is useless.

You can imagine that things can get out of hand when one party (party can be an Operating System, Email client, etc.) uses EUC-JP, and the other Shift_JIS for example. They both represent Japanese characters, but in a completely different way.

Encodings can either (to a certain degree) be auto-detected, or specified as some sort of meta information. Below is a HTML page with the same Japanese word, Shift_JIS encoded:

<HTML>
    <TITLE>Shift_JIS Encoded Page</TITLE>

    <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=Shift_JIS">
    <BODY>
            さようなら
    </BODY>
</HTML>

You can paste this into an editor, save it has a .html file, and then view it in your favorite browser. Try changing "Shift_JIS" to "EUC-JP", fun things await you.

But I am getting carried away, after all this post is about Unicode, not encodings. So, Unicode solves these problems by giving every character from every language a unique code point. No more "Shift_JIS", no more "EUC-JP" (not even to mention all the other encodings out there), just UNICODE.

Once a document is encoded in Unicode, specifying a code page is no longer necessary - as long as the client (reader) supports the particular Unicode encoding (e.g. UTF-8) the text is encoded with.

The five major Unicode encodings are:

UTF-8
UCS-2
UTF-16 (an extension of UCS-2)
UTF-32
UTF-7

All of these encodings are Unicode, and represent Unicode characters. That is, UTF-8 is just as capable as UTF-16 or UTF-32. The number in the encoding name represents the minimum number of bits that are required to store a single Unicode code point. As such, UTF-32 can potentially require 4 x as much storage as UTF-8 - depending on the text that is being encoded. I will be ignoring UTF-7 going forward, as its use is not recommended and it's not widely used anymore.

The biggest difference between UTF-8 and UCS-2/UTF-16/UTF-32 is that UTF-8 is a variable length encoding, opposed to the others being fixed-length encodings. OK, that was a lie. UCS-2, the predecessor of UTF-16, is indeed a fixed length encoding, whereas UTF-16 is a variable length encoding. In most use cases however, UTF-16 uses 2 bytes and is essentially a fixed length encoding. UTF-32 on the other hand, and that is not a lie, is a fixed-length encoding that always uses 4 bytes to store a character.

Let's look at this table which lists the 4 major encodings and some of their properties:

Encoding	Variable/Fixed	Min Bytes	Max Bytes
UTF-8	variable	1	4
UCS-2	fixed	2	2
UTF-16	variable	2	4
UTF-32	fixed	4	4

What this means, is that in order to represent a Unicode character (e.g. さ), a variable length encoding might require more than 1 byte, and in UTF-8's case up to 4 bytes. UTF-8 needs potentially more bytes, since it maintains backward-compatibility with ASCII, and as such loses 7 bits.

Windows uses UTF-16 to store strings internally, as do most Unicode frameworks such as ICU and Qt's QString. Most Unixes on the other hand use UTF-8, and it's also the most commonly found encoding on the web. Mac OSX is a bit of a different beast; due to it using a BSD kernel, all BSD system functions use UTF-8, whereas Apple's Cocoa framework uses UTF-16.

UCS-2 or UTF-16
I had already mentioned that UTF-16 is an extension of UCS-2, so how does it extend it and why does it extend it?

You see, Unicode is so comprehensive now that it encompasses more than what you can store in 2 bytes. All characters (code points) from 0x0000 to 0xFFFF are in the "BMP", the "Basic Multilingual Plane". This is the plane that uses most of the character assignments, but additional planes exist, and here is a list of all planes:

•    The "BMP", "Basic Multilingual Plane", 0x0000 -> 0xFFFF
•    The "SMP", "Supplementary Multilingual Plane", 0x10000 -> 0x1FFFF
•    The "SIP", "Supplementary Ideographic Plane", 0x20000 -> 0x2FFFF
•    The "SSP", "Supplementary Special-purpose Plane", 0xE0000 -> 0xEFFFF

So technically, having 2 bytes available is not even enough anymore to cover all the available code points, you can only cover the BMP. And this is the main difference between UCS-2 and UTF-16, UCS-2 only supports code points in the BMP, whereas UTF-16 supports code points in the supplementary planes as well, through something called "surrogate pairs".

Representation in Unicode
So let's look at the above sample text in Unicode, shall we? Sayonara Shift_JIS & EUC-JP! The site http://rishida.net/tools/conversion/ has some great online tools for Unicode, one of which is called "Uniview". It shows us the actual Unicode code points, the symbol itself and the official description:

The official Unicode notation (U+hex) for the above characters uses the U+ syntax, so for the above letters we would write:

U+3055 U+3088 U+3046 U+306A U+3089

With this information, we can now apply one of the UTF encodings to see the difference:

UTF-8
E3 81 95 E3 82 88 E3 81 86 E3 81 AA E3 82 89

UTF-16
30 55 30 88 30 46 30 6A 30 89

UTF-32
00 00 30 55 00 00 30 88 00 00 30 46 00 00 30 6A 00 00 30 89

So UTF-8 uses 5 more bytes than UCS-2/UTF-16 to represent the same exact characters. Remember that UCS-2 and UTF-16 would be identical for this text since all characters are in the BMP. UTF-32 uses yet 5 more bytes then UTF-8 and would be require the most storage space, as to be expected.

What you can also see here, is that UTF-16 essentially mirrors the U+ notation.

Fixed Length or Variable Length?
Both encoding types have their advantages and disadvantages, and I will be comparing the most popular UTF encodings, UTF-8 and UCS-2, here:

Variable Length UTF-8:
•   ASCII-compatible
•   Uses potentially less space, especially when storing ASCII
•   String analysis/manipulation (e.g. length calculation) is more CPU-intensive

Fixed Length UCS-2:
•   Potentially wastes space, since it always uses fixed amount of storage
•   String analysis/manipulation is usually less CPU intensive

Which encoding to use will depend on the application. If you are creating a web site, then you should probably choose UTF-8. If you are storing data in a database however, then it will depend on the type of strings that will be stored. For example, if you are only storing languages that cannot be represented through ASCII, then it is probably better to use UCS-2. If you are storing both ASCII and languages that require Unicode, then UTF-8 is probably a better choice. An extreme example would be storing English-Only text in a UCS-2 database - it would essentially use twice as much storage as an ASCII version, without any tangible benefits.

One of the strongest suits of UTF-8, at least in my opinion, is its backward compatibility with ASCII. UTF-8 doesn't use any numbers below 127 (0x7F), which are - well - reserved for ASCII characters. This means that all ASCII text is automatically UTF-8 compatible, since any UTF-8 parser will automatically recognized those characters as being ASCII and render them appropriately.

The BOM
And this brings us to the next topic - the BOM (header). BOM stands for "Byte Order Mark", and is usually a 2-4 byte long header in the beginning of a Unicode text stream, e.g. a text file. If a text editor does not recognize a BOM header, then it will usually display the BOM header as either the þÿ or ÿþ characters.

The purpose of the BOM header is to describe the Unicode encoding, including the endianess, of the document. Note that a BOM is usually not used for UTF-8.

Let's revisit the example from earlier, the UTF-16 encoding looked like this:

30 55 30 88 30 46 30 6A 30 89

If we wanted to store this text in a file, including a BOM header, then it could look also look like this:

FF FE 55 30 88 30 46 30 6A 30 89 30

"FF FE" is the BOM header, and in this case indicates that a UTF-16 Little Endian encoding is used. The same text in UTF-16 Big Endian would look like this:

FE FF 30 55 30 88 30 46 30 6A 30 89

The BOM header is generally only useful when Unicode encoded documents are being exchanged between systems that use different Unicode encodings, but given the extremely little overhead it certainly doesn't hurt to add it to any UTF-16 encoded document. As such, Windows always adds a 2-byte BOM header to all Unicode text documents. It is the responsibility of the text reader (e.g. an editor) to interpret the BOM header correctly. Linux on the other hand, being a UTF-8 fan and all, does not need to (and does not) use a BOM header - at least not by default.

Tools & Resources
There are a variety of resources and tools available to help with Unicode authoring, conversions, and so forth.

I personally like Ultraedit, which lets me convert documents to and from UTF-8 and UTF-16, and also supports the BOM headers. GEdit on Linux is also very capable, and supports different code pages (if you ever need to use those) as well. Babelpad is an editor designed specifically for Unicode, and seems to support every possible encoding. I have not actually used this editor though.

A nifty online converter that I already mentioned earlier can be found at http://rishida.net/tools/conversion/, and also check out UniView: http://rishida.net/scripts/uniview/.

The official Unicode website is of course a great resource too, though potentially overwhelming to mere mortals that only have to deal with Unicode occasionally. The best place to start is probably their basic FAQ: http://www.unicode.org/faq/basic_q.html.

I hope this provides some clarification for those who know that Unicode exists, but are not entirely comfortable with the details.

さようなら!

How to REALLY monitor SMTP, POP3 and IMAP on Exchange 2003

Mon, 01 Feb 2010 07:15:46 -0600

Even though Microsoft Exchange Server 2010 has already been released, many organizations still use Exchange 2003. In this article I'll explain how to thoroughly monitor the various Internet protocols that Exchange 2003 offers, including SMTP, POP3, IMAP (and NNTP for that matter). The reason why I'll only be looking at Exchange 2003 is because there is a significant difference in architecture between Exchange 2003 and later versions.

It is a common misconception that you can effectively monitor the W3SVC service (commonly referred to as IIS, though IIS encompasses a lot more than just a web server) and other services provided through IIS, such as SMTP and POP3, by simply monitoring their associated service. It's a misconception, because a given IIS-based service may contain multiple instances - most commonly the case with the World Wide Web Service which often hosts multiple independent web sites. The status of these instances can be controlled independently of the hosting service, though that service needs to be running of course.

Don't despair though, most server-based windows applications, fortunately, can be monitored by ensuring that their respective service is - well - running. For example, to ensure that the Apache service is up, you "simply" make sure that the Apache service is running. The same goes for countless other services such as MySQL - even SQL Server (of course you can still detach individual databases in SQL Server).

Exchange 2003, due its partnership with the Internet Information Services 6.0, is different though. Yes, IIS and Exchange 2003 are tightly coupled, and if you intend to have your Exchange Server 2003 communicate with any other server using a standard Internet protocol such as SMTP, then you will need IIS.

The screenshot above shows that the inetinfo.exe process hosts all the major services (bold name), and that each service can host one or more instance. For more details please see http://technet.microsoft.com/en-us/library/bb124674(EXCHG.65).aspx.

The three most common Internet services your Exchange 2003 server is running are probably SMTP, POP3 and IMAP4. While a lot of attention is being paid to the core Exchange services such as

• Microsoft Exchange Information Store (MSExchangeIS)
• Microsoft Exchange System Attendant (MSExchangeSA)

The services providing SMTP, POP3 and IMAP4 connectivity are usually similarly important, especially the SMTP service. Looking at the EventSentry service status page immediately reveals that the SMTP, POP3 and IMAP4 services are managed by IIS:

As you can see, IMAP4Svc, POP3Svc and SMTPSvc all use inetinfo.exe (Executable column) for their host process. So why is this important again?

Since all of these services support multiple instances INSIDE the service (inetinfo.exe), the host process will continue to run even when one or more instances inside the service are stopped. Since most installations only have one instance, stopping that one instance inside the service will still leave the service up and running. The effect of course is the same; the service is not available to the end users while the Windows service will happily continue to run.

A screen shot from the System Manager application shows instances listed inside:

As you can see with the IMAP4 protocol, we have two virtual servers setup that are both hosted inside the "Microsoft Exchange IMAP4" service. To stubbornly illustrate my point further I took a screenshot that shows both IMAP4 instances stopped while the service itself is running:

So I think we're all in agreement now that monitoring the POP3, SMTP etc. services in Exchange 2003 is not enough if you want to ensure that these services are actually available. So how do we monitor all of these instances?
The easiest way is actually with a VBScript, which is included below. VBScript works well since the cscript.exe interpreter is readily installed on Windows 2003, so no additional installation of tools is required. The script enumerates all instances of a given protocol, and checks whether they are running or not. If at least one instance is not running, the tool will return 1, thus setting the ERRORLEVEL to 1.

This VBScript can then be embedded into EventSentry, which will then run the script at set intervals using the application scheduler, notifying you via email (with the proper filter setup) when an instance is stopped. There's a screencast for that, you can view it at http://www.eventsentry.com/screencasts/eventsentry-application-scheduler/eventsentry-application-scheduler.htm. It shows you how to create an embedded script and setup EventSentry to notify you when the scripts returns an error. Note that the screencast uses an older version of the script which only monitored web sites (not SMTP, IMAP4, ...), but the process of setting up the script with EventSentry is exactly the same.

You should be able to use the script as-is, just configure which protocols are monitored by adjusting the values in the "Define which protocols to monitor here" section. The script always prints all installed instances and their status, and any stopped instance is prefixed with an asterisk. Below is what an email from EventSentry looks like:

The line with the stopped instance won't be yellow in the actual email, I just added this for readability. The script can also easily be modified to automatically start any stopped instances - simply add the line

Instance.Start

after line 102. This will still trigger an email (or error) to notify you that it was stopped, but a subsequent run of the script at the next monitoring interval should not trigger an error again if the start was successful.

A note of caution here though - I have seen the script hang indefinitely with this line added when an instance that is currently stopped can't be started because it's not configured correctly. Hence, it's not included by default.

' Lists the state of all IIS protocols configured on the local machine
' and returns an %ERRORLEVEL% of 1, if at least one instance is not in
' the "Started" state.
'
' When scheduling this script with EventSentry's application scheduler,
' make sure that the interpreter is set to "cscript.exe"

Option Explicit

Dim allInstancesAreRunning

Dim monitorSMTP, monitorPOP3, monitorIMAP4, monitorNNTP, monitorFTP, monitorWWW

' Define which protocols to monitor here

monitorSMTP = 1
monitorPOP3 = 1
monitorIMAP4 = 1
monitorNNTP = 1
monitorFTP   = 1
monitorWWW   = 1

' Define which protocols to monitor here

' ==================== EXECUTION STARTS HERE ====================
allInstancesAreRunning = EnumerateAllInstances

If allInstancesAreRunning = 0 Then
    WScript.Echo vbCRLF & "WARNING: One or more IIS components are not running" & vbCRLF
End If

If allInstancesAreRunning = 0 Then
    WScript.Quit 1
End If

' ==================== FUNCTIONS ====================
Function EnumerateAllInstances

EnumerateAllInstances = 1

If monitorSMTP = 1 Then
    EnumerateAllInstances = EnumerateAllInstances And EnumerateInstances("localhost", "SMTPSVC")
End If

If monitorPOP3 = 1 Then
    EnumerateAllInstances = EnumerateAllInstances And EnumerateInstances("localhost", "POP3SVC")
End If

If monitorIMAP4 = 1 Then
    EnumerateAllInstances = EnumerateAllInstances And EnumerateInstances("localhost", "IMAP4SVC")
End If

If monitorNNTP = 1 Then
    EnumerateAllInstances = EnumerateAllInstances And EnumerateInstances("localhost", "NNTPSVC")
End If

If monitorFTP = 1 Then
    EnumerateAllInstances = EnumerateAllInstances And EnumerateInstances("localhost", "FTPSVC")
End If

If monitorWWW = 1 Then
    EnumerateAllInstances = EnumerateAllInstances And EnumerateInstances("localhost", "W3SVC")
End If

End Function

Function MapServiceToInstance( Service )

    If Service = "SMTPSVC" Then
        MapServiceToInstance = "IIsSmtpServer"
    ElseIf Service = "POP3SVC" Then
        MapServiceToInstance = "IIsPop3Server"
    ElseIf Service = "IMAP4SVC" Then
        MapServiceToInstance = "IIsImapServer"
    ElseIf Service = "W3SVC" Then
        MapServiceToInstance = "IIsWebServer"
    ElseIf Service = "NNTPSVC" Then
        MapServiceToInstance = "IIsNntpServer"
    ElseIf Service = "FTPSVC" Then
        MapServiceToInstance = "IIsFtpServer"
    End If

End Function

Function EnumerateInstances( Server, Service )
    On Error Resume Next

    Dim VirtualServerService
    Dim Instance, InstanceID

    EnumerateInstances = 1

    Set VirtualServerService = GetObject("IIS://" & Server & "/" & Service)

    If Err.Number = 0 Then
        InstanceID = MapServiceToInstance(Service)

        For Each Instance in VirtualServerService

        If Instance.KeyType = InstanceID Then

                If SiteIsNotRunning(Instance.ServerState) Then
                    WScript.StdOut.Write "*"
                    EnumerateInstances = 0
                End If

                WScript.StdOut.Write Instance.ServerComment & " (" & Service & "): " & State2Desc(Instance.ServerState) & vbCRLF
        End If
        Next
    End If

End Function

Function SiteIsNotRunning( nState )

    If nState <> 2 Then
        SiteIsNotRunning = 1
    Else
        SiteIsNotRunning = 0
    End If

End Function

Function State2Desc( nState )

    Select Case nState
    Case 1
        'MD_SERVER_STATE_STARTING
        State2Desc = "Starting"
    Case 2
        'MD_SERVER_STATE_STARTED
        State2Desc = "Running"
    Case 3
        'MD_SERVER_STATE_STOPPING
        State2Desc = "Stopping"
    Case 4
        'MD_SERVER_STATE_STOPPED
        State2Desc = "Stopped"
    Case 5
        'MD_SERVER_STATE_PAUSING
        State2Desc = "Pausing"
    Case 6
        'MD_SERVER_STATE_PAUSED
        State2Desc = "Paused"
    Case 7
        'MD_SERVER_STATE_CONTINUING
        State2Desc = "Continuing"
    Case Else
        State2Desc = "Unknown state"
    End Select

End Function

You can also download the script from here.

Until next time,
Ingmar.

Announcing EventSentry v2.91

Mon, 30 Nov 2009 17:09:05 -0600

Now that EventSentry v2.91 has been released, I'm happy to have the opportunity to blog about our monitoring solution again.

The most significant new feature in EventSentry is the Health Matrix, a new way to see your network status in a space-efficient way. In fact, you can see the overall health status of your entire network on a single screen, even if it consists of hundreds of hosts.

We also made numerous other changes throughout the web reports, and added some exciting new filtering capabilities with our event log filters, as well as improved speed with the event log engine and file checksum generations.

EventSentry v2.91 also includes many minor improvements throughout the application, including service monitoring, process tracking and more. We have also updated EventSentry Light, and a new version will be released in the coming days after we have completed testing.

But now to the new features in version 2.91:

Health Matrix
In the health matrix, each host is displayed as a colored square, circle or rectangle, with the color indicating the overall health of the monitored computer. When all of the monitored components of a host are in an OK status, the color of the square is green. The color will change to orange or red when a problem is detected, depending on the number or severity of the issue.

The health matrix is highly customizable, for example both the size and shape of the icons can be adjusted depending on the size of the network (and your monitor).

Event Log Monitoring
In 2.91, the event log filtering engine was improved, resulting in reduced CPU usage of the event log monitoring component. Since the CPU usage of the EventSentry agent is already quite low, you will most likely only notice this improvement on hosts that generate an extremely large number of events, such as domain controllers.

Also new is the ability to filter events based on insertion strings in addition to just filtering based on the event message text. This means that one can now match individual strings inside event messages against strings, numbers, file checksums and group memberships. If you are not familiar with the term "insertion string", then I highly recommend my previous post about event message files before you read on.

Consider the following hypothetical example: The environment-monitoring component of EventSentry logs event id 10908:

The temperature (78.21 degrees F) has fallen outside the configured range (60F to 76F).

which is defined as:

The temperature (%3 degrees %4) has fallen outside the configured range (%1%4 to %2%4).

This event obviously informs us, that the current temperature has exceeded a set limit. Now let's say that we wanted to get an email when the temperature exceeds the limit, but also send a page when the temperature exceeds 90 degrees.

The new filtering feature allows you to do just that, by using the numerical comparison functionality with insertion strings (of course you would also need to set the hour/day properties). Assuming that you already have a filter in place for regular email notifications, you would simply setup an additional include filter that would evaluate insertion string 3 (%3) and only match if the number is above 90. See the screen shot below for the example. The result is a filter that only matches when then the temperature exceeds 90 degrees.

2.91 also includes two more comparison options, file checksums and group membership. So, if an insertion string represents a filename (e.g. from a security event), then EventSentry can create a SHA checksum from the specified file and compare it with the value that you specified. Another example would be a security event that includes a username in an insertion string, in which case you could setup a filter that would only match if that user is a member of particular group you specify. Both examples are mostly applicable for security events, since those are most likely to contain either filenames or usernames.
Using file checksums, you can be notified whenever a user plays solitaire, even when the user renames the executable.

Simply create a checksum of the file first using shachecksum.exe (included in the free NTToolkit, make sure you account for different OS versions and platforms) and intercept the corresponding 4688 event.

Service Monitoring
Service Monitoring now collects the username as well as the executable of a service. These additional properties are available in the web reports and in events generated, for example when the username of a service changes.

Software Monitoring
Software monitoring has been overhauled in 2.91, and some limitations and bugs have been removed. On Vista, Win2k8 and later, Windows patches are now monitored and included in the software inventory. 64-bit software is now classified as such and searchable, and searching for installed Windows Updated patches has also been simplified.

SNMP Traps
EventSentry can now send version 2c and version 3 traps, previously only version 1 traps were sent by the agent. The SNMP trap daemon was originally set to be released as part of 2.91, but this feature has been pushed back to v2.92.

Web Reporting
We have made a number of improvements in the web reporting to make using our web-based interface easier:

•    Reports are now easily accessible from every page, in addition to the reports page.
•    The database usage page now shows the actual page name in addition to the table name.
•    The dashboard page has been overhauled
•    The network status page can be customized (performance counters & disks)

Miscellaneous Improvements

There have of course been other improvements across the board, such as:

•    Notes can now be applied to computers
•    AD-linked groups can be sorted, and authentication properties can be set globally
•    Hardware monitoring now includes the IP address of an interface
•    Process tracking can capture the command line of a process
•    Logon tracking includes group information
•    File checksum generation has been optimized and will now use fewer CPU resources (affects file monitoring and file access tracking)
•    The minimum database interval for environment monitoring has been reduced to 5 minutes from 15 minutes
•    Software uninstallation events now include the same information as software installation events

If you have an active maintenance agreement, then this 2.91 release will of course be free of charge. If you are not already using EventSentry, then you can download a free 30-day evaluation version from http://www.eventsentry.com/downloads_downloadtrial.php.

Happy Holidays,
Ingmar.

Group Policy Software Deployment: Targeting the right computers with WMI filters

Mon, 19 Oct 2009 22:04:03 -0600

Group policy was introduced with Windows 2000, and is an easy way of centralizing many Windows settings. In addition to centralizing event log and firewall settings, I personally like the ability to deploy MSI-based software applications with Group Policy, since it makes it extremely easy to deploy new software packages.

Even though Software Installation only works only with MSI-based packages, it does make deploying MSI-based software packages extremely easy. Here is a short list of software (mostly tools for sysadmins) that you can deploy using Active Directory:

7-Zip
Firefox (from FrontMotion)
Foxit Reader (a PDF reader)
TortoiseSVN
ActivePerl
Python
SourceGear Vault Client
OpenOffice (I have not tried this)
Adobe Flash (requires custom download)

There are of course many more, and you can distribute most Microsoft client applications, such as Microsoft Office, through Group Policy as well.

We generally deploy software through Group Policy when three or more computers use it, since it's very easy to create a new package (if you already have a network share etc. setup, then you can literally do it in 2 minutes).

Before I list some of the useful WMI queries we use to target certain operating systems or computer types, there are a couple of things to note for those who are new to software deployment via group policy:

Software packages are always installed right after a reboot, so they're mostly suitable for workstations.
The network share which hosts the MSI files needs to give the computer accounts (e.g. DESKTOP1$) at least read access. Generally, giving EveryOne Read access works well unless you have a reason to restrict access to the software packages that you distribute.

Since the mechanism to distribute software is based on group policies, any sort of software package you create inside a group policy, will need to be assigned to an organizational unit (OU).

Since OUs can contain a large amount of computers that might not all need that particular software package, you can use two techniques to narrow down which computers receive the software:

Security Filtering
WMI Filtering

Security Filtering
With this method, you create a security group in ActiveDirectory, place the computers that should get a particular software package into the group, and then specify this group in the Security Filtering list.

The screen shot below shows a group policy that will only be applied to members of the "Source Control Computers" group:

WMI Filtering
With this method, you can filter the computers which are affected by your policy, based on common properties of the Operating System. For example, some packages might distinguish between 32-bit and 64-bit, some packages might only work on Vista or later, whereas other packages apply only to servers. With WMI, you can target the right computers without having to mess with group memberships (though you will probably still need to do that). For example:

32-bit vs. 64-bit computers
only workstations
only computers running a certain OS
only computers with a certain amount of RAM
only computers of a certain brand

With WMI filtering, you just create the software group policy, for example:

7-Zip 32-bit
7-Zip 64-bit

and then apply the respective WMI filter to them. But lets cut to the chase, here are a few WMI queries that you can cut & paste:

Operating System 32-bit
Select * from Win32_Processor where AddressWidth = '32'
Operating System 64-bit
Select * from Win32_Processor where AddressWidth = '64'

Workstation
Select * from WIN32_OperatingSystem where ProductType=1
Domain Controller
Select * from WIN32_OperatingSystem where ProductType=2
Server
Select * from WIN32_OperatingSystem where ProductType=3

Some filters require multiple WMI queries, which are just chained together.

Workstation 32-bit
Select * from WIN32_OperatingSystem where ProductType=1
Select * from Win32_Processor where AddressWidth = '32'
Workstation 64-bit
Select * from WIN32_OperatingSystem where ProductType=1
Select * from Win32_Processor where AddressWidth = '64'

Windows XP
Select * from WIN32_OperatingSystem where Version like '5.1.%' and ProductType=1
Windows Vista
Select * from WIN32_OperatingSystem where Version like '6.0.%' and ProductType=1
Windows 7
Select * from WIN32_OperatingSystem where Version like '6.1.%' and ProductType=1

Windows 2003
Select * from WIN32_OperatingSystem where Version like '5.2.%' and ProductType>1
Windows 2008
Select * from WIN32_OperatingSystem where Version like '6.0.%' and ProductType>1
Windows 2008 R2
Select * from WIN32_OperatingSystem where Version like '6.1.%' and ProductType>1

WIN32_OperatingSystem of course includes more information that can be useful for WMI queries, such as a descriptive name of the installed OS ("Name") as well as the service pack installed ("ServicePackMajorVersion").

Manufacturer (e.g. DELL)
Select * from WIN32_ComputerSystem where Manufacturer = 'DELL'

Installed Memory (e.g. more than 1Gb)
Select * from WIN32_ComputerSystem where TotalPhysicalMemory >= 1073741824

Like I mentioned earlier, this is merely a small sample of the possible WMI queries one can use to filter group policies, but they should cover most relevant scenarios. Feel free to suggest other useful WMI queries and I will include them here.

For more information, check out these resources on WMI:

WMI
Secrets of Windows Management Instrumentation
Scriptomatic (Vista/Win2k8/Win7: run as administrator!)

Happy querying,
Ingmar.

Get your KIX on route 66 - Powerful (login) scripts made easy with KiXtart

Thu, 17 Sep 2009 16:07:39 -0600

Route 66 was a US highway that connected Chicago with Los Angeles (or vice versa), with a total length of almost 2500 miles (for the rest of world using the metric system: almost 4000 km). It was established in 1926 and Nat King Cole first recorded the song "(Get Your Kicks On) Route 66" in 1946.

Completely unrelated to Route 66 of course is KiXtart, a free, free-format scripting language for Windows.

I first ran across KiXtart back in '99, when I was looking for a scripting language that I could use to write login scripts in a NT4 network. My goals back then were simple, and included the ability to map printers and shares depending on the user and/or group membership.

I was already familiar with Perl back then, and would have preferred to use that, if it wouldn't have been for the requirement to install Perl on every workstation. Things have changed since then of course, and installing Perl today on every workstation in your domain would be rather simple with GroupPolicy (ActivePerl provides a MSI).

Still, KiXtart is a surprisingly simple and flexible scripting language that will allow you to accomplish most anything (not only in regards to login scripts) with extremely little effort. KiXtart also supports Windows 9x clients, if you are in the unfortunate position to take advantage of that functionality.

So what can you do with KiXtart? Here is an overview:

Read and/or write to the registry
Manage the event log
Add printer or network share connections
Create shortcuts, program groups etc.
Read and/or write from/to files
Retrieve system information (memory, hostname, IP address, ...)
Get group information
And much more ...

There really is little you cannot do, and most tasks can be accomplished with as little as one or two lines of code. How about some practical examples of what you can do with KiXtart:

Map the color laser printer to all members of the "Marketing" group at logon.

Map a network share depending on the network location (e.g. IP address) of a user.

Query registry values or log information to the event log.

Add a shortcut or program group

Change the wallpaper :-)

The KiXtart web site has the complete documentation for all commands and functions that are at your disposal, and you can download them in a variety of formats (I recommend the CHM format) from http://www.kixtart.org/?p=manual.

But that's all nothing but dry theory, so I will show you how to create a KiXtart script that accomplishes the following:

Creates a printer connection depending on the group membership of the user
Maps network shares depending on the group membership
Displays a warning message if the latest service pack is not installed
Maps another network share only if the user is in a certain IP network
Display a warning if the password is older than 180 days

1. Creating a printer connection

IF INGROUP("MARKETING")
        ? "Connecting to color laser ..."
        ADDPRINTERCONNECTION("\\PRINTSERVER\COLOR_LASER_1")
ENDIF

In this example we are taking advantage of two functions, INGROUP and ADDPRINTERCONNECTION. I think they are fairly self-explanatory. If the currently logged on user is in the MARKETING group, then a printer connection to \\PRINTSERVER\COLOR_LASER_1 will be established.

2. Mapping network shares

The same INGROUP feature can be used to add network connections as well, so here is how you can control connections to network shares based on group membership

;Map Home Directory
USE G: "\\FILESERVER\@USERID"

IF INGROUP("Marketing")
        USE I: "\\FILESERVER\Marketing"
ENDIF

IF INGROUP("SALES")
        USE J: "\\FILESERVER\Sales"
ENDIF

In this example I introduced macros (@USERID), another powerful feature of KiXtart. By default, pretty much any system property is available as a macro (macros always start with the @ symbol). @USERID contains the user name of the currently logged on user, but there are others, such as:

@ProductType (OS type, e.g. "Windows Vista Ultimate"
@Wksta (computer name)
@LDomain (logon domain)
@CSD (service pack information)
@CPU (CPU information)
@Address (MAC address of network adapter)
@IPaddress0, @IPaddress1, ... @IPaddress3 (IP address of xth network adapter)
@PWAge (password age)

Lines that contain comments in KiXtart start with a semi-colon.

3. Display a warning based on the service pack number

KiXtart also includes a variety of functions for handling strings. We can use the @CSD variable to get the service pack information:

? "Service Pack: " + @CSD

which will yield something similar to

Service Pack: Service Pack 1

In order to display a dynamic message, we can get the last character and evaluate it. So let's display a warning message if a user is running Vista with a service pack smaller than SP 2:

IF INSTR(@ProductType, "Vista") > 0
IF RIGHT(@CSD, 1) < 2
    MESSAGEBOX("Important Message from your IT Department" + @CRLF + @CRLF + "Your computer is not running the latest service pack, and will be upgraded tomorrow automatically at 10am. The upgrade will take approximately 30 minutes, and you will not be able to use your computer at that time." + @CRLF + @CRLF + "Thank you for your understanding.", "Service Pack Installation", 48)
ENDIF
ENDIF

The INSTR() function checks whether a string appears inside another string, and the LEFT() function retrieves the specified number of characters from the beginning of a string.

4. Map a network share depending on the IP address

Let's imagine that we have a network share with lots of really large files (e.g. corporate training videos and more) and that we only want to map this share if a user is in the headquarter, opposed to a satellite location which has a slow access speed.

IF LEFT(@IPADDRESS0, 11) = " 10. 10. 0"
         USE Z: "\\FILESERVER\TrainingVideos"
ENDIF

Now the network share is only mapped if the user is in the 10.10.0.0/24 subnet. You can also use the EnumIPInfo() if you need to get more information from the network adapter.

5. Display a warning if a password is old

Most networks require users to change their passwords on a regular basis, but wouldn't it be nice if we could give our users a one-time warning before they are faced with the inevitable prompt that requires them to change their password?

$PasswordWarningThreshold = 170

IF @PWAge = $PasswordWarningThreshold
   MESSAGEBOX("Your password is " + $PasswordWarningThreshold + " days old and will have to be changed in 10 days. Please think of a really good password in the meantime.", "Password Expiration", 64)
ENDIF

In this example I also introduced variables, which are specified with the dollar sign. These are simplified examples, and there is a lot more you can do. For example, using the registry functions, you can save user responses and previous alerts in the registry, and later read them again.

So, forget multiple logon scripts or batch scripts using "NET USE" commands. With KiXtart, you can have one central login script that can adjust dynamically to the user, location, operating system or even the computer itself.

To get started, simply follow these steps:

Create a batch file (e.g. login.cmd) with the following line:

%0\..\WKix32.exe %0\..\login.kix
Create the actual login script for KiXtart, e.g. "login.kix"
Assign the login script login.cmd to all user accounts that require them

That's it. You don't have to install anything on the client computers, and you now have a single login script for your entire network.

Until next time,
Ingmar.

Wish Sandwich - 5 (free) tools we wish Windows had

Thu, 13 Aug 2009 06:53:20 -0600

"Have you ever heard of a wish sandwich? A wish sandwich is the kind of a sandwich where you have two slices of bread and you, hee hee hee, wish you had some meat..."

These are part of the lyrics from the "Rubber Biscuit" song by "The Chips", covered by the Blues Brothers in 1978. At the time, UNIX was almost 10 years old, the first version of BSD had been released, and Microsoft had their office in Albuquerque with Bill Gates being 23 years old.

It would take almost another 20 years before Windows NT 4 would be released. But back to the future now.

Well, after working with Windows for about 15 years now, I also wish that the base set of utilities that ship as part of Windows would have been updated and improved. It might seem odd, but one of the first things I do when I install a new release of Windows, is to open notepad, the calculator and paint - to see if they have improved.

Somewhat surprisingly, Windows 7 brings a lot of improvements to the core utilities that ship with Windows. Microsoft not only spiced up Paint and Wordpad by giving them the "Ribbon", but also improved the calculator in ways never imagined before. Yeah!

But these improvements do not satisfy a long-time Windows user! Having worked with Linux, OS X and Windows since the 3.0 days, I have my own list of apps that I use to substitute or extend some of the archaic tools that ship with Windows.

And here they are:

1. Notepad
What we know today as "Notepad", was first seen in Windows NT 4.0. When Windows 2000 came out, notepad hadn't changed. Well, fair enough - it had only been 4 years after all. A short while later Windows XP was released with a bang, but notepad was still the same. Windows 2003 showed that an upgrade to notepad obviously had low priority, and the release of Vista confirmed to me that notepad was clearly no longer under development. The recent release of Windows 7 crushed my hopes of Microsoft ever releasing an updated version of Notepad. Sigh.

So, why was Notepad left behind? Well, I have a few theories:

a)    The developer who originally developed Notepad has left the company, and nobody at Microsoft understands the existing code enough to make modifications.
b)    Companies developing third-party editors formed a powerful, mafia-like lobby, threatening Microsoft (presumably kidnappings) to never ever release an update to notepad, to ensure that third-party editors will continue to sell well.
c)    Microsoft deems Notepad complete, and cannot imagine how this robust application could be improved.
d)    Windows applications do not use text files, since all configuration is stored in the registry or databases. Even though unneeded, Notepad is provided as a courtesy and might be excluded from future version (like, telnet.exe!).

Whatever the reason (I may never find out), fact is that Notepad hasn't been updated in 13 years, and since Windows 8 won't be out until 2012, probably won't change in 16 years. That's a lot of years for a software program.

Line Numbers?
Basic syntax highlighting?
Anyone?

So what could replace Notepad? Why, Notepad2 of course! To be fair, there are more powerful editors out there than Notepad2, but it's free, light-weight and fast. Florian's Notepad2 supports line numbers, syntax highlighting, line highlighting, encodings, Unix/Windows line endings, transparency and much more. An extended version from Kai Liu is also available here, most notably including code folding abilities. Tabs are not supported in either version, unfortunately.

My other favorite editor is Ultraedit, as it includes pretty much any feature you could ever want from an editor. A nice feature, for sysadmins in particular, is the ability to switch environments. The "System Administrator" view for example, allows you to show SSH/telnet/FTP windows along with the editor windows.

2. Command Prompt
Yeah, this hasn't changed much since the early days either, though the introduction of the PowerShell deserves some credit. Using Linux regularly though, I miss some of the features like tabs, transparency and so forth.

The good news is, there is an excellent substitution out there called Console. The latest beta of version 2 features transparency, multiple tabs, appearance options and is free. It's so free, that they even give you the source code if you want it! It works on all the machines I use (mostly Vista, soon to be Win7) and I'm very happy with it overall - though it is a beta still and you might run into a glitch every now and then. I sit around in the command line a lot, and having multiple tabs open is nice.

One option I really like is the ability to show the currently executing command as the tab title, which is useful because you can see when a long-running process finishes (see screenshot above where fping is running in the 2nd tab).

When downloading, get the latest beta and simply extract all files from the

Console2\bin\release

folder to a directory of your choice.

3. Desktops / Spaces
Linux, and Unix, had multiple desktops since the industrial revolution. Well, at least it seems that way. I'm not sure why this hasn't been added to Windows yet, given that:

•   Every major Operating System OTHER than Windows includes it
•   Microsoft provides a tool (part of Sysinternals) that offers this functionality

Yes, in the age of affordable large monitors, multiple desktops aren't really that necessary anymore. But, many of us still work on laptops and having multiple virtual desktops can help group different work into different workspaces.

I recommend Sysinternals' Desktops, but there are more tools out there that do the same thing - though they are not all free.

4. Launchy
It indexes all of your applications in the start menu, and you can simply launch them by typing their name - or part of their name. No longer do you have to wade through dozens and dozens of menu items just to find a shortcut. Simply launch Launchy with ALT+SPACE and type a couple of letters. Voila!

5. PuTTY
If you work with Windows and UNIX/Linux machines, then it's pretty much impossible that you haven't heard of PuTTY. It's a free SSH client that no only provides SSH/Telnet functionality, but also comes with other SSH-related utilities like PSCP, PSFTP and PLINK (see previous post on this).

I'd love it if Windows would ship with a command-line SSH client, just like all UNIX and Linux distributions do.

Of course there is more, but these are the tools "desktop" that we really use on a daily basis.

If your computers are in an Active Directory domain and you want to roll out some of these tools with your computers by default, then I recommend reading our previous post: Your favorite tools and utilities always available everywhere.

What do you want for nothing?

Rubber Biscuit?

Firefox .NET Framework Assistant Paranoia

Fri, 05 Jun 2009 10:09:58 -0600

There has been a lot of concern and uproar recently about the .NET Framework Assistant Firefox Add-On (plug-in), that Microsoft silently installs with the Microsoft .NET Framework 3.5 Service Pack 1 (which was pushed in early 2009 with Windows Update). As such, if you are using Firefox, then there this is a very high probability that you have this Firefox Add-On installed, maybe even without knowing it.

To quote Microsoft: "In the .NET Framework 3.5 SP1, the .NET Framework Assistant enables Firefox to use the ClickOnce technology that is included in the .NET Framework."

There are dozens of blogs that complain about the security implications, how the Add-On cannot be uninstalled and eventually post instructions on how to remove the Add-On from your computer, essentially implying that the AddOn harbors major security risks. Contrary to most Firefox Add-Ons, this one can't be uninstalled through the browser since it was installed at the "computer level". As such, you have to remove files from the file system and modify the Firefox configuration to disable it.

I'd have to admit that I haven't heard much about the ClickOnce technology before this sneaky little AddOn was set free, and the buzz words one reads in all the blogs, newspapers etc. certainly have the potential to make one uneasy and follow the surgical removal procedure without much hesitation:

Microsoft installs .NET AddOn without user approval!
AddOn can't be uninstalled
AddOn silently runs .NET applications without user knowledge!
ActiveX security hell is back!

So is the AddOn a security risk and do you have scramble to rip it out? Not in my opinion, and I will explain why.

In this post I will clear up some misconceptions about the ClickOnce technology, but also show you how to remove the AddOn from any number of computers with a few clicks - using our new AutoAdministrator 2.0 - just in case you do want to rip it out :-).

What most people don't know, is that the ClickOnce "technology" is already present in Internet Explorer, and is not even close to what was/is possible with ActiveX applets.

ClickOnce applications run in a sandbox, similar to Java, and - by default - do not have any permission outside the sandbox. As such, a web site can't just install a trojan horse or spam client on your computer - at least not using ClickOnce. The users permission is asked before elevated permissions are assigned to the application, and software that's being installed can be signed - just like Windows applications are. Please see the Microsoft article below for more information on ClickOnce deployment and security:

ClickOnce Deployment and Security

So the AddOn is really just a gateway into something that is already on your system in the first place - .NET. Java does the same thing, and the AddOn Microsoft provides is likely much leaner than the Java plugins - and doesn't register a new plugin with every new Java update that is released.

Don't get me wrong - Microsoft could have handled this much better, and the inability to uninstall the AddOn really doesn't help their case.

Oh, and by the way, to see a sample ClickOnce application then you can click here. It's hosted by the author of the FFClickOnce Firefox AddOn, a predecessor of the .NET Framework Assistant if you will.

However, Microsoft has recently provided information on their site that outlines the required steps to remove the Add-In from Firefox, and has also released an update that will allow you to uninstall it on a per-user basis. Keep in mind that even with this update, every user would have to uninstall the Add-On manually:

Update to .NET Framework 3.5 SP1 for the .NET Framework Assistant 1.0 for Firefox

Having said all that, you might still want or have to remove the AddOn from multiple computers if you need to remove the ability for your users to run ClickOnce applications from Firefox. The good news is that you can remove all files as well as all registry entries that are associated with this Add-On from any number of computers within a matter of minutes -- using AutoAdministrator.

AutoAdministrator integrates with ActiveDirectory, and lets you query/modify files, services, registry entries and more on any number of computers with the click of a few buttons. Read on to find out more.

Microsoft states that you need to perform three steps to remove the Add-On (official removal instructions - KB963707):

1. Delete the registry key HKEY_LOCAL_MACHINE\SOFTWARE\Mozilla\Firefox\Extensions\{20a82645-c095-46ed-80e3-08825760534b}

2. In the Firefox preferences (about:config), right-click the general.useragent.extra.microsoftdotnet property and select "reset".

3. Delete the folder %SYSTEMDRIVE%\Windows\Microsoft.NET\Framework\v3.5\Windows Presentation Foundation\DotNetAssistantExtension\DotNetAssistantExtension.

We can accomplish (1) and (3) with AutoAdministrator, which does remove the Add-On. It doesn't reset the setting inside Firefox (2), but that should be merely a formality without the actual plug in. Our tests have shown that the plug in is gone after deleting the registry key and the directory on the file system.

There are two prerequisites for this to work: Your remote machines need to have the remote registry service running (you can temporary toggle that too with AutoAdministrator if it's not running!) and the ADMIN$ share needs to exist.

As with all things you can do with AutoAdministrator, you should be very careful. We cannot take any responsibilities if you end up corrupting your Firefox installations, or worse, the Windows OS.

So, fire up AutoAdministrator and select the computers you want to uninstall the pesky Add-On from in the right pane. Then, select "Registry" from the toolbar and paste the key from step one in there and select "Delete key".

The screen shot above shows the result list, using the "Read Value" option. To actually delete the key, you would need to select "Delete key". Machines that are turned off are displayed as "Ping Failure: ...", and machines that don't have the Add-On installed show a Windows API error message.

When you are doing ripping the registry settings out, you can delete the folder as well. This time, select "File Management" from the toolbar, and paste the directory in there. Note that the remote path should start with ADMIN$, as shown in the screen shot below:

You can also save these settings as a preset, so that you can retrieve these settings at any point in the future with the click of a button.

I hope this information helps you make an informed decision as to how to proceed with the AddOn if it's already installed in your network. You can

Leave it
Give your users instructions on how to disable it
Roll-out the Microsoft patch to give your users the ability to uninstall it ( arguably identical to (2) )
Remove it from all systems with AutoAdministrator or scripts

I think if this exercise reveals anything, then it's that Firefox's AddOn framework leaves some room for improvement. For example, why did Firefox not inform me that this AddOn had been installed? Skype also silently installs an AddOn, though that can be removed easily.

And if you're really serious about browser security, then you might want to check out the Flashblock AddOn. It disables all flash animations by default, leaving placeholders that you can click to load any flash animation. This improves page load times, can help suppress annoying flash-based ads and of course helps security. I haven't tested it on many sites yet, but it can quickly get annoying if you're accessing a lot of web sites that contain reporting widgets that are flash-based.

So long,
Ingmar.

Auditing Changes to Microsoft SQL Server Database Tables

Fri, 15 May 2009 12:11:21 -0600

Database servers store massive amounts of data, often including sensitive information. It is not uncommon for there to be databases holding millions of rows of data, where a small subset of rows are considered critical or sensitive. This could be anything from a Social Security number to an EventSentry entry of a security event. Being notified when existing data in your database changes is crucial for log data, and can be accomplished by using triggers with Microsoft SQL Server.

For those of you not familiar with triggers, a database trigger executes code in response to events on a table or database. Triggers are essentially hooks into a table, and they usually execute SQL statements as a response to another SQL statement.

Since we love the windows event log, we'll take advantage of SQL Server's ability for triggers to log an event to the event log when a row in a table is modified. This allows us to not only log that activity, but also get notified immediately when suspicious or important activity occurs in the EventSentry database.

In EventSentry, we have a table named ESEventlogMain that stores Windows event information. This table constantly gets new data inserted into it, and it often gets purged as well to manage the size of the database. However, there is no reason this data should ever be modified. If it is, then we know that something is amiss and we want to trigger an event in the event log. It is also useful to know what account made that change.

The first step is to create the message in SQL. You can use this SQL statement to create it:

sp_addmessage 80000, 10, 'Data Integrity Alert: %s', @with_log = TRUE

The first argument is a unique SQL server message ID that should be 50001 or higher, you can delete it again using sp_dropmessage. The number 10 is the severity level, but you can read more about the different options for sp_addmessage here.

Now we create the trigger that will use this message:

CREATE TRIGGER Trigger_ESEventlogMain_Modified ON
ESEventlogMain
FOR UPDATE
AS

IF UPDATE(eventmessage) OR UPDATE(eventid) OR UPDATE(eventtime) OR UPDATE(eventcomputer)
BEGIN

     DECLARE @Msg VARCHAR(8000)
     DECLARE @EventNumber INT
     DECLARE @EventID INT
     DECLARE @Computer VARCHAR(255)
     DECLARE @EventMessageOld VARCHAR(8000)
     DECLARE @EventMessageNew VARCHAR(8000)

     SET @EventNumber = (SELECT eventnumber from deleted)
     SET @EventID = (SELECT eventid from deleted)
     SET @Computer = (SELECT A.eventcomputer from ESEventlogComputer as A, deleted as B WHERE A.id = B.eventcomputer)
     SET @EventMessageOld = (SELECT eventmessage from deleted)
     SET @EventMessageNew = (SELECT eventmessage from inserted)

     SET @Msg = 'ESEventlogMain modified by ' + CONVERT(VARCHAR(20), USER_NAME(USER_ID())) + ' at ' + CONVERT(VARCHAR(20), GETDATE()) + '. Computer: ' + @Computer + ', Event ID: ' + CONVERT(VARCHAR(8), @EventID) + ', Event Number: ' + CONVERT(VARCHAR(16), @EventNumber) + ', EventMessage (old) =' + @EventMessageOld + ', EventMessage (new) = ' + @EventMessageNew

     RAISERROR( 80000, 10, 1, @Msg)
END

This creates a trigger which will generate an event when the eventmessage column in the ESEventlogMain table is modified. You can remove the "IF UPDATE(eventmessage) ..." clause (as well as the BEGIN & END statements) if you want to be notified of any changes to that table, this might however create some noise since acknowledging events will also perform an UPDATE on this table.

FYI: "deleted" and "inserted" are keywords that refer to either the old record that was updated (=deleted) or the new data (=inserted).

As you can see from the screen shot above, the message text from a logoff event was renamed to "Trigger Test". So now that the event is in the event log, we can set up a filter in EventSentry to alert us:

Events generated from triggers always have the event id 17061, so it's a good idea to restrict the filter further using the "Content Filter" field. From now on, when the ESEventlogMain table is modified, we will get an entry in the event log as well as an email.

Just remember that any database administrator can delete or modify triggers, so it's crucial that you keep dba access to your database as restricted as possible.

Please see the Table Relationships topic in the EventSentry help file for more information on the database tables used by EventSentry.

Best,
Tames, Ingmar + Ryan.

Running Linux applications on Windows - over the network with Xming

Sun, 19 Apr 2009 08:19:10 -0600

I always find it interesting to see clothes and accessories that were in fashion 30 years ago, make it back into the mainstream. It seems like the computer industry also goes in cycles every now and then.

Back in the early days of computing – before the dawn of the glorious PC era – there were few powerful servers that were accessed by dumb terminals. The emergence of the IBM PC changed all that and eventually led to the rich clients that most of us have under our desks today. The traditional PC desktop however causes quite a bit of management overhead – especially in large organizations – which appears to be leading to the re-emergence of “dumb” terminals that access a powerful – well – terminal server. Only this time we have a fancy user interface.

If you have worked with Unix-like operating systems before, then you’re probably familiar with the X windows system, though most people don’t know about the X Windows system’s (from now on referenced to as X11) network transparency. In essence, you can run an application on host A, but actually display and interact with the application on host B. Furthermore, you can actually utilize X11 to remotely log into a host running X11 without the need to install additional software on that host – provided that X11 is configured to support this. The screenshot below shows this a bit better.So what does this mean in practice? You can install a resource-hungry application on a dedicated and powerful Linux host, yet run and execute the application on a different, less powerful Linux machine – even if that machine is not even running Linux. What’s even better is that those remote applications appear just like any other application on your desktop. Citrix calls this “application publishing”, and Microsoft introduced “TS RemoteApp” with the Windows Server 2008 platform. Yet, X Windows has offered this functionality for decades - from the very start.

But what makes this feature really interesting for us windows admins (or Unix admins that, for whatever reason, have to use a Windows workstation), is the fact that you can install an X server on your windows machine and run Linux applications “natively” on it – thanks to the open-source project Xming).

Xming, according to the project web site, is the “leading free unlimited X Window Server for Microsoft Windows® (XP/2003/Vista)”. There have been security concerns in the past when using X11 remotely, but by tunneling X11 traffic through SSH, Xming is actually quite secure and doesn’t usually require any configuration changes on the host running X11 (phew!).

When tasked with either cross-platform system administration or development, the discovery of Xming opens up a door of possibilities. For example, you can edit remote configuration files conveniently by running your favorite Linux editor on your Windows desktop, or run a terminal like gnome-terminal. Why run a terminal through X-Windows when you can just use an SSH app like PuTTY? For one thing, you can launch GUI applications directly from the terminal (e.g. ‘gedit &’) on your Windows desktop. Of course, you can also play a Linux game on Windows that way.

If you’re a cross-platform developer, then you can execute a Linux/Unix development studio (e.g. eclipse) on your Windows box – and it appears just like any other Windows app. And since it’s technically running on the Linux box, compiling on your Windows app really compiles it on the remote platform (e.g. Linux). The responsiveness of applications is also quite good, at least over an Ethernet connection.

This technique also works for multiple end users, so it’s also possible to connect to one Linux machine from multiple Windows machines and run Linux apps. The Linux machine really acts like a terminal server in this case.

Let’s look at how to run a Linux app on a Windows desktop. I used Ubuntu 8.10 and installed Xming on a Vista laptop. So, download & install the following Xming packages from http://sourceforge.net/project/showfiles.php?group_id=156984:

Xming
Xming-fonts

Then, start XLaunch from the start menu and select the following options:

Multiple Windows
Start a program
Start program: Enter the application you want to launch there. E.g. gnome-terminal, gedit, mahjongg or whichever remote application you want to run “locally”
Run remote – using PuTTY: Select this option and specify the computer name, user name and password.
On the next step, simply leave the default options in place, click “Next” and “Finish”.

You should now have a little X icon on the tray, and the application you selected should be running on your desktop. The screenshot below shows gnome-terminal and gnome-text-editor running on my Vista machine.

Xming uses plink.exe (see also: http://www.eventlogblog.com/blog/2007/12/plink-or-issuing-ssh-command-o.html) internally to execute apps, whose display is then redirected to our local Windows client, on the remote host. You can also save these settings in a configuration file and create a shortcut on your desktop or start menu.

If the XDMCP protocol is enabled on the Linux/Unix host (disabled by default on most distributions for security reasons), then you can log into the remote host for a complete remote session similar to VNC or other remote desktop applications. But again, keep in mind that XDMCP transmits data in clear text over the wire (using both TCP and UDP), and as such is an insecure protocol that should only be enabled in trusted networks. To log in remotely with Xming, select the following options after starting XLaunch:

One Window
Open session via XDMCP
Specify the remote host name

One last tip regarding Xming: If, at some point down the line, you are unable to launch remote apps on your desktop, even though the X tray icon from Xming is present, then try to reset the X server by right-clicking the tray icon and choosing "Exit".

Well, I hope this gives you a starting point and helps ease the pain when maintaining heterogeneous network environments.

Until next time,
Ingmar.