Jon Bernard

A Specific Module

2013-09-15T00:00:00-04:00

I'm sure you're chomping at the bit to compile kernel modules. Certainly the simplest way is to build a kernel from source, which will compile all of the requested modules in the process.

But what if you don't want to build the entire kernel? Instead you'd like to just build a single module in the source tree without building the whole damn thing. Why would you do this? Perhaps your vendor did not include a needed module in their stock kernel. Or perhaps you intend to send a patch to a linux kernel driver and you would like to test it before submission. Or perhaps you have an ancient laptop that can barely handle browsing the web without needing more RAM and making you scream and throw things all while trying to not draw attention to yourself in the coffee shop.

Whatever the reason, here's how to do it:

First, configure the kernel to build everything as a module:

make allmodconfig

There are several ways to configure the linux kernel, but if you only intend to build a single module then this is probably what you want. You may already have a .config in your source tree and so you can skip this step. If not, you certainly will after you run the above command.

Then you must prepare the build environment:

make modules_prepare

Once that command completes, you're ready to build a module! The command is simple, you need only know the subdirectory that holds the module you want to build. For instance, say I would like to build the newly added fwserial module (TTY over FireWire). The source for the module lives in drivers/staging/fwserial. So the appropriate command to build only that module would be:

make M=drivers/staging/fwserial

You can replace my hypothetical path to fwserial with whatever you want. After compilation completes, you could find this module at drivers/staging/fwserial/fwserial.ko.

Decoding Hexidecimal Strings

2013-07-16T00:00:00-04:00

I was looking through job postings today and I came across a LoveWithFood posting that really got my attention. Not the posting itself so much (although the job looks really cool), but because they included a small programming question inside the post. It was a hexadecimal-encoded string that contained a message that should be decoded and included in the job submission. The encoded string is this:

e299a5205765204d616b652054756d6d792048617070792120e299a5

I don't know about you, but I love secret messages. I just couldn't help myself.

In python, this is really easy because it has encode() and decode() that both take a format as a parameter. So the quick solution is this:

>>> mystring = "e299a5205765204d616b652054756d6d792048617070792120e299a5"
>>> print mystring.decode('hex')

But that's really boring. And decode() gets to do all of the work, so I set out to implement the decode operation on my own (in python), and I made some interesting discoveries along the way.

Character Encoding

Let me first explain what I've learned about character encoding. Well, there is too much to explain, so let me sum up. You're certainly familiar with ASCII, which is a 7-bit encoding scheme developed by the U.S. to represent character symbols. This means only 128 unique characters can be represented and this is an obvious problem for languages that don't share our relatively small alphabet.

So unicode was created to solve this problem. Unicode is a standard, but not an implementation. Utf-8 is a unicode implementation that uses 32 bits to represent character symbols (some of the higher bits aren't used but that's beyond this post). Utf-8 will use fewer bytes if it can, so not all characters encode to a 32 bit value. One neat fact is that utf-8 uses the same byte value for ASCII characters which has the same value as ASCII encoding, so the character values are the same.

Let's take a look at a few examples:

>>> 'a'.encode('hex')
'61'

>>> 'é'.encode('hex')
'c3a9'

>>> '…'.encode('hex')
'e280a6'

I just glossed over a huge amount of information, but bear with me. The basic gist is this: given a long hexadecimal-encoded string, decoding it means breaking it apart into individual bytes and decoding each byte by itself. In some cases, decoding a particular byte may require the value of the previous byte. Since a byte is represented as two hexadecimal digits, we need to separate the string into pairings and decode each pairing.

Decoding without decode()

In the encoded string we are given, each pair of two characters contains the encoding for a single printable character, so the goal is to separate this large array into pairings of two that we can individually decoded and then combine to see the final result.

Python's :: array syntax

Python provides a really powerful array index notation that allows you to quickly slice and dice an array with very little effort. One of these is the :: operator which allows you to take each nth entry starting at some starting point. So, for example, if you wanted to pull out all of the even-indexed elements in an array called A you would say A[0::2]. That is, starting at index 0, take every second element. You can get the odd-indexed elements by changing the start index to 1 like so: A[1::2].

This allows us to separate the large array A into two smaller arrays B and C with the following propery: at each index i from 0 to n, B[i] and C[i] contain the two characters we want to pair together. If only there was a way to zip the two arrays together...

The zip() function

Of course python has a zip() function, which does exactly what we want. Here's the documented definition:

Return a list of tuples, where each tuple contains the i-th element from each of the argument sequences. The returned list is truncated in length to the length of the shortest argument sequence.

So acquiring a set of tuples that we can then decode individually is as simple as:

>>> [''.join(c) for c in zip(INPUT[0::2], INPUT[1::2])]
['e2', '99', 'a5', '20', '57', '65', '20', '4d', '61', '6b', '65', '20', '54', '75', '6d', '6d', '79', '20', '48', '61', '70', '70', '79', '21', '20', 'e2', '99', 'a5']

The above is a list comprehension and is one of the many features that make Python a joy to work with. If you're not familiar with list comprehensions, I highly encourage you to learn more about them.

Now that we can take the string apart into pieces, it's easy to convert each byte to decimal:

>>> [int(''.join(c), 16) for c in zip(INPUT[0::2], INPUT[1::2])]
[226, 153, 165, 32, 87, 101, 32, 77, 97, 107, 101, 32, 84, 117, 109, 109, 121, 32, 72, 97, 112, 112, 121, 33, 32, 226, 153, 165]

And now we can interpret each decimal value as a character:

>>> [chr(int(''.join(c), 16)) for c in zip(foo[0::2], foo[1::2])]
['\xe2', '\x99', '\xa5', ' ', 'W', 'e', ' ', 'M', 'a', 'k', 'e', ' ', 'T', 'u', 'm', 'm', 'y', ' ', 'H', 'a', 'p', 'p', 'y', '!', ' ', '\xe2', '\x99', '\xa5']

And now the message is starting to show. It appears there are unicode characters at the beginning and end of the string. The print function knows how to correctly interpret those sequences, so decoding the string is now:

>>> print ''.join([chr(int(''.join(c), 16)) for c in zip(INPUT[0::2], INPUT[1::2])])
♥ We Make Tummy Happy! ♥

So there you have it. Decoding secret messages is fun, but I think the really cool part is the use of the :: array index operator, the zip function, and python's list comprehensions.

What about Ruby?

The job requested the answer in Ruby, and it turns out the answer is not so different from what we've just done in Python. Here is a Ruby solution:

>> INPUT = "e299a5205765204d616b652054756d6d792048617070792120e299a5"
=> "e299a5205765204d616b652054756d6d792048617070792120e299a5"

>> INPUT.scan(/../)
=> ["e2", "99", "a5", "20", "57", "65", "20", "4d", "61", "6b", "65", "20", "54", "75", "6d", "6d", "79", "20", "48", "61", "70", "70", "79", "21", "20", "e2", "99", "a5"]

>> INPUT.scan(/../).map(&:hex)
=> [226, 153, 165, 32, 87, 101, 32, 77, 97, 107, 101, 32, 84, 117, 109, 109, 121, 32, 72, 97, 112, 112, 121, 33, 32, 226, 153, 165]

>> INPUT.scan(/../).map(&:hex).map(&:chr)
=> ["\xE2", "\x99", "\xA5", " ", "W", "e", " ", "M", "a", "k", "e", " ", "T", "u", "m", "m", "y", " ", "H", "a", "p", "p", "y", "!", " ", "\xE2", "\x99", "\xA5"]

>> INPUT.scan(/../).map(&:hex).map(&:chr).join
=> "\xE2\x99\xA5 We Make Tummy Happy! \xE2\x99\xA5"

>> print INPUT.scan(/../).map(&:hex).map(&:chr).join
♥ We Make Tummy Happy! ♥

In the absense of decode(), I personally find the Ruby solution to be more concise.

The GPL and Derived Works

2012-12-10T00:00:00-05:00

For the longest time I thought I had a decent understanding of what constitutes a derived work of source code released under the GPL. I work with GPL code every day, so it's a concept that I generally take for granted. My personal view is that any influence from GPL source, whether directly copying it into your text editor or reading it carefully and then reproducing it in some form is considered a derived work (for my definition of "derived").

I realize my position is a bit extreme, but I thought this was the general consensus. For any creation, I feel all influences and sources should be duly credited. If properly crediting a source means releasing your creation under the GPL, then it's the right thing to do. If this conclusion is unacceptable, then the problem must be solved independently without the use of information contained in relevant GPL solutions. Those are the two conclusions that I see for how to properly conduct myself as a software creator.

But this view is not entirely accurate.

It turns out, what constitutes a derived work is much more vague than I imagined. Having been an employee of companies that create and sell proprietary software, this same issue comes up every now and then: "I know I cannot just copy the code, but what if I read it and think about it, and then create a solution that's similar?" The answer to this question seems obvious to me, but as I've learned recently, it's actually a very difficult question to answer.

The GPL does not define exactly what a derived work is, which leads to the ambiguity in question. So I contacted the Free Software Foundation and got the following response:

In fact, the issue has nothing to do with the GPL per-se, but about copyright law in general. For example, If I read an interesting book and am inspired to write one myself, should that new book be considered a derivative? Ultimately, this is something that only a court of law can decide. There are methods to help reduce the risk of legal trouble when writing software based on foreknowledge of some other code, but those are context dependent.
Yoni Rabkin, Free Software Foundation

As much as I hate it, the answer seems to be "it depends on context", rather than black and white. I don't intend to change the way I conduct myself personally, but it is nice to have a better understanding of the issue.

Logging to the Serial Console

2012-07-19T00:00:00-04:00

There are several ways to capture kernel log output on a remote machine. One way is to configure the kernel to use a serial port as console. If you have a machine that has a serial port, or you're running a VM, this method can be very helpful in capturing kernel log data when a panic occurs.

On Debian (or Ubuntu) this is really easy to setup. Simply edit /etc/default/grub and change the line:

GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"

to:

GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8 debug ignore_loglevel"

and then execute:

update-grub

and reboot. Let's examine this a bit closer:

console=tty0 instructs the kernel to continue using the standard console device so you'll still see the usual boot messages on an attached monitor.

console=ttyS0, instructs the kernel to also use the first serial device as a console. If you have multiple serial ports, you can change the number to match the device you wish to use.

115200n8 sets the serial speed to 115200 bits per second with no parity and 8 data bits per character. The other end of the serial line must be configured to match these settings.

debug ignore_loglevel allows you to see ALL kernel messages on the configured console(s). If you're trying to debug your kernel, you will most certainly want this.

If your machine is running in a VM, the management utility likely has a configuration that will allow you to write the console data to a file on the host machine. If you intend to use a real serial port and cable, you can use any serial communication program. I use minicom, the command I use is:

minicom -D /dev/ttyUSB0 -b 115200n8 -m -o -w

I use a serial-to-usb converter as my laptop does not have a proper serial port. This device communicates on /dev/ttyUSB0. See the manpage for minicom for additional options.

Scrolling Log Output

2012-06-10T00:00:00-04:00

Related to the earlier article on kernel development, here's one thing that I find very useful. During development I often compile, load, unload, modify, and recompile in a loop. I rarely do this on my development machine but rather in a VM or on a nearby test machine. It's a pain to keep typing dmesg all the time. So I keep a separate terminal open with dmesg running in a loop. Actually it's a split pain within Tmux, but that's not important yet. Here's what I do.

while true; do sudo dmesg -c | tee -a /tmp/dmesg.log; sleep 1; done

With this running in it's own terminal, you can now immediately see the effect of a module load or unload without having to go through the tedium of typing dmesg and figuring out what's been added since the last invocation. I find this quite helpful.

Also, tee is a really useful tool, read up on it if you haven't already.

Linux Kernel Development

2012-05-29T00:00:00-04:00

My intent with this, and related articles to come, is to explain how I develop software that runs as part of the Linux kernel. Not so much the way my code works, but rather the tools, configuration, and techniques that allow me to create kernel code quickly (within reason).

My hope is that you can use some of these ideas to bring you closer to doing actual work, and spend less time monkeying around with your environment. Or at least this can serve as a reference so I don't keep reinventing every wheel I come across.

Techniques like testing your module within UML (Usermode Linux) so you don't destroy your machine when things go wrong, or how to use objdump and some basic algebra to determine exactly where an oops occurred in a given piece of code. Perhaps we could cover common bit-shift operations and how they're used to save computational resources. These and other basic debugging tools can help you navigate the world of kernel development more quickly and increase productivity immensely.

This exercise will walk you through creating a simple module, compiling it, loading it into your running kernel, and verifying that it has been loaded. This is the most basic form of the Modify-Compile-Test loop that allows you to begin to iterate over a piece of code. Also, I have to get this intro stuff out of the way so I can later talk about things that build upon it.

I assume a good general understanding of the Linux ecosystem as well a proficiency in C and other related tools like make, gdb, and vim (or something equivalent).

Your First Module

Create a directory to hold your kernel source

$ mkdir ~/my-module
$ cd ~/my-module

Let's start with a boilerplate makefile. Put the following in your Makefile

ifneq ($(KERNELRELEASE),)

        obj-m += my-module.o

else

        KERNELDIR ?= /lib/modules/$(shell uname -r)/build
        PWD := $(shell pwd)

all: my-module

%.o: %.c
        @echo "  CC      $<"
        @$(CC) -D_GNU_SOURCE -ggdb3 -Wall -Wextra -I. -c $< -o $@

my-module:
        $(MAKE) -C $(KERNELDIR) M=$(PWD) modules

tags: my-module.c
        @echo "  GEN     $@"
        @ctags -R

clean:
        $(MAKE) -C $(KERNELDIR) M=$(PWD) clean
        rm -f *.o tags

endif

Make sure to preserve the indentation using tabs or make will complain.

The above makefile gives you three targets. An all target (just typing make will launch this) that builds your module. A tags target that will generate a ctags file of all the source code contained in your directory. Vim will pick this up automatically and allow you to jump to tags from within your editor. This is a must-have feature. And a clean target that will remove any auto-generated files.

Another nice thing about this makefile is that it can be referenced within the kernel tree itself, or it can be used to build an out-of-tree module. For this exercise, we will build an out-of-tree module, meaning we're not building our code as part of a larger kernel compile. This code will be built against the headers for the running kernel and produce just a kernel module that can be inserted into the running kernel.

We create a simple test module in my-module.c

#include <linux/module.h>

#define AUTHOR      "My Name <me@some-host.com>"
#define LICENSE     "GPL"

static int __init my_module_init(void)
{
        printk(KERN_INFO "ZOMG I'M ALIVE ! ! !\n");
        return 0;
}

static void __exit my_module_exit(void)
{
        printk(KERN_INFO "We've been hit captain, abandon ship!\n");
}

module_init(my_module_init);
module_exit(my_module_exit);

MODULE_AUTHOR(AUTHOR);
MODULE_LICENSE(LICENSE);

You'll likely need to fix the single and double quotes if you're pasting into your editor from the browser.

Building a kernel module from this should be as simple as

$ make

If this fails, make sure you have the headers for the currently running kernel installed on your system.

Upon successful compilation, load the resulting module into your kernel

$ sudo insmod ./my-module.ko

If the prompt returns with no error message, you're good to go. The kernel logs informational messages to a ring buffer that can be accessed with dmesg. You should see your init string contained in the output of that command.

The module is removed from the kernel with

$ sudo rmmod my-module

The output of dmesg should now contain your exit string.

It's important to note that rmmod does not guarantee you return to the state before the module was loaded. The code you compiled and loaded is executed within the context of the kernel and not within any sort of sandboxed environment. This means that any public data within the kernel could have been modified. It is not always the case that a module cleans up after itself properly, so although rmmod succeeded, you may still require a reboot to return to a sane running state.

The above is essentially what you'll read an any Linux development book. Nothing special, but it's the most basic foundation so I feel it's worth covering. Everything builds on this first step.

Get More Out of Vim's makeprg

2011-09-30T00:00:00-04:00

This started out as a small post on how to preview markdown documents in Vim using makeprg, but somehow I got carried away. More on the markdown thing in a bit.

When you type :make in Vim, the value of makeprg is executed in a subprocess. Traditionally this value is set to make, but you can set this to anything you like. You can see the current value by typing :set makeprg. When you combine this with Vim's support for filetype plugins, there's a decent amount of value that I think is sometimes overlooked. So this article is an attempt to explain some of the things you can do with makeprg and filetype detection.

Filetype Plugins

A filetype plugin is like a global plugin, except that it sets options and defines mappings for the current buffer only.

In order to make use of filetype plugins, vim must be told to enable this feature. This is enabled by default on many installations, but some (perhaps MacOS X) require the user to enable it explicitly. Place this line in your vimrc:

filetype plugin on

Using a filetype plugin, we can set makeprg to something useful depending on the type of a file it happens to be. Adding your own filetype plugin is easy, just create a file of the form: <filetype>.vim inside the ~/.vim/after/ftplugin directory. The after subdirectory is technically optional, but I prefer it as it keeps things isolated from other installed plugins. The after subdirectory is sourced last as Vim initializes itself, so putting your options there ensures they aren't clobbered by other plugins.

For a given Vim buffer, you can see the filetype by typing :set ft. Some people put this information in their statusline, which can be helpful if you're writing Vim plugins or just want to make sure things are being detected correctly.

For example, if I want to define custom settings for the C filetype, I would create ~/.vim/after/ftplugin/c.vim and put my settings there. Any time a file of type C is loaded, my settings are applied. We can use this mechanism to set a custom makeprg for certain filetypes.

Customizing makeprg for Markdown

I take all of my notes in Markdown, which is nearly plain text with a tiny amount of markup. This allows me to quickly convert my personal notes into decent looking html documentation. It seems to be a regular occurrence that my notes become the initial documentation for the project or subproject that I'm working on. Many wiki's support Markdown, so it ends up saving me work by just keeping everything in that format.

Even though Markdown's syntax is very lean and simple, I still end up forgetting some of it from time to time. I find it useful to generate and view the HTML output for the buffer that I'm working on periodically to make sure things look right. This is really easy by setting a custom makeprg for the markdown filetype.

The Hammer Plugin

Robert Gleeson wrote a Vim plugin called Hammer that basically does exactly what I explain here and quite a bit more. It might be perfect for you, but for me it's much more than I need. I've had problems when the version of Ruby on the system is not the same as what Vim was compiled with. Check it out though, you might love it.

A Simpler Approach

All I want is to type :make and have Vim generate an HTML version in /tmp that I can point my browser to. So I put the following in ~/.vim/after/ftplugin/markdown.vim:

set makeprg=redcarpet\ %\ >/tmp/%<.html

Let's take a quick look at what's going on here:

The above setting is applied only when editing a markdown buffer. Personally, I use redcarpet to parse markdown, but there are many others that you can substitute here.
Spaces in option strings must be escaped so Vim knows when to stop parsing.
% expands to the name of the file currently being edited.
%< expands to the name of the file without the trailing extension. This allows us to change the name of the file and add the html extension.

So the above command calls redcarpet on the filename and redirects the output to /tmp/<filename>.html. That's it, super simple. Once this completes, I point my browser to file:/tmp and load the file I'm interested in.

This same approach can be taken with ReStructured Text by putting the following in ~/.vim/after/ftplugin/rst.vim:

set makeprg=rst2html.py\ %\ /tmp/%<.html`

Extra Credit

For C, we could set makeprg to call gcc directly instead of make if a Makefile is not found in the current working directory. I find this very helpful when creating a scratch source file to test certain behaviour or a quick theory. Here's what the contents of ~/.vim/after/ftplugin/c.vim might look like:

if !filereadable(expand("%:p:h")."/Makefile")
    setlocal makeprg=gcc\ -Wall\ -Wextra\ -o\ %<\ %
endif

Now when you load a file of type C and there is no Makefile, typing :make will compile your source file into an executable of the same name (minus the extension). I use this all the time. You could do the same thing for C++ by changing the filename to cpp.vim and changing gcc to g++.

Future Work

It would be cool to copy the target filename to the clipboard for easy pasting into the browser's URL bar.

Update

I handle this with the following
setlocal makeprg=markdown\ %\ >%<.html

" Open corresponding html file
nmap <Leader>v :!google-chrome %<.html<CR><CR>
which may be replaced by the browser and file position of your preference.
Bijan Chokoufe

New GPG Key

2010-07-20T00:00:00-04:00

I've recently set up a new GPG key, and will be transitioning away from my old one. I have done this in order to migrate to a larger RSA key and stronger hash functions, and NOT due to any known key compromise. The old key will continue to be valid for some time, but future correspondence should use the new one wherever possible.

I have created a transition document and signed it with both keys.

Don't Clobber Your Registers

2010-07-02T00:00:00-04:00

I recently needed to call the cpuid instruction from some C++ code. This is well documented on wikipedia and everything went fine until I hit a snag moving the code from x86_64 to i386.

It seems that %ebx is the PIC register for i386, and so you must take care to not clobber it when calling cpuid. This is accompolished by explictly saving and restoring the register before and after the call to cpuid. An additional step is required to tell gcc that %ebx will not be clobbered by the call to cpuid. The last lines of assembly define which registers were potentially clobbered so gcc can generate appropriate code. By manually saving and restoring %ebx, it can be saftely omitted from that list. I did not find any information documenting this specifically, so here it is! The following code returns the result of cpuid and runs on both x86_64 and i386.

void
cpuid(int regs[4], int cpuid_leaf)
{
        int eax, ebx, ecx, edx;
        asm volatile (
#if defined(i386)
             "pushl %%ebx;\n\t"
#endif
             "movl %4, %%eax;\n\t"
             "cpuid;\n\t"
             "movl %%eax, %0;\n\t"
             "movl %%ebx, %1;\n\t"
             "movl %%ecx, %2;\n\t"
             "movl %%edx, %3;\n\t"
#if defined(i386)
             "popl %%ebx;\n\t"
#endif
             :"=m" (eax), "=m" (ebx), "=m" (ecx), "=m" (edx)
             :"r" (cpuid_leaf)
             :"%eax",
#if !defined(i386)
             "%ebx",
#endif
             "%ecx", "%edx");

    <span class="n">regs</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">=</span> <span class="n">eax</span><span class="p">;</span>
    <span class="n">regs</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">=</span> <span class="n">ebx</span><span class="p">;</span>
    <span class="n">regs</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span> <span class="o">=</span> <span class="n">ecx</span><span class="p">;</span>
    <span class="n">regs</span><span class="p">[</span><span class="mi">3</span><span class="p">]</span> <span class="o">=</span> <span class="n">edx</span><span class="p">;</span>


}

For additional information, I found Sam Hocovar's post to be quite informative.

Find command in $PATH

2010-04-19T00:00:00-04:00

I make calls to external commands from Python quite often. Sometimes it is desirable to know if a command is available on the running system before making a call to subprocess.call(). Here's the code I came up with for doing that:

import os

def find_in_path(cmd):
    for path in os.environ["PATH"].split(os.pathsep):
        full_cmd = os.path.join(path, cmd)
        if os.path.exists(full_cmd) and os.access(full_cmd, os.X_OK):
            return full_cmd
    return None

If there's a better way to do this, or even a builtin python way, don't hesitate to mention it.

Periodic timers in PyGTK

2010-04-16T00:00:00-04:00

I've been writing a ton of Python code lately and I'm absolutely loving it. As a C programmer for the better part of the last 15 years, I find the syntax and ease of expression to be refreshing and fun.

I've recently been tasked with writing a GUI interface to an existing product we have at MiserWare called Granola -- a free (as in beer) version of our enterprise power management solution.

I found myself needing a periodic timer to update the window with new statistics coming in from the daemon. You can do this in regular python, but PyGTK, specifically gobject, makes it very easy. Here's an example that draws a simple window with a label displaying the current counter.

#!/usr/bin/env python

import gtk, gobject

class PerodicTimer:
    def init(self, timeout):

    <span class="c"># create a simple window with a label</span>
    <span class="bp">self</span><span class="o">.</span><span class="n">window</span> <span class="o">=</span> <span class="n">gtk</span><span class="o">.</span><span class="n">Window</span><span class="p">()</span>
    <span class="bp">self</span><span class="o">.</span><span class="n">window</span><span class="o">.</span><span class="n">connect</span><span class="p">(</span><span class="s">'destroy'</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">wid</span><span class="p">:</span> <span class="n">gtk</span><span class="o">.</span><span class="n">main_quit</span><span class="p">())</span>
    <span class="bp">self</span><span class="o">.</span><span class="n">window</span><span class="o">.</span><span class="n">connect</span><span class="p">(</span><span class="s">'delete_event'</span><span class="p">,</span> <span class="k">lambda</span> <span class="n">a1</span><span class="p">,</span><span class="n">a2</span><span class="p">:</span><span class="n">gtk</span><span class="o">.</span><span class="n">main_quit</span><span class="p">())</span>
    <span class="n">vbox</span> <span class="o">=</span> <span class="n">gtk</span><span class="o">.</span><span class="n">VBox</span><span class="p">()</span>
    <span class="bp">self</span><span class="o">.</span><span class="n">window</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">vbox</span><span class="p">)</span>
    <span class="bp">self</span><span class="o">.</span><span class="n">label</span> <span class="o">=</span> <span class="n">gtk</span><span class="o">.</span><span class="n">Label</span><span class="p">(</span><span class="s">'Periodic Timer'</span><span class="p">)</span>
    <span class="n">vbox</span><span class="o">.</span><span class="n">pack_start</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">label</span><span class="p">)</span>
    <span class="bp">self</span><span class="o">.</span><span class="n">window</span><span class="o">.</span><span class="n">show_all</span><span class="p">()</span>

    <span class="c"># register a periodic timer</span>
    <span class="bp">self</span><span class="o">.</span><span class="n">counter</span> <span class="o">=</span> <span class="mi">0</span>
    <span class="n">gobject</span><span class="o">.</span><span class="n">timeout_add_seconds</span><span class="p">(</span><span class="n">timeout</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">callback</span><span class="p">)</span>

<span class="k">def</span> <span class="nf">callback</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
    <span class="bp">self</span><span class="o">.</span><span class="n">label</span><span class="o">.</span><span class="n">set_text</span><span class="p">(</span><span class="s">'Counter: '</span> <span class="o">+</span> <span class="nb">str</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">counter</span><span class="p">))</span>
    <span class="bp">self</span><span class="o">.</span><span class="n">counter</span> <span class="o">+=</span> <span class="mi">1</span>
    <span class="k">return</span> <span class="bp">True</span>


if name == 'main':
    periodic_timer = PerodicTimer(1)
    gtk.main()

The bulk of the code is just setting up a gtk window, the timer code is only 5 lines or so. As long as callback() returns True, the timer gets re-registered for the original timeout value (1 second in this example). One caveat I found, gobject.timeout_add_seconds() seems to depend on the GTK main loop, so you cannot use it from a regular non-GTK python application.

Byobu Rackspace Cost Plugin

2010-01-23T00:00:00-05:00

If you haven't heard of Byobu you're missing out. It's an enhancement to GNU screen that, among many cool features, allows users to define their own plugins and have them displayed in the statusbar along the bottom of the terminal.

The default installation comes with a plugin that displays the approximate Amazon EC2 cost of the running instance. I wanted a plugin that worked for Rackspace servers, so I modified it and implemented the Rackspace Cloudserver pricing model to create byobu-rcs-cost. Documentation is in the README, it's pretty straightforward to install and use.

Now I can quickly see the estimated cost of each server I'm logged into, pretty cool.

Update: This plugin was merged into the official byobu release as of version 2.73. See the byobu project page for more details.

Ubuntu Resource Management

2009-10-13T00:00:00-04:00

I recently wanted to limit CPU usage on my Ubuntu Karmic workstation. Since libcgroup is included in the upcoming Karmic release, setting up cgroups on an Ubuntu server has never been easier.

The Goal

My needs were as follows:

My machine is dedicated to running an Apache Web Server and I would like for it to have priority on the CPU whenever it's needed.
I have an occasional user on my system that chews up CPU like it's going out of style and slows down my web server. In this example, we'll call this user kirkland.

The goal is to only allow kirkland CPU cycles when no other process is requesting them. Similar to a sponge, kirkland will be throttled to 0 if my apache daemon is really busy.

Getting There

Install libcgroup:

$ sudo aptitude install libcgroup cgroup-bin

Edit /etc/cgconfig.conf, define a sponge group:

group sponge {
        perm {
            task {
                uid = kirkland;
                gid = kirkland;
            }
            admin {
                uid = root;
                gid = root;
            }
        }
        cpu {
            cpu.shares = 1;
        }
    }
    mount {
        cpu = /mnt/cgroups/cpu;
        cpuacct = /mnt/cgroups/cpuacct;
}

Edit /etc/cgrules.conf, assign kirkland to the sponge group:

kirkland         cpu             sponge/
*                *               sysdefault/

Restart cgconfig and cgred services:

$ sudo service cgconfig restart
$ sudo service cgred restart

Processes created by kirkland should now automatically get moved into the sponge cgroup and will only recieve CPU cycles when no other processes are in need.

For more information about libgroup, you can visit the libcgroup project page. And for cgroups in general see the current kernel documentation.