From the Canyon Edge - :-Dustin Kirkland

Gazzang Presents: Sh*t IT Security Guys Say

2012-02-07T11:59:00.004-06:00

We had a blast at the Gazzang offices last week shooting this fun video, Sh*t IT Security Guys Say. What a great way to kick back and have a little fun on a Friday afternoon ;-)

We worked with Austin filmmaker Brandon Stephens who took some time away from work on his feature film, Enemy of the Mind, to hack on this little project. Our CEO Larry Warnock (Mr. Backdoor) called the shots and our new Marketing Director, David Tishgart (Mr. Redbull) handled the script. Also featured in the short: Ben First (Marketing, aka Mr. Ruby), Liz Britain (Marketing, aka Ms. Slashdot), Rob Balena (Sales, aka Mr. Millennium Falcon), Sergio Pena (Mr. $*&%!#), Eddie Garcia (Engineering, aka Mr. IT), and I guess I'm Mr. Wingdings ;-)

As many of my fellow hackers, I predictably cringe when I watch a movie or a tv show and the hapless IT characters attempt to interface with a computer or discuss technology. The Matrix, The Net, Swordfish, whatever, it's all painful to hear. And funny enough, our little video is no different, and this time I actually share the blame :-) Most of our one-liners make no IT sense whatsoever. And while some of the one-liners I proposed made perfect IT/Security sense, but they just didn't play well on the screen.

In any case, for my hacker/dev/IT peeps, here's my full list of one-liners I proposed for our project:

- Right, RSA 4096 is definitely the way to go - Ubuntu or Fedora? - Did you read Bruce Schneier's post today? - Wow, check Slashdot! - Open a new terminal - Emacs or Vi? - Grab my public key - apt-get dist-upgrade - Sure, I encrypt my home directory - Hang on, I'm recompiling my kernel - PC Load letter???? The f*ck does that mean? - Yeah, I need to merge those changes - We're moving from MD5 to SHA512 hashes - Of course I've rooted my Android! - Chef or Puppet? - There's an XKCD about that :-) - Users, I swear...add it to the FAQ - Buffer overflow, uh oh... - Python or Perl? Ruby!?! -- you gotta be kidding me :-( - You don't have to forward me that email. I've already seen it. You don't use email encryption :-) - Would you sign my public key? - Fire up an instance in EC2 - My kernel oops'd - TCP or UDP? - There's not enough entropy on this friggin machine! - You haven't rooted your phone? - No open access points? I see 12 running WEP. Give me a minute... Okay, I'm in. - Where's your public key? - Drop that in a pastebin - Okay, I have it. What's your fingerprint? - Java or C++? - What do you think of Unity? - OpenStack or Eucalyptus? - Check StackExchange - Shit, not another core dump...

I hope you enjoy watching it as much as we enjoyed making it!

Cheers!
:-Dustin

bootmail encryption and shutdown messages now supported

2012-02-02T09:03:00.000-06:00

I've made two pretty cool changes to the bootmail utility...

Bootmail now sends a message on both boot, and shutdown, using an upstart job. Big thanks to Clint Byrum for a bit of help on that one!
Bootmail has always sent GPG-signed email. But now, it will actually send GPG-encrypted email too! All you need to do is set the RECIPIENT_KEYID variable in /etc/bootmail/gpg.conf to your GPG key id, and bootmail will send you GPG encrypted AND signed boot and shutdown messages!

Now, perhaps you wondering why, or how one would use this...

Actually, I have all of my EC2 instances set to install and use bootmail. With this, I get an email when I start, reboot, and shutdown an instance. I find it helps me remember what instances I have have running at any one time, by keeping the email in my Inbox (I practice Inbox Zero).

Moreover, I use cr-gpg with Gmail, so that I can read GPG encrypted email and verify GPG signatures within my Gmail web interface. Check out this post for more information on how to set that up!

:-Dustin

ssh-import-id gaining some steam

2012-02-01T11:46:00.000-06:00

My Google Alerts and IRC highlights have been firing almost daily with references to ssh-import-id, a handy utility I co-authored with my buddy Scott Moser a couple of years ago.

That's quite exciting to me actually, as I find the tool really, really useful, and I wish more people knew about it. I tried in vain to contribute it to the OpenSSH project, as a complement to ssh-copy-id, but it never landed there. Oh well. There's rarely a day that goes by that I don't use it, actually. I frequently use virtual machines in public clouds; usually EC2 but not exclusively. I often want to share that machine with a colleague. Rather than sharing a password, I simply:

$ ssh-import-id edygarcia sergio-pena
INFO: Successfully authorized [edygarcia] 
INFO: Successfully authorized [sergio-pena]

And now, I just share the hostname or IP with Eddie and Sergio and they can SSH into this machine and authenticate using their SSH keypair.

Reviewing what actually happened...

ssh-import-id looped over each of the arguments on the command line, which are typically Launchpad user IDs
Fetched each user's public keys from https://launchpad.net/~/+sshkeys
Validated each key's syntax
And concatenated the results to the local ~/.ssh/authorized_keys file

The methodology is secure in that:

I know what each of my colleague's Launchpad IDs are, and that's easier to remember than their SSH fingerprints
I know that they had to authenticate with Launchpad to upload their SSH public keys
I know that the communication between my system and Launchpad was authenticated and private as it used https with a valid SSL certificate

Note that I've uploaded a couple of minor fixes to ssh-import-id in the last 2 weeks that more accurately validates the contents of the public keys retrieved from Launchpad (thanks, Soren for one of those).

You can always grab the latest version from ppa:launchpad/ssh-import-id, though perhaps I should SRU some of these changes to Lucid/Natty/Oneiric. Anyone willing to test and validate those SRUs, if I propose and upload them?

Cheers,
:-Dustin

"Harvey" the Honey Badger

2012-01-30T11:24:00.000-06:00

The feedback on eCryptfs' new mascot and logo has been just awesome :-) At the bottom of the last post, we opened a call for name suggestions.

As it turns out, my mom reads my blog from time to time, and with that post, she saw an exercise and opportunity for one of her high school classes. She tasked them with researching eCryptfs and the reasoning behind the new logo. As an extra credit assignment, they were invited to propose names for our tenacious new mascot. These are so much fun, we'll share all of them with you now!

Leading by example, Mrs. Kirkland (aka, my Mom) writes:

I think you should call him...Honey, but play on the "e" ... and the quotes actually look like claws. Hon"e"y. The "e" fits perfectly in the hand. Plus, when he is standing, he really forms an H. I am no artist but I am sure you can see the H in its body. I know you are just looking for a name... but I wanted to show you why I thought the name fit.

Thanks, Mom! The quotes around the "e" do look like claws, and it does refer back to the "e" in eCryptfs.

One of her students, Christopher Bordelon suggests:

I think the name Henry would be the perfect name for the honey badger. The name Henry refers to the noble politician Henry Clay. Henry led a defensive army when it came to the war of 1812. By naming the honey badger Henry it will set the tone of the project to have a well strengthened background. By being a member of the war hawks, Henry was always ready for a battle. He knew he would not be able to be defeated. I feel this is a great name for the honey badger because Henry Clay is a well known political leader in United States history. When people hear this name they will be drawn to it because of the historical accomplishments of Henry Clay. Henry was also known for living a very long time. By Henry living a long time this means that the project will be around for a long time too. He outlived the majority of his fellow leaders. By using his historical context the project will have a face that will never be forgotten. The project will have a mascot with a defensive output that will make customers want to bring their services there.

Thanks, Christopher. We just loved the historical references!

Another of her students, Kristin Seneca, had a different idea:

I really appreciate the new logo; it is a major upgrade to the last. It has more vibrant colors and an all around better design. It isn’t as plain or boring as the key overlapping the pie chart. With a great logo, the project should want a great name to go along with it. This is why Boris the Badger would be a perfect name for your project's new honey badger logo. Boris the Badger would be a great name for eCryptfs' new honey badger logo. This name has significance to the honey badger. A honey badger is fierce and strong, much like what the name means. According to 2000names.com, the name Boris means a battle or fighter. It was derived from the name Bogoris, meaning small. Since the honey badger is one of the smallest and fiercest warrior animals around, I believe that this name definitely suits the eCryptfs honey badger logo. He certainly looks fierce! eCryptfs is a great project and should have a really awesome name to go along with their new logo. Boris the Badger would be the best name for the honey badger because it represents the idea of a warrior or fierce battle. The honey badger is a ferocious warrior animal and will go to great lengths to defend itself, much like the project will go to great lengths to protect files and software.

But our unanimous favorite here at the Gazzang offices was from Mrs. Kirkland's student Blane Palazzo, who wrote:

Reviewing possible names for the new honey badger design, I've decided that "Harvey, the Honey Badger" sounds the best. Not only is this name appropriate because of its beginning with an "H," but the name "Harvey" also means, "battle worthy." When determining which names would be possible for the new logo's design, I kept in mind that defense was a major part of choosing the "fitting" name. Having a defensive name, while at the same time establishing trust, was very important. Not only does the name "Harvey" build trust, it also has a background that allows for an understanding that he "means business." Like a true honey badger, he "takes what's his!" Paul Harvey is famously known for his “Rest of the Story” segment, which was watched by millions until his death at the age of 90. The name Harvey can be related to many things, including the stamina held by Paul Harvey Himself, and the impact of his life felt by millions of Americans. This “Harvey” could be looked to for the “rest of the story” when it comes to protecting software and programs being used. The finality of such a name could be applied to the logo of a project that protects and defends. eCryptfs is software that thrives on protection, and as mentioned in the blog, is a “vibrant and open-sourced” project. Having a fitting name is appropriate when it comes to any new idea or project. In order to be remembered as a project that strives for excellence, the logo has to be focused on and perfected. The name “Harvey” is a name that is not easily forgotten, and provides a significant enough meaning to the new logo.

Well done, Blane. We, here at the Gazzang offices, absolutely love it! And so, I'm pleased to introduce Harvey, the Honey Badger!

:-Dustin

UDW: Pair Programming and Code Reviews in the Cloud

2012-01-26T12:36:00.000-06:00

Next week is yet another installment of the Ubuntu Developer Week education series. If you been wanting to get involved in Ubuntu or Free Software development, or perhaps just hone your existing skill set, please join us in #ubuntu-classroom on irc.freenode.net Tuesday/Wednesday/Thursday next week. Check the schedule, and hopefully you'll find something that piques your interest.

I'm pleased to note that each member of Gazzang's engineering team will be attending at least two sessions per day! With today's shrinking education budgets, perhaps you can convince your employer to let you attend some excellent, continuing technical education at no additional expense to them. Should be an easy sell ;-)

I will be leading an hour long session on Thursday, February 2nd from 18:30-19:30 UTC -- that's 12:30pm-1:30pm in my local Central Standard Time. My session is on Pair Programming and Code Review in the Cloud.

I've used Pair Programming for years -- ever since I was introduced to the Extreme Programming methodologies in the Tivoli Bootcamp as an intern in 2000. Pair Programming is a relatively simple concept -- two people, one keyboard and screen. It's a great way to teach, learn, and review code. Back then, we were a couple of developers, sitting side by side in the Arboretum in Austin, Texas.

But times have changed! It's highly unlikely that I'm sitting next to the person I need to pair program with. Rather, they're sitting somewhere far across the world.

Welcome to 2012! I'll spend an hour, sharing a screen with a few dozen of you, showing you how some Ubuntu developers work with colleagues across the world, through the Cloud!

I'm going to fire up Amazon's largest instance splurging $2.10 an hour for 60GB of RAM and 16 CPUs. You hardly need this, but I thought it would be fun. If nothing else, drop in and have a look at what this kind of hardware looks like :-) We'll import SSH public keys and users will SSH into a shared Byobu/Tmux session, where I'll demonstrate how to make the most use of our screen resources. We'll split the window horizontally and vertically, look at code side by side, while still tailing log files and conducting builds.

Prerequisites:

A terminal and an SSH client with Internet access

And to maximize your experience:

Please run your terminal/SSH client maximized/full-screen

I'll open up the classroom IRC channel in there, which you'll be able to read

Open an account at Launchpad.net and add your public SSH keys
Print out a copy of the Byobu keyboard shortcuts for quick reference

As a teaser, here's what my terminal currently looks like, and a taste of where we'll get to, in this session. This session can be detached and reattached later, or even by multiple users at the same time.

I have 8 panes open in a single Byobu session. The first two windows have some eCryptfs source code (mount.ecryptfs_private.c and pam_ecryptfs.c). Next, I have a little test window where I'm checking my changes, with a foobar@x220 user logged in, and it's just above a small window where I'm reading some manpage documentation. To the far right, I'm re-compiling the new ecryptfs sources. Across the bottom, I'm tailing 4 log files (kern.log, dmesg, auth.log, syslog). Note that I'm using tail -f and ccze for colorized log files -- which really helps separate warnings and errors (in warm reds and yellows) from the rest (in cool blues and greens).

Hope to Pair Program with you on Thursday!

Cheers,
:-Dustin

Video Explanation of ACTA / SOPA

2012-01-18T10:14:00.000-06:00

I think it's important for everyone to understand what's at risk here today. This is a must-see, and well worth 7 minutes of your time...

Dustin

Automatically Swapping Launchpad and Bazaar Identities

2012-01-16T12:59:00.000-06:00

I've been a Launchpad.net member since 2006-10-11, when I first created an account to add some debugging information and submit a patch to a bug affecting the xserver on iMac G3s and the Ubuntu 6.06 PowerPC LiveCD, which my wife, Kim, used in her 4th grade classroom. Wow, those were the days! I see that that bug is still open :-) I can't imagine that hardware is even functional anymore....is it?

I was thoroughly impressed with the shear elegance, look, feel, and usability of Launchpad.net. I was a long time user of SourceForge.net and Bugzilla, and had brushed by at least a dozen other bug trackers. No other bug tracker or source code system could hold a candle to Launchpad, in my opinion.

In my ~4 years at Canonical, Launchpad.net and Bazaar became the cornerstone and foundation of my day to day development and productivity. I was absolutely thrilled when Launchpad was open sourced (to relatively little fanfare, sadly).

I've filed and fixed a few minor issues, and worked around some others, and leveraged Launchpad for tools of my own (like ssh-import-id). And today, I still think Launchpad.net and Bazaar are the best combination of bug tracking, source code management, binary package builders, team building, blueprint tracking out there!

I continue to use Launchpad and Bazaar to manage more than two dozen open source projects. And now, we're also using commercial Launchpad here at Gazzang now, actively committing to both public and private projects every day.

This introduced a new challenge, for me, though. I want to make ensure that my commits to Bazaar when I'm "at the office" and working on Gazzang projects are correctly credited to my work email address and identity, and otherwise, they're credited to my personal email address.

This email address is stored in ~/.bazaar/bazzar.conf. For me, the logic is pretty easy... I generally work from the office where we have a (mostly) static IP address. I simply run a cronjob every five minutes that checks my external IP address, and updates ~/.bazaar/bazzar.conf accordingly. Your logic might differ (perhaps time of day, etc.). Does anyone know how I might perhaps hook bzr to check the project's name at commit time? Also, any ideas about how to update $DEBEMAIL in a similar manner? It's an environment variable, so it's pretty hard/impossible to update that in all of my shells and byobu sessions/windows/splits, and the Debian maintainer rejected a few requests to support $DEBEMAIL in ~/.devscripts. Other ideas?

My script currently looks something like this:

#!/bin/sh
# $HOME/bin/update-email

work_email="dustin.kirkland@work.example.com"
home_email="dustin@home.example.com"

work_ip="10.9.8.7"
current_ip=$(wget -q -O- http://v4.ipv6-test.com/api/myip.php 2>/dev/null)

if [ "$current_ip" = "$work_ip" ]; then
        sed -i -e "s/<.*>/<$work_email>/g" $HOME/.bazaar/bazaar.conf
else
        sed -i -e "s/<.*>/<$home_email>/g" $HOME/.bazaar/bazaar.conf
fi

And it runs in this cronjob:

*/5 * * * *  run-one $HOME/bin/update-email

Suggestions for improvement? Leave a note!

Enjoy!
:-Dustin

Introducing the Ken Burns Effect in Pictor's Slideshow

2012-01-11T14:43:00.000-06:00

I'm a big fan of documentaries. Especially Ken Burns documentaries. The National Parks, Jazz, Baseball, Huey Long -- I love them all!

Ken Burns is particularly famous for a special effect (known as The Ken Burns Effect) of zooming and panning across still pictures. It's both brilliant and beautiful.

I recently came across an implementation of The Ken Burns Effect written in Javascript, by Will McGugan, introduced in his blog here:

http://www.willmcgugan.com/blog/tech/2011/2/26/ken-burns-effect-with-javascript-and-canvas/

I contacted Will and he was gracious enough to put an AGPL header on his source and I have added it to Pictor. Now, if you click on the screensaver link on a Pictor picture page, you can cycle through your album in order and your pictures are panned and zoomed like a Ken Burns documentary!

All you need to do is:

Install Pictor
```
sudo apt-get install pictor
```
Symbolically link some albums into /usr/share/pictor/pictures
```
sudo ln -sf $HOME/Pictures /usr/share/pictor/pictures
```
Point a browser to your server's pictor instance
http://localhost/pictor
Click on an album, click on a picture, and then click screensaver, as below:

Will released his code here as kenburns-1.0.0.zip, which I was thinking of packaging for Ubuntu (rather than statically including it in Pictor). Would that be useful to anyone else?

:-Dustin

Introducing New Branding and Logos for eCryptfs

2012-01-11T14:09:00.000-06:00

The time I have spent around Ubuntu has given me a deep appreciation for the finer points of design, branding, themes, and artwork around software development and user interfaces. From Ubuntu's elegant color schemes and meticulously kerned fonts, to the careful placement and balance of Ubuntu's logos and Canonical's brandmarks, Ubuntu exudes an exquisite level of polish and professionalism, particularly among free software projects.

With this as a backdrop, I have long wanted to refresh and modernize the logos and branding associated with eCryptfs, as an upstream open source project. For over six years, the eCryptfs logo has been an ever-so-trite yellow key overlayed on top of a pie chart.

Yawn :-o I'm pretty sure that "logo" was pulled off of a slide deck for IBM management, when Michael Halcrow and Emily Ratliff originally presented the idea of eCryptfs back in 2004.

So I pitched this idea to my new employer, Gazzang, who, as it turns out, has considerable interest in a healthy eCryptfs community, as it forms the basis for several of our cloud encryption products. Our CEO, Larry, was thrilled by the idea, and gave Heidi (our director of marketing) financial approval to commission the new art from a professional graphic designer.

We felt that eCryptfs, as an active and vibrant open source project, deserved a logo and a mascot that reflects just that. Everyone uses a lock or a key to represent encryption, so we thought we'd do something different. We decided we wanted a stylized animal, in the spirit of Linux's Tux, BSD's Daemon, OpenBSD's Puffy, and of course Ubuntu's every growing zoo!

We settled on the honey badger (Mellivora capensis), inspired by its thick skin and ferocious defensive traits, much as eCryptfs adamantly defends your data against even the most determined attackers (honey badger don't care!) We are, of course, also saluting the running honey badger Internet meme :-)

And the font is modern, crisp, clean, and perhaps a little "techy" even. The "fs" is highlighted, to note the relationship to the filesystem, as well as help demonstrate the pronunciation of the word -- "ecrypt" and then "fs".

Gazzang has contributed all of this artwork to the eCryptfs project under the Creative Commons CC BY-SA 3.0 license. We hope you enjoy it as much as we do :-) Let us know what you think!

Now there is one piece we're still missing. We don't yet have a name for our snarling cryptographic honey badger. So we're putting it out to you... Suggestions? Drop us a comment below!

:-Dustin

Gmail and GPG in Chromium, with cr-gpg!

2012-01-10T16:51:00.000-06:00

Once upon a time, I used FireGPG to sign, encrypt, decrypt, and check GPG and PGP messages in Gmail. Sadly, FireGPG eventually withered away, dropping support for Gmail altogether. Encrypted, authenticated email messaging is very important to me in some specific situations, and for those, I've been using Gmail, but then copying and pasting the data manually back and forth to a command line, and using GPG by hand.

However, I have been extremely excited to see the cr-gpg plugin for Chromium coming along very rapidly, and finally re-enabling email encryption for Gmail! The plugin is still in an Alpha state, but I've reported a few bugs here, and worked with the developers and helped test others.

I'm pleased to say that I'm now using cr-gpg on a daily basis for encryption/decryption with:

Signature verification is still a little broken in a couple of scenarios, but I've reported these bugs upstream and they're actively being worked.

Here's a couple of screen shots of it in action, first decrypting a message, then encrypting one.

Enjoy,

:-Dustin

Ubuntu Quick Installation Preseed Link Updated!

2012-01-09T12:42:00.000-06:00

priority=critical locale=en_US url=http://bit.ly/uinstall

I've maintained for some time now a preseed file (previously called "uquick") that you can use with your Ubuntu Server and Alternate ISO boot options, to quickly and automatically install an Ubuntu server.

That url (http://bit.ly/uquick), has been broken for a month or so since I've left Canonical and all my people.canonical.com/~kirkland/* pages have been purged.

I've actually set up a new bit.ly link at http://bit.ly/uinstall. This is hosted on a Google Sites page that I own, so I'll be able to maintain it there permanently. Note that the source is actually under bzr revision control under lp:bikeshed here.

So now, you can boot an Ubuntu Server ISO, press F6 for Other Options, and append the following::

local=en_US priority=critical url=http://bit.ly/uinstall

Enjoy!
:-Dustin

Gazzang Bang!

2012-01-08T14:18:00.000-06:00

With my friends and former colleagues sprinting in Budapest at Ubuntu's Distro Rally, I already miss our nightly Ubuntu All-Stars jam sessions :-( It was always such a great outlet for creativity and right-brain activity, after long days devoted to left-brain engineering work. I think some of my best memories over the last four years are jamming with Tony, Pete, Amber, Jorge, Matt, Hugh, Luke, Jono, Barry, and countless others!

Searching for some of the same camaraderie and revelry, we at Gazzang have formed our own band -- Gazzang Bang! And remarkably, we actually got a byline of press coverage and a band portrait in our local Austin American Statesman newspaper this weekend! The article was actually about the background of some of the funny/silly/unusual names of current Austin tech start-ups, of which Gazzang was one (along with Infochimps, Springbox, and UShip).

As you can see, we're a little drum-heavy at the moment (hence, the Bang part of the name). However, we do have a few job openings in Austin, Texas, (Senior Security Software Engineer, Director of Information Security) -- special preference given to keyboardists and vocalists! ;-)

:-Dustin

Gazzang officially an AWS Solution Provider!

2012-01-06T15:23:00.000-06:00

Gazzang is now an officially Amazon Web Services (AWS) Solution Provider! I'm quite excited about this, as our information security and data encryption products are such a great match for the cloud, where you don't actually have physical control over where and how your data gets written to disk. Encryption is absolutely essential here.

Seriously...if you or your organization is using AWS and storing sensitive data of any kind in a MySQL or PostgreSQL (especially on the rockin' Ubuntu Server), you should really take a close look at Gazzang's ezNcrypt for Databases. I've recently showed how its possible, though complex, to use eCryptfs and setup Encrypted Home Directories on EC2. But if you're looking for something completely seamless, and turn-key for transparently encrypting all or some of your databases, ezNcrypt is just that.

Cheers,
:-Dustin

Byobu 5 Released!

2011-12-26T10:08:00.000-06:00

Happy Holidays everyone! And for you, I have a gift -- Byobu 5.0!

I've been working hard over the last few months pulling together some big changes in the 4.x series, culminating in yesterday's release. I gave an early preview during a lightning talk (31:02 mark) at the Ubuntu Developer Summit in Orlando, Florida a couple of months ago.

History

The Byobu project started a little over 3 years ago as a set of best practices and configuration profiles for GNU Screen with the screen-profiles package. Byobu builds on top of existing text-based window managers and adds real-time dynamic status reporting, helper configuration utilities, and convenient keybindings.

Earlier this year, we started to reach the limits of what we could do with GNU Screen within Byobu. The GNU Screen project hasn't been officially released in over 3 years, and Ubuntu is currently carrying nearly 20,000 lines in 48 patches to the upstream source. I started looking into alternatives and learned a bit about Tmux, a newly redesigned and actively maintained window manager springing from the OpenBSD project. The code is modern and elegant, and has an excellent programmable interface. In June, I polled some Byobu users, asking of their interest in Tmux and the response was overwhelming! I started porting Byobu to Tmux almost immediately.

New in 5.0

The most significant change that Byobu 5.0 introduces is a shift from GNU Screen to Tmux as the default backend. You can still run Byobu in Screen-mode, but the default experience now uses Tmux.

Selecting your Back end

You can select your default back end using:

$ byobu-select-backend 

Select the byobu backend:
  1. tmux
  2. screen

Choose 1-2 [1]:

After which, just running byobu will use your selected back end. Alternatively, you can run byobu-screen or byobu-tmux at any time, to launch Byobu with a particular back end.

The New Byobu Look

When you start Byobu 5.0, you may notice a couple of immediate changes. For starters, there's only one line of status at the bottom. Your windows and status items are all in the same line. You can set multiple status combinations in your ~/.byobu/status line, and cycle through them using Shift-F5. Personally, I run Byobu maximized and use horizontal and vertical splits for efficiency (more on that in a minute!).

Tmux offers several advantages in the status line, namely: UTF8 characters and 256 colors. If you look at the lower left of the screen shot, you should see the Ubuntu brandmark, u, as well as other nice symbols in the status bar, such as "▴2.0Mb ▾53kb". Also, with 256 colors, we can get much closer to the right aubergine and orange.

The Help Menu

You can bring up Byobu's new help menu any time by pressing Shift-F1, with which you can find a comprehensive list of Byobu's keybindings.

Creating Windows, Splits, and Sessions

All of the "creation" actions are conveniently found under the F2 key.

Create new windows with F2
Create new horizontal splits with Shift-F2
Create new vertical splits with Ctrl-F2
Create new sessions with Ctrl-Shift-F2

Navigating Windows, Splits, and Sessions

As in previous versions, you can use F3 and F4 to move right and left among windows.

But far more intuitively, you can also use the up/down/left/right arrow keys with the alt/ctrl/shift modifiers.

Move between windows with Ctrl-Shift-Left and Ctrl-Shift-Right
Move between sessions with Alt-Up and Alt-Down
Move focus among splits with Shift-Up, Shift-Down, Shift-Left, and Shift-Right

Note that the split with the focus will be highlighted in purple

Re-size a split using Ctrl-Up, Ctrl-Down, Ctrl-Left, and Ctrl-Right

You can also:

Move a split using Ctrl-F3 and Ctrl-F4
Move a window using Ctrl-Shift-F3 and Ctrl-Shift-F4

The Status Bar

As usual, the F5 key deals with your status line.

Refresh all status and reload your profile with F5
Toggle through multiple status configurations with Shift-F5
Reconnect ssh, gpg, dbus, and X sessions with Ctrl-F5

sometimes, these connections become stale on session disconnect/reconnect

Randomly select the background color of the status line with Ctrl-Shift-F5

visually identify each system by its unique color

Disconnecting and Reconnecting Sessions

The F6 key handles disconnecting and detaching.

Detach the current session and logout with F6
Detach the current session, but do not logout with Shift-F6
Kill the current split with Ctrl-F6

Running byobu will automatically prompt you to select a session, if there are more than one running. Or running byobu-select-session will also list the available sessions and prompt for selection.

$ byobu-select-session 

Byobu sessions...

  1. tmux: 0: 8 windows (created Sun Dec 25 09:59:05 2011) [170x42]
  2. tmux: 1: 1 windows (created Sun Dec 25 10:00:46 2011) [170x42]
  3. tmux: 3: 2 windows (created Sun Dec 25 12:30:55 2011) [136x36]
  4. Create a new Byobu session (tmux)
  5. Run a shell without Byobu (/bin/bash)

Choose 1-5 [1]:

Scroll back and History

Each window and each split has an independent history buffer that can be scrolled and even searched, as usual with F7.

Enter scroll back with F7
Enter and navigate scroll back with Alt-PageUp and Alt-PageDown
Exit scroll back with Enter
Search scroll back with / and ? and then typing your search term

Window and Split Arrangement

As in previous versions, you can change a window's name with F8, but F8 also provides some advanced features around split arrangements.

Rename a window with F8
Cycle through preset split arrangements with Shift-F8
Restart a saved split layout with Ctrl-F8
Save the current split layout with Ctrl-Shift-F8

Configuration Window

As always, you launch the Byobu configuration menu with F9. It's greatly simplified from previous versions. (I'm actually hoping to deprecate it entirely one day, as the dependency on python-newt here has always been a little inconvenient from an upstream perspective. I'm trying to make most of the features usable from key bindings. Getting there eventually...)

Full Screen, Joining, and Breaking Out Splits

The F11 key is probably used by your X window manager to toggle a window from full screen and back. Byobu uses Alt, Shift, and Ctrl and F11 to provide a few other features.

Break the current split out into a full window of its own with Alt-F11
Join the current window into a horizontal split with Shift-F11
Join the current window into a vertical split with Ctrl-F11

Escapes, Toggling Key bindings, and Piet Mondrian

The default escape sequence in Tmux is actually Ctrl-B. To maintain consistency with Byobu and Screen, Byobu changes this back to Ctrl-A. Byobu also loads a set of key bindings that operate Tmux with the same commands that are familiar to Screen users.

The F12 key is actually an alias for the escape sequence
Toggle on and off Byobu's key bindings with Shift-F12

this is useful when running programs that conflict with Byobu's keys, such as mc

For Piet Mondrian inspired fun, press Ctrl-Shift-F12

And with that, I'll leave you for now. Give Byobu 5.0 a shot and let me know what you think. Cheers everyone! Hope you're having a wonderful holiday!

:-Dustin

Using eCryptfs and Ubuntu Encrypted Home in EC2

2011-12-22T13:03:00.000-06:00

Admittedly, using eCryptfs and Ubuntu's Encrypted Home feature in EC2 is a bit circumlocutious. At Gazzang, we're working on making that a bit more seamless, and a lot more secure. But in the meantime, here are some handy instructions on how you can set it up manually for yourself.

But first, why would you want to do this? Good question! Bear in mind that by using EC2 and storing any data there, you're putting a considerable amount of trust in Amazon already. They own the hardware and the hypervisor. They are running a modified Linux/Xen kernel that you cannot even audit, if you wanted to. They haven't released the sourced to that modified Linux kernel, so don't deceive yourself -- their instrumented kernels could be logging your every keystroke. Hopefully not. But you don't know that.

So what can you do? What good is eCryptfs here? Well, if you transparently read and write your data through an eCryptfs encryption/decryption layer, you can add a measurable amount of confidence and security that your data will at least be encrypted when it's at rest, once it lands on a spinning hard disk somewhere in an Amazon data center. In this world of cloud trust, you're explicitly trusting Amazon to "do the right thing" and take reasonable precautions. Amazon is huge, and has a tremendous amount to lose by acting deceptively. But you can't say the same for every single individual between you and your data. In other words, you don't necessarily trust every individual that might brush past your data. Hard disks get stolen and sold on eBay, they're returned to the manufacturer for repair, donated to Goodwill or schools, recycled, repurposed, and reused. So if you could trivially ensure that your bytes are encrypted before being written to disk, would you? Well, as you see below, it's not quite trivial yet, but it is very much possible. Stay tuned here and watch this area of technology evolve. In the meantime, give this a shot...

First, start an Ubuntu VM in EC2. I use the cloud-sandbox command from lp:bikeshed. I'm sure you have your own methods.

Next, SSH into your new VM and install ecryptfs-utils.

sudo apt-get install ecryptfs-utils

Next, you must set a login password for the Ubuntu user. Note that you do not have to enable PasswordAuthentication in /etc/ssh/sshd_config (though you might choose to). As always, make sure you choose a strong passphrase. I recommend at the very least 12 characters, with upper case, lower case, and numbers. You know how to choose a good password. The more important it is that your data stay private, the better the password should be ;-)

sudo passwd ubuntu

Exit byobu, or any other programs you might be running as your ubuntu user, and change out of your $HOME directory, and migrate your home directory. However, if you've encrypted all of your $HOME, you MUST move your .ssh directory out, so that your authorized keys file is not encrypted!!! Make sure you run all of the following commands sequentially, and without terminating your SSH connection, or else you might find yourself locked out of your instance :-)

cd / ; sudo ecryptfs-migrate-home -u ubuntu

sudo ln -s /home/.ecryptfs/ubuntu/.ssh $HOME/
su - ubuntu
ecryptfs-mount-private
cd $HOME
mv $HOME/.ssh /home/.ecryptfs/ubuntu/
ln -s /home/.ecryptfs/ubuntu/.ssh $HOME/

If that completes successfully, we can clean up our backup of our unencrypted home directory.

sudo rm -rf /home/ubuntu.*

Alternatively, might might choose just to encrypt one private directory, instead of migrating all of your home. To do so, use:

ecryptfs-setup-private

Finally, we will want to be prompted for our login password at every login to automatically mount our home directory, so let's also create a ".profile" in our unencrypted home directory.

ecryptfs-umount-private
echo "ecryptfs-mount-private; . $HOME/.profile; cd" | sudo tee $HOME/.profile
ecryptfs-mount-private

Alright! At this point, we should be able to exit all of our shells and SSH back into our EC2 instance. The SSH public key authentication will get us onto the machine, and then our .profile script should prompt us for our login passphrase and automatically mount our encrypted home directory.

The data that actually gets written to your root ext4 filesystem on /dev/xvda1 are the files that you can find in /home/.ecryptfs/ubuntu/.Private/, which should look something like this:

ubuntu@ip-10-194-246-143:~$ ll /home/.ecryptfs/ubuntu/.Private/
total 68
drwx------ 3 ubuntu ubuntu  4096 Dec 22 18:54 ./
drwxr-xr-x 5 ubuntu ubuntu  4096 Dec 22 18:46 ../
lrwxrwxrwx 1 ubuntu ubuntu   124 Dec 22 18:42 ECRYPTFS_FNEK_ENCRYPTED.FWbSgDSRezlYtETTxmAwbGjiN4WOMkt-2hHi1R07VV4a9quAsP3ATb2JK--- -> ECRYPTFS_FNEK_ENCRYPTED.FYbSgDSRezlYtETTxmAwbGjiN4WOMkt-2hHiSRA-6SgbLQ.LtWP2pwGZY57PtU2wAgzLn-ECMilfrp9dp0YUYlTDNwY6P764.gPo
-rw-r--r-- 1 ubuntu ubuntu 12288 Dec  1 12:50 ECRYPTFS_FNEK_ENCRYPTED.FWbSgDSRezlYtETTxmAwbGjiN4WOMkt-2hHi9KCXyAtK1PsV4KirBxb8fk--
drwx------ 2 ubuntu ubuntu  4096 Dec 22 17:32 ECRYPTFS_FNEK_ENCRYPTED.FWbSgDSRezlYtETTxmAwbGjiN4WOMkt-2hHicFvfubbvnebsd2N8jh9vRU--/
-rw-r--r-- 1 ubuntu ubuntu 12288 Dec  1 12:50 ECRYPTFS_FNEK_ENCRYPTED.FWbSgDSRezlYtETTxmAwbGjiN4WOMkt-2hHifCuJCnlfaXjU4QlrUWfhIU--
-rw-r--r-- 1 ubuntu ubuntu 12288 Dec  1 12:50 ECRYPTFS_FNEK_ENCRYPTED.FWbSgDSRezlYtETTxmAwbGjiN4WOMkt-2hHiNgxmEEQUk9nI3uOlsQkCHk--
lrwxrwxrwx 1 ubuntu ubuntu   104 Dec 22 18:42 ECRYPTFS_FNEK_ENCRYPTED.FWbSgDSRezlYtETTxmAwbGjiN4WOMkt-2hHipvXKHoAMUybcfPOQYgm1WE-- -> ECRYPTFS_FNEK_ENCRYPTED.FXbSgDSRezlYtETTxmAwbGjiN4WOMkt-2hHif.b7-V31EJPzRLnx.vfW9dIwfbnZuIcdSIqqNTvonyo-
lrwxrwxrwx 1 root   root     104 Dec 22 18:54 ECRYPTFS_FNEK_ENCRYPTED.FWbSgDSRezlYtETTxmAwbGjiN4WOMkt-2hHisXvcg5obbXbibbufq7QjyE-- -> ECRYPTFS_FNEK_ENCRYPTED.FXbSgDSRezlYtETTxmAwbGjiN4WOMkt-2hHiGNrAq2Ud8N9P5xVz2YssSWo-.u4wRtBbZLQLIeG-0I2-
-rw-r--r-- 1 ubuntu ubuntu  8192 Dec 22 17:32 ECRYPTFS_FNEK_ENCRYPTED.FXbSgDSRezlYtETTxmAwbGjiN4WOMkt-2hHivZ3-rM86jHHkrHcJAXqMkfoOaMkowIPainVLMFWajCg-

This is what you're hoping your attacker, the unsavory individual who comes into contact with one of those magic cloud hard drives containing your data, sees. These are the encrypted file names, and the file contents are just as unreadable without the necessary keys!

Enjoy,
:-Dustin

Released ecryptfs-utils 94 and 95

2011-12-14T14:34:00.000-06:00

Howdy!

I've done quite a bit of work in the last few days to get on top of the eCryptfs bug backlog. I've managed to at least triage all of the upstream New/Undecided bugs, and managed to digest all of the High/Medium/Low ones. I haven't gotten to the Wishlist ones yet, but I'll do so soon. Next week, I'll try to tackle the Ubuntu ecryptfs-utils bug backlog and do the same (triage New/Undecided, and process High/Medium/Low).

In doing so, I've fixed a handful of bugs, tested, and released ecryptfs-utils-94 and ecryptfs-utils-95. These have been uploaded to Ubuntu precise already, and other distros can find the release tarballs here.

The release notes are below. Thanks to Tyler for help with the testing, and to all of the contributors noted below. Happy Crypting!

ecryptfs-utils (95-0ubuntu1) precise; urgency=low

  [ Serge Hallyn ]
  * fix infinite loop on arm: fgetc returns an int, and -1 at end of
    options.  Arm makes char unsigned. (LP: #884407)

  [ Dustin Kirkland ]
  * debian/compat, debian/control, debian/ecryptfs-utils.install,
    debian/ecryptfs-utils.lintian-overrides,
    debian/libecryptfs0.install, debian/libecryptfs-dev.install,
    debian/lintian/ecryptfs-utils, debian/python-ecryptfs.install,
    debian/rules, debian/source/options, doc/ecryptfs-pam-doc.txt,
    doc/manpage/ecryptfs-setup-private.1, lintian/ecryptfs-utils, ===
    removed directory debian/lintian:
    - merge a bunch of packaging changes from Debian's Daniel Baumann
  * scripts/release.sh:
    - minor release fixes

 -- Dustin Kirkland   Wed, 14 Dec 2011 14:21:34 -0600

ecryptfs-utils (94-0ubuntu1) precise; urgency=low

  [ Dustin Kirkland ]
  * scripts/release.sh:
    - fix release script
    - bump ubuntu release
  * doc/manpage/ecryptfs-recover-private.1, src/utils/ecryptfs-migrate-
    home (properties changed: -x to +x), src/utils/ecryptfs-recover-
    private:
    - add a --rw option for ecryptfs-recover-private
  * src/utils/ecryptfs-migrate-home: LP: #820416
    - show progress on rsync
  * debian/ecryptfs-utils.ecryptfs-utils-restore.upstart,
    debian/ecryptfs-utils.ecryptfs-utils-save.upstart,
    src/utils/ecryptfs-migrate-home,
    src/utils/ecryptfs-setup-private: LP: #883238
    - remove 2 upstart scripts, which attempted to "save" users who didn't
      login after migrating their home; instead, we now require the root
      user to enter user passwords at migration time
  * debian/copyright, debian/ecryptfs-utils.ecryptfs-utils-
    restore.upstart, debian/ecryptfs-utils.ecryptfs-utils-save.upstart,
    doc/manpage/ecryptfs.7, doc/manpage/ecryptfs-add-passphrase.1,
    doc/manpage/ecryptfs-generate-tpm-key.1, doc/manpage/ecryptfs-
    insert-wrapped-passphrase-into-keyring.1, doc/manpage/ecryptfs-
    mount-private.1, doc/manpage/ecryptfs-recover-private.1,
    doc/manpage/ecryptfs-rewrap-passphrase.1, doc/manpage/ecryptfs-
    rewrite-file.1, doc/manpage/ecryptfs-setup-private.1,
    doc/manpage/ecryptfs-setup-swap.1, doc/manpage/ecryptfs-stat.1,
    doc/manpage/ecryptfs-umount-private.1, doc/manpage/ecryptfs-unwrap-
    passphrase.1, doc/manpage/ecryptfs-wrap-passphrase.1,
    doc/manpage/fr/ecryptfs-add-passphrase.1, doc/manpage/fr/ecryptfs-
    generate-tpm-key.1, doc/manpage/fr/ecryptfs-insert-wrapped-
    passphrase-into-keyring.1, doc/manpage/fr/ecryptfs-mount-private.1,
    doc/manpage/fr/ecryptfs-rewrap-passphrase.1,
    doc/manpage/fr/ecryptfs-setup-private.1, doc/manpage/fr/ecryptfs-
    umount-private.1, doc/manpage/fr/ecryptfs-unwrap-passphrase.1,
    doc/manpage/fr/ecryptfs-wrap-passphrase.1, doc/manpage/fr/ecryptfs-
    zombie-kill.1, doc/manpage/fr/ecryptfs-zombie-list.1,
    doc/manpage/mount.ecryptfs_private.1, doc/manpage/pam_ecryptfs.8,
    doc/manpage/umount.ecryptfs.8,
    doc/manpage/umount.ecryptfs_private.1,
    src/pam_ecryptfs/pam_ecryptfs.c,
    src/utils/ecryptfs_add_passphrase.c,
    src/utils/ecryptfs_insert_wrapped_passphrase_into_keyring.c,
    src/utils/ecryptfs-migrate-home, src/utils/ecryptfs-mount-private,
    src/utils/ecryptfs-recover-private,
    src/utils/ecryptfs_rewrap_passphrase.c, src/utils/ecryptfs-rewrite-
    file, src/utils/ecryptfs-setup-private, src/utils/ecryptfs-setup-
    swap, src/utils/ecryptfs-umount-private,
    src/utils/ecryptfs_unwrap_passphrase.c,
    src/utils/ecryptfs_wrap_passphrase.c:
    - update some email addresses, moving kirkland@canonical.com ->
      kirkland@ubuntu.com (which I can still read)
  * src/libecryptfs/key_management.c: LP: #715066
    - fix 2 places where we were handling
      ecryptfs_add_passphrase_key_to_keyring() inconsistently
    - if we're trying to add a key to the keyring, and it's already there,
      treat that as "success"
  * debian/control:
    - ecryptfs-setup-swap is strongly recommended, which depends on
      cryptsetup; so promote cryptsetup from suggests -> recommends

  [ Stephan Ritscher and Tyler Hicks ]
  * src/libecryptfs/cmd_ln_parser.c: LP: #683535
    - fix passphrase_passwd_fd for pipes
    - handle memory allocation failures
    - free memory in error paths

  [ Arfrever Frehtes Taifersar Arahesis ]
  * configure.ac: LP: #893327
    - no need to check for python, if --disable-pywrap is passed

 -- Dustin Kirkland   Thu, 27 Oct 2011 10:58:47 -0500

:-Dustin

I've Joined the Gazzang Team!

2011-12-12T10:24:00.000-06:00

A few weeks ago, I joined a fun, new start-up company here in Austin called Gazzang. I was a little surprised that this was published in the form of a rather flattering press release :-) Let's just say that my Mom was very proud!

I know that some of you in the Ubuntu community are wondering how that career change will affect my responsibilities and contributions to Ubuntu. I'm delighted to say that I'll most certainly continue to contribute to Ubuntu and many of my upstream projects. Gazzang is quite supportive of my work in both Ubuntu and open source.

Most directly, you should see me being far more active in my regular maintenance, development, bug triage, and support of eCryptfs. Gazzang's core business is in building information privacy and data security solutions for the Cloud. eCryptfs is at the heart of their current products, and in my new role as Gazzang's Chief Architect, we're working on some interesting innovations in and around eCryptfs. A healthy, high-quality, feature-filled, high-performance eCryptfs is essential to Gazzang's objectives, and I'm looking forward to working on one of my real passions in eCryptfs!

More specifically, looking at the projects I maintain, I expect to continue to be very active in:

eCryptfs (essential to my new job)
byobu (mostly around tmux, and because hacking on byobu is fun and awesome :-)
manpages.ubuntu.com and manpg.es (because that's how I read manpages)
musica (because that's how I've streamed music since 1998)
pictor (because that's how I've managed and shared pictures since 1998)

You'll probably see opportunistic development (nothing active, but when an opportunity or bugs spring up), including the usual bzr/launchpad dance, developing, testing, upstream releasing, packaging, and uploading to Ubuntu, of:

bikeshed (apply-patch, bch, bzrp, col*, dman, pbget, pbput, pbputs, release, release-build, what-provides, what-repo, what-source, wifi-status)
bogosec
bootmail, rootsign
cloud-utils (along with smoser)
dotdee
errno
kvm-ok
run-one, run-this-one, keep-one-running
service
ssh-import-id
... (plus some new stuff I haven't released yet!)

And finally, as prescribed by the Ubuntu Code of Conduct, I'm gracefully stepping away from a few other projects I've founded or maintained in the past. I'll help out if and when I can, but for now I've transferred all of the necessary rights, responsibilities and ownership of:

awstrial (to Canonical ISD)
cloud-live (to Ante Karamatic)
cloudfoundry (to the Canonical SI Team)
orchestra (to the Ubuntu Platform Server Team)
powernap (to Andres Rodriguez)
testdrive (to Andres Rodriguez)

Finally, I must say that the last 4 years have been the most amazing 4 years of my entire 12 year professional career. It's been quite rewarding to witness the fledgling Ubuntu Server of February 2008 (when I joined Canonical), and the tiny team of 5 grow and evolve to the 20+ amazing people now working directly on the Ubuntu Server. And that list doesn't even remotely cover the dozens (if not hundreds!) of others around Canonical and the Ubuntu Community who contribute and depend on the amazing Server and Cloud distribution that is Ubuntu.

I'm really looking forward to my new opportunities around Gazzang and eCryptfs, but you'll still most certainly see me around Ubuntu too :-) As crooned by The Beatles...

You say "Yes", I say "No". \\ You say "Stop" and I say "Go, go, go". \\ Oh no. \\ You say "Goodbye" and I say "Hello, hello, hello". \\ I don't know why you say "Goodbye", I say "Hello, hello, hello". \\ I don't know why you say goodbye, I say hello!

Cheers,
:-Dustinhttp://www.gazzang.com

Getting Started with Ubuntu Orchestra -- Servers in Concert!

2011-10-27T16:12:00.000-05:00

Servers in Concert!

Ubuntu Orchestra is one of the most exciting features of the Ubuntu 11.10 Server release, and we're already improving upon it for the big 12.04 LTS!

I've previously given an architectural introduction to the design of Orchestra. Now, let's take a practical look at it in this how-to guide.

Prerequisites

To follow this particular guide, you'll need at least two physical systems and administrative access rights on your local DHCP server (perhaps on your network's router). With a little ingenuity, you can probably use two virtual machines and work around the router configuration. I'll follow this guide with another one using entirely virtual machines.

To build this demonstration, I'm using two older ASUS (P1AH2) desktop systems. They're both dual-core 2.4GHz AMD processors and 2GB of RAM each. I'm also using a Linksys WRT310n router flashed with DD-WRT. Most importantly, at least one of the systems must be able to boot over the network using PXE.

Orchestra Installation

You will need to manually install Ubuntu 11.10 Server on one of the systems, using an ISO or a USB flash disk. I used the 64-bit Ubuntu 11.10 Server ISO, and my no-questions-asked uquick installation method. This took me a little less than 10 minutes.

After this system reboots, update and upgrade all packages on the system, and then install the ubuntu-orchestra-server package.

sudo apt-get update
sudo apt-get dist-upgrade -y
sudo apt-get install -y ubuntu-orchestra-server

You'll be prompted to enter a couple of configuration parameters, such as setting the cobbler user's password. It's important to read and understand each question. The default values are probably acceptable, except for one, which you'll want to be very careful about...the one that asks about DHCP/DNS management.

In this post, I selected "No", as I want my DD-WRT router to continue handling DHCP/DNS. However, in a production environment (and if you want to use Orchestra with Juju), you might need to select "Yes" here.

And a about five minutes later, you should have an Ubuntu Orchestra Server up and running!

Target System Setup

Once your Orchestra Server is installed, you're ready to prepare your target system for installation. You will need to enter your target system's BIOS settings, and ensure that the system is set to first boot from PXE (netboot), and then to local disk (hdd). Orchestra uses Cobbler (a project maintained by our friends at Fedora) to prepare the network installation using PXE and TFTP, and thus your machine needs to boot from the network. While you're in your BIOS configuration, you might also ensure that Wake on LAN (WoL) is also enabled.

Next, you'll need to obtain the MAC address of the network card in your target system. One of many ways to obtain this is by booting that Ubuntu ISO, pressing ctrl-alt-F2, and running ip addr show.

Now, you should add the system to Cobbler. Ubuntu 11.10 ships a feature called cobbler-enlist that automates this, however, for this guide, we'll use the Cobbler web interface. Give the system a hostname (e.g., asus1), select its profile (e.g., oneiric-x86_64), IP address (e.g. 192.168.1.70), and MAC address (e.g., 00:1a:92:88:b7:d9). In the case of this system, I needed to tweak the Kernel Options, since this machine has more than one attached hard drive, and I want to ensure that Ubuntu installs onto /dev/sdc, so I set the Kernel Options to partman-auto/disk=/dev/sdc. You might have other tweaks on a system-by-system basis that you need or want to adjust here (like IPMI configuration).

Finally, I adjusted my DD-WRT router to add a static lease for my target system, and point dnsmasq to PXE boot against the Orchestra Server. You'll need to do something similar-but-different here, depending on how your network handles DHCP.

NOTE: As of October 27, 2011, Bug #882726 must be manually worked around, though this should be fixed in oneiric-updates any day now. To work around this bug, login to the Orchestra Server and run:

RELEASES=$(distro-info --supported)
ARCHES="x86_64 i386"
KSDIR="/var/lib/orchestra/kickstarts"
for r in $RELEASES; do
  for a in $ARCHES; do
    sudo cobbler profile edit --name="$r-$a" \
        --kickstart="$KSDIR/orchestra.preseed"
  done
done

Target Installation

All set! Now, let's trigger the installation. In the web interface, enable the machine for netbooting.

If you have WoL working for this system, you can even use the web interface to power the system on. If not, you'll need to press the power button yourself.

Now, we can watch the installation remotely, from an SSH session into our Orchestra Server! For extra bling, install these two packages:

sudo apt-get install -y tmux ccze

Now launch byobu-tmux (which handles splits much better than byobu-screen). In the current window, run:

tail -f /var/log/syslog | ccze

Now, split the screen vertically with ctrl-F2. In the new split, run:

sudo tail -f /var/log/squid/access.log | ccze

Move back and forth between splits with shift-F3 and shift-F4. The ccze command colorizes log files.

syslog progress of your installation scrolling by. In the right split, you'll see your squid logs, as your Orchestra server caches the binary deb files it downloads. On your first installation, you'll see a lot of TCP_MISS messages. But if you try this installation a second time, subsequent installs will roll along much faster and you should see lots of TCP_HIT messages.

It takes me about 5 minutes to install these machines with a warm squid cache (and maybe 10 mintues to do that first installation downloading all of those debs over the Internet). More importantly, I have installed as many as 30 machines simultaneously in a little over 5 minutes with a warm cache! I'd love to try more, but that's as much hardware as I've had concurrent access to, at this point.

Post Installation

Most of what you've seen above is the provisioning aspect of Orchestra -- how to get the Ubuntu Server installed to bare metal, over the network, and at scale. Cobbler does much of the hard work there, but remarkably, that's only the first pillar of Orchestra.

What you can do after the system is installed is even more exciting! Each system installed by Orchestra automatically uses rsyslog to push logs back to the Orchestra server. To keep the logs of multiple clients in sync, NTP is installed and running on every Orchestra managed system. The Orchestra Server also includes the Nagios web front end, and each installed client runs a Nagios client. We're working on improving the out-of-the-box Nagios experience for 12.04, but the fundamentals are already there. Orchestra clients are running PowerNap in power-save mode, by default, so that Orchestra installed servers operate as energy efficiently as possible.

Perhaps most importantly, Orchestra can actually serve as a machine provider to Juju, which can then offer complete Service Orchestration to your physical servers. I'll explain in another post soon how to point Juju to your Orchestra infrastructure, and deploy services directly to your bare metal servers.

Questions? Comments?

I won't be able to offer support in the comments below, but if you have questions or comments, drop by the friendly #ubuntu-server IRC channel on irc.freenode.net, where we have at least a dozen Ubuntu Server developers with Orchestra expertise, hanging around and happy to help!

Cheers,

:-Dustin

The Magic Number 4

2011-10-19T13:57:00.000-05:00

We're less than two weeks away from the next Ubuntu Developer Summit, in Orlando, Florida, where nearly 700 techies will define the enterprise Linux landscape for the next decade.

You: "Come on, Dustin, you're being a bit melodramatic, here, no?"

Me: "Heh, if anything, I may be understating the importance of the Ubuntu 12.04 LTS!"

When it comes to enterprise operating systems, there's a certain magic aurora that surrounds the number, "4". Let's take a stroll through enterprise operating systems history...

Anyone here remember Windows NT4? You can hate Microsoft and Windows all you want, but in 1996, NT4 became the first Windows release in 11 years that delivered an enterprise-ready server. I was in high school working for a little PC outfit called Alpha Computer Company in Plaquemine, Louisiana, and we installed NT4 servers by the hundreds. For all its faults and security vulnerabilities, server administration had never been point-and-click easier.

I have infinite respect for RHEL4! I was a Red Hat and Fedora user for 10 years between 1997 and 2006 (when I switched to Ubuntu), and ran nearly every version from Red Hat 5 through Fedora Core 5, as well as RHEL2.1 and RHEL3. It was RHEL4 in 2005 that was pure gold! The features, the stability -- this was the first enterprise Linux release anywhere that was ready for prime time. And it's still a great OS nearly 7 years later. There's no shortage of hosting companies still running RHEL4.x + cPanel out there.

I dabbled in Solaris just a little in high school and eventually in my Computer Science courses at Texas A&M University. Guess what Solaris was called, before it was rebranded in 1993? Yep, SunOS4 became the first Solaris! I dare say that Sun cranked out the dominant UNIX implementation right up until OpenSolaris tanked spectacularly and the aforementioned RHEL4 stole the Linux/UNIX show.

I also served 8 years hard time at IBM, where we danced to a slightly different UNIX tune -- that of AIX. Once again, it was the AIX4 release series that established AIX as a UNIX mainstay and rose to the level of expectations of IBM customers. AIX4 shifted the focus to IBM's innovative PowerPC processors, introduced CDE, IPv6 (remarkably in 1997!), and everyone's favorite text-based system management utility, smitty ;-)

With all this talk about UNIX, we certainly cannot overlook SVR4. UNIX System V Release 4.0 in 1988 was basically the last (SVR5 was a SCO disaster, and SVR6 was cancelled) of the great UNIX specification releases, feeding into all of the proprietary and open UNIX distributions, from Sun, to HP, to IBM, to DEC, to the various BSD derived distributions. SVR4 was the beginning of a new era of UNIX computing, and its legacy runs right up to our doorsteps today.

And here we are, just 6 months away from the fourth Ubuntu LTS. Reflecting back a bit, Ubuntu 6.06 LTS (Dapper) was the first long term supported, enterprise release, and the introduction of Ubuntu as a Server platform. Support for Dapper just ended in June of this year (2011), and provided Ubuntu users with some rock-solid stability, if lacking a bit on some modern Linux features. The Ubuntu 8.04 LTS (Hardy) release (the first cycle on which I worked the Ubuntu Server for Canonical) introduced the enterprise Linux industry to KVM as a hypervisor and refined our ability to deliver a long term supported, heavily QA'd server release. Hardy is still supported for another 1.5 years, and I know of many Ubuntu Server installations happily cranking along on Hardy (including my own divitup.com). Ubuntu 10.04 LTS defined the IaaS cloud market, providing a fully-functional, 100% open source cloud infrastructure with UEC, and absolutely rewrote the industry's books on Linux as a cloud guest operating system.

It's quite easy to see the progression of the Ubuntu LTS Server, from 6.06 to 8.04 to 10.04. With that kind of momentum behind us, coupled with history's emphasis on "4th" releases of operating systems, can you imagine the quality, features, and industry impact of Ubuntu's LTS4? I'm just beginning to wrap my head around it, and it's damn exciting!

Personally, I can't wait for UDS, to help get that chapter of history underway.

Cheers,
:-Dustin

The email I received from Dennis Ritchie (by way of maddog)

2011-10-13T18:46:00.000-05:00

I learned earlier this morning that Dennis Ritchie, one of the fathers of the C programming and UNIX as we know it, passed away. Thank you so much, Mr. Ritchie, for the immeasurable contributions you've made to the modern world of computing! I think I'm gainfully employed and love computer technology in the way I do, and am in no small ways indebted to your innovation and open contributions to that world.

Sadly, I've never met "dmr", but I did have a very small conversation with him, via a mutual friend -- Jon "maddog" Hall (who wrote his own farewell in this heartfelt article).

A couple of years ago, I created the update-motd utility for Ubuntu systems, whereby the "message of the day", traditionally located at /etc/motd could be dynamically generated, rather than a static message composed by the system's administrator. The initial driver for this was Canonical's Landscape project, but numerous others have found it useful, especially in Cloud environments.

A while back, a colleague of mine complemented the sheer simplicity of the idea of placing executable scripts in /etc/update-motd.d/ and collating the results at login into /etc/motd. He asked if any Linux or UNIX distribution had ever provided a simple framework for dynamically generating the MOTD. I've only been around Linux/UNIX for ~15 years, so I really had no idea. This would take a bit of old school research into the origins of the MOTD!

I easily traced it back through every FHS release, back to the old fsstnd-1.0. The earliest reference I could find in print that specifically referred to the path /etc/motd was Using the Unix System by Richard L. Gauthier (1981).

At this point, I reached out to colleagues Rusty Russell and Jon "maddog" Hall, and asked if they could help me a bit more with my search. Rusty said that I would specifically need someone with a beard, and CC'd "maddog" (who I had also emailed :-)

Maddog did a bit of digging himself...if by "digging" you mean emailing the author of C and Unix! I had a smile from ear to ear when this message appeared in my inbox:

Jon 'maddog' Hall to Dustin on Tue, Apr 20, 2010 at 10:08 PM:

> A young friend of mine is investigating the origins of /etc/motd. I
> think he is working on a mechanism to easily update that file.
>
> I think I can remember it in AT&T Unix of 1977, when I joined the labs,
> but we do not know how long it was in Unix before that, and if it was
> inspired by some other system.
>
> Can you help us out with this piece of trivia?

Ah, a softball!
MOTD is quite old. The same thing was in CTSS and then
Multics, and doubtless in other systems. I suspect
even the name is pretty old. It came into Unix early on.

I haven't looked for the best citation, but I bet it's easily
findable: one of the startling things that happened
on CTSS was that someone was editing the password
file (at that time with no encryption) and managed
to save the password file as the MOTD.

Hope you're well,
Regards,
Dennis

Well sure enough, Dennis was (of course) right. The "message of the day" does actually predate UNIX itself! I would eventually find Time-sharing Computer Systems, by Maurice Wilkes (1968), which says:

"There is usually also a message of the day, a feature designed to keep users in touch with new facilities introduced and with other changes in the system"

As well as the Second National Symposium on Engineering Information, New York, October 27, 1965 proceedings:

"When a user sits down at his desk (console), he finds a "message of the day". It is tailored to his specific interests, which are of course known by the system."

Brilliant! So it wasn't so much that update-motd had introduced something that no one had ever thought of, but rather that it had re-introduced an old idea that had long since been forgotten in the annals of UNIX history.

I must express a belated "thank you" to Dennis (and maddog), for the nudges in the right direction. Thank you for so many years of C and UNIX innovation. Few complex technologies have stood the test of time as well as C, UNIX and the internal combustion engine.

RIP, Dennis.

-Dustin

Ubuntu Cloud Live

2011-10-07T13:00:00.000-05:00

This morning, Canonical's CEO Jane Silber is delivering the first keynote address at the incredible OpenStack Conference in Boston, MA. I've spent the entire week here in Boston -- Monday, Tuesday, and Wednesday were dedicated to an Ubuntu-style developer summit, focusing on the next OpenStack release (code named Essex), set for release in early April. This version of OpenStack will form the IaaS basis for the Ubuntu 12.04 LTS server in April 2012.

I saw a preview of Jane's slides yesterday evening, and I'm quite sad that I'm missing her talk (I'm writing this from the Boston/Logan airport on my way back to Austin, TX). Jorge Castro will be posting a video of her talk as soon as he can. I think you'll hear about Jane's vision of a Ubuntu's history of leadership as the best Host and Guest OS in the Cloud, and our revolutionary approach Service Orchestration in the Cloud.

I've also seen a sneak preview of a demo given at the end of the talk. Clint Byrum and Adam Gandelman have worked around the clock producing a spectacular visualization of an Ubuntu Cloud at work. In the front of the stage, we have a portable rack of servers (a 40-core Intel Emerald Ridge, a 24-core HP Proliant, a 16-core Dell Precision, with a System76 local Ubuntu mirror, and Cisco networking hardware). We've used Ubuntu Orchestra to remotely install the systems, and we've deployed OpenStack to the rack. Once OpenStack is running, Clint has a series of Hadoop jobs that he spins up and runs against dozens of instances on the local Nova compute node. And for the real whiz-bang, Clint uses gource for dynamic visualization of the Hadoop cluster, the various nodes, and their relationships. It is absolutely stunning to behold!

We are also giving away a few hundred top notch USB sticks, rubber coated with the Ubuntu brandmark. Ask Robbie Williamson how much he enjoyed dd'ing several hundred ISO images :-) What was he loading onto the stick, you ask?

Rewind back to May 2010, in a 5-minute lightning talk at UDS-Brussels, I demonstrated an Ubuntu LiveISO running the Ubuntu Enterprise Cloud and called it Cloud in your Pocket. A bit later, I reworked that image to support OpenStack too and showed that at the OpenStack Design Summit in San Antonio. I was delighted when a couple of the Canonical OEM Server developers (Ante Karamatic, Dave Medberry, and Al Stone) have picked that work up, and ported it forward to Ubuntu 11.10, Unity, and OpenStack Diablo.

So this morning's OpenStack Conference attendees are walking away with the Ubuntu Cloud Live USB experience! For the rest of you, you can freely download the image yourself, and write that to your own USB stick, or even run it in a virtual machine!

To get started download the image from:

http://people.canonical.com/~akarama/binary.img

We're going to re-roll that image for the 11.10 official GA release. Next, write that image to a USB stick (assuming that USB drive is sdb):

sudo dd if=binary.img of=/dev/sdb

Or just run that image in a virtual machine using TestDrive:

testdrive -u ./binary.img

The image should boot much like an Ubuntu Desktop Live, and you should end up in a very minimal Unity environment, with a command line and a web browser, and not much else. On the desktop, there's a text document with instructions for getting started. We could have automated all of the cloud creation, but we figured it would be educational to leave a few steps for you (key generation, image registration, instance running).

You can watch it here:

http://people.canonical.com/~kirkland/cloud-live.mpg

I'm hoping we contribute Ubuntu Cloud Live to the OpenStack Satellite projects (akin to Ubuntu Universe -- it's not part of Core OpenStack, but it's related and useful to some OpenStack users).

It's quite easy for you to modify and rebuild the Ubuntu Cloud Live image to your uses! That looks something like this...

Install the live-build tools and grab the source code from launchpad.net/cloud-live.

sudo apt-get install live-build
bzr branch lp:cloud-live

Make your changes, if any. And then build.

lb clean
lb build

You'll wait a while. Internet connection speed and CPU/Memory will determine how long the build takes. Eventually, you'll see a file called binary.img. And there you go! You have just re-built the Ubuntu Cloud Live image.

Enjoy!
:-Dustin

eCryptfs in the Wild

2011-09-16T11:57:00.000-05:00

Perhaps you're aware of my involvement in the eCryptfs project, as the maintainer of the ecryptfs-utils userspace tools...

This post is just a collection of some recent news and headlines about the project...

I'm thrilled that eCryptfs' kernel maintainer, Tyler Hicks, joined Canonical's Ubuntu Security Team last month! He'll be working on the usual Security Updates for stable Ubuntu releases, but he'll also be helping develop, triage and fix eCryptfs kernel bugs, both in the Upstream Linux Kernel, and in Ubuntu's downstream Linux kernel packages. Welcome Tyler!
More and more and more products seem to be landing in the market, using eCryptfs encryption! This is, all at the same time, impressive/intimidating/frightening to me :-)

Google's ChromeOS uses eCryptfs to securely store browser cache locally (this feature was in fact modeled after Ubuntu Encrypted Private Directory feature, and the guys over at Google even sent me a Cr48 to play with!)
We've spotted several NAS solutions on the market running eCryptfs, such as this Synology DS1010+ and the BlackArmor NAS 220 from Seagate
Do you know of any others?

I've had several conversations with Android developers recently, who are also quite interested in using eCryptfs to efficiently secure local storage on their devices. As an avid Android user, I'd love to see this!
There's a company here in Austin, called Gazzang, that's developing Cloud Storage solutions (mostly database backends) backed by eCryptfs.
And there's a start-up in the Bay Area investingating eCryptfs + LXC + MongoDB for added security to their personal storage solution.

Exciting times in eCryptfs-land, for sure!

Which brings me to the point of this post... We could really use some more community interaction and developer involvement around eCryptfs!

Do you know anything about encryption?
What about Linux filesystems?
Perhaps you're a user who's interested in helping with some bug triage, or willing to help support some other users?
We have both kernel, and user space bug-fixing and new development to be done!
There's code in both C and Shell that need some love.
Heck, even our documentation has plenty of room for improvement!

If you'd like to get involved, drop by #ecryptfs in irc.oftc.net, and poke kirkland or tyhicks.

Cheers,

:-Dustin

Ubuntu Monospace Font

2011-09-16T08:02:00.000-05:00

At long last, we have a Beta of the Ubuntu Monospace font available! (Request membership to the ubuntu-typeface-interest team in Launchpad for access.)

Here's a screenshot of some code open in Byobu in the new font!

It really has a light, modern feel to it. I like the distinct differences between "0" and "O", and "1" and "l", which are often tricky with monospace fonts. Cheers to the team working on this -- I really appreciate the efforts, and hope these land on the console/tty at some point too!

I've only encountered one bug so far, which looks to have been filed already, so I added a comment to: https://bugs.launchpad.net/ubuntu-font-family/+bug/677134 I think the "i" and "l" are a little too similar. if-fi statements in shell are kind of hard to read.

Anyway, nice job -- looking forward to using this font more in the future!

Enjoy,
:-Dustin

Enterprise Software History

2011-09-15T22:59:00.000-05:00

I've visited the Computer History Museum in Mountain View three times now. I love reading Steven Levy's dramatic biographies of the unsung heroes of technology. Heck, I even took an independent study class at Texas A&M on the History of Mathematics :-) Geek: yes.

Anyway, I recently came across a nice little series of articles about the history of software, specifically:

http://blog.softwareadvice.com/articles/enterprise/software-history-pt1-1082411/
http://blog.softwareadvice.com/articles/enterprise/software-history-part-2-109062011/
http://blog.softwareadvice.com/articles/enterprise/software-history-part-3-109082011/
http://blog.softwareadvice.com/articles/enterprise/software-history-part-4-109142011/

Linux got a mention, but no sign of Ubuntu yet, in the annals of Software History. Best start working harder ;-)

:-Dustin

5.1 Ubuntu Login Sound now in a PPA!

2011-09-03T19:33:00.000-05:00

Thanks for all the positive feedback to my last post! I have made a couple of updates to the 5.1 channel Ubuntu login sound, namely:

Remastered based on the original wav files, since my previous version was based on the lossy, compressed ogg files.
Adjusted a couple of levels, having actually tested it on as many different 5.1 and 2-channel stereo environments I could find.
Updated the ubuntu-sounds package and pushed to bzr and a PPA for easier installation on lucid, maverick, natty, or oneiric!

So now, you can install the 5.1 channel Ubuntu login sound easily from this PPA to any supported Ubuntu release with:

sudo apt-add-repository ppa:kirkland/sound
sudo apt-get update
sudo apt-get install ubuntu-sounds

Log out, and then log back in. If your Ubuntu system is hooked up (correctly) to a 5.1 stereo receiver, then you should hear the login sound start in the center speaker, then spread outwards to the front left and right channels, with the sound moving from the front to the rear for the whoosh and crickets at the end. Oh, and the bongos should be bumpin' in your sub woofer :-)

If you're interested in the sources, they're in bzr too:

bzr branch lp:~kirkland/ubuntu-sounds/834802

Finally, if you'd like to see this land in Ubuntu, mark bug #834802 as "affects me too"!

I'll embed the audio here, but it sounds really different in the various browsers I've tested (Firefox, Chromium, Chrome). Sounds like the the multi-channel OGG is being correctly passed to Pulse Audio for proper downmixing/discrete playback in Firefox, but not in Chrome/Chromium. So your mileage may vary! :-)

Doh! Your browser doesn't support embedded audio :-(

Cheers!
:-Dustin