Contentment

My Very Own Phone System

2010-09-20T01:05:41Z

Okay, so I’ve now hacked my home phone system. The ultimate goal here is to make it so that we don’t pay so much for a land line we hardly use. However, we still want a home phone system. By popular demand (HT: Jim, Melissa), I’m posting what I’ve done so far and what I plan to do still.

Phase 1: Build a PBX

I have this old desktop machine sitting around that has been my guinea pig over time for various projects. It’s a P2 933MHz (or something) and works pretty well in this role for being 8-10 years old. It’s last job was to run Windows XP so I could runt he upgrade software to install Android 2.1 on my phone, though that actually failed because the computer only has USB 1.0 and the upgrade required USB 2.0.

Before that, it was an experiment to setup MythTV, which worked, but I decided I didn’t want to buy the hardware required to make a box work when we almost never watch anything on TV with Netflix and Hulu. That would only be really beneficial if I was more into sports.

Anyway, I digress…

I started off using The Incredible PBX, which is a distribution of Centos Linux, Asterisk, and FreePBX. Setting that up with Google Voice and SIPgate was pretty easy. I just used the instructions at The Incredible PBX.

Once I could get that done. I was ready to move on to phase two.

Phase 2: Installing a VoIP Gateway

I purchased a LinkSys SPA-3102 to handle the VoIP gateway. This box now runs a little over $60 and works as a router with one WAN port and one LAN port (I really only need one of these ports) and two voice ports. One FXS port for connecting to the PSTN (Public Switched Telephone Network) and one FXO port for power the house phone line. Utlimately, the FXS port will be unused, but by having this available, I’m able to install the VoIP system in phases without the risk of the phone system being down for a long time while I get everything figured out. If nothing else, I can always reset the box to the factory original settings and it works just like it wasn’t there.

In the process, I had to correct some deficiencies in my home phone network, which is pretty badly spliced together. It’s still badly spliced, but it’s a little better now.

Once it arrived, I installed the VoIP gateway and it just worked, which was pretty nice of it.

Phase 3: VoIP Gateway and PBX Together

I have just completed phase 3 of the process, which is to set up the trunk, extension, and routes in FreePBX/Asterisk and then setup the SPA-3102 so that it talks through the Asterisk server. This was pretty complicated. I found a couple different sites to help, but a HOWTO on FreePBX’s site got me about 95% there.

It did take some experimentation and I did need to setup the Line 1 interface as an extension too, but it works now. When a call comes in, it gets routed into the 700 extension group, which rings the house extension and any softphones we want to setup. I have it setup so that dialing out routes back out of this line, and it works. I also made sure to create a special emergency route that allows 911 calls to be routed out properly as well.

Still to Come

Okay, so this gets me to the point where I can start to experiment with the PBX to see what we can do with it. Things that come to mind are setting up night mode, where we can keep the phones from ringing after the kids are in bed unless the caller dials through a screening process. I can also setup dial-out through other services, which I’ve started to do, but haven’t finished. We also have fancy voice mail if we want to use it instead of the answering machine, we can have music on hold (woo-woo), etc. I haven’t figured out quite all I want to do with the box yet.

However, the ultimate goal is to reduce the cost of this line, which means I need to port the phone number to some other phone service that’s cheaper than the PSTN. The prime candidate at the moment is Twilio. They have an app called OpenVBX, which sets up a private service that resembles Google Voice, but much more configurable. I can use that to ring a my phone at SIPgate, which will ring the house phone. Then, I can configure even more fancy things on the OpenVBX server, which will give me voicemail with SMS and email integration and transcriptions. I can ring our cell phones when someone calls the house and get all kinds of other whizbangs, for a $1/mo phone number + a few cents a minute in charges.

So, that’s what I’m experimenting with now.

Cheers.

OWASP Top Ten

2010-07-31T12:20:17Z

If you do any kind of application programming, particular network oriented applications or web applications (who isn’t?), you should definitely be aware of OWASP. They keep a list of the Top Ten security concerns when developing software and I have found the reference to be excellent.

Site Link: http://www.owasp.org/index.php/Category:OWASPTopTen_Project
PDF Link: http://owasptop10.googlecode.com/files/OWASP%20Top%2010%20-%202010.pdf

I think a printed version of the updated Top Ten is going to become a part of my standard reference kit to look over from time to time. The guide is organized and color coded in such away that just about anyone could understand why you should be concerned and what the remedies for each of the top ten issues. It provides additional links off to further information and even a standard reference library that provides an implementation reference for best practices.

Cheers.

The Danger of DWIM

2010-07-26T19:44:57Z

Perl libraries often attempt to do what you want (or DWIM, Do What I Mean) when it comes to deciding how to handle input and output. However, this can have unintended consequences if you’re not careful with these conveniences. I’m going to pick on an example I ran into today:

use Data::Dumper;
use XML::Simple;
my $data = XMLin($ARGV[0]);
print Dumper($data);

What’s that do? It takes the input from @ARGV and parses it as XML, right? It returns that information as a hash, right? Maybe.

If this were a little script named test.pl and I ran this:

./test.pl "foo"

I would end up with output of:

$VAR1 = { value => 'foo' };

However, if you just ran:

./test.pl

what would happen? It would try to find and open a file named test.xml.

Or I could run it like this:

cat some.xml | ./test.pl -

That would cause it to read in standard input. Okay, great! That’s pretty handy, except…

Imagine if you were using this in a web application. Suppose you have an upload widget where someone uploads an XML file to your app and then you pass that value straight into XMLin() without checking, this could be very bad.

my $xml = $c->parameters->{xml_file};
my $data = XMLin($xml);
print Dumper($data);

What if your user uploads a file that contains the string “/root/secret.xml” and that happens to refer to a real file that is secret?! You just passed back the information in that file.

Beware of the DWIM! Always check your inputs! In the case of the web app, I would recommend against any use of XMLin() directly and create some sort of wrapper that will only allow the input you expect:

sub parse_xml_content { 
    my ($xml, @options) = @_;
    die 'input does not look like XML' unless $xml =~ /<.*?>/; 
    return XMLin($xml, @options);
}

This doesn’t go just for XML::Simple, but for just about anything that is smart with input.

Cheers.

YAPC 2010 in Review

2010-06-26T16:06:44Z

A few days with the JAPHs has reminded me of a few things:

I’m behind. I need to do a better job of keeping up with Perl things.
Perl has a unique culture and community that is both interesting and valuable to what it produces.
I’m so glad that the people I usually hang out with do not use a certain expletive as their favorite adjective.

Okay, so let’s review.

Sunday

I traveled to Columbus, Ohio on Sunday. My travel was mostly uneventful, other than an hour delay in Dallas to replace the brakes on the plane. Because of that, I didn’t get to have dinner with Terri’s cousins, just her uncle. However, I enjoyed a delicious meatloaf and lemonade with Keith at the Cap City Diner (warning the web site plays music). I arrived at The Blackwell that evening, which is apparently named after a professor at OSU who has been convicted of insider trading.

My first impression of Ohio State campus is that they must have had a beautiful football stadium at one time, but now appears to be just a modern monstrosity with a beautiful arch on the end facing The Blackwell. I suppose this stadium will soon be featuring regular visits from our northern neighbors in Nebraska… I’m not an OSU fan, but I hope OSU beats NU every time they play here.

While traveling and late into Sunday evening, I finished up my notes for my first talk.

Monday

I met my colleagues in the lobby and we walked together over to the Ohio Union, where the conference was being held. The Union is knew and was a very nice place to conference, particular one priced at the value that YAPC is—though, my conference registration was paid for by the low-low price of giving two talks, which is great since I was just talking about two subjects I like to talk to others about anyway.

The keynote on Monday was given by Jesse Vincent, who was the Perl5 v12 pumpking, which has been released and is well into it’s maintenance cycle now. For those that may not know, this marks the second major release of Perl in the last 3 years and is part of a new effort to consistently make a major release of the language every year. This is good news for at least a couple reasons.

Perl language developing will stay fresher with new features coming regularly.
It will help dispel the message of the naysayers and show that Perl is Alive.

After that, I went to a couple talks on Catalyst that I didn’t pay much attention to. This had far less to do with the quality of the talks and more to do with me thinking about my talk.

I gave my first talk at 10:00 am on Telecommuting. I will post the audio/video when Krishna gets it online. I may post the slides too, but there’s really not much to show in them. Most of it was me telling a few stories from note cards. I feel like I was a bit dull, but I got some good questions at the end, which is usually a good sign. As I said on Twitter, I hope it was useful.

After breathing a sigh of relief I spent most of the rest of the day in talks and working on a project for work. I attended Util’s talk on Amazon EC2. I learned that EC2 hasn’t changed significantly since I used it. It’s still the same, though there are more instance configuration options, Windows instances, and different storage engines. This is mostly just an extension of what was there when I was last using it a couple years ago.

We then grabbed some lunch from The Flying Pizza, which was cheap and greasy and pretty good.

After lunch, I ended up in Moose for Managers, which wasn’t a stand-out talk in my mind since I can’t remember anything about it. However, I was working on something related to work at the time, so that’s probably just me not paying close enough attention. I stayed in the Grant Street Room to listen to Paul Fenwick’s talk on Awesome things you’ve missed in Perl. Paul is a great speaker and showed a number of things I’ve put on my todo list to investigate.

I ended the regular talks for the day by attending Dave Rolsky’s talk on Fey and Fey::ORM, which I’m mildly curious about since I once tried to write an ORM and because Fey seems like something which could be useful for something at work. By the way, my attempt at an ORM was awful and Dave actually wrote a scathing review of it on CPAN Ratings, which is part of the reason it has been disappeared from CPAN…I think that may have been my first CPAN module. Oh well. No hard feelings.

Finally, there were the lightning talks. They were enjoyable, as usual, but I’m not sure there are any that stand out in my mind from Monday.

After that, my fellow Grant Streeters headed out to dinner. We went to a local, cheap Chinese place, which was pretty decent. I stayed up late working after that, though after all the talks and travel I was pretty spent, so I got done far less than I wanted.

Tuesday

I ended up sleeping in a bit and missed the first round of talks and caught only the last part of Michael Schwern’s talk on the coding cycle. I stayed in the Grant Street room to see Gabor Szabo’s talk on Padre, which I’ve long been interested in, but still haven’t looked at much. If you want to learn Perl, using this editor sounds like the best way to get started.

Then it was lunch again, at Chipotle this time. Mmmm… Chipotle.

I attended scrottie’s talk on running Perl in a cheap distributed computing system (i.e., a pile of cheap computers working together to form a supercomputer of sorts). This was followed by Nick Perez giving a talk on how he’s building apps using POE and Plack.

I moved downstairs to another Schwern talk, this time on perl5i, which is a pretty interesting project. He’s basically taken to pulling in his favorite language extensions and piled them into a single module, perl5i. I’m still thinking about how it works and wondering if I like it or not. I certainly like most of the modules he’s brought in and how it works. I’m just not certain I like the mo and mc meta-accessors… still considering it.

After that talk, I stepped out for a while and worked a bit and chatted with some folks in the lobby of the Union. I met a lot of great folks while here, too many to mention all of them.

Then, I attended the keynote, which was given by Stevan Little. This talk explored the history of Perl and how we got to Modern Perl today. Using some concepts by Larry Wall, he wanted to show how Perl had filled a void that once existed in Unix. That is, it used to be if you wanted manipulexity (lots of deep functionality) you wrote something in C. If you needed something with whipuptitude (built quickly and easily) you used shell script. Perl was an attempt to fill in the difference so you could have both manipulexity and whipuptitude. Modern Perl is the outcome and successful implementation of this.

This is also a good opportunity to explain that while Perl has long had the slogan TMTOWTDI (There’s More Than One Way To Do It) from Larry Wall, Modern Perl makes it longer with TMTOWTDI BSCINABTE. This is pronounced Tim Toady Bicarbonate, which acronymizes, “There’s More Than One Way To Do It But Sometimes Consistency Is Not A Bad Thing Either.” This is an extension to Perl thinking that basically adds a little more consistency to programming (via Moose and related thinking) without actually losing any flexibility. (In fact, Modern Perl is really much more flexible in many ways, it just adds a common foundation that makes that flexibility a little more solid and less re-inventive.)

After this came the second round of lightning talks. There were again several good ones, but I think the talk that outshined them all was the one by Makoto Nozaki titled “How I mastered English with Perl.” It was great. I hope the video of it is posted so I can watch it again.

After this, we had the banquet, where I sat with one of my coworkers, Dieter, some of the folks from Best Practical (Shawn Moore, Kevin Falcone, and Jesse Vincent) and got to meet a couple others, Elliot and Ingy. I had a pretty decent time learning a bit about others, talking with Elliot and Jonathan Rockway in the food line, and made a smallish bid on some coasters during the auction. Pathetic, I know, but I did by a new Perl T-shirt since my Perl polo I bought at OSCON is starting to wear.

After that, I went with some folks and ended up talking about work with my coworkers at a local bar before heading back to my room where I stayed up late inventing a new talk to replace the one I wasn’t able to get together in time.

Wednesday

I started the morning by attending another Schwern talk on gitPAN, which is a really cool way of mining information about CPAN using github. This was followed by a talk by Tatsuhiko Miyagawa on cpanminus, which I had only heard rumors of up until this point. Now that I’ve tried it out, I’m not sure why I haven’t been using it for months now. It’s a very nice (simple!) CPAN client that handles most of what you need when you just want to install a module to try it out.

I then gave my second and final talk on Form::Factory, which showed a bit on how to extend Moose and how I like to build forms, but I don’t think it’s earth shattering. I would like it to be, but my time to hack is constrained by so many things.

I followed that with mst’s talk on The Troll, the God, and the Mountain, which was a very unusual introduction to DBIx::Data::Store. The punchline is that the Troll, Thog and his friend Al, are renamed and made great by the god v10 (“Voton,” i.e., Perl 8 v10) to become “Orthog-an-Al Persisten-T roles.” That’s about all I remember, to be honest. I’ve been hearing a few rumors about this project via Rob Kinyon (I think), though, so I’m keeping an eye open for when it hits CPAN.

I had one more lunch at YAPC, with a bunch of folks I didn’t know and Rob. I confess to have forgotten all names at this lunch, but they were good company for lunch.

Next, I attended Cool Perl 6 today by Patrick Michaud, who is the lead on Rakudo, which is the engine for Perl 6. I’m really thinking Perl 6 should be renamed to something else because it’s like Perl, but it’s not Perl. I like to watch these talks mostly for fun, though, things are getting to the point now that you can write Perl 6 applications, which is tempting to try.

I then attended a really excellent talk on autodie, titled “The Art of Klingon Programming” by Paul Fenwick. The talk featured Paul in a TNG uniform talking about his time as an exchange officer in the Klingon empire where he was testing the new universal translator. The joke is this. If you’ve programmed very much Perl, you know a common idiom is to write “open or die.” The reason is that if the file doesn’t open it returns a false value, so the short circuit “or” operator allows you to skip the die on success or run the die on failure. This is a good way to ask for blood wine in Klingon, since trying to ask nicely won’t get you any. The autodie module comes along to make this kind of talk automatic (making open die without the “or die” on the end).

Prior to the final keynote by mst, I attended two more Moose talks, one by doy and the other by perigrin. Both had way more information than I was really able to consume and the major thing that I took away was that I need to look into the new declarative syntax and immerse myself more deeply into the Moose type system, which I’ve really barely touched.

The final keynote was given by mst, which was another summary of the history of Perl as well as a general summary of some of the good things that have happened in the world of Perl in the past year.

After that, I went over to the Blue Nile Ethiopian Restaurant with the other speakers (and hangers on) and experienced my first taste of Ethiopian food and had one more social event. I got to speak a bit with chromatic, whom had edited the articles O’Reilly published to OnLamp. I got to chat with Ricardo Signes, Dave Rolsky, Rob’s wife, Randall Schwartz, and several others. It was a good time.

Finally, it was time to pack and head home.

In Summary

Perl has a really interesting culture. Between the libraries on CPAN, the dedication of many developers to the language and the community, and the way the leaders in the community interact with it, Perl has a lot going for it. It almost seems like there is a bit of religious fervor to it, though, that’s partially just the concentration of people getting together to talk about Perl. Most of these folks work in other languages too and have important concerns outside of Perl.

On my list of things I need to look into from the conference:

Perlbrew: A nice way of installing Perl VMs for development and testing.
local::lib: A nice way of locating and using CPAN modules installed to your home directory.
cpanminus: A tool that simplifies installing modules from CPAN.
Bread::Board: A framework for inversion of control, which is something I want to do better with in Form::Factory
CPAN Explorer: A visualization of CPAN.
Reflex: A new syntax for doing POE-like things with Moose.

There are others, but I don’t want the list to get too long.

Anyway, that’s my summary of everything I have to say about YAPC::NA 2010. Lord willing, I hope to attend again.

Cheers.

Samsung Moment: Best Phone I Ever Had

2010-06-03T23:00:00Z

I have no idea how this phone compares to other Android-based phones. I’m not going to compare it. However, compared to the Motorola Q9c running Windows Mobile, the Treo running Palm OS, and the Nokia 3650 running Symbian, this is clearly the best phone I’ve owned. I’ve now been using it for a couple months and I still make good use of it on a daily basis. I think it of it more as a small laptop than as a cell phone. I can definitely see what the appeal is for having a tablet like this with a larger screen now. My next laptop may be one of these mini-laptop/tablets running Android I see coming out soon.

So far the only downside has been the battery life, which is mostly a problem when I travel. However, I can use my mini laptop (with it’s very nice 6-cell battery) as a charger. The other downside I experienced was that Sprint installs a bunch of apps I don’t need or want running in the background. However, since I rooted the phone, I’ve solved that problem rather nicely and have extended my battery life a bit in the process.

One of the main thing that makes Android nice is all the apps. So, I’m going to list my favorite apps now. I have added QR codes to each in the Android Market, so you can get to them easily if you have a barcode scanner on your phone. If you don’t, I recommend ShopSavvy below or Barcode Scanner also works.

Auto Memory Manager

By Mad Squirrel. This app is, on it’s own, worth the effort of rooting your phone. Android allows you to run multiple apps at once. You can see the last 6 or so such apps by holding down the Home key. When too many apps are running, Android will kill of the lowest priority ones to keep the phone from getting too hot and running down the battery. Sometimes this isn’t aggressive enough.

There are apps you can get (such as the EStrongs Task Manager) that will kill off running apps on top of the built-in task manager, but why run an app to do that when you can just adjust the built-in one? That’s what Auto Memory Manager does. It lets you tweak the settings of the built-in task manager. You need root to your phone to do it, but it’s worth it. I keep AMM on the minimum settings (only slightly more aggressive than the factory settings) and it makes a big difference.

LauncherPro

By Federico Carnales. Android’s equivalent to the Start Menu is the launcher menu. It’s a little drawer you slide up with your finger from the bottom. The built in one is kind of boring. LauncherPro replaces this with one that gives you customizable buttons at the bottom and gives you the option to have a larger home screen. The animations and such are very smooth and I think it brings up the menu faster than the built-in one.

Bible (YouVersion)

By LifeChurch.tv. This is one of the two Bible programs I have installed. This one uses the Internet connection to pull Bible verses from online servers. I don’t like the reader as much as the one from Olive Tree, but it has some nice reading plan helpers that I use.

ESV for BibleReader

By Olive Tree BibleReader. This is a full blown Bible study program for the phone. I paid $10 for the ESV version, which is now installed on the phone directly. You can also purchase additional books and study helps if you chooes. I’ve installed a few of the free ones. Navigating the Bible is very easy with this and the text is nicely formatted. The main downside is navigating any other text is not easy and it has a few rough edges, like not remembering what you were last looking at if you don’t quit through the menu and the bookmarks are nearly useless.

WeatherBug Elite

By WeatherBug Mobile. I’ve tried Weather.com (which comes with the phone) and AccuWeather and WeatherBug and WeatherBug wins, easily, as the best weather app and widget set. I can now get a nice weather map on my home screen, the current conditions, and 2 day forecast at a glimpse. It also keeps the current temp up in the corner, which is nice. This one is less quirky than the other weather apps I’ve tried too. It was easily worth the $3 I paid for the version without ads and the extra widgets.

ColorDict Dictionary

By Notes. I’m a word geek. I look up words in the dictionary almost daily. This provides a dictionary installed on the phone so it works even if I don’t have a connection. It has a variety dictionaries available and a Wikipedia add-on that will look things up there when the Internet connection is available.

K-9 Mail

By K-9 Dog Walkers. Best IMAP mail reader on Android. This was the only app I knew about before I got an Android phone. One of my Perl-geek acquaintances, Jesse Vincent, wrote this to replace the built-in Mail app. It is very configurable and much nicer in nearly every way than the built-in IMAP mail program. I didn’t like some features of it at first until I learned that I’d just missed one of the settings screens.

Google Voice

By Google Inc. This is another excellent app for Android. This allows me to send all of my calls from my phone through my Google Voice account automatically. I already use this for voicemail, so I get my voicemail online and I get a reasonably useful text transcript of the voicemail. And that’s just some of the features.

AnyPost

By skAmped. I use this to send pictures and status updates to Twitter and Facebook from my phone. It connects to ping.fm. I prefer it to the Hootsuite App (which still only does Twitter) and just about anything else I’ve tried. This app is very simple and just does what it needs. Good enough.

Dolphin Browser HD

By Dolphin Browser. This is a new browser for the phone. The built-in browser is fine, but this one just adds a few extras that make it worth it. For example, with an add-on it will save bookmarks to the SD card so that you won’t lose them when you do an OS update or get the phone repaired.

Google Maps / Navigation

By Google Inc. The phone comes with Sprint Navigation. That application does the job, barely. Google Maps with Navigation does the job well, very well. I bought a dash-mount to put into the van because this app actually works to navigate me. With Sprint Navigation, I was always waiting forever for it to finish loading the route from the online service and if it thought you got off track (which is inevitable with the weak GPS chips in these phones) it would lock up trying to reroute. Google Navigation loads the map and then is smart enough to reroute in a second or two on it’s own without contact the server again.

Genial Writing

By zenpie studio. When I first downloaded this app, I just wanted to see what it was since it looked interesting. Now, I use it to take notes more often than my other note taker app. It lets you take notes using your own “handwriting.” This is very appealing for a number of reasons. This is a purely emotionally satisfying solution, even though it lacks certain practical features like being able to search (how can it search when all it does is record your writing). I still recommend it. It’s fun to use.

ColorNote Notepad

By Notes. This is my favorite general note taker app. I use it for shopping lists and reminders that I want posted to the home screen. It’s handy.

Handcent SMS

By handcent_admin. I’m not totally sold on this app yet. It adds a bunch of extra junk that isn’t related to SMS (font packs, it’s own voice input, it’s own notification system, etc.) I also find the default colors and theming to be very ugly. I didn’t buy an iPhone and I don’t really think apps on my phone need to look like one.

It is an improvement over the built-in SMS Messenger in some small ways, like the fact that it gives you a popup for quick replies and lets you do some nice per-contact customizations.

StopWatch

By sportstracklive.com. I use this for lots of things. If I need to time something or if I need a countdown to something (like Terri has asked me to take something out of the oven), this is a very nice app. It has a nice large timer display, allows for multiple countdowns, and has a button for recording laps (which I mostly use to help me track time while grilling).

Droid48

By [shagrath])(http://twitter.com/shagr4th). The HP 48 is the Best. Calculator. Ever. This provides the classic calculator with RPN and everything. I wish it handled the (real) keyboard a little better, but I like the app anyway.

Congress

By Sunlight Labs. This is a beautiful app for keeping track of what your local Congressmen (or anyone’s Congressmen) are up to. It provides news about them, easy access to their Twitter and Youtube feeds, and information about bills they are currently sponsoring. If you want to stay up to date on national politics, this is great.

EStrongs File Explorer

By EStrongs. I’ve tried a few different SD card browsers and this is probably my favorite, though Astro is a close second.

EStrongs Task Manager

By EStrongs. This is the task manager that goes with the file browser. It’s pretty standard and I use it to kill off apps every now and then.

ConnectBot

By Kenny Root and Jeffrey Sharkey. Gives you a terminal on the local machine (i.e., on the phone) so I can execute Linux programs and such locally. It also gives me SSH in case I need to manage one of my servers.

TripIt - Travel Organizer

By TripIt, Inc. I organize all my trip itineraries on TripIt.com. This brings the web site to my phone, which is very helpful on occasion when I need to remember where I’m going, what my confirmation numbers are, or whatever.

Twitter

By Twitter, Inc. The official Twitter app is better for checking my Twitter account and getting updates when someone mentions me than anything else I’ve tried. It works really well, actually. I wish the Facebook app was half as nice.

ShopSavvy

By Big In Japan, Inc. This is the best bar code reader and shopping helper I’ve found for Android. It’s better than Google Shopper. You can use it to enter a search for a product or scan a bar code and then it will search online for Internet stores and local shops that have it and help you compare prices. Very nice.

Aldiko Book Reader

By Aldiko Limited. I don’t have a ton of e-books, but this has a nice library of public domain books, many of which are on my list of books I want to read. Currently, I’m reading Alice in Wonderland. This has a very nice interface and works well. I wish the Bible readers I had worked like this app does.

AndChat

By Al. R. This is an IRC client. It works very well. I don’t use it quite as often as I would like to, but it is a good client in addition to my usual lurker connection (I log a number of IRC channels that I rarely interact with anymore).

eBuddy Messenger

By eBuddy. This is a decent IM client. I do not like the user interface of this one as much as the Meebo IM client. However, it does not use up battery like Meebo does and Meebo does not seem to maintain the Android app very well.

Movies

By Flixster Inc. I use this to help organize our Netflix account and check for movies in the unusual case I’m actually thinking of going to one. This actually provides a nicer interface to Netflix than the Netflix web site (which is what I say about just about every alternative web site for accessing Netflix since I find the main web site of Netflix to be impossible to navigate).

Facebook for Android

By Facebook. This is the official Facebook app. I use it because it’s about the best there is, but it’s pretty bad. The notifications work every month or so. The UI uses a home page rather than the more usual menu or tab-based navigation. Clicking on notifications (when they come) or on many of the links within the app take you to the Facebook web site in the browser instead of the screen that exists within the app. When it takes you to the web site, it doesn’t even take you to the touch-site (the one made for iPhones and Android, etc.), but to the standard mobile site.

Robo Defense

By Lupis Labs Software. This was the first game I tried on the phone and the one I’ve liked the best. It is extremely repetitive, but I’ve enjoyed in anyway. You have to protect one or more gates to your base by placing defense towers around it. Each game has 100 waves to it that steadily get harder. The thing that made me keep playing is that there are number of achievements you can get for doing certain things and you use points earned to buy upgrades. I’ve mostly exhausted these, so I haven’t played much recently, but it was a fun game.

Blow Up

By Camel Games. This is another fun game. In this game, there are little stick buildings that you blow up with dynamite. The goal is to get all the pieces to settle in as small a heap as possible. In addition, you have a stuffed panda that you need to get to fall toward a certain spot on each level. Gabe likes to push the button that causes the dynamite to blow and especially likes it if I put the dynamite on the panda and blow it up.

YAPC 2010 Telecommute Talk

2010-05-13T02:25:14Z

Help! Help! I need help!

I’m giving a talk at YAPC 2010 on telecommuting as a software developer. For the talk to be successful, I’m going to need more than just my own experience to pull from. If you are a telecommuter and have a moment, I would appreciate your help in filling out this survey.

The goal of the talk is two-fold:

I want to explain the benefits and risks of telecommuting as a Perl developer.
I want to provide some practical tips to help fellow telecommuters.

I want the format to be oriented towards telling a number of stories about telecommuters with practical advice interspersed.

Perl Telecommute Survey

Books are Better

2009-12-20T22:08:22Z

I was asked the other day why I prefer real paper books over electronic books the other day and realized I’d never really posted on this subject, which I feel somewhat strongly about. I have a fundamental distrust of computers and technology. That may seem a little odd coming from a software developer, but the more I learn about software development, the more this distrust grows. When it comes to books and printed materials, I can condense this distrust into three basic problems:

Digital technology changes too much.
Digitized data is fragile and transient.
Digital printing is not self-sufficient.

Ever-Changing Technology

Ever tried to open a document you created 10 or 20 or 30 years ago? Did you succeed? Chances are you didn’t. Ten years ago, data was saved on 3.5” floppy disks or a CD-R if you had the money for a burner. (The USB drive didn’t come into being until 2000.) Do you even have a 3.5” floppy disk drive in your house? Is the disk or CD-R even readable anymore (since the usual half-life of such things has already past)?

Twenty years ago, you would have used a 3.5” or 5.25” disk. Have you even seen a 5.25” drive in the past 10 years?

Thirty years ago, you would have used… who knows what. I was two years old. I’d have to look at Wikipedia to even get a good guess going. Audio cassettes on your Atari? Yeah, like you have something to read one of those.

Even if you read the media, do you have a program that works anymore to read the format? Do you have something that will read whatever word processor you had then? This is possible and maybe even likely, but even so there’s the risk it won’t work or you don’t have one. Formats change. Programs must be made to read a format. If you don’t have any such program, you might be able to get it to someone who can extra something out of it, but it’s probably gone.

However, I can pick up a book from any of those eras and read them today. A notebook, a journal, a magazine, a newspaper are all quite readable with modern day technology. If you printed your document and placed it somewhere dry and clean, you can still read it.

Fragile and Transient

I already mentioned that the old media fails rather frequently, but it gets worse. Digital data is, by necessity, encoded in an unnatural way. That is, digital data is very robust and easy to process is many ways, but the way we choose to represent digital data is made up.

To demonstrate this problem, let’s consider a manuscript that someone finds in the desert from 2,000 years ago. Let’s say that it’s in a script that no one has ever seen before, will it be understood? Actually, it might be. If that script happens to be similar enough to a known script, a linguist might be able to extrapolate at least something about the contents of the document. All languages and scripts are related, even constructed languages like Esperanto.

Now, imagine the same scenario 2,000 years in the future. Someone picks up a magnetic tape that still has some good sections on it. In the future, they know something about ASCII encoding, but not some of the other encodings that were used today. This one is one of those other encodings. They’d have to use some sort of statistically analysis and hope it’s written in a language they know with text strings they can find or something. They’re going to have a much harder time of it reconstructing the data.

Digital formats add another layer of encoding upon already encoded information. Usually multiple layers of encoding are involved. This makes maintenance and upkeep much higher than for books.

A printed book’s encodings are just the script and language it’s in without the extra layers of encoding, file formats, compression, etc. Digital data has many more things to fail along the way to prevent easy reading in the future.

Self-Insufficient

This is really the most important reason I like books. I can pick one off the shelf, open it to a page, and immediately examine it’s contents. If I have a book in my computer, I have to boot the computer, load the right program for reading it, and read it. If I put it on a flash drive or SD card, I have the full contents of the text with me, but I can’t do anything with it until I get a computer or cell phone or a Kindle or something capable of showing me the data.

A book requires no translator or reader to explain it to me. It requires no special equipment to read other than my hands and eyes.

In the end, a book is something I am sure I can pass on to my children as their inheritance or share with someone else without any additional effort. I rarely have that kind of guarantee when sharing a digital file or something. The digital copy of a book gives me the ability to search or process data in it, but if I just want to read, reference, or consume the content of a book, the printed page is always better.

Cheers.

Form Factory: A Better Way (I Think)

2009-12-09T17:53:33Z

Recently, I’ve started backing away from Jifty as my web framework of favor. There are a few reasons for this, but mostly it’s related to performance. Jifty is includes everything, Ajax, dispatch, ORM, Comet/PubSub, REST, forms, HTML, templating, and a pony (probably a kitchen sink too). Basically, with Jifty you don’t need to spend very much time working on building up your own utilities or framework upon framework to fit your style or needs. Jifty hands you a style as well, one that I mostly like.

However, the cost to performance on my Qublog project was too much. It was easier for me to port it to Catalyst sans-Ajax than it was to try and correct the performance issues. In addition, I no longer use Jifty for work, wanted to use features of DBIx::Class, Moose, etc. I’m not trying to bag on Jifty here, just explain why I’m moving away from it general. I very well may find a use for it for something in the future.

There is, however, a major feature of Jifty I miss after having moved to Catalyst. Jifty has a collection of classes for encapsulating form rendering and processing. At the center of this is the action class, which is really a glorified functor, but with all the accoutrement required to render an HTML form and to clean up, verify, and process submission. I like this. I like this a lot. I decided to use the flexibility afforded by Moose with an implementation of something like these action classes to create a new form handling system. Over my Thanksgiving vacation, I was able to build up what has become Form::Factory.

An Example

Okay, I thought about showing you the process I worked through to get here, but you probably want to see it in action first. I’ll work backwards from there.

In order to use Form::Factory, you must first define an action class. Here’s a simple login action you might use to present a login form to your users in a web application:

package MyApp::Action::Login;
use Form::Factory::Processor;

has_control login_name => (
    control => 'text',
    features => {
        required => 1,
        trim => 1,
        length => {
            minimum => 3,
            maximum => 20,
        },
        match_regex => {
            regex => qr{.@.}, # really naive check for email address-ish thing
            message => qw{your %s should be an email address},
        },
    },
);

has_control password => (
    control => 'password',
    features => {
        required => 1,
        trim => 1,
    },
);

has c => (
    is => 'ro',
    isa => 'MyApp',
    required => 1,
);

sub run {
    my $self = shift;

    if ($self->c->authenticate({ name => $self->login_name, password => $self->password })) {
        $self->result->success("welcome back,  @{[$self->login_name]}!");
    }
    else {
        $self->result->failure('the login name and password you entered is wrong');
    }
}

The has_control calls establish the inputs expected for our login action and tell the action what features those controls have. For example, the required features will result in a form that reports an error for that control when no value is given during a submission.

The run method here actually performs the action. (For this example, I’m using something Catalyst-ish, but this could just as easily be used with CGI, CGI::Application, Plack, Jifty, or whatever you prefer.)

To use the login action in our web application, we’ll now need to send the form to the user. Here’s a Template::Declare template that should work:

template 'user/login' => sub {
    my ($self, $c) = @_;

    page_wrapper {
        form { { action is 'user/check_login', method is 'POST' }
            my $interface = Form::Factory->new_interface(HTML => {
                renderer => sub { outs_raw(@_) },
            });
            my $action = $interface->new_action('MyApp::Action::Login', { c => $c });
            $action->unstash('login');
            $action->render;
            $action->clear_messages;
            $action->stash('login');
            $action->render_control(button => { name => 'login' });
        };
    };
};

That would render an HTML form that the user can then fill in and submit. The new_interface call creates an instance of an interface class. These are responsible for connecting an object to the user interface. In this case, it’s an HTML form.

The new_action call then creates an instance of the action class associated with our interface class. Note that we can pass a hashref of arguments that will be passed to the constructor.

The unstash and stash calls are handy for remembering certain things about the form. This includes any error or success messages associated with processing the form. For example, if the user entered their login name, but the incorrect password. The processing bit (which we’ll see in a moment) would stash the error from trying to authenticate and would also stash the login name itself (but not the password since passwords controls don’t ask to stash their value). Then, when the form draws again, teh error message will be shown and the login name box will come prefilled with the previous entry.

The clear_messages call clears the messages from the action so that we can stash a clean slate for the action (we don’t want to see a message we already displayed again).

The render method renders all the controls associated with the action. (By passing the controls option you can output only a subset if you wish.) The render_control is used to output a button that is not directly associated with the action (in this case we just use it to get the user to send the form back to us).

Once the user clicks on our “Login” button, we’ll then want to process the input. That looks something like this:

sub check_user_login :Path('user/check_login') :Args(0) {
    my ($self, $c) = @_;

    my $interface = Form::Factory->new_interface('HTML');
    my $action = $interface->new_action('MyApp::Action::Login', { c => $c });
    $action->unstash('login');

    $action->consume_and_clean_and_check_and_process(
        request => $c->request->params,
    );

    if ($action->is_valid and $action->is_success) {
        $c->response->redirect('/index');
    }
    else {
        $action->stash('login');
        $c->response->redirect('/user/login');
    }
}

We get the interface and action objects same as before. We unstash the saved information again. Then, we do that long method, which is a shortcut for saying:

$action->consume( request => $c->request->params );
$action->clean;
$action->check;
$action->process;

The consume method takes the input and applies that to each control object.

The clean method then cleans up the values given to each control. In this case, for example, the login_name will have whitespace trimmed from the beginning and end since it uses the trim feature.

The check method then validates the inputs to determine whether or not we can reasonably expect the user’s input to have a chance at working. Here’s where most of the features added above will do their work. If the user does not enter anything for the username, the required feature will cause the control to be set as invalid.

Then, the process method will cause the action to process. Actually, the process method itself first checks to see if the input is valid and does nothing if it is not. If it is valid, it will copy the values from the control objects into the action attributes and run the run method defined on the action.

Finally, we can check the success or failure of the action and react accordingly. That’s how Form::Factory works in a nutshell.

Goals

Now that you’ve seen an example, here’s what I want to achieve in a nutshell:

Actions should be well-structured, but very simple to build.
Actions are tied to the user interface by controls, which are easy to extend.
Common features of actions should be simple to define and associate with actions.
User interfaces should be easy to build for hosting actions.

Actions are Well-Structured, Simple

Actions are just functors, function objects. At the center of each is a run method responsible for taking some action. To get there, however, you need input and after you get there you need output. Form::Factory uses Moose attributes to specify what input the action takes. Each has_control builds an attribute onto the action, but also associates more information about how it interacts with the user.

The output is a little less formally specified at this point, but comes out through a result object. The result object is a list of messages associated with the action along with some status information. It can also return a hash of other information for more general output. However, I’m assuming that most of the output of the action is the side-effects the action performs.

When an action is run, it goes through four basic phases to process:

Consume. The user input is delivered to each control associated with each attribute on the action.
Clean. The input in each control is filtered and corrected, allowing whitespace to be removed, numbers to be reformatted, etc.
Check. The input of each control is validated and any errors discovered are reported. Processing stops here if there are errors.
Process. The input in each control is copied into the action attributes directly and the action is run.

In addition to the features attached to the controls, you can specify general subroutines which will run during these phases. For example, if you have a form for changing a password, you might want a check that looks like this:

has_checker check_that_passwords_match => sub {
    my $self = shift;
    my $new_password = $self->controls->{new_password}->current_value;
    my $confirm_password = $self->controls->{confirm_password}->current_value;

    if ($new_password ne $confirm_password) {
        $self->result->is_valid(0);
        $self->result->error('the New Password and Confirm Passwords do not match');
    }
};

This allows you action’s run method to focus on the task it performs and nicely segments off all the checks. You can also logically group your checks and filters so that they are separate from each other. This makes it easier to subclass your actions or compose them from roles.

Controls are the Bridge

An attribute added to your action with has_control has a control object associated with it. This control does most of the work until right before processing. The control receives the initial input, the cleaned input, and is where the value sits when checked. It is not until it is checked that the value enters the attribute instance slot.

Without the control in place, i.e., if we assigned to the attribute directly, Moose would object to the input before we have a chance to do anything. I want to avoid exceptions in this case as well because input validation failures are not exceptional, they’re expected. We need a way to easily send back notes the user about corrections needed and would like to avoid spewing call stacks unnecessarily.

Controls also suggest how the user interface object should present them to the user. An interface is not bound to use these suggestion in exactly the same way as its peers. For example, a text control in HTML might just be a text input box while in a CLI interface might be presented as a command-line option that takes an argument. A check box, on the other hand, will be a check box input in HTML, but would be an option that takes no arguments in a CLI (it’s presence suggesting “on” and it’s absence suggesting “off”).

Features Modify Actions

A common feature of form handling is filtering and input verification. In Jifty these were called canonicalization and validation. Horrible. I’ve called them cleaning and checking. In general, though, the cleaning and checking you want to do is extremely repetitive. On almost everything I want to trim the whitespace and verify that something that is required is present. I often want to check input length for something too long or too short. Sometimes I want to check that email addresses or phone numbers are sane, etc. In Form::Factory, all of these are called “features.”

A feature is a generic object that modifies an action. These can be anything from a feature that is just used to flag the role for some reason to features that actually modify the structure and processing of the form. Features can also be attached to particular controls to clean or check an individual control’s input.

Interfaces Talk to the User

And interfaces are probably what make Form::Factory most unique. An interface looks at the controls for an object and presents them somehow to the user. Currently, I’ve written an HTML interface and a CLI interface. The HTML interface presents them using the matching HTML form controls. The CLI interface presents them as a usage snippet describing the command-line options the action accepts. You can define a single action class and use that action to define a web form or a command-line interface.

I plan to add a REST interface as well, which will be a variation of the HTML interface. Other interfaces I’ve considered (but have no plans for at this time) could include things like a Wx GUI, a PDF interface that builds a PDF with form fields and then consumes FDF documents created from that, an XForms interface, a XUL interface, or a curses interface.

Okay, so that’s my current contribution to the forms parsing arena. I’m still working the kinks out, so be careful if you give Form::Factory a try. Things will change. Otherwise, I’d love some feedback and help ironing out some of the wrinkles.

Cheers.

Bluga WebThumbs 1.0 for Drupal 6

2009-11-04T04:02:47Z

If you use Drupal and would like to automatically generate thumbnails of web sites, you might want to take a look at the Bluga WebThumbs module. Once that module is installed on Drupal and you have given the module your Bluga WebThumb API key, you can then inject a PHP snippet like this into a node using the PHP filter or into a theme file:

Now, when you view the page containing that snippet, you will see a thumbnail generated using the Bluga WebThumb service within a few seconds that shows that web site. The thumbnail is cached locally on your server.

You can further customize your thumbnail with many options. You can read the full documentation for the module at Drupal’s web site as well. New features since March of last year also include a script for updating the thumbnail in place without requiring a page reload and a nicer looking (and CSS-styled) place holder, which is an improvement over the graphic previously used.

Last week, I was approached by the owner of Konigi.com to do some Drupal work. He’s a user of the Bluga WebThumbs module I wrote in March of last year and wanted to see it work with Drupal 6. I have finished the update and have made a full release of the Drupal 5 version of the module after making a few improvements and then ported the module to Drupal 6.

Cheers.

Google Wave is Nothing Special

2009-10-15T23:28:10Z

In the past, I’ve done a fair bit of work on Jifty with Jesse Vincent by way of my previous job. He had some Google Wave invites and I begged one off of him. After waiting a week, it came through and splat. Google Wave is a wasteland of nothing. Playing with some posts to myself led to about 10 minutes of use and then I hung up.

A colleague of mine, who (last I knew) does not have a Google Wave account had come across a LifeHacker post describe the “with:public” search that provides at least some content for Google Wave. However, that content is mostly just a dearth of “What can I do?” and “Anybody else here from Canada?” and “What happens if I post porn?” It was initially interesting, but I got about another 10 minutes and hung up again.

Okay, so there’s not much to do, but it’s brand spanking new. Thus, criticizing it now is like calling a bridge useless when the engineers have only just finished putting up the supports. There are couple things I’d like to note, though.

First, there’s not just a whole lot new here. Google Wave is merely a new combination of social networking, email, wiki, forums, chat, document and image sharing, and widgetry. If you’ve used a Wiki, Twitter, IM, and Email before, you have a good idea what Google Wave is already. They’ve just taken the next step and attempted to combine things in a way that will be challenging to scale, but I think Google is up to the task.

Second, that all said there is something I hope that can be achieved here, if not by Wave, by something like it. They consider Google Wave to be a reinvention of email. They have the idea that they might be able to replace email with it. Maybe. I think email has already been replaced by SMS, Twitter, and Facebook for many people. I end up using Facebook to send messages to folks much more often than Email these days and would probably use SMS if I weren’t so cheap and had someone other than my wife that I communicated with regularly via cell phone.

Problem: Services like Twitter and Facebook have a very significant problem. They’ve taken the Internet backwards by providing a single hosted service in the cloud. Email has an important advantage in that if my email provider has an outage, everyone else on the Internet is probably fine. If my email provider goes out of business or provides terrible service, I have the alternative to go somewhere else. Twitter and Facebook have competitors, but unless you can convince all your family and friends to move with you, you can’t leave them unless you’re willing to sever your ability to communicate with them.

Solution: Something like Google Wave or Twitter or a consortium of social networking sites needs to come up with a new decentralized mechanism for communicating between people. Whether that means you can switch services but they have hooks between one another to send messages and share friend lists or Google Wave provides some sort of decentralized platform for doing this independent of the social networking sites, something needs to happen here.

There are other dangers here to privacy and such that I haven’t even touched on either. Something like Google Wave (assuming Google Wave can become decentralized like email) should happen if social networking is going to continue to develop. That’s my thought anyway.

Cheers.

Driving an Empty Bus or a Clown Car?

2009-09-11T02:44:30Z

This morning I had an epiphany about a difference in project management style between the two major development jobs I’ve held. One style was like driving an empty bus and the other was like riding in a clown car. I am going to examine both as anecdotes from my perspective and try to avoid grandiose analysis.

The Empty Bus

So, I start the job and the first thing the company does is hand me the keys to the bus. Actually, they dither on what kind of bus to give me for several months before getting me a suitable one and give me a loaner to drive in the meantime. However, once on the bus driving, I am pretty much on my own. I have a destination to reach that has been vaguely described on a scribbled piece of paper. The directions are unclear and no one in the company has been there before. They keep changing the directions. But I get to drive. That is fun.

Every now and then, I pick somebody from the company up, they make changes to the directions and then they get off again before I go very far. Every six months, everybody climbs on to the bus and sits in the very back. They do a lot of yelling while I park and then they take away my scribbled directions and give me new scribbles to follow and a new destination to reach. But I get to drive. That’s usually fun.

All in all, I am asked to develop software with very little cooperation or direction. I am left on my own to make almost all the decisions. Even though I have weekly meetings with my manager, I am not really given much feedback on whether I’m going the right way. He’s not a developer, so he doesn’t really know enough about what I do to give me useful feedback. My quarterly reviews aren’t very cooperative or helpful, they are more about the manager wishing I would drive faster and make fewer mistakes (mutually exclusive goals when you think about it).

I nearly get into a wreck a couple times, but there’s no one on the bus to help me out. Don’t get me wrong, I’m not an excellent driver. I’m still learning, but some help should help things go faster in process, you’d think. Usually, though, my directions are so unclear and difficult to follow that I am directed to get into wrecks. This is not actually all that fun as time goes on.

The Clown Car

Clown cars are funny. They drive around in circles and then the doors pop open and an absurd number of hilarious characters hop out of the little car. This job is not quite like this. It’s actually more like a really crowded minivan, like the trip I took the other day with my wife, son, dad, mom, brother, sister, and her boyfriend, all crammed into our little Pontiac Montana. But now, imagine, that all of these people have a stake in where the van goes and have a slightly role and different idea about how to go about getting there. Now, we’ve got a good analogy. Clown cars are fun, though.

The CEO’s seat is next to mine and he gets in and out of the van whenever he feels the need. He’s a busy man: lots of vans to help. Usually, he gets in right before we wreck or near the major turns to make sure we drive carefully at those point and turn the right direction. Directly behind me sits an analyst whose job it is to navigate. He tells me where to go and annotates those instructions pretty regularly. Beside him sits another analyst whose job it is to talk to the customer and figure out what they want. He then tells the first analyst and me where we need to think about going next. I get to drive, though, they keep reaching from the back for the wheel and the pedals. That’s annoying, but still fun.

Behind the analysts sit a whole team of project managers, executives, sales people and between them and between the front seats sit a bunch of other engineers. Sometimes the other engineers help drive, make suggestions, and they often critique or commend my driving. There are a lot of people in this minivan, sometimes there’s a lot of yelling about what to do next. All the activity does make for quite a bit of fun, even if it gets a bit distracting at times.

I get to drive. As I mentioned, sometimes the analysts and engineers have their hand on the wheel and help push the pedals for me. This is pretty fun too, unlike the car though, this actually gives us a lot more control. We seem to be getting places in a much more controlled way, though we do have to control our speed much more carefully. It might take us longer, but the drive is fun along the way.

Software development in this style takes away some of my freedom as a coder. That’s a bummer in some ways, not as much fun. I like control. Yet, it also lets me focus on my strengths while others worry about talking to customers, making sure we have a plan that does what the customer wants, and while the constant feedback makes it hard to see the big picture, I usually have a lot of warning before I drive off the road or get into a wreck. Overall, this is more fun.

So far, I prefer the “clown car”/stuffed minivan to the empty bus. It’s less bipolar and more slow, steady, and directed.

Cheers.

Mario AI Competition and Perl

2009-08-17T14:15:30Z

Last week, a coworker let me know about this. It’s been on Slashdot and such, but I’m a little behind on my feeds. For some reason, the competition intrigued me so I’ve put together a set of Perl classes that will allow agents written in Perl to control the Mario AI simulator using the server agent. I considered trying Java, but I started and quickly remembered why I hate Java, so I quit that and wrote this up over the weekend instead. You can fork it and give it a look here.

http://github.com/zostay/AI-Mario/tree/master

It includes a very stupid and blind robot for testing at this point. However, it is able to get through most levels I’ve tried on difficulty 0 (though, not many on any higher difficulty). Once you install that and fetch the latest JAR file from the Mario AI competition site:

http://groups.google.com/group/mariocompetition/files

You can then run an agent via:

java -jar iMario.jar -server on

to start the server, then:

cd AI-Mario
bin/agent.pl

This will run the blind Perl agent (named AI::Mario::Agent::Simple) on the same level over and over. A more interesting run might be:

bin/agent.pl --config AI::Mario::Config::Random -o level_difficulty=2

This will run the same agent through a random series of levels (restarting each time the agent wins or loses) and sets the level difficulty to 2—which means, the simple agent nearly always loses.

The system is built with Modern Perl (i.e. Moose) and uses POE and POE::Declarative to talk to the Mario AI server agent. You can implement new configuration (to customize how options are set at the start of each run) and agent and set them to run using the arguments available on the CLI.

To write an agent, for example, you just need to create a class that does the AI::Mario::Agent role and implements the required methods.

name: should return the agent name to be sent to the server agent as greetings
reset: a method called to reset the agent when a simulation is about to start
update: a method called during each frame, given an observation object (see AI::Mario::Observation) and should set the left, right, duck, jump, and run controls appropriately to keep moving.
fitness: this is called at the end of the simulation with a fitness report to tell the agent how well it did (see AI::Mario::Fitness)

Then, you can run your agent via:

bin/agent.pl --agent MyAgentClass -p foo=1

where you can pass a series of parameters to the agent constructor using the -p option.

I plan to add a bit more to it, but the interface is mostly set.

I’m not even sure they’ll accept a Perl submission, but it seems to be implied by the instructions on the site. I’ve asked to be sure. If I can confirm that, I’ll also add a Makefile.PL and such so that the solution can be packaged with all the required libraries to make it run.

I still haven’t decided whether or not to enter or just goof around, but whatever. :)

Cheers.

Update: Julian already replied. Yes, Perl submissions would be accepted. I’ll definitely try to add a submission script which will help build all the requirements to submit.

Goofing Around with HTML 5 Canvas

2009-08-08T04:53:04Z

I was goofing around with HTML 5 canvas for giggles this week. The canvas allows you to draw using JavaScript to a section of the page. HTML 5 defines a “2d” context, which allows you to draw lines, polygons, images, boxes, etc. You can style your stroke, fill, and shadow, etc. It’s pretty easy to goof around with. Oh, and if you’re using Internet Explorer, these won’t work. You’re out of luck because Microsoft is too cool for canvas.

Box Fractal Renderer

First, I built a box fractal renderer, which are my favorite things to doodle on graph paper.

Fractal Coastline

Then, I played some more with fractals and eventually ended up with this fractal coastline builder. Each view of this will be a little different. Some look more like coastlines than others.

Anyway, just some random goofing off I thought I’d share.

Cheers.

Time Estimation Sucks

2009-07-29T01:43:22Z

From the perspective of myself, a software developer, time estimation always sucks. I believe I’m working in an environment which has a better view of time estimation than those I’ve worked in before, but we still do it and I think that it’s sucking is inherent. I’ll explain why in a minute.

However, I want to say that time estimation is probably essential for project managers, product managers, executives, sales, and the like. For them, perhaps they can provide a barometer giving them a good indication of where things will be in the future. However, I don’t speak for any of them, I’m speaking from the selfish position in which I stand (as we all do and would admit when we’re being honest).

Time Estimates are Guesses

A time estimate is, by definition, going to be inaccurate. Until someone finds a prophet willing to use his gift for mundane predictions like, “How long will this new feature take to implement?” Or, “What’s the timeline on this fix?” The answer will be wrong.

To a person without the training it may seem like a software developer works magic and comes up with solutions, but that’s not the case. She’s merely working through developing an explicit set of directions, logic, and math to allow the computer to do something. However, the complexity of a problem is rarely well-known until you try to work through it. Thus, you don’t know what problems you are going to run into.

This is further complicated by the fact that you have to work within some framework including programming language, available libraries, code already in built up, customer expectations, code maintainability, solution flexibility, etc. A software developer often has to make decisions and compromises along the way and it’s not always clear what the ramifications of each of these decisions will have.

Implied Margin of Error

All time estimates include an implied, but rarely recorded or even knowable, margin of error. When I estimate the time it will take to do something, I will often say something like, “Well, since I just did a very similar process in importing that fail for client X, it will take about 2/3rd’s as much time to do it for client Y since I’m familiar with the code and added some pieces there to make doing this task easier.” I’m implying a narrow margin of error.

On the other hand, I might say, “Well, the code in that part of the application was not well future-proofed and this will build on there. I may have to do some refactorings, so I’d say we’re looking at 2 days, plus or minus a week.” I’m even stating it here, the large margin error.

Yet, what if the first task turns out to take 5 times as long as I said, does that mean my margin of error was wrong or does that mean I was operating on incomplete information?

When a meteorologist says, “50%” chance of rain, is he wrong when it rains on your house? Or when it doesn’t? No, it was just an estimate, which is a statistical entity that cannot be analyzed on those terms as a data point. Only after you have a collection of data points that shows that when the meteorologist suggested 50% chance of rain, it actually rained 60% of the time. Then, he’s been wrong (and even in this analysis, you have to be careful about data bias and all the other statistical issues your analysis may contain).

Large Estimates

Getting into the actual business case, we find that making a large time estimate on something will get you into trouble when you make the estimate.

I recall a case when I suggested I was starting on something during the last week of December after Christmas. I broke the task down into all the steps I would have to do to complete the task and concluded that an reasonable estimate of time would be the end of the second week of February. I told this to the executive in charge of the product and he said, “We’d really like to be able to tell the clients the start of the third week in January.” I said, “No. I don’t think that’s reasonable.” He said, “We’d really like to be able to tell the clients the start of the third week in January.” In other words, “I don’t care what your estimate is, this is the estimate we’ll use.” My response, “I will give you something at that time, but it won’t be finished.”

In this case, it actually took all the way through April to get it done, largely because the project ended up having twice as many features as originally spec’d.

As a software developer, I am strongly motivated to report short estimates to avoid pressure from above. Sometimes, however, a long estimate is the only one that will work, but I always expect to be criticized if I give a long estimate on something.

Short Estimates

If I end up giving an estimate for a project that actually takes me 2 to 5 times as long to complete, then I’m in big trouble. These are the times when executives yell at people. Directors and project leads ask you what lessons you’ve learned, and if someone really gets upset, you have weekly phone calls with someone in HR listening in.

So here’s where making a long estimate for a short task seems like a good idea. But then, what happened in the last section? There’s no win in this game.

Accurate Estimates

So what happens when you get it dead on? Do you get a big promotion, cigars and cognac with the executives, and a fat bonus? Nope. You just plod on. You might get such if you made a long estimate, got away with it, and then delivered way early, but that might just mean you’re a good con-artist, not necessarily good at estimating your time.

What’s the Answer?

I don’t have one for managers. That’s not my problem, but managing my managers is. So, here’s the solution (at least what seems to work best for me at this point):

Communicate frequently with the bosses. Anyone you know to be a stakeholder in your project should get something from you often enough to keep them in the loop, but not so often they get annoyed. Hard to balance, but in generally you ought to be talking to your supervisor at least once every day, your project/product managers should get information at least twice a week and as much as once every day, and executives should get direct feedback from you once a week if they have a stake.

And if you find out something is really critical to someone, update them as often as your supervisor. If it’s an emergency or something critical to a sales pitch being made tomorrow, send out an update at every stopping point.

Regular status reports help managers feel in control and you want them to feel in control. If they feel out of control, expect them to blame you for not telling them what was going on.
Make your guesses as good as you can. On complex tasks, break the task down as far as you can ahead of time and estimate each piece. Give yourself an explicit margin of error and percentage of accuracy you’d place on that margin. Sum up the time estimates and margins and use that to present you estimate. Take some time to do this well, don’t rush and spend more time considering the harder to estimate parts.

When you give your estimate, you might share some or all of this with your supervisor (depending on how well you think he’ll be able to cope with the information). Don’t give this to anyone else, but use this information to determine how to talk about it when reporting on your estimate. Use vague, fuzzy language for parts you aren’t sure about and use concise, direct language on the parts you are pretty sure about.
Stand by your estimate. If your boss says, “Unacceptable,” or doubts you, listen to them and explain more about your estimate. Especially if you’ve worked through the time it will take in details, wavering in a moment of doubt is probably not going to serve you or anyone.
Be flexible. If the bosses need a shorter estimate for some reason or even expected a longer one, use that to your advantage. If you’ve done your homework and got it with you, perhaps you can negotiate parts of the spec away that aren’t as critical in order to cut down on time. Or perhaps there are things you can iron out a little better and get working really well in the time you have. Or perhaps you can use that opportunity at the tail end to get better testing and quality in place.
Meekness wins when all else fails. Never ever burn your bridges. When it hits the fan because you prove to be the woeful prophet you are, stand by your mistake, but stay calm. Be humble. You might have a really good idea what went wrong, but your bosses are pointing at something else. Point that out, but don’t beat them with it if they disagree.

Other than that, be you. I work really hard to never blame someone for a mistake, even if they made it. I do not tell on fellow employees when they aren’t doing their job. However, I try to make it a point to point out excellence in my fellows whenever I can. I don’t feel like it’s my job to do management (which, in my mind, means correcting and rebuking employees when they fail at something), but pointing out excellence and success is everyone’s business. If I really appreciate something a fellow developer has done or someone on the business side writing a really great spec, I try to make it a point to say so in conversation. Unfortunately, being the loner that I am, I don’t think I notice these things as often as I could.

Time estimation sucks, but it’s pretty much inevitable. The key is understanding why it sucks and how to deal with that.

Cheers.

Weight Loss, My Way

2009-06-25T17:24:04Z

Disclaimer: IANAD: I AM NOT A DOCTOR. Please consult one prior to engaging in any attempt at weight loss. I provide no recommendation you do things my way and take no responsibility for your health.

Disclaimer 2: I am not affiliated with, have never given money to, and have no relationship at all with the Unnamed Diet System I have based my diet plan upon. I do not recommend for or against their system, I am just sharing the fact that I my own variation of it and have had success.

Since my birthday this year, I’ve been working to lose a little extra weight. I now weigh around what I did when I was in high school and am still losing. I did this to show solidarity with my wife who wanted to lose a bit, but I needed to do this for myself as well.

Wanting to lose weight presented a few problems for me. First, I can not stand to do what somebody else tells me to do. My wife calls that being “obstinate-defiant.” Depending on my mood I usually either say I’m just selfish or I have a built-in distrust of the crowd. In any case, it was clear that unless I could tweak my plan a bit, I wasn’t going to be be happy with it. I’ve ended up tweaking less than I thought I would, but I still can, so there.

My next problem is that I refuse to do cruel or unusual dieting. I’m not going in for any fad or diet that drastically changes things. I’m not giving up cookies or eating grapefruit or doing Adkins. Regardless of what science there is or is not backing these things up, anything that changes what I eat is going to make me grumpy. I love food and there is no shame in that. The shame comes in consistently eating more than I need. I also find the idea of using a pill or surgery repulsive (no offense to those who do such, I won’t). For me, this process is about developing self-control, which means I need to learn to do it and my wife provides enough accountability to that end.

While metabolism and other factors adjust a person’s dietary needs, failure to consume enough calories to maintain a person’s current weight will cause a reduction. (Unless, which I suppose is possible, a person’s body is somehow capable of storing fat, but incapable of using it. I don’t know if any such disease exists, but I don’t have it if it does, so it’s not my problem.) Therefore, my diet plan would have to be something as mundane as journaling what I eat.

Which brings up my next problem, counting calories is too easy and not really addressing the full magnitude of the problem. Because not only should I eat less, but I should encourage myself to eat healthier. The system ought to take other factors into account.

My final problem is that whatever it is I do must be something I can do on a computer. I sit in front of one for around 8-12 hours per day. I can set my computer to remind of things, I can share things between myself and my wife to provide accountability on my computer, and while I like writing down notes, particularly when I’m brainstorming, I really don’t want to do all the math we’re talking about in my head all the time. It’s too tedious.

Fortunately, my wife previously went on and successfully completed a plan using Unnamed Diet System for which you may have seen ads. This system met all my basic requirements. Yet, other than they way they count points, they don’t provide any value to me, at least none I would pay for. Fortunately, everything I needed was published at various places on the Internet and I built myself a Google Docs spreadsheet to do it.

I provide a link to a version of it here for anyone interested in weight loss on similar terms. Go re-read the disclaimers again now. I’ll wait… Done? Okay, I don’t recommend this plan to you, but if you find the spreadsheet useful, great. I’m providing it under a Creative Commons 3.0 license.

Food Points Sheet

If you have a Google account, you can create a copy of the spreadsheet to use it or you can download it in another format to use with Excel or OpenOffice (I think, haven’t tried that).

To use it, I first scroll right until I fill in some information about myself. This sets up the basic tolerances for my diet plan based upon sex, age, current weight, height, daily activity, etc. Then scroll back left and log my consumption. Under each meal, the wide column is for a description of the item eaten and the narrow for recording the points. Once I have “0” points left for the day, I stop eating. The spreadsheet does have a weekly allowance of extra points that I can use as well to indulge in something or just allow me to consume all the points for a day without worrying about going over by a couple. I use all the points I have for a day unless I’m really not hungry. This is not a starvation diet, so I try to use up as many of my daily points as possible. I do not worry too much about using or not using the weekly points. I often consume most of them.

The formula for calculating this points is simple, but elegant in that it encourages me to get more fiber and avoid fatty foods, while consuming fewer calories than I need to maintain my weight:

Points = Calories / 50 + Fat (g) / 12 + MIN(Fiber (g) / 5, 1)

I’ve been told that Unnamed Diet System actually divides Fiber by 4, but whatever. I have embedded a couple calculators in the spreadsheet for the times when I’m too lazy to do the math in my head.

I weigh myself once a week to track my progress (on a separate spreadsheet).Every 10 pounds, I adjust the chart to the right since the spreadsheet gives one less point per day for each 10 pounds I lose. I also copy the spreadsheet (actually, Terri manages this part) each week and blank it out to use the next week.

Eventually, I should reach my goal weight (I haven’t really decided what that is). When that happens, I’ll need to adjust the spreadsheet to deal with maintenance. When that happens, I will give myself more points until my weight stabilizes. I plan to continue recording points for the foreseeable future this way.

Cheers.