過労死 Coderoshi

On Eggheads

2012-09-19T01:33:00.001-05:00

Some people are too smart for their own good.

I don't mean this in any mean or derogatory way. I'm a firm believer that it's always better to be more, rather than less smart--even if it causes you some degree of discomfort when attempting to communicate with others.

However. You cannot ignore the oft-quoted "IQ gap"--the theory that a 30-point difference in IQ renders communication difficult--goes both ways. Not only do those of a lower IQ find it hard to follow a conversation lead by the more intelligent agent (or as I prefer the more technical nomenclature "Egghead"), so too does the higher IQ person find it difficult to know what it's like to be lower.

In other words, you don't know what you don't know.

I find this to be the case in certain organizations as well. An organization can actually be too smart for its own good... they can be too Egghead to talk to your average person. If your group is run by ten 160 IQ people, you're probably going to do amazing work. Sadly, most people probably won't ever know it.

Just as the Dunning-Kruger Effect points out, it takes knowledge to recognize genuine knowledge in others. If the average person is 100 IQ (and they always are), your 160 IQ company will have a hard sell. Sure, you'll get a few adopters. They'll sing your praises. But when people ask: "yes, yes... but what do you do?", you might end up like ZeroMQ, and so many other academic curiosities. Something brilliant, amazing, and very, very hard to explain.

I'm not advocating hiring dumber people. Far from it. I just wanted to bring up the point that, oh you rare brilliant companies... pay close attention. Don't discount what the average person thinks about you. Make an effort to help them see your brilliance. In software, this is documentation, tools, and constant communication.

On Resurgence

2012-09-16T01:08:00.003-05:00

It's been quite a while since I visited this blog. A lot has changed.

I moved to Portland. I changed careers. I took up glass blowing, published a book, traveled the world, spoke at and organized some conferences, and built a community. Where I used to obsess about the minutiae of project management and building an optimized development team, I now care much more about the people around me, and building a better community. In some ways I'm probably better known than when I once obsessed about being well known -- when I stopped writing so much about doing, and simply just did.

But at my core I still crave the act of authorship, no longer for any overtly self-serving purposes, but simply because I care enough to contribute. In truth, I have been writing a bit, not on this blog, but rather here and here and here and here.

So what should I write about? NoSQL? I'm uniquely positioned to have worked professionally with and for a number of NoSQL databases. Tech writing? I've certainly done more than most coders will ever consider by their own volition. Community building? My favorite thankless task.

This is just another space, adding to the noise, contributing to the information heat death of the internet. Maybe I'll return, maybe my one-time 10,000+ readers will return. Maybe not. But I felt compelled to announce to this space that I have not forgotten about this site I once so eagerly built (seemingly) a lifetime ago, now populated exclusively by stragglers and bots.

Code Te Ching - Verse 58

2010-09-02T11:47:00.002-05:00

The ten thousand things are presented by Code.
Code is, and is presented by, abstraction.
Abstraction is presented by code.

Code helps the ten thousand things by entailing understanding.
abstraction helps understanding by entailing structure.
code helps abstraction by giving definition.
comments help code by giving direction.

Like the mountain directs the storm without restraining the sun,
Comments direct the coder without restraining the code.

Multiple to Single Select

2010-05-23T11:28:00.002-05:00

(If you're in a hurry - you can get the Multi Single Select code from Github. I also have a Multi Single Select example on GAE).

I have a problem. I've become addicted to progressive enhancement. I can't bring myself to write a single line of CSS or Javascript until I'm certain my web application could work in Lynx. But this is 2010 - and since 99% of users have every right to expect a little more zazz than standard control widgets, I've been cranking out UJS libs. This is one of them.

Multi Single Select is an onload library which uses jquery to turn a boring old multi select tag like this:

Into a sexy single select box, where all choices become tags (which can be removed by clicking the X)

Build your multi select as normal html. Then in your header add the jquery.multi_single_select.js library, and code similar to this in your script tag:

jQuery(document).ready(function() {
  $('#whopper').multi_single_select({list_class:'toppings', title:'Topping'});
});

Selecting from the "add" group in the drop-down adds the item to the submit list. Selecting from the "remove" group will remove them (or just click the external tag's "x").

This library is completely unobtrusive. It submits values in the same way a normal multi select would. Your server-side actions will know no difference.

Also, the Multi Single Select code on github contains a sample css lib to make your tags look all pretty.

Code Te Ching - Verse 57

2010-03-31T07:23:00.000-05:00

When managers manage teams, management manages to manage management.
When teams manage management, the means manage to manage management.

Thus when managers do not manage management, the team manages to manage.

On Eraser Stains

2010-03-23T16:15:00.005-05:00

Have you ever written something wrong in pencil on a piece of paper? Of course. That's why erasers were invented. Have you ever done it twice? Three times? Four? Eventually, the ability of your eraser to function properly will wear out - leaving a dark greasy stain on the paper.

Have you ever made a mistake designing software? Of course. That's why refactoring was invented. In much the same way, the constant writing and rewriting of software creates artifacts over time which can't be blamed on any one person, or idea, or decision. It's simply a function of imposing incremental development changes on the substrate of the real world, without a clear idea in mind.

These are eraser stains in development. Sometimes the best course of action is a new piece of paper (I mean, a rewrite).

Before you 'static process' people use this as an example for why agile development has flaws - you have your own problems: you write with a pen.

Code Te Ching - Verse 56

2010-03-12T07:21:00.000-06:00

Over-dependence on coding rules?
    Uncreative.
Under-appreciation of coding rules?
    Unwise.

Take the middle path.

But what if a fork must
be chosen? Which way to walk?

Know this:
    Remain in the center where understanding lies.
    The path will
        adjust
            its
                self.

Pivotal Tracker Scrum Script

2010-03-10T15:01:00.005-06:00

At Mobi we use Pivotal Tracker to keep track of new features, chores, bugs, etc. (if you don't use it, drop everything and SIGN UP NOW... I'll wait). Like many PT users, we also have daily Scrum updates.

Since much of what we talk about in our standup meeting is information that is readily available in PT, I wrote this handy little script to spit out what I've done in the past 24 hours. You'll need to populate your own PT project id, token, and have ruby and curl installed. Enjoy.

#!/usr/bin/env ruby
require 'date'

PROJECT_ID = 'YOUR_PROJECT_ID'
TOKEN = 'YOUR_PT_TOKEN'
NAME = 'YOUR PT NAME'

date = (Date.today - 1).strftime('%m/%d/%Y').gsub('/', '%2F')
name = NAME.gsub(' ', '%20')

val = `curl -s -H "X-TrackerToken: #{TOKEN}" -X GET https://www.pivotaltracker.com/services/v3/projects/#{PROJECT_ID}/stories?filter=state%3Afinished%2Cdelivered%2Caccepted%20owner%3A%22#{name}%22%20modified_since:#{date}`

totes = []
by_story_type = {}
val.scan(/\<story\>(.*?)\<\/story\>/m){|story|
  story = story.to_s
  id = story.scan(/\<id[^>]*\>(.*?)\<\/id\>/m)[-1].to_s
  name = story.scan(/\<name\>(.*?)\<\/name\>/m).to_s
  desc = story.scan(/\<description\>(.*?)\<\/description\>/m).to_s
  state = story.scan(/\<current_state\>(.*?)\<\/current_state\>/m).to_s
  story_type = story.scan(/\<story_type\>(.*?)\<\/story_type\>/m).to_s
  next if totes.include?(id)
  (by_story_type[story_type] ||= []) << { :state => state, :name => name, :desc => desc}
  totes << id
}

for story_type, datas in by_story_type
  puts "#{story_type.capitalize}s"
  for data in datas
    puts "  #{data[:state]}: #{data[:name]}"
  end
end

Code Te Ching - Verse 55

2010-02-26T07:19:00.001-06:00

Can anyone give a pattern for refactoring?
Then there is no algorithm to follow?
Yet there it happens, without steps?

Order exists beyond our ability to capture it.

A wise man once knew true granularity:
  1.  Write down the problem
  2.  Think real hard
  3.  Write down the solution

Drop Down Like It's Hot

2010-02-20T18:33:00.001-06:00

I've never run across a search drop-down box that functioned exactly how I wanted. Most code for drop-down boxes seem bound up in a particular heavyweight framework (like dojo, or exjs), which are fine if you use them, but I don't. I use jQuery (an altogether different heavyweight framework).

(Before getting too far along, grab the js and test libs from github)

Another problem is they always seem bent on a particular setup (a text input rather than a textarea) - or performing a particular function (load via ajax or pre-loaded array data) or assume a particular use-case (drop down for a search box). I want the flexibility to define my own client-side datasource parser, to have multiple drop-downs on a single page, or search from several data-sources at once, separating each type of data by title, or work like the gmail email text-area, where all non-word chars after the last comma performs a search of contacts - choosing an item appends to the list.

Anyway, I have a big itch to scratch - to channel the Cathedral and Bazaar. Here is the problem scope, broken down:

Must work in text input fields and textareas

Must allow single or multiple elements chosen (separated by custom char)

Must allow datasource from a local array, object, or remote (ajax, on-demand javascript, json, etc)

Must allow custom parsing of datasource

Must allow multiple simulates datasources of various types (eg. 1 local + 2 ajax)

Must optionally limit result sets, both by total and per datasource

Must optionally seperate datasources with a non-clickable header

Must optionally pre-parse input data (eg. ignore all non-word characters for purposes of searching)

Must optionally suspend drop-down until a minimum number of chars are typed

Must allow for submission throttling

Must allow for caching return values

Must be able to move though elements via keyboard up/down, tab/enter to choose item, esc to close

Must close the drop-down when bluring the input field or clicking outside the drop-down

Must highlight matching typed characters in the dropdown data set

Must work on as many browsers as possible, otherwise degrade

Must use no toolkit beyond standard jquery

Must be unobtrusive javascript

Must look nice (or hot!), and feel natural to the user (not heavyweight or klunky)

Must be easy to use (every choice must have reasonable defaults and at least one implementation)

All above items must be optional and customizable

Must be well documented so I don’t have to talk to anyone

Basically, I'm taking the shotgun approach: cram as many features into this thing before it becomes absurd, and stop just shy of that.

Here are several examples on usage. I appreciate any thoughts.

Code Te Ching - Verse 54

2010-02-03T07:17:00.000-06:00

The lamer prides himself on his skill.
He spends ten times longer than necessary and says:
"This way is the best!"
How much more has gone undone?

The elite does not dwell on his skill.
He leans into the job, code falls forth:
“This way will work.”
What more can he do?

Because he focuses on the use, his work is never useless.

Code Te Ching - Verse 53

2010-01-14T07:24:00.002-06:00

A manager is soft, like a buffer
A buffer in memory?
VPs hammer down hard
developers rustle up.

The man stands up, like a prized warrior, to an army

Subdomains on Github

2009-09-11T09:12:00.012-05:00

I've been using Github for a while, and it didn't really occur to me to write about it. It's just something that I assume everyone uses, but don't talk much about. Like toilet paper.

But I just discovered some two-ply goodness that I had to share. Github link juice; or more correctly - instant SEO credibility by owning a github subdomain. If you visit http://coderoshi.github.com, you'll be redirected to this site. But how?

There are two way of going about it. If you visit a subdomain that hasn't been used you'll get this page:
Try it: http://alice.github.com/

That's most of the trick. Create a new public repository (don't worry, it's free) of "yourusername.github.com" (don't forget the ".github.com part). Add an index.html file and push to master. Viola.

(for you non-git people, when you create the repository, github gives you clearer instructions than I could here)

But there's more. To do a simple redirect, just paste in some html with a meta refresh tag, or to keep the nice subdomain url to mask your GeoCities site (look it up, kids) - just upload a page with a blank iframe, which contains your site.

<html>
<style>body,html{padding:0px;margin:0px;}</style>
<body>
<iframe src="http://www.coderoshi.com" width="100%" height="100%"
  style="border:0px" marginheight="0" marginwidth="0" frameborder="0">
  <a href="http://www.coderoshi.com">Coderoshi - Death By Overcoding</a>
</iframe>
</body>

But there's more! If you don't want something so sloppy, and merely want a URL redirect to your site, instead of an index.html file, upload a file named CNAME, which contains your domain. Mine looks like this:

coderoshi.com

That's it! Instant credibility from google (you're a real subdomain of the very popular github.com domain).

I Hate Sass, but I Love Compass

2009-09-02T14:46:00.023-05:00

Generally speaking, I have a rule about generators. Don't ever use markup that generates more markup if the final product is straightforward anyway, and doesn't require too much typing.

I have another rule: don't get involved with code generators that remove your ability to use good existing tooling, unless they generate orders of magnitude more code and are orders of magnitude easier to understand. This is why I never got into Modello - not even when working on the Maven or Plexus. It takes nearly as much XML to generate a trivial amount of fairly trivial code.

For these reasons, amongst others, I hate SASS. It takes something trivial and straightforward with great tooling (CSS) and turns it into marginally less, but equally readable code (YAMLish). It also convolutes two styles of markup (stylesheet language which describes markup, and some YAMLish code which is markup). It just reeks of those xml-based scripting languages - this is a type of recursion up with which I cannot put. Or so I thought.

Enter: Compass.

It was for frameworks like this that frameworks like Blueprint CSS were created. It's symbiotic - without both, each are only marginally interesting. But together they become something more - like the Wonder Twins.

The Problem With CSS Frameworks

I'm going to pick on Blueprint, though the problem exists universally. The benefits of a pre-defined CSS framework are severalfold: attractive defaults across several browsers, nice typography for printing, and a common reference-point for merging other toolkits into your application (plugins, eg, jquery). Blueprint is great for HTML element defaults, and even for defining simple CSS class names like "highlight" or "hide". But an entire framework? Blueprint (and others) also have a rich set of definitions for layouts (be it "columns", "grids", "table", etc.). The nightmare is using them.

Here is an example of a value "23" set in the last column, with a specific width, prepended by 20 columns, and appended by 1, from Blueprint's own documentation:

<div class="prepend-20 span-1 append-1 last">23</div>

This harkens back to the dark-ages of the web: mixing content and layout. Though the mixing is done via css, the problem is that the CSS (more flexible than, say, a table) is still describing how the HTML should be displayed (however vaguely), not what it is.

Keeping Semantic

Up until now, we are stuck. With only HTML and CSS available, it is difficult to utilize Blueprint grids without giving something up. We can either do the above (describe layout in HTML's class attributes), or attempt to define a friendlier CSS. Let's pretend (with a stretch of the imagination) that the div declared above contains my age. A descriptive, layout neutral HTML would appear thusly.

<div class="my-age">23</div>

(Or better yet, in HAML:)

.my-age 23

Now that's separation! Nice, semantic HTML. No layout. Of course, now we have a problem: Blueprint knows nothing about the new class "my-age". How can I explain to Blueprint that the CSS class I just created should act like "prepend-20 span-1 append-1 last"? In short: you cannot. Not without hacking up your Blueprint CSS file (non-portable - you'll lose a lot of the benefit of using a CSS framework), or simply copying the Blueprint code into your own custom CSS class (better, but you've now completely missed the benefit of using Blueprint).

How Compass Aids Separation

Compass is a CSS generation framework. Actually, let me back up. SASS is a CSS generation framework, while Compass is the glue between Blueprint (and other CSS frameworks) and SASS. Remember me saying I hated SASS? It's true. But Compass pre-packages these CSS frameworks and makes them available as mixin modules... and that I love. So let's get on with it.

Assuming I have Compass/SASS installed and operational, fixing the above problem is trivial.

@import blueprint
body
  div.my-age:
    +prepend(20)
    +span(1)
    +append(1)
    +last

What Compass does is provide those Blueprint mixins (+prepend, +span, +append, +last - complements of @import blueprint), while SASS provides much of the rest, compiles the given markup to CSS (yuck). What you get is the kind of inheritance lacking in CSS proper.

Opening the Blueprint Grid CSS file we see the classes we used are defined as this:

.prepend-20 { padding-left: 800px;}
.span-1 {width: 30px;}
.append-1 { padding-right: 40px;}
.last { margin-right: 0;}

But after Compass/SASS we get a new CSS file generated, which contains the following:

body div.my-age {
  padding-left: 800px;
  width: 30px;
  padding-right: 40px;
  margin-right: 0;
}

We get to keep our Blueprint pristine, while able to keep markup semantic and layout separate. What's more, is that beyond the brute-force method of copying Blueprint class-for-class, Compass provides a host of other useful mixins. In fact, the above SASS can be simplified as something more succinct:

@import blueprint
body
  div.my-age:
    +column(20, true)
    +append(1)

Wrap Up

What I stated in the intro is still true. Markup that generates markup often lacks any sort of real tooling, and this is no exception. What I find myself doing is writing CSS using CSSEdit against the SASS-generated CSS. Once I'm happy with the changes visually, I make changes in the SASS file. Is it extra effort? Yes - but not much. And the payoff to project bootstrapping speed, along with the overall maintainability, makes it worth it. Try it on your next small Rails project (along with HAML) - you may find it hard to go back.

Codeoshi on Github

2009-09-01T19:48:00.003-05:00

Here's the link: http://github.com/coderoshi

Sure, it's been there for a while - it just recently occurred to me that I never linked to it.

Enjoy.

Code Te Ching - Verse 52

2009-08-19T00:00:00.000-05:00

Like one cast away on a desert island,
    the rolling tide keeps his head.
Like one working on the dark prairie plains,
    the chirping insects keep him company.
Listen to the hum of the machine, for that is the coder’s solitude,
    that keeps his company ahead.

The elite understands his value;
    does not lord it over his subordinates,
    does not embellish it to his superiors.

His opportunity flies like a gentle bee’s.
Alights on the first flower it sees,
And leaves just as suddenly, without a trace
But benefits the flower in endless ways.

The elite has no home but Code,
And because of this, is never homeless.

Ruby is Java and Java is C

2009-08-17T10:29:00.012-05:00

Remember 10 years ago? As "Java 2" (jdk 1.2) exploded in popularity throughout 1999, scores of developers dropped their perl/c CGI web server-side methods in favor of the new hotness Java. And why not? The language was maturing, the simplicity of ASP was ported to Java as JSPs in June, and the servlet spec was hammered out with the advent of WARs in August. Those of us lucky enough to get paid to be on this forefront found plenty of work still needed to be done - by way of tooling and optimization.

But with this new fervor of excitement rose an equal amount of riffing from the "real" server-side language: C/C++. "It may seem like you can write apps faster", they'd say, "but there's no way you can beat our tools - look at our libraries! Look at the power of our pre-processor, compile optimizations, and emacs! And - you'll be sorry when you have to spend all that time optimizing, and eventually re-write your code in C anyway." It was a fair argument, and there were certainly anecdotes over the years of companies pulling the plugin on their Java projects to revert back to C (hell, I worked on one at Cerner as late as 2004). Some Javanistas attempted to argue back - but most of us quietly hammered on, bending Java to our will - content in the knowledge that hardware would close the performance gap, and our tools would do nothing but grow. Yet still they scorned - for a while. Ten short years later, you rarely hear from that camp anymore concerning web server-side productivity or reliability. We clearly - emphatically - won. (It's a shame, really, since I kind of miss hacking C.)

But times have changed. Despite incessant claims that Moore's law is dead - it has held strong, driving forward. Hardware is cheap. Disks are cheap. Memory is cheap. And in many systems, bandwidth is cheap. Yet now, a short 10 years later, the Ruby camp suffers the slings and arrows of a myopic community - in this case Java steps into the role of C. The old, general purpose stand-by refuses to fade away in favor of it's lighter, easier, friendlier replacement. The arguments levied against Ruby these days are similar to the old arguments against Java. It's slower. It doesn't have powerful tools. It doesn't concern the needs of the "enterprise". "Real" programmers understand the difference between interfaces and abstract classes - and the value of each being distinct (I still don't know). Yadda yadda yadda...

So I'll make two predictions here. Not so bold now, but in fairness to me, I've been saying it for 4 years: Ruby will win this battle for the web over Java. Java is just too clunky of a language to survive, with too much baggage to match the agility of Ruby - or even Python for that matter.

Java - desperate to remain relevant - has at least learned something from the past. Java, as a language will die, but hopefully the JVM - arguably the greatest VM ever created (not so much by design, but by sheer effort alone) won't fade into that dark night. Hopefully EngineYard's recent investment into JRuby is just the right medicine - lest it completely drop from the web server-side market. And that really would be a shame.

Build a Business with Google

2009-07-29T08:55:00.040-05:00

Hi, I'm Eric. A C++ coder turned Java zealot turned Ruby on Rails adopter turned Django turned... whatever. They're all fine. I'm not here to talk about superior architecture, or the best language, or design or whatever. I'm here to talk about getting your web app idea out the fucking door. This is exactly what we did with MicroTasking.net. A year ago this was hard, but could be done for free. Two years ago it could be done cheaply. Five years ago you needed some funding and a lot more time. This is today - so I'll talk about the lowest barrier to entry technical stack on the market: Google.

Note - I don't work for Google, I don't get kickbacks, I don't even own any stock. But I have bootstrapped several companies, products, enterprises, forgettable services, and open source products. I also make iPhone apps.

Are you afraid of "putting all of your eggs in one Google basket"? Don't. When speaking of risk assessment, the largest risk of a startup is failing to launch. Google going away isn't your biggest concern. Scaling isn't either. Just push it out the door - then worry about those things. Once you have customers, you'd be amazed all of the problems you can pay to have solved. Like migrating to dedicated machines, if it makes you feel better.

So what do you need, from the technical side, to start a web application company? A domain name. You need a server that doesn't suck. Email/office software is nice. A blog is good. A phone number to call. What else? You also need customers. You get them through advertising and campaign optimization (which you achieve via tracking). They also need to be able to make purchases. But first thing's first:

Domain Registration/Host/Email/Office Apps

Let's start with the basics. Register a domain and get a host through Google Apps signup. Click "I want to buy a domain name" and search for one. It costs $10/year, which is more than Go Daddy. What they give is worth the extra price. You'll automatically be set up with a standard version of GA - which provides you with 50 domain-specific email accounts integrated with the Calendar, a sandboxed office suite with IM/Video Chat, Documents, Slideshows, Spreadsheets - also integrated Forms, which you can put on your Site or email, and collect results as a spreadsheet. Oh, also, they provide a website CMS.

Money down: $10

Phone Management

Sign up for Google Voice. It's not public yet - but it will be soon enough. I use it daily... it has damn near everything you need for a small company PBX. I live in Indianapolis, but my business number is San Francisco - which forwards to my cellphone. Twofold benefit. 1) you don't want to give out your private number, and 2) you don't want to give out your private number. It's free for now. You can even pay extra for a toll-free (800, 888, 866) number.

More on Email

This is a nice trick mentioned by Tim Ferriss in the 4-Hour Work Week. Since you're eventually going to grow and outsource all of your work (right?) it's not a bad idea to segment your communications in advance. Your company may just be you and a buddy for now, but you want the illusion of a larger team. Sales. Info. Webmaster. Whatever. You can do this on the Google Apps dashboard by clicking "Users and Groups" on the top, then choosing "Groups". Create as many groups as you need and just have them forward to the appropriate people. As you grow, it's easy to reroute the traffic - much easier than changing a well-known email address. Our list for MicroTasking.net is:

contact@microtasking.net, help@microtasking.net, info@microtasking.net, refund@microtasking.net, sales@microtasking.net

They all forward to me and Jim.

Blog

Get a blog via Blogger. Just sign-up with the Google Account you made above. Follow the steps to create a blog. That's easy enough, eh? What you probably want here though, is not a blog URL http://mynewblog.blogspot.com , but rather http://blog.mynewcompany.com. This is a two-step process.

First, create the blog subdomain. In the Google Apps dashboard, click "Domain Settings" above. Then click "Domain Names", then "Advanced DNS settings". Sign into enom. "Edit Host Records" and "Add New" CNAME named "blog" (or whatever). Set the address to "ghs.google.com." (don't forget the period at the end). That's it for the DNS routing table.

Next log back into Blogger. Click "Settings" and then "Publishing". Fill out Your Domain with the new subdomain (eg. http://blog.mynewcompany.com). That's it! No more lame blogspot domain.

This should cover you on the business/IT end. Let's make the app!

The Application

For years Google skirted around the edges - instead focusing on toolkits, APIs, and company tools like email. With the launch of Google App Engine (GAE) last year, they've taken a huge leap toward Amazon. What is App Engine? Think EC2+SimpleDB+S3+deployment tools+data management. It's to Python (and Java) what Heroku is for Ruby on Rails. In other words - a dream.

Sign up for a Google App Engine account. Again, just use your new gmail account. There are a lot of posts about App Engine, so I won't cover it here. But what I will tell you is, if you choose to write in Python, use Django. There a great patch project here. Follow the instructions. Use it. I've written 3 GAE projects so far - two of them without it. One with. If your application will be non-trivial, just make the investment to learn Django and use it - you'll thank me later.

Note that your application lives at http://mynewapp.appspot.com/. Again, we want to put the application under our single domain - which is painfully easy. This is where I diverge from the Google party-line. I don't use the Google Site stuff - but by default, your domain will forward to use Google Site. So I instead hijack the "www" subdomain to point to my app-engine app.

First, we need to tell Google about our application. Go back to the Dashboard, and click on "Add More Services". Under "Other services", you will see "Google App Engine" - "Enter App ID:". Do what it says (this is the unique name you gave your app - the *.appspot.com subdomain). It will then forward you to another settings page - but ignore this for now and return to the main Dashboard page. Click on the "Sites" link, then "Web address mapping". Delete the "www" mapping. Then return to your Dashboard.

Select the new "app engine" link - the one you just added. Choose "Add new URL", and enter in the subdomain "www". Viola! Now your application will be the main page. I recommend doing this because Google Sites is pretty limited, both in style and customizability. Your applications index page doesn't have to be anything more than a static template page - which you can 100% customize like any other website. Some people are taking advantage of this, and using App Engine as a pure content delivery network (CDN) to host static pages, images, etc for free. Actually, I host ZiggyPod this way. It' fast, scales, and the price is right :). It's "freemium", so you can pay for more later.

Payments

There are many services Google provides, which you can easily hook into via app-engine. Maps, Email, etc. But there's nothing special there. The real cool piece is hooking into Google Checkout. Sign up for an account. There are many ways you can hook into GC, but my favorite way is to set up your account to automatically update the application when a sale is made - such as making more services available. It's nearly immediate.

Head to "Settings", "Integration" in the GC page. Under "API callback URL" enter in the URL that will accept the callback. Note that this URL must be https, and that app engine only supports certificates from appspot.com. Meaning, here is one place where you will have to enter in your appspot URL. For example:

https://mynewapp.appspot.com/googlepaynotify

This assumes your app has a path "/googlepaynotify" which handles the callback. Google Checkout will send the URL XML or name=value pairs. I prefer XML - just parse it and handle it. There's plenty of documentation on the API.

This handles the callback, but how do you make the sale begin? The easiest way is to create a "Buy Now Button" - you'll see it if you click "Tools" on the top. It will generate the client-side code where you can put the button on your site. Congratulations! You can now accept sales. You can make it as complex as you want, of course, which requires more integration with the API. But that's beyond the scope here.

So far we have a homepage, full office stack, a blog, application hosting, and the ability to accept payments. What more do we need... oh yeah, customers! Let's integrate with AdWords.

AdWords

AdWords is older than dirt. It's Google's first and most profitable business unit - and with good reason. I worked for over a year as a keyword marketer, and I have one piece of advise: Don't try and game the system. All of the obvious ideas have been tried, and the cleverer ones are dominated by a few players who do that, and only that, for a living.

However - there is something you should know: Turn off keywords. OK, perhaps not completely, but your focus should be on optimizing Content Network. Two reasons for this 1) since the keyword space is basically dominated, you'll pay out the nose for decent keywords. Sure, grab a few - your company name, for example, but for the most part, take a cue from Paul McCartney. Let it be. 2) You'll have far more impression on the content network than keywords. You only pay per click, so if someone sees your website name and don't click, you just got a free impression. I get 100,000 impressions a day. Sure - most don't click, or see it. But some do - and it didn't cost all that much.

I could go on. Forever. The point being, sign up for adwords - it's a decent driver of traffic and a way to bootstrap your advertising for cheap. Just be wary of the expensive words. I never pay more than 0.50 per click. This is a secret of the pros. If the keywords you've chosen are expensive, they have high traffic, sure, but you can rather visit https://adwords.google.com/select/KeywordToolExternal, type in the good keyword, and take all of the lesser quality words instead. Let's do the math.

1 keyword @ $5.00/click * 1,000 click each = $5,000
500 (related) keywords @ $0.10/click * 2 clicks each = $10

For a savings of $4,990, you still got 1000 clicks. Of course, this takes more management. But there are several of these kinds of tricks available.

Analytics

Google Analytics is great. Before this gem, most people either rolled their own traffic trackers, or payed huge sums of money for less-awesome analytics tools. I don't have much to say about it other than sign up, put it on your home page (landing page). If you use the same account for AdWords - Analytics will track your traffic which comes from clicking on ads. A handy feature to track conversions.

Integration/Optimization

There are so many advanced tools available, it's hard to list them all here. But here are some

Conversion Tracking: From ad to successful "conversion" (visiting whatever page you consider to be a success), AdWords can self-optimize where/when/who to place your ads.

Conversion Optimizer: After a month of tracking conversions, you can start paying per action - this means rather than paying Google for keywords/content which may not succeed at sales - you can pay them only when a sale happens.

Checkout Badge: Increase AdWord conversions by letting Google notify and target people with Checkout accounts.

Many people use some aspect of Google services: gmail, applications, analytics. No need to complicate the matter - just wire these services together to create a full-stack for your new web-service company. Most anything you need (short of legal or tax advice) can be done for free or cheap.

Google App Engine Scales and Develops Fast

2009-05-06T19:01:00.019-05:00

Really. It's amazing. After a quick chat about the concept with Jim, I bet my wife I could get a web app knocked out which accepted text messages, stored and printed them in an hour (www.textbarf.com). It took 35 minutes. In fact, I have a feeling this write-up will take longer. Now THAT's progress.

(FYI - sauce: http://github.com/vocmsg/textbarf/tree/master)

Getting started
First, create a Google App Engine account. If you already have gmail, just use that. You can have up to 10 apps.

If you download the SDK and install it for your OS, you'll get a nice little launcher with it. Right-click new, and you're on your way. Launching is just as simple... just click "Deploy", enter your account login, and your app is launched. In my case, http://textbarf.appspot.com (all GAE apps are subdomains of appspot).

Code
The source code couldn't be simpler. First, appengine db models are mapped to Google's BigTable - a columnar (not relational) database. The nice thing about this kind of dataset is that OO models map fairly directly to the data on the backend (no ORM). The downside is that you'll have lots of duplicate data - since joins are really expensive. But I digress. All we need to do is store a text message, the phone number it came from, and the datetime the txt was made. Ready? Java people cover your eyes - you may cry from jealousy:

class Barf(db.Model):
  text = db.StringProperty(required=True)
  phone = db.PhoneNumberProperty()
  date = db.DateTimeProperty(auto_now_add=True)

That's it. GAE creates a BigTable type which maps to this design (as well as automatically create any necessary indexes based on given queries, which then populates the index.yaml file. I could have put this in an external module (normally I'd put it in a file called models.py and import it), but for the sake of speed and simplicity, I just put it in the main.py file generated by the GAE SDK tools.

Next, we need to deal with request. GAE webapp maps URLs to files (in app.yaml - which already points root to main.py by default), and uses 'WSGIApplication' to map internally URLs to RequestHandler classes. In this case, there are two urls: '/' and '/txtback' - one is the website URL, the other is so the SMS service can ping back data to the server. They are mapped in the main module

def main():
  app = webapp.WSGIApplication([('/', MainHandler),
                                ('/txtback', TxtBackHandler)])
  wsgiref.handlers.CGIHandler().run(app)

Let's look at the root URL.

class MainHandler(webapp.RequestHandler):
  def get(self):
    barfs = Barf.all().order('-date').fetch(250)
    
    path = os.path.join(os.path.dirname(__file__), 'templates/index.html')
    self.response.out.write(template.render(path, {'barfs': barfs}))

Again - hopefully straightforward (isn't python so readable?). If a GET request happens, first fetch 250 Barf objects reverse ordered by date. Next, get the path to an external template file and write the rendered template (passing in the barfs objects) to the response output. Most of the magic is in the template.

Template
The template is mostly a standard HTML file, with a little bit of server-side markup.

{% for barf in barfs %}
  {% ifchanged barf.date.date %}
    <li><div class="date">{{ barf.date.date }}</div></li>
  {% endifchanged %}
  <li>
    <div class="time">{{ barf.date|date:"P" }}</div>
    <div class="quote">{{ barf.text|escape }}</div>
  </li>
{% endfor %}

Again - it should be self explanatory. Iterate through each "barf" object. If the date has changed since the previous loop, output it. Then, output the barf.date formatted by the date formatter with type "P" (simple time) and the given text. That's all you need to output stored data. Google manages the rest for you - all with a scalable base.

Accepting Text Messages
Next, we want to accept text messages to populate the text Barf objects. There's no need to create a template here, a simple text response it sufficient. Our main goal is to store the input.

class TxtBackHandler(webapp.RequestHandler):
  def get(self):
    text = self.request.get('message')
    if text: text = text.strip().lower()
    phone = self.request.get('min')
    if phone: phone = phone[-10:]
    
    b = Barf(text=text, phone=phone)
    b.put()
    
    self.response.out.write('http://textbarf.com')

We expect two request attributes, create a Barf object, save it and write back the URL. Simple as that.

But that was only the callback. To accept text messages, I signed up for a 41411 shortcode account (named "tbarf") through TextMarks.com. Once you sign up and verify the account, you need to provide the callback URL. Before you can do that, there must be a URL to hit. So, I click the deploy button. 5 seconds later, the app is up an running. All that's left is to give the URL to TextMarks - in my case: http://textbarf.appspot.com/txtback?min=\p&message=\0 (where \p and \0 are interpolated with the calling phone and message data). 41411 is great, free, and they make money by ads.

Seriously - that's it. That took a grand total of 30 minutes. So, I registered a URL and pointed it at the app (http://www.textbarf.com), then just for good measure, slapped in Google Analytics. This whole app cost a total of $10, for the sweet, sweet domain.

Voilà! Eat it, EC2! Just for the record - this post took about an hour.

Code Te Ching - Verse 51

2009-04-28T23:07:00.000-05:00

The ancients followed the way of the machine.
Dark, wondrous, complex.

Difficult beyond knowing.

Yet for all their knowing, they could not control it.
Commanding:
    Machine do this!
Storing:
    Machine hold this!
Jumping:
    Machine go there!
Linking:
    Machine go here!
Finally, Yielding:
    Machine, please help!

And they taught the machine to help.

Long did they suffer for the sake of their children.
Those who return to such thought do the ancients a disservice.

So embrace what has been done.
Though the path may wind, it should never go back.

Honor their memory, and their command of memory.

    Then forget.

There is no instruction for honor.

Code Te Ching - Verse 50

2009-04-24T12:06:00.000-05:00

Learners!
Great men of learning toil lifetimes
for a glimpse of what the elite know.

Great, unconditional love flows from the master.
Great, conditional knowledge flows from the learned.

When a labor of the mind toils a lifetime to fall short of perfection,
the inaction of love sits silently, content in its experienced and subtle understanding of Code.

Google App Engine Java Support

2009-04-09T19:16:00.003-05:00

I know, I know. Envy me. Not only have I gotten to beta test the iPhone 3.0 notification and payment frameworks (incidentally, OS3.0 is piiiimmmppp), I now get to beta test the GAE J6 support. Although I'm still shocked, and slightly dismayed that the first language supported beyond Python is Java, it's clear they did it based on the very vocal rantings of the Javanistas (http://code.google.com/p/googleappengine/issues/detail?id=1) with gems such as "Java/Groovy would make this service much more serious for the individuals and development companies to work with it". Ah, the egocentric hubris of the Java-first crowd.

Well, enough about the that. The great news isn't Java support - it's JVM support. Since now I can finally run Rails (well, JRails), it may be worth a go. But I still don't know how they're going to handle the lack of opening ports.

Sadly, I've been so busy knocking the hell out of vocmsg, everything must relate to that. So, I think I'll write my iPhone notifier on GAE using J6, and dump the overview here. ¡Me parece bien!

Finally! Practical Maven Skinny Wars

2009-03-23T15:27:00.005-05:00

So, Maven 2.1 was finally released. I can't say I'm overly impressed with the pace, but considering it was driven almost exclusively by John Casey, it's a great achievement, nonetheless. Deterministic Build Lifecycle Planning (or DeBLiP, as we [well, just me] in the industry call it) is such a huge leap forward in build repeatability, it's hard to comment on at this stage. It's something best held to the clearer lens of retrospect.

But enough about that. Something of immediate practical value: real skinny WARs. Until now the maven-war-plugin left you with two choices: no jars, or most jars. Sure, you could manually exclude all but a set, but it was a PITA and unpractical. Behold my surprise to find the newest WAR config addition to 2.1-beta-1: packagingIncludes.

Use case:
I want a mostly skinny WAR that filters out all JARs except for struts.

Solution:

<project>
  ...
  <build>
    <plugins>
      <plugin>
        <artifactId>maven-war-plugin</artifactId>
        <version>2.1-beta-1</version>
        <configuration>
          <packagingIncludes>WEB-INF/lib/struts*.jar</packagingIncludes>
          <archive>
            <manifest>
              <addClasspath>true</addClasspath>
              <classpathPrefix>lib/</classpathPrefix>
            </manifest>
          </archive>
        </configuration>
      </plugin>
    </plugins>
  </build>
  ...
</project>

Thanks WAR plugin guys! Now just fix the EAR problem and I'll be set.

Code Te Ching - Verse 49

2009-03-13T12:05:00.000-05:00

The elite rests entirely, his mind is balanced and at ease.
He is not attached to methods or language, but he knows them well anyway.

A pebble cannot halt a stream, but can block a dam ten thousand times larger.
So too does the elite know, that though a language may have no use to the masses,
There are times where it simply outshines the most elaborate of methods.

The wise respect the direction of the elite,
The wise knows he is not blind to the world.

Maven Hack: Simple Skinny WARs

2009-03-01T22:04:00.005-06:00

Reading Maven's skinny-war documentation, it might turn you off to the idea. A skinny war is where your WAR does not contain any jar files, but instead get's packaged within an EAR than instead contains the jars. So rather than a zipped up EAR that looks like this:

myear-1.0.ear
\__mywar-1.0.war
     \__WEB-INF/lib/
          |__ log4j-1.2.8.jar
          \__ commons-logging-1.1.jar

It will instead look like this:

myear-1.0.ear
|__mywar-1.0.war
\__lib/
     |__ log4j-1.2.8.jar
     \__ commons-logging-1.1.jar

By default, the maven-ear-plugin assumes that any WAR dependencies already have the required jars packaged into it (in other words, it assumes a "fat" war). Because of this assumption, the EAR plugin will not package the WAR's transitive dependencies (why should it? If it did, you'd have duplicate dependencies, and the EAR would be twice the size).

The problem is, because of this assumption, when you instead provide it with a "skinny" war (one without JARs), the EAR plugin cannot know that the WAR it is given doesn't have those dependencies already. If you give the EAR as skinny WAR, the package will look like this:

myear-1.0.ear
\__mywar-1.0.war

Hardly what we wanted! The fix according to the documentation is to duplicate the WAR's dependencies in the EAR, so the EAR can download and install them. Note that the maven-ear-plugin only ignores WAR transitive dependencies, not JARs or EJBs. But duplicating all WAR dependencies isn't only a lot of typing, it breaks encapsulation of the WAR projects. What to do!? (NOTE: "breaks encapsulation" means, if your skinny WAR project depends on log4j:1.2.8 and commons-logging:1.1, then your EAR project will need to also add the dependencies log4j:1.2.8 and commons-logging:1.1. If you upgrade your WAR to use log4j:1.2.12, then you must make the change in your EAR too. Fail.)

Luckily, we can fix this egregious conundrum.

The maven-ear-plugin will package up transitive dependencies of all projects except for WARs (and possibly EAR - I've never really tried it). This means, we can add the WAR project's POM as it's own dependency.

So, your EAR dependency list will now contain two dependencies for each WAR only:

mywar:1.0:war and mywar:1.0:pom. Although the EAR won't package up your skinny WAR's dependencies, it _will_ package the POM's dependencies - which just so happen to be exactly the same as the WAR's! Now your EAR POM will look something like this.

<dependency>
  <groupId>com.myorg</groupId>
  <artifactId>mywar</artifactId>
  <version>1.0</version>
  <type>war</type>
</dependency>
<dependency>
  <groupId>com.myorg</groupId>
  <artifactId>mywar</artifactId>
  <version>1.0</version>
  <type>pom</type>
</dependency>

Yatta!