過労死 Coderoshi

Subdomains on Github

2009-09-11T09:12:00.012-05:00

I've been using Github for a while, and it didn't really occur to me to write about it. It's just something that I assume everyone uses, but don't talk much about. Like toilet paper.

But I just discovered some two-ply goodness that I had to share. Github link juice; or more correctly - instant SEO credibility by owning a github subdomain. If you visit http://coderoshi.github.com, you'll be redirected to this site. But how?

There are two way of going about it. If you visit a subdomain that hasn't been used you'll get this page:
Try it: http://alice.github.com/

That's most of the trick. Create a new public repository (don't worry, it's free) of "yourusername.github.com" (don't forget the ".github.com part). Add an index.html file and push to master. Viola.

(for you non-git people, when you create the repository, github gives you clearer instructions than I could here)

But there's more. To do a simple redirect, just paste in some html with a meta refresh tag, or to keep the nice subdomain url to mask your GeoCities site (look it up, kids) - just upload a page with a blank iframe, which contains your site.

<html>
<style>body,html{padding:0px;margin:0px;}</style>
<body>
<iframe src="http://www.coderoshi.com" width="100%" height="100%"
  style="border:0px" marginheight="0" marginwidth="0" frameborder="0">
  <a href="http://www.coderoshi.com">Coderoshi - Death By Overcoding</a>
</iframe>
</body>

But there's more! If you don't want something so sloppy, and merely want a URL redirect to your site, instead of an index.html file, upload a file named CNAME, which contains your domain. Mine looks like this:

coderoshi.com

That's it! Instant credibility from google (you're a real subdomain of the very popular github.com domain).

Codeoshi on Github

2009-09-01T19:48:00.003-05:00

Here's the link: http://github.com/coderoshi

Sure, it's been there for a while - it just recently occurred to me that I never linked to it.

Enjoy.

Code Te Ching - Verse 52

2009-08-19T00:00:00.000-05:00

Like one cast away on a desert island,
    the rolling tide keeps his head.
Like one working on the dark prairie plains,
    the chirping insects keep him company.
Listen to the hum of the machine, for that is the coder’s solitude,
    that keeps his company ahead.

The elite understands his value;
    does not lord it over his subordinates,
    does not embellish it to his superiors.

His opportunity flies like a gentle bee’s.
Alights on the first flower it sees,
And leaves just as suddenly, without a trace
But benefits the flower in endless ways.

The elite has no home but Code,
And because of this, is never homeless.

Ruby is Java and Java is C

2009-08-17T10:29:00.012-05:00

Remember 10 years ago? As "Java 2" (jdk 1.2) exploded in popularity throughout 1999, scores of developers dropped their perl/c CGI web server-side methods in favor of the new hotness Java. And why not? The language was maturing, the simplicity of ASP was ported to Java as JSPs in June, and the servlet spec was hammered out with the advent of WARs in August. Those of us lucky enough to get paid to be on this forefront found plenty of work still needed to be done - by way of tooling and optimization.

But with this new fervor of excitement rose an equal amount of riffing from the "real" server-side language: C/C++. "It may seem like you can write apps faster", they'd say, "but there's no way you can beat our tools - look at our libraries! Look at the power of our pre-processor, compile optimizations, and emacs! And - you'll be sorry when you have to spend all that time optimizing, and eventually re-write your code in C anyway." It was a fair argument, and there were certainly anecdotes over the years of companies pulling the plugin on their Java projects to revert back to C (hell, I worked on one at Cerner as late as 2004). Some Javanistas attempted to argue back - but most of us quietly hammered on, bending Java to our will - content in the knowledge that hardware would close the performance gap, and our tools would do nothing but grow. Yet still they scorned - for a while. Ten short years later, you rarely hear from that camp anymore concerning web server-side productivity or reliability. We clearly - emphatically - won. (It's a shame, really, since I kind of miss hacking C.)

But times have changed. Despite incessant claims that Moore's law is dead - it has held strong, driving forward. Hardware is cheap. Disks are cheap. Memory is cheap. And in many systems, bandwidth is cheap. Yet now, a short 10 years later, the Ruby camp suffers the slings and arrows of a myopic community - in this case Java steps into the role of C. The old, general purpose stand-by refuses to fade away in favor of it's lighter, easier, friendlier replacement. The arguments levied against Ruby these days are similar to the old arguments against Java. It's slower. It doesn't have powerful tools. It doesn't concern the needs of the "enterprise". "Real" programmers understand the difference between interfaces and abstract classes - and the value of each being distinct (I still don't know). Yadda yadda yadda...

So I'll make two predictions here. Not so bold now, but in fairness to me, I've been saying it for 4 years: Ruby will win this battle for the web over Java. Java is just too clunky of a language to survive, with too much baggage to match the agility of Ruby - or even Python for that matter.

Java - desperate to remain relevant - has at least learned something from the past. Java, as a language will die, but hopefully the JVM - arguably the greatest VM ever created (not so much by design, but by sheer effort alone) won't fade into that dark night. Hopefully EngineYard's recent investment into JRuby is just the right medicine - lest it completely drop from the web server-side market. And that really would be a shame.

Build a Business with Google

2009-07-29T08:55:00.040-05:00

Hi, I'm Eric. A C++ coder turned Java zealot turned Ruby on Rails adopter turned Django turned... whatever. They're all fine. I'm not here to talk about superior architecture, or the best language, or design or whatever. I'm here to talk about getting your web app idea out the fucking door. This is exactly what we did with MicroTasking.net. A year ago this was hard, but could be done for free. Two years ago it could be done cheaply. Five years ago you needed some funding and a lot more time. This is today - so I'll talk about the lowest barrier to entry technical stack on the market: Google.

Note - I don't work for Google, I don't get kickbacks, I don't even own any stock. But I have bootstrapped several companies, products, enterprises, forgettable services, and open source products. I also make iPhone apps.

Are you afraid of "putting all of your eggs in one Google basket"? Don't. When speaking of risk assessment, the largest risk of a startup is failing to launch. Google going away isn't your biggest concern. Scaling isn't either. Just push it out the door - then worry about those things. Once you have customers, you'd be amazed all of the problems you can pay to have solved. Like migrating to dedicated machines, if it makes you feel better.

So what do you need, from the technical side, to start a web application company? A domain name. You need a server that doesn't suck. Email/office software is nice. A blog is good. A phone number to call. What else? You also need customers. You get them through advertising and campaign optimization (which you achieve via tracking). They also need to be able to make purchases. But first thing's first:

Domain Registration/Host/Email/Office Apps

Let's start with the basics. Register a domain and get a host through Google Apps signup. Click "I want to buy a domain name" and search for one. It costs $10/year, which is more than Go Daddy. What they give is worth the extra price. You'll automatically be set up with a standard version of GA - which provides you with 50 domain-specific email accounts integrated with the Calendar, a sandboxed office suite with IM/Video Chat, Documents, Slideshows, Spreadsheets - also integrated Forms, which you can put on your Site or email, and collect results as a spreadsheet. Oh, also, they provide a website CMS.

Money down: $10

Phone Management

Sign up for Google Voice. It's not public yet - but it will be soon enough. I use it daily... it has damn near everything you need for a small company PBX. I live in Indianapolis, but my business number is San Francisco - which forwards to my cellphone. Twofold benefit. 1) you don't want to give out your private number, and 2) you don't want to give out your private number. It's free for now. You can even pay extra for a toll-free (800, 888, 866) number.

More on Email

This is a nice trick mentioned by Tim Ferriss in the 4-Hour Work Week. Since you're eventually going to grow and outsource all of your work (right?) it's not a bad idea to segment your communications in advance. Your company may just be you and a buddy for now, but you want the illusion of a larger team. Sales. Info. Webmaster. Whatever. You can do this on the Google Apps dashboard by clicking "Users and Groups" on the top, then choosing "Groups". Create as many groups as you need and just have them forward to the appropriate people. As you grow, it's easy to reroute the traffic - much easier than changing a well-known email address. Our list for MicroTasking.net is:

contact@microtasking.net, help@microtasking.net, info@microtasking.net, refund@microtasking.net, sales@microtasking.net

They all forward to me and Jim.

Blog

Get a blog via Blogger. Just sign-up with the Google Account you made above. Follow the steps to create a blog. That's easy enough, eh? What you probably want here though, is not a blog URL http://mynewblog.blogspot.com , but rather http://blog.mynewcompany.com. This is a two-step process.

First, create the blog subdomain. In the Google Apps dashboard, click "Domain Settings" above. Then click "Domain Names", then "Advanced DNS settings". Sign into enom. "Edit Host Records" and "Add New" CNAME named "blog" (or whatever). Set the address to "ghs.google.com." (don't forget the period at the end). That's it for the DNS routing table.

Next log back into Blogger. Click "Settings" and then "Publishing". Fill out Your Domain with the new subdomain (eg. http://blog.mynewcompany.com). That's it! No more lame blogspot domain.

This should cover you on the business/IT end. Let's make the app!

The Application

For years Google skirted around the edges - instead focusing on toolkits, APIs, and company tools like email. With the launch of Google App Engine (GAE) last year, they've taken a huge leap toward Amazon. What is App Engine? Think EC2+SimpleDB+S3+deployment tools+data management. It's to Python (and Java) what Heroku is for Ruby on Rails. In other words - a dream.

Sign up for a Google App Engine account. Again, just use your new gmail account. There are a lot of posts about App Engine, so I won't cover it here. But what I will tell you is, if you choose to write in Python, use Django. There a great patch project here. Follow the instructions. Use it. I've written 3 GAE projects so far - two of them without it. One with. If your application will be non-trivial, just make the investment to learn Django and use it - you'll thank me later.

Note that your application lives at http://mynewapp.appspot.com/. Again, we want to put the application under our single domain - which is painfully easy. This is where I diverge from the Google party-line. I don't use the Google Site stuff - but by default, your domain will forward to use Google Site. So I instead hijack the "www" subdomain to point to my app-engine app.

First, we need to tell Google about our application. Go back to the Dashboard, and click on "Add More Services". Under "Other services", you will see "Google App Engine" - "Enter App ID:". Do what it says (this is the unique name you gave your app - the *.appspot.com subdomain). It will then forward you to another settings page - but ignore this for now and return to the main Dashboard page. Click on the "Sites" link, then "Web address mapping". Delete the "www" mapping. Then return to your Dashboard.

Select the new "app engine" link - the one you just added. Choose "Add new URL", and enter in the subdomain "www". Viola! Now your application will be the main page. I recommend doing this because Google Sites is pretty limited, both in style and customizability. Your applications index page doesn't have to be anything more than a static template page - which you can 100% customize like any other website. Some people are taking advantage of this, and using App Engine as a pure content delivery network (CDN) to host static pages, images, etc for free. Actually, I host ZiggyPod this way. It' fast, scales, and the price is right :). It's "freemium", so you can pay for more later.

Payments

There are many services Google provides, which you can easily hook into via app-engine. Maps, Email, etc. But there's nothing special there. The real cool piece is hooking into Google Checkout. Sign up for an account. There are many ways you can hook into GC, but my favorite way is to set up your account to automatically update the application when a sale is made - such as making more services available. It's nearly immediate.

Head to "Settings", "Integration" in the GC page. Under "API callback URL" enter in the URL that will accept the callback. Note that this URL must be https, and that app engine only supports certificates from appspot.com. Meaning, here is one place where you will have to enter in your appspot URL. For example:

https://mynewapp.appspot.com/googlepaynotify

This assumes your app has a path "/googlepaynotify" which handles the callback. Google Checkout will send the URL XML or name=value pairs. I prefer XML - just parse it and handle it. There's plenty of documentation on the API.

This handles the callback, but how do you make the sale begin? The easiest way is to create a "Buy Now Button" - you'll see it if you click "Tools" on the top. It will generate the client-side code where you can put the button on your site. Congratulations! You can now accept sales. You can make it as complex as you want, of course, which requires more integration with the API. But that's beyond the scope here.

So far we have a homepage, full office stack, a blog, application hosting, and the ability to accept payments. What more do we need... oh yeah, customers! Let's integrate with AdWords.

AdWords

AdWords is older than dirt. It's Google's first and most profitable business unit - and with good reason. I worked for over a year as a keyword marketer, and I have one piece of advise: Don't try and game the system. All of the obvious ideas have been tried, and the cleverer ones are dominated by a few players who do that, and only that, for a living.

However - there is something you should know: Turn off keywords. OK, perhaps not completely, but your focus should be on optimizing Content Network. Two reasons for this 1) since the keyword space is basically dominated, you'll pay out the nose for decent keywords. Sure, grab a few - your company name, for example, but for the most part, take a cue from Paul McCartney. Let it be. 2) You'll have far more impression on the content network than keywords. You only pay per click, so if someone sees your website name and don't click, you just got a free impression. I get 100,000 impressions a day. Sure - most don't click, or see it. But some do - and it didn't cost all that much.

I could go on. Forever. The point being, sign up for adwords - it's a decent driver of traffic and a way to bootstrap your advertising for cheap. Just be wary of the expensive words. I never pay more than 0.50 per click. This is a secret of the pros. If the keywords you've chosen are expensive, they have high traffic, sure, but you can rather visit https://adwords.google.com/select/KeywordToolExternal, type in the good keyword, and take all of the lesser quality words instead. Let's do the math.

1 keyword @ $5.00/click * 1,000 click each = $5,000
500 (related) keywords @ $0.10/click * 2 clicks each = $10

For a savings of $4,990, you still got 1000 clicks. Of course, this takes more management. But there are several of these kinds of tricks available.

Analytics

Google Analytics is great. Before this gem, most people either rolled their own traffic trackers, or payed huge sums of money for less-awesome analytics tools. I don't have much to say about it other than sign up, put it on your home page (landing page). If you use the same account for AdWords - Analytics will track your traffic which comes from clicking on ads. A handy feature to track conversions.

Integration/Optimization

There are so many advanced tools available, it's hard to list them all here. But here are some

Conversion Tracking: From ad to successful "conversion" (visiting whatever page you consider to be a success), AdWords can self-optimize where/when/who to place your ads.

Conversion Optimizer: After a month of tracking conversions, you can start paying per action - this means rather than paying Google for keywords/content which may not succeed at sales - you can pay them only when a sale happens.

Checkout Badge: Increase AdWord conversions by letting Google notify and target people with Checkout accounts.

Many people use some aspect of Google services: gmail, applications, analytics. No need to complicate the matter - just wire these services together to create a full-stack for your new web-service company. Most anything you need (short of legal or tax advice) can be done for free or cheap.

Google App Engine Scales and Develops Fast

2009-05-06T19:01:00.019-05:00

Really. It's amazing. After a quick chat about the concept with Jim, I bet my wife I could get a web app knocked out which accepted text messages, stored and printed them in an hour (www.textbarf.com). It took 35 minutes. In fact, I have a feeling this write-up will take longer. Now THAT's progress.

(FYI - sauce: http://github.com/vocmsg/textbarf/tree/master)

Getting started
First, create a Google App Engine account. If you already have gmail, just use that. You can have up to 10 apps.

If you download the SDK and install it for your OS, you'll get a nice little launcher with it. Right-click new, and you're on your way. Launching is just as simple... just click "Deploy", enter your account login, and your app is launched. In my case, http://textbarf.appspot.com (all GAE apps are subdomains of appspot).

Code
The source code couldn't be simpler. First, appengine db models are mapped to Google's BigTable - a columnar (not relational) database. The nice thing about this kind of dataset is that OO models map fairly directly to the data on the backend (no ORM). The downside is that you'll have lots of duplicate data - since joins are really expensive. But I digress. All we need to do is store a text message, the phone number it came from, and the datetime the txt was made. Ready? Java people cover your eyes - you may cry from jealousy:

class Barf(db.Model):
  text = db.StringProperty(required=True)
  phone = db.PhoneNumberProperty()
  date = db.DateTimeProperty(auto_now_add=True)

That's it. GAE creates a BigTable type which maps to this design (as well as automatically create any necessary indexes based on given queries, which then populates the index.yaml file. I could have put this in an external module (normally I'd put it in a file called models.py and import it), but for the sake of speed and simplicity, I just put it in the main.py file generated by the GAE SDK tools.

Next, we need to deal with request. GAE webapp maps URLs to files (in app.yaml - which already points root to main.py by default), and uses 'WSGIApplication' to map internally URLs to RequestHandler classes. In this case, there are two urls: '/' and '/txtback' - one is the website URL, the other is so the SMS service can ping back data to the server. They are mapped in the main module

def main():
  app = webapp.WSGIApplication([('/', MainHandler),
                                ('/txtback', TxtBackHandler)])
  wsgiref.handlers.CGIHandler().run(app)

Let's look at the root URL.

class MainHandler(webapp.RequestHandler):
  def get(self):
    barfs = Barf.all().order('-date').fetch(250)
    
    path = os.path.join(os.path.dirname(__file__), 'templates/index.html')
    self.response.out.write(template.render(path, {'barfs': barfs}))

Again - hopefully straightforward (isn't python so readable?). If a GET request happens, first fetch 250 Barf objects reverse ordered by date. Next, get the path to an external template file and write the rendered template (passing in the barfs objects) to the response output. Most of the magic is in the template.

Template
The template is mostly a standard HTML file, with a little bit of server-side markup.

{% for barf in barfs %}
  {% ifchanged barf.date.date %}
    <li><div class="date">{{ barf.date.date }}</div></li>
  {% endifchanged %}
  <li>
    <div class="time">{{ barf.date|date:"P" }}</div>
    <div class="quote">{{ barf.text|escape }}</div>
  </li>
{% endfor %}

Again - it should be self explanatory. Iterate through each "barf" object. If the date has changed since the previous loop, output it. Then, output the barf.date formatted by the date formatter with type "P" (simple time) and the given text. That's all you need to output stored data. Google manages the rest for you - all with a scalable base.

Accepting Text Messages
Next, we want to accept text messages to populate the text Barf objects. There's no need to create a template here, a simple text response it sufficient. Our main goal is to store the input.

class TxtBackHandler(webapp.RequestHandler):
  def get(self):
    text = self.request.get('message')
    if text: text = text.strip().lower()
    phone = self.request.get('min')
    if phone: phone = phone[-10:]
    
    b = Barf(text=text, phone=phone)
    b.put()
    
    self.response.out.write('http://textbarf.com')

We expect two request attributes, create a Barf object, save it and write back the URL. Simple as that.

But that was only the callback. To accept text messages, I signed up for a 41411 shortcode account (named "tbarf") through TextMarks.com. Once you sign up and verify the account, you need to provide the callback URL. Before you can do that, there must be a URL to hit. So, I click the deploy button. 5 seconds later, the app is up an running. All that's left is to give the URL to TextMarks - in my case: http://textbarf.appspot.com/txtback?min=\p&message=\0 (where \p and \0 are interpolated with the calling phone and message data). 41411 is great, free, and they make money by ads.

Seriously - that's it. That took a grand total of 30 minutes. So, I registered a URL and pointed it at the app (http://www.textbarf.com), then just for good measure, slapped in Google Analytics. This whole app cost a total of $10, for the sweet, sweet domain.

Voilà! Eat it, EC2! Just for the record - this post took about an hour.

Code Te Ching - Verse 51

2009-04-28T23:07:00.000-05:00

The ancients followed the way of the machine.
Dark, wondrous, complex.

Difficult beyond knowing.

Yet for all their knowing, they could not control it.
Commanding:
    Machine do this!
Storing:
    Machine hold this!
Jumping:
    Machine go there!
Linking:
    Machine go here!
Finally, Yielding:
    Machine, please help!

And they taught the machine to help.

Long did they suffer for the sake of their children.
Those who return to such thought do the ancients a disservice.

So embrace what has been done.
Though the path may wind, it should never go back.

Honor their memory, and their command of memory.

    Then forget.

There is no instruction for honor.

Code Te Ching - Verse 50

2009-04-24T12:06:00.000-05:00

Learners!
Great men of learning toil lifetimes
for a glimpse of what the elite know.

Great, unconditional love flows from the master.
Great, conditional knowledge flows from the learned.

When a labor of the mind toils a lifetime to fall short of perfection,
the inaction of love sits silently, content in its experienced and subtle understanding of Code.

Google App Engine Java Support

2009-04-09T19:16:00.003-05:00

I know, I know. Envy me. Not only have I gotten to beta test the iPhone 3.0 notification and payment frameworks (incidentally, OS3.0 is piiiimmmppp), I now get to beta test the GAE J6 support. Although I'm still shocked, and slightly dismayed that the first language supported beyond Python is Java, it's clear they did it based on the very vocal rantings of the Javanistas (http://code.google.com/p/googleappengine/issues/detail?id=1) with gems such as "Java/Groovy would make this service much more serious for the individuals and development companies to work with it". Ah, the egocentric hubris of the Java-first crowd.

Well, enough about the that. The great news isn't Java support - it's JVM support. Since now I can finally run Rails (well, JRails), it may be worth a go. But I still don't know how they're going to handle the lack of opening ports.

Sadly, I've been so busy knocking the hell out of vocmsg, everything must relate to that. So, I think I'll write my iPhone notifier on GAE using J6, and dump the overview here. ¡Me parece bien!

Finally! Practical Maven Skinny Wars

2009-03-23T15:27:00.005-05:00

So, Maven 2.1 was finally released. I can't say I'm overly impressed with the pace, but considering it was driven almost exclusively by John Casey, it's a great achievement, nonetheless. Deterministic Build Lifecycle Planning (or DeBLiP, as we [well, just me] in the industry call it) is such a huge leap forward in build repeatability, it's hard to comment on at this stage. It's something best held to the clearer lens of retrospect.

But enough about that. Something of immediate practical value: real skinny WARs. Until now the maven-war-plugin left you with two choices: no jars, or most jars. Sure, you could manually exclude all but a set, but it was a PITA and unpractical. Behold my surprise to find the newest WAR config addition to 2.1-beta-1: packagingIncludes.

Use case:
I want a mostly skinny WAR that filters out all JARs except for struts.

Solution:

<project>
  ...
  <build>
    <plugins>
      <plugin>
        <artifactId>maven-war-plugin</artifactId>
        <version>2.1-beta-1</version>
        <configuration>
          <packagingIncludes>WEB-INF/lib/struts*.jar</packagingIncludes>
          <archive>
            <manifest>
              <addClasspath>true</addClasspath>
              <classpathPrefix>lib/</classpathPrefix>
            </manifest>
          </archive>
        </configuration>
      </plugin>
    </plugins>
  </build>
  ...
</project>

Thanks WAR plugin guys! Now just fix the EAR problem and I'll be set.

Code Te Ching - Verse 49

2009-03-13T12:05:00.000-05:00

The elite rests entirely, his mind is balanced and at ease.
He is not attached to methods or language, but he knows them well anyway.

A pebble cannot halt a stream, but can block a dam ten thousand times larger.
So too does the elite know, that though a language may have no use to the masses,
There are times where it simply outshines the most elaborate of methods.

The wise respect the direction of the elite,
The wise knows he is not blind to the world.

Maven Hack: Simple Skinny WARs

2009-03-01T22:04:00.005-06:00

Reading Maven's skinny-war documentation, it might turn you off to the idea. A skinny war is where your WAR does not contain any jar files, but instead get's packaged within an EAR than instead contains the jars. So rather than a zipped up EAR that looks like this:

myear-1.0.ear
\__mywar-1.0.war
     \__WEB-INF/lib/
          |__ log4j-1.2.8.jar
          \__ commons-logging-1.1.jar

It will instead look like this:

myear-1.0.ear
|__mywar-1.0.war
\__lib/
     |__ log4j-1.2.8.jar
     \__ commons-logging-1.1.jar

By default, the maven-ear-plugin assumes that any WAR dependencies already have the required jars packaged into it (in other words, it assumes a "fat" war). Because of this assumption, the EAR plugin will not package the WAR's transitive dependencies (why should it? If it did, you'd have duplicate dependencies, and the EAR would be twice the size).

The problem is, because of this assumption, when you instead provide it with a "skinny" war (one without JARs), the EAR plugin cannot know that the WAR it is given doesn't have those dependencies already. If you give the EAR as skinny WAR, the package will look like this:

myear-1.0.ear
\__mywar-1.0.war

Hardly what we wanted! The fix according to the documentation is to duplicate the WAR's dependencies in the EAR, so the EAR can download and install them. Note that the maven-ear-plugin only ignores WAR transitive dependencies, not JARs or EJBs. But duplicating all WAR dependencies isn't only a lot of typing, it breaks encapsulation of the WAR projects. What to do!? (NOTE: "breaks encapsulation" means, if your skinny WAR project depends on log4j:1.2.8 and commons-logging:1.1, then your EAR project will need to also add the dependencies log4j:1.2.8 and commons-logging:1.1. If you upgrade your WAR to use log4j:1.2.12, then you must make the change in your EAR too. Fail.)

Luckily, we can fix this egregious conundrum.

The maven-ear-plugin will package up transitive dependencies of all projects except for WARs (and possibly EAR - I've never really tried it). This means, we can add the WAR project's POM as it's own dependency.

So, your EAR dependency list will now contain two dependencies for each WAR only:

mywar:1.0:war and mywar:1.0:pom. Although the EAR won't package up your skinny WAR's dependencies, it _will_ package the POM's dependencies - which just so happen to be exactly the same as the WAR's! Now your EAR POM will look something like this.

<dependency>
  <groupId>com.myorg</groupId>
  <artifactId>mywar</artifactId>
  <version>1.0</version>
  <type>war</type>
</dependency>
<dependency>
  <groupId>com.myorg</groupId>
  <artifactId>mywar</artifactId>
  <version>1.0</version>
  <type>pom</type>
</dependency>

Yatta!

The Organ Grinder: Cubicals

2009-01-19T15:05:00.001-06:00

The Cube is a Lie. Cubicles are amongst the most insidious creations ever, erm, created - and have gotten worse.

A short history of the cube:
  Working in the open
  Working inside
  Working in an office
  Office shrinks
  Invent cubicles, put people in boxes
  Shrink them
  Start removing walls

The most uncomfortable layout is the cubical where 1) your boss can see your back and 2) you see a wall. He can stand behind you, watch you work, and you have no idea he's even there. This is fucking retarded. Sure, I check my email 4 times a day - and I swear to Jesus - that every time I do so, my boss shows up... it makes me nervous. And a nervous coder is not a good coder.

So, this common problem has a common solution: mirrors.

The best is a small mirror. I'm a fan of CHIMP. It's kind of obvious, but most bosses won't catch on, and if they do, won't say much about it. It's kind of a dick move to tell your employees "Take that mirror down, because I want to keep sneaking up on you." But what if you can't have a mirror?

Here's what you do (RE: what I did): CDs. Store-bought CDs work best, since they tend to be more mirror-ish - burnable CD's tend to have a colored hugh. BluRay discs are actually glossier, but that's a lot of cash hanging on your wall.

Anyway, there's not a point to this post other than, it's always good to know when people are standing behind you.

Ahh shit! Caught blogging... alt-tab. AAAHHHH.... THAT WAS CAPS LOCK! Tab.... taaaabbb!!!!

Code Te Ching - Verse 48

2009-01-02T12:04:00.000-06:00

The lamer desires to be the hacker.
The hacker desires to be the electrician.
The electrician desires to be the mathematician.
The mathematician desires to be pure.

The pure is free from desire.

Never be clever in two ways where one will do.

Long Time, NP

2008-12-13T11:33:00.003-06:00

The title has nothing to do with this post, let me get that straight. It just sounded funny, as all things do after 19 hour days. No, that isn't a typo.

Some folks have asked where I've been, and the answer is, blogging on Vontoo's corporate blog. Can't I do both? Well, yes, but I'd rather have one good quality than two half quality posts. Like all good bloggers, my upcoming new years resolution is to post more. We'll see how NP-hard that is to do... lord, I need some sleep.

Annotation Hell

2008-09-08T15:24:00.014-05:00

XML Hell, the phenomenon that killed struts, and fear of which launch Seam. Annotations were the savior, so the prophets proclaimed. But have we gone too far? You decide.

  /**
   * @return related details for this person
   */
  @ManyToOne( fetch = FetchType.EAGER, cascade = 
    { javax.persistence.CascadeType.PERSIST, javax.persistence.CascadeType.MERGE } )
  @JoinColumn( name = "PERSON_DETAIL", nullable = false )
  @Where( clause = "ACTV_ID = 'T'" )
  @Cascade( { CascadeType.SAVE_UPDATE, CascadeType.MERGE, CascadeType.DELETE } )
  @Cache( usage = CacheConcurrencyStrategy.NONSTRICT_READ_WRITE )
  @EqualsMember
  @NotNull
  @Valid
  @Message( "${msg.invalid.person.detail}" )
  @XmlElement( name = "person-detail", required = true )
  public PersonDetail getPersonDetail()
  {
      return personDetail;
  }

This is a method of a Hibernate ORM object which can be serialized via JAXB. Also utilizes Hibernate validation, an i18n Message manager and EqualsMember, which means this element is a member of the set of fields denoting object equality. Somehow, I'm starting to miss XML...

The Organ Grinder: Firewall Tunneling via EC2 Proxy

2008-09-03T12:20:00.001-05:00

Teaching monkeys new tricks.

I've been around. Let's get that straight. Sure, I no longer work in a corporate environment yet still, on occasion, feel the ache of an old scar. The wounds of the corporate blade just cut too deep. This world is hell to anyone who doesn't thrive in such artificial environments. This is a new segment for a new year (well, fiscal year) - The Organ Grinder: Teaching (Code)Monkeys New Tricks. Perhaps not new, but useful for those who need to survive in this wilderness. Today's trick: getting around corporate firewalls via tunneling.

I understand why some corporations wish to limit internet access to the common folk: ensuring that the receptionist does his job instead of playing WoW, blocking an old-lady manager from downloading the new forward "cute-kittens.jpg.exe", stopping sales guys from surfing for porn all day. However, when you are a software developer, blocking your access to the wild, wild web is like keeping a goldfish in one of those colored baggies of water they give out at carnivals; sure the fish live - sort of - only to die a day later when you take it home. Get the analogy? The fish is your soul.

1) Set up EC2

Firstly, you must have Java 1.5 or greater installed, and the JAVA_HOME env var set.
in unix

$ export JAVA_HOME=<PATH>

in windows

C:\> set JAVA_HOME=<PATH>

Create an EC2 account on Amazon (all of these steps can be found in detail here).

Set up the account and grab your X.509 certs.

Download and unzip the EC2 command-line tools - works for *nix, osx and windows (afaik).

Set up your *nix/osx paths:

$ export EC2_HOME=<path-to-tools>
$ export PATH=$PATH:$EC2_HOME/bin
$ export EC2_PRIVATE_KEY=~/.ec2/pk-HKZYKTAIG2ECMXYIBH3HXV4ZBZQ55CLO.pem 
$ export EC2_CERT=~/.ec2/cert-HKZYKTAIG2ECMXYIBH3HXV4ZBZQ55CLO.pem

Or, set up your windows paths:

C:\> set EC2_HOME=<path-to-tools>
C:\> set PATH=%PATH%;%EC2_HOME%\bin
C:\> set EC2_PRIVATE_KEY=c:\ec2\pk-HKZYKTAIG2ECMXYIBH3HXV4ZBZQ55CLO.pem 
C:\> set EC2_CERT=c:\ec2\cert-HKZYKTAIG2ECMXYIBH3HXV4ZBZQ55CLO.pem

run:

$ ec2-add-keypair my-proxy

This will output a key similar in structure to this:

-----BEGIN RSA PRIVATE KEY-----
MIIEoQIBAAKCAQBuLFg5ujHrtm1jnutSuoO8Xe56LlT+HM8v/xkaa39EstM3/aFxTHgElQiJLChp
... blah blah blah, more data ...
-----END RSA PRIVATE KEY-----

copy/paste everything between and including "-----BEGIN RSA PRIVATE KEY-----" to "-----END RSA PRIVATE KEY----- " into a file named:

id_rsa-my-proxy

Then change the permissions of the file

chmod 600 id_rsa-my-proxy

2) EC2 Instance
Use the the Ubuntu 7.1 ami.

ec2-run-instances ami-b111f4d8 -k my-proxy

Which will output a line similar to the following:

INSTANCE        i-10a64379   ami-b111f4d8     pending   my-proxy  0

It will take a few minutes for the instance to launch. To check on its status:

ec2-describe-instances

When the instance is finally running, you will see the instance domain as something like the following:

domU-12-34-31-00-00-05.usma1.compute.amazonaws.com

You can ssh in as you wold expect, using the domain. Root has no password.

ssh -i id_rsa-my-proxy root@domU-12-34-31-00-00-05.usma1.compute.amazonaws.com

3) SSH Proxy

If you can connect out through port 443, set up sshd to listen on it. Avoid port 80. Your network may filter encrypted data on that port. If not, you'll have to drop down to port 80 and pray.

ssh -ND 1337 root@domU-12-34-31-00-00-05.usma1.compute.amazonaws.com

What this does is routes all localhost requests from port 1337 (or whatever port you want) to your EC2 ssh server (over port 22).

4) Important! Shutdown!

Amazon charges you $0.10 for every hour your instance is running. When you are done with your proxy, don't forget to shut it off! Ten cents isn't much money, but it adds up to approx. $72/mo if you leave it running all the time. Pretty expensive for a simple proxy! But in reality, you'll probably only need it a few times a week, costing you maybe $2 per month (20 hours). Use the instance id to terminate, not the domain (ec2-describe-instances if you have forgotten the id). Something like:

ec2-terminate-instances i-10a64379

Of course, if you get caught, don't blame me. Or do blame me, I don't really care.

Code Te Ching - Verse 47

2008-09-01T12:02:00.000-05:00

Old technology is old, but not useless.
New technology is new, but does not devalue the old.

The greatest code was work of the ancients.
Subtle. Efficient. Genius.

Like the days of old where the weakest would not survive,
Come the days of today where many weak survive on ancient efforts.

The weak rest blindly on their fathers' graves.
Knowing nothing about what lies beneath the earth.
The wise know what lies beneath, but do not dwell upon it.
Because of this, the ghosts of the past will walk with them in times of need.

He occupies his time:
    Studying the ancient ways.
    Understanding the future direction.
    Cultivating his skill.
    Remaining faithful to the past without resting upon it.
From his gentle actions the world remains powered.

In this way: the ancients are not lost.
In this way: the elite is immortal.

Ruby on Rails on REST on Java

2008-08-05T18:50:00.072-05:00

(or Rails can Scales!)

In this 'very special issue' of Coderoshi, I'll go over how to create a scalable REST service in Java using CXF that can be consumed by Rails via ActiveResource. I do this based on the following assumptions:

* Rails sucks at scaling, but rocks at creating quick, slick presentations.
* Java sucks at writing presentation code, but scales like a mofo.

If you disagree, and really do believe Rails can scale with ActiveRecord's simple abstraction of an RDBMS, or that Java is awesome for presentation, best to stop reading now. But if you know enough about these technologies to muster a knowing nod, then hold onto your face - it's about to be rocked off.

What's all this, then?

Let us begin with an overview: Ruby on Rails ActiveResource consuming Java REST via CXF defined by JAXB. Whaa??

ActiveResource (not to be confused with ActiveRecord, Ruby's ORM) is a simple REST consumer. That's it. It wraps structured REST actions and managed data (xml or json) in active objects. Just like ActiveRecord, executing methods on an ARes object triggers backend actions or accessed data. For example:

@customer = Customer.find( 1 )

will translate into executing some defined REST service to populate @customer, internally constructing a URL like "http://server/customers/1.xml", getting the xml returned:

<customer>
  <id>1</id>
  <name>Joe</name>
</customer>

And making the value accessible by the Ruby program

#outputs: Joe
puts @customer.name

It's probably the drop-dead easiest method of consuming a remote service - assuming the remote service conforms to ActiveResource's standards (there's always a catch).

Happily, this isn't too difficult to do with the magic ingredient of the server-side: CXF. CXF is the evolution of Dan Diephouse's XFire (merged with IONA's Celtix). It's primarily considered to be a simple annotated WS-* impl, but also has some mean REST hooks. We'll be using the JSR 311 implementation rather than CXF's custom annotations.

Getting Started
Although I suggest you follow along, if you just plain hate writing code, then download both projects here. The Ruby on Rails project is called "ror" and the REST on Java project is "roj".

I hear it time and time again: Ruby on Rails can't scale. This is, of course, an absurd statement. If your software gets to the point that this is really a problem, well, then take your millions and rearchitect it. You can dry your eyes on hundred dollar bills while your Java counterparts are still fighting to get GWT to work. Most projects fail anyway - and I don't have enough lives to waste on them.

People complain that Java sucks at presentation, and for good reason. Sure, you can write webservices that scale out to thousands of servers, but try and write JSF and you have two choices: drink the koolaid, or pray for death.

With those points in mind, I'm going to go over how to create a scalable REST service in Java using CXF that can be consumed by Rails new ActiveResource.

Ingredients

Like a good cook, I like to gather my ingredients up front. Here is what we'll need to bake our little cake (I'm bolding the versions in case you can't read:

* Ruby >= 1.8.6
* Ruby on Rails >= 2.1 (2.0 is required for ActiveResource, but 2.1 has tons of bug fixes)
* Java >= 6 (or v5 - I haven't tried it)
* Maven >= 2.0.9 (you need the new graph resolution stuff, or dep overload may not work)

I presume you understand these technologies, at least in a cursory way. If not, there are plenty of good guides to get you started. Go ahead. I'll wait...

Oh, you're back? Then let's begin.

REST on Java

Although this is about scalability, the focus here isn't about handling multiple concurrent nodes. If you need a super-scalable data-management system check out HBase. If you want to cluster thousands of JVMs, use Terracotta. Wire it all up with Spring, and deploy using Maven. There's your scalability, buddy.

Let's begin with the Java service side. In a nutshell, we're going to use Maven to manage our dependencies (naturally), Spring to wire our components (of course), JAXB to define our XML (what else?), CXF as our REST service and Jetty as our web server. The easy way to start (besides downloading the finished product) is use a Maven archetype to generate a basic WAR.

mvn archetype:generate

Choose "maven-archetype-webapp" (mine was #18), make up a groupId, artifactId and version, and "packaging: war". Next, replace the generated pom with this one:

<project>
  <modelVersion>4.0.0</modelVersion>
  <groupId>com.coderoshi</groupId>
  <artifactId>roj</artifactId>
  <version>1.0-SNAPSHOT</version>
  <packaging>war</packaging>

  <build>
    <plugins>
      <plugin>
        <artifactId>maven-compiler-plugin</artifactId>
        <configuration>
          <source>1.6</source>
          <target>1.6</target>
        </configuration>
      </plugin>
      <plugin>
        <groupId>org.mortbay.jetty</groupId>
        <artifactId>maven-jetty-plugin</artifactId>
        <version>6.1.11</version>
        <configuration>
          <contextPath>/</contextPath>
        </configuration>
      </plugin>
    </plugins>
  </build>

  <dependencies>
    <dependency>
      <groupId>org.apache.cxf</groupId>
      <artifactId>cxf-rt-frontend-jaxrs</artifactId>
      <version>2.1.1</version>
    </dependency>
    <dependency>
      <groupId>org.apache.cxf</groupId>
      <artifactId>cxf-rt-transports-http-jetty</artifactId>
      <version>2.1.1</version>
    </dependency>
    <dependency>
      <groupId>org.apache.cxf</groupId>
      <artifactId>cxf-rt-bindings-http</artifactId>
      <version>2.1.1</version>
    </dependency>
    <dependency>
      <groupId>org.springframework</groupId>
      <artifactId>spring-core</artifactId>
      <version>2.5.5</version>
    </dependency>
    <dependency>
      <groupId>org.springframework</groupId>
      <artifactId>spring-context</artifactId>
      <version>2.5.5</version>
    </dependency>
  </dependencies>
</project>

This pom does two important things: gives us the spring and CXF dependencies that we need, and configures the plugins that we require. Namely, that we assume JDK 1.6, and that we want to use the Jetty plugin to run our war (eventually).

Next we'll set up our Spring's context file. This describes to spring how much we love CXF and really want to use it. Spring replies by constructing our components correctly for us. Isn't that nice? Create a new file under "src/main/webapp/WEB-INF/" named "beans.xml", and paste the following:

<beans xmlns="http://www.springframework.org/schema/beans"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xmlns:jaxrs="http://cxf.apache.org/jaxrs"
  xmlns:cxf="http://cxf.apache.org/core"
  xsi:schemaLocation="
  http://cxf.apache.org/core http://cxf.apache.org/schemas/core.xsd
  http://cxf.apache.org/jaxrs http://cxf.apache.org/schemas/jaxrs.xsd
  http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd">

  <import resource="classpath:META-INF/cxf/cxf.xml" />
  <import resource="classpath:META-INF/cxf/cxf-extension-jaxrs-binding.xml" />
  <import resource="classpath:META-INF/cxf/cxf-servlet.xml" />

  <jaxrs:server id="campaignREST" address="/">
    <jaxrs:serviceBeans>
      <bean class="com.coderoshi.service.CampaignService" />
    </jaxrs:serviceBeans>
    <jaxrs:entityProviders>
      <bean class="com.coderoshi.service.JAXBCollectionProvider" />
    </jaxrs:entityProviders>
    <jaxrs:features><cxf:logging /></jaxrs:features>
  </jaxrs:server>
</beans>

Cryptic? Well, no, not if you already know Spring. If not, here's the skinny: We are importing some built-in CXF Spring configurations (thanks CXF!) and using them to define our own REST server (we haven't written it yet) named "campaignREST". The "jaxrs:serviceBeans" element contains a list of services we define, in this case, one we are about to write called "com.coderoshi.service.CampaignService". The block between "jaxrs:features" turns on logging, so we can view incomming/outgoing messages and errors. Finally, the "jaxrs:entityProviders" contains a bean we are about to write to convert a JAXB Collection into an XML form consumable by Rails.

Finally, we need to tell the web server (Jetty) that it needs to load up Spring and also the CXF servlet (which handles requests and passes them off to our REST service).

<?xml version="1.0" encoding="ISO-8859-1"?>
<web-app>
  <context-param>
    <param-name>contextConfigLocation</param-name>
    <param-value>WEB-INF/beans.xml</param-value>
  </context-param>

  <listener>
    <listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
  </listener>

  <servlet>
    <servlet-name>CXFServlet</servlet-name>
    <servlet-class>org.apache.cxf.transport.servlet.CXFServlet</servlet-class>
    <load-on-startup>1</load-on-startup>
  </servlet>

  <servlet-mapping>
    <servlet-name>CXFServlet</servlet-name>
    <url-pattern>/*</url-pattern>
  </servlet-mapping>
</web-app>

Whew! With configuration out of the way, let's get to the fun part... coding!

First, we'll start with the JAXB bean we eventually want to serialize. Just to keep it simple, we'll make public fields and forgo getters and setters. We're using Maven to build, so you'll need to create the package under src/main/java (where "java" is a sibling of the "webapp" directory.

package com.coderoshi.service;

@javax.xml.bind.annotation.XmlRootElement( name = "campaign" )
public class Campaign
{
  public long id;
  public String name;
  public long budget;

  public Campaign() {}
  public Campaign( long i, String n, long b ) { id=i; name=n; budget=b; }
}

Ah, the magic of JAXB! That's all you need to create an XML-marshallable object. Eat it, Rails (and you're about to...). Now we get to make our REST server. Create a new class named "com.coderoshi.service.CampaignService". Next we need to annotate the class with two annotations:

@javax.ws.rs.Path( "/campaignService" )
@javax.ws.rs.ProduceMime( "application/xml" )

These tell the server that this is a service available at the HTTP path "/campaignService", and that by default, calling this service will return XML (it sets the HTTP header's mime-type on response).

Let's write our standard CRUD operations (which Rails' ActiveResource will presume exist by default).

It's important to note that we're faking data and operations. If you downloaded the zip provided, then you'll see I made the service database-backed (HSQLDB), but that's just for fun, and really is beyond the code here.

Create (addCampaign)

@Path( "/campaignService" )
@ProduceMime( "application/xml" )
public class CampaignService
{
  // ... define actions ...
  @POST
  @Path("/campaigns.xml")
  @ConsumeMime( "application/xml" )
  public Response addCampaign( Campaign c )
  {
    System.out.println( "Adding Campaign: " + c.name );
    c.id = 99;
    // Adding header Location=/{id} is required so Rails can extract an id
    return Response.status( 200 ).header( "Location", "/" + c.id ).build();
  }

Rails presumes all adds are "POST" operations. It also assumes the path is the plural name of the resource ("campaigns") and post-fixed by the produced mime-type. In our case XML. On a successful update ActiveResource will assume a 200 (successful) response. We can play with error handling at a later time. Since we are expecting data is in XML form, we explicitly enforce it to be so with @ConsumeMime. Note that all of these method annotations are in the JSR 311 as the "javax.ws.rs.*" package.

Read (getCampaign, getCampaigns)

  @GET
  @Path("/campaigns.xml")
  public List<Campaign> getCampaigns()
  {
    return Arrays.asList( new Campaign(1, "campaign #1", 500), new Campaign(2, "campaign #2", 600) );
  }

  @GET
  @Path("/campaigns/{id}.xml")
  public Campaign getCampaign( @PathParam( "id" ) long id )
  {
    return new Campaign(id, "campaign #"+id, 900);
  }

In Rails REST reads are GET operations. It also presumes the pathnames begin with "campaigns", but notice this time the singular "getCampaign" has a param called "id". In CXF/JSR311 you can define that you expect a path to contain a variable - in this case, we expect an ID. However, since Java doesn't keep track of parameter names after compilation, the best thing you can do is name the parameter you want CXF to populate for you. Most basic types are supported, like String or int - in our case long. So when someone "GETs" our url "http://localhost:8080/campaignService/campaigns/35.xml", the "getCampaign" method is called and the value "35" is passed in as parameter "id".

Also notice that our getCampaigns path is the same as addCampaign. How can this be? Because although the URLs are the same, different operations take place on them (POST versus GET). This is how CXF differentiates between the operations and knows when to expect XML that conforms to our Campaign JAXB, or no arguments at all.

Update (updateCampaign)

  @PUT
  @Path("/campaigns/{id}.xml")
  @ConsumeMime( "application/xml" )
  public Response updateCampaign( @PathParam( "id" ) long id, Campaign c )
  {
    System.out.println( "Updating Campaign: " + id );
    return Response.status( 200 ).build();
  }

Here we PUT Campaign XML to the server, along with the ID. Although technically we shouldn't require the ID (since it should already be in the Campaign data), Rails builds URLs that way, so we need to also. Don't ask me why - I don't know.

Delete (deleteCampaign)

  @DELETE
  @Path("/campaigns/{id}.xml")
  public Response deleteCampaign( @PathParam( "id" ) long id )
  {
    System.out.println( "Deleting Campaign: " + id );
    return Response.status( 200 ).build();
  }
}

Finally we handle delete. To delete an object you merely need to pass in the ID as a DELETE operation. Here we don't really do anything, just print to the console so you know it worked.

Yeah yeah, I know REST is not necessarily CRUD, but for our purposes the mapping is close enough, and we're going to run with it.

Yay! We wrote our Java REST server - but we have a problem. Our "getCampaigns" method returns a "List" (list of JAXB Campaigns) - but CXF doesn't know how to handle a list like this. And it certainly wouldn't handle a JAXB list in a way friendly to ActiveResource, which has it's own expectations.

Luckily for us JSR-311/CXF allows you to create your own custom providers. Providers are responsible for translating to and from one kind of data (an JAXB object, for example) from and to another (an XML stream). The one we're about to write has already been defined in the Spring context above under "jaxrs:entityProvider". Without too much ado, here it is in it's entirety:

package com.coderoshi.service;

import java.io.*;
import java.util.*;
import javax.ws.rs.*;
import javax.ws.rs.core.*;
import javax.ws.rs.ext.*;
import javax.xml.bind.*;

import org.apache.cxf.binding.http.strategy.EnglishInflector;

@Provider
@ProduceMime( "application/xml" )
public class JAXBCollectionProvider
  implements MessageBodyWriter<Collection<?>>
{
  public boolean isWriteable( Class<?> type )
  {
    return Collection.class.isAssignableFrom( type );
  }

  public long getSize( Collection<?> l )
  {
    return l == null ? 0 : l.size();
  }

  public void writeTo( Collection<?> li, MediaType mt, MultivaluedMap<String, Object> headers, OutputStream os )
    throws IOException
  {
    try
    {
      if( li == null || li.isEmpty() )
      {
        os.write( "<nil type=\"array\" />".getBytes() );
        return;
      }

      Marshaller marshaller = null;
      String envelopeName = null;
      for ( Object object : li )
      {
        if ( object != null )
        {
          if ( marshaller == null )
          {
            JAXBContext context = JAXBContext.newInstance( object.getClass() );
            marshaller = context.createMarshaller();
            marshaller.setProperty( Marshaller.JAXB_FRAGMENT, true );

            envelopeName = context.createJAXBIntrospector().getElementName( object ).getLocalPart();

            // pluralize the collection
            envelopeName = new EnglishInflector().pluralize( envelopeName );

            os.write( "<".getBytes() );
            os.write( envelopeName.getBytes() );
            os.write( " type=\"array\">".getBytes() );
          }
          marshaller.marshal( object, os );
        }
      }
      if ( envelopeName != null )
      {
        os.write( "</".getBytes() );
        os.write( envelopeName.getBytes() );
        os.write( ">".getBytes() );
      }
    }
    catch ( JAXBException e )
    {
      throw new IOException( "broke", e );
    }
  }
}

Whew! That's some chunk of code, where to begin? Let's start at the top.

Notice that we annotate our class with

@Provider
@ProduceMime( "application/xml" )

These tell the framework that this is an available provider implementation and that it produces XML. We implement "MessageBodyWriter", which contains hook methods for the REST container. "isWriteable" checks if outbound object classes can utilize this Provider. In our case, we can use anything that is a Collection (which, of course, our List<Campaign> most certainly is).

The "writeTo" method does the actual work of marshaling any approved Collection object directly into the output stream. Here is where things get weird. First we ensure that the collection contains objects. If not, we just output "<nil type=\"array\" />". This is a flag to Rails' ActiveResource that we would like to return an array, but we don't have any data to give it. We call the element "nil" since we don't have any objects, we can't exactly know what types they are.

Assuming our collection contains value, we first wrap the collection within a pluralized version of the elements we envelope. For example, since our List contains two Campaign JAXB objects named "campaign", we wrap them in a root element named "campaigns". How can we know how to correctly pluralize words? We use a built-in CXF class called "EnglishInflector". We then define the type as "array" for the benefit of the ActiveResource consumer.

That's it! If it seems like a lot, don't worry. When it's all together and chugging along, it's amazing how little code there really is (considering we're dealing with Java here).

Running the REST Service

Assuming no typos or environment problems, you can now run the server using the Maven Jetty plugin. Don't worry about building it, the plugin will ensure we are built before launching. So go to the project base dir, and type:

mvn clean jetty:run-war

Navigate your browser to "http://localhost:8080/campaignService/campaigns.xml", and you should be treated to a list of your campaigns:

<campaigns type="array">
  <campaign>
    <id>1</id>
    <name>campaign #1</name>
    <budget>500</budget>
  </campaign>
  <campaign>
    <id>2</id>
    <name>campaign #2</name>
    <budget>600</budget>
  </campaign>
</campaigns>

Ruby on Rails

Now on to the Ruby on Rails frontend, which gladly, is simpler. Would you expect less from Rails? Like the Java backend, we can build the project from scratch, or you can download the completed project here.

rails ror
cd ror

Next, create the model file "app/models/campaign.rb" with the contents (assuming your server is at port 8080:

class Campaign < ActiveResource::Base
  self.site = "http://localhost:8080/campaignService"
end

That's it for the ActiveResource. Rails presumes the rest (REST, get it? ha ha). It points the ActiveResource to the given site as a base URL, and from there constructs URLs based upon a pluralized version of the classname. For each of the following ActiveResource actions:

Campaign.find(:all)        # GET http://localhost:8080/campaignService/campaigns.xml
Campaign.find(7)           # GET http://localhost:8080/campaignService/campaigns/7.xml
Campaign.create(data).save # POST http://localhost:8080/campaignService/campaigns.xml
Campaign.find(7).save      # PUT http://localhost:8080/campaignService/campaigns/7.xml
Campaign.delete(7)         # DELETE http://localhost:8080/campaignService/campaigns.xml

This covers it for the ActiveResource, but we need something to show the user. Luckily Rails makes this simple with scaffold generation.

./script/generate scaffold Campaign name:string budget:integer

This generates a controller, resources in the routing table, test cases, and attempt to generate a model in the form of an activerecord object. But since we already created our campain ActiveResource there, it just skips it.

Next, we need to set up the server. Since we don't actually need Rails to run on a database anymore, we can configure the server to skip it. Open up your "config/environment.rb" file, and within "RailsInitializer" add the line:

  config.frameworks -= [ :active_record ]

To stop active_record from loading (and thus any database requirements).

Now, start up the rails server:

./script/server

Did that work? Probably not. Unless you are running edge Rails, there is a current bug (gasp!) that forces you to comment out the lines from "new_rails_defaults.rb" (4th line in the server startup stack trace):

# ActiveRecord::Base.include_root_in_json = true
# ActiveRecord::Base.store_full_sti_class = true

Try and start the server again. It should work this time. If you visit "http://localhost:3000/campaign", it will probably work. This is just a coincidence.

Next make a few tweaks to your new rails CampaignController. This is an issue with Rails scaffold
generation. It generated a Controller for an ActiveRecord, not an ActiveResource, so you
need to modify for using ActiveResource objects.

Under "def update", replace the AR.update_attributes method with the ARes.load and save methods. Like so:

      if @campaign.update_attributes(params[:campaign])

with

      @campaign.load(params[:campaign])
      if @campaign.save

and replace

    @campaign = Campaign.find(params[:id])
    @campaign.destroy

with

    Campaign.delete(params[:id])

Better yet, let's just rip out all the stuff we don't need. Here's the complete Controller:

class CampaignsController < ApplicationController
  # GET /campaigns
  def index
    @campaigns = Campaign.find(:all)
  end

  # GET /campaigns/1
  def show
    @campaign = Campaign.find(params[:id])
  end

  # GET /campaigns/new
  def new
    @campaign = Campaign.new()
  end

  # GET /campaigns/1/edit
  def edit
    @campaign = Campaign.find(params[:id])
  end

  # POST /campaigns
  def create
    @campaign = Campaign.new(params[:campaign])
    if @campaign.save
      flash[:notice] = 'Campaign was successfully created.'
      redirect_to(@campaign)
    else
      render :action => "new"
    end
  end

  # PUT /campaigns/1
  def update
    @campaign = Campaign.find(params[:id])
    @campaign.load(params[:campaign])
    if @campaign.save
      flash[:notice] = 'Campaign was successfully updated.'
      redirect_to(@campaign)
    else
      render :action => "edit"
    end
  end

  # DELETE /campaigns/1
  def destroy
    Campaign.delete(params[:id])
    redirect_to(campaigns_url)
  end
end

You'll also need to make a slight change to the generated "new.html.erb" file. Again, this is because the scaffold generator we used presumes it's dealing with an ActiveRecord object, and we provide an ActiveResource. (more info if you care: The "form_for(@campaign)" line generated, given an ARec object, will generate a hidden "_method=put" field in the form. This tells the controller that you are going to update an object - which isn't true - you are creating one. So you really want to POST, not PUT)

So, as long as I'm giving away free answers, put this in place of the new.html.rb form generated (don't forget your authenticity token):

<form action="/campaigns/" method="post">
  <%= hidden_field_tag "authenticity_token", form_authenticity_token %>

  <p>
    <label for="campaign_name">Name</label><br />
    <%= text_field_tag "campaign[name]", "" %>
  </p>
  <p>
    <label for="campaign_budget">Budget</label><br />
    <%= text_field_tag "campaign[budget]", 0 %>
  </p>
  <p>
    <%= submit_tag "Create" %>
  </p>
</form>

Navigate to "http://localhost:3000/campaigns" and check out your list of two campaigns. click "New campaign" to add one. Fill out the form and "Create". This then forwards you to "show" the newly created campaign, which is retrieved from the Java's REST service. Click "back" to return to the list of campaigns, and you should see your new one sitting there. Click "Edit" to make a change to the campaign, and "Destroy" to delete it. These actions don't actually do anything since we hard-coded our REST service.

However, if you downloaded the sample projects and run them, then the REST service is backed by an HSQLDB - so adding/editing/removing values actually works.

That's it! This convergence of technologies is precisely what ActiveResource was created to deal with. Simple REST serviced is why JSR311 (aka, JAX-RS) exists for. Simplicity of domains - what more could you want?

RoRoRoJ? Ro3J?

Cheap Tricks XVI - Terse Equals

2008-07-21T09:57:00.003-05:00

I've been running across a lot of code like this lately. It's something I've brought up with my own developers, but suppose it's just something generally useful. When testing for equality of object, the precursing null check. Here's the code I'm seeing everywhere:

  @Override
  public boolean equals( Object obj )
  {
    if ( !( obj instanceof Account ) )
    {
      return false;
    }

    Account that = (Account) obj;
    if ( that.id == null || this.id == null )
    {
      return false;
    }
    if ( !that.id.equals( this.id ) )
    {
      return false;
    }
    return true;
  }

I'm a fan of short concise code - and even if you're not... come on... it's an equals method! Let's be frank:

  @Override
  public boolean equals( Object obj )
  {
    if ( !( obj instanceof Account ) )
    {
      return false;
    }

    Account that = (Account) obj;
    return id == null ? that.id == null : id.equals( that.id );
  }

If this id is null, then from the tertiary operator, you only need to know if that.id is null. If so, then they are equal. If not, then you just need to execute the equals method on them. Now if only Java would retrofit equals to use generics...

Code Naming Rules

2008-07-08T10:30:00.008-05:00

I believe it is a fundamental misunderstanding of nature to assume that rules are not made to be broken. This is the sticking point where breakdowns in communications occur - so I'm letting you know what side of the debate I'm on right away. That said - here are some rules I adhere to when naming things - giving real (poor) examples from the system I currently work on:

Example: CallRequestRecipientObj
Reason this is bad: Classnames should never contain the word "Object" or "Class" or their derivatives - unless you are writing framework code and referring specifically to an object or class type. We know it's an object or class we're talking about. Adding "Class" or "Object" into a classname is just redundantly redundant.
Rename to: CallRequestRecipient is sufficient.

Example: GenericPersistableClass
Reason this is bad: For the love of Jesus, don't use the word Generic or Framework in your classnames. Yes yes - your little framework is very clever and flexible and "generic". Stop beating me over the head with it. Especially when naming interfaces... which are, by definition, generic. Also: remove "Class" (see above).
Rename to: Persistable ftw.

Example: interface Account { int getAccountId(); }
Reason this is bad: Don't prepend the object name to an ID. Again, it's just redundant. The argument used to be surrounding database joins. But that's precisely what aliases are for:

select a.id, d.id from Account, Details where a.id=d.accountId

Rename to: getID() is terse and descriptive. It's also the same for every object.

These are a few good naming rules I stand by. Do you have any more? I'd love to hear them. Again, rules are made to be broken - but should be followed, like, 99% of the time - give or take, depending on your rules. Next on the docket: package, project, and namespace naming rules.

Code Te Ching - Verse 46

2008-06-28T12:00:00.001-05:00

When a tiger begins to run to parts unknown,
    she must first lean forward.
This is called instability.

When a tiger's paws land on the earth,
    they find their place of comfort.
This is called self-organizing.

When a tiger's eyes see, ears hear,
    the feet know to move.
This is called multi-learning.

When a tiger has her senses about her
    no trap can capture her feet.
This is called subtle control.

When a cub watches his mother run,
    he will mimic her movements.
This is called knowledge transfer.

Instability, self-organizing, multi-learning, subtle control, knowledge transfer.
This is the natural way of action.

Forced unnatural natural force? Madness!
Artificial process? Confusion!
The sage guides but does not manage.

Cheap Tricks XV - Static Collections in Java

2008-06-26T16:24:00.016-05:00

Being back in the Java game from Ruby, makes me miss a lot of the shortcuts. Despite the crushing pain I feel whenever I recognize a 5 line block of Java code that can be reproduced with one short Ruby line, being a professional Ruby developer was overall a great experience. It has made me so lazy, I have amassed quite the little toolkit to save precious lines. Here's another cheap trick around reducing the common need for a static block populated collections (NOTE: this isn't limited to Enums - if you are only dealing with Enums, just use EnumSet.of):

public class StateMachine
{
  enum State { STARTED, COMPLETED, CANCELLED, ABORTED }

  public static final Set<State> FINAL_STATES;
  static
  {
    HashSet<State> fs = new HashSet<State>();
    fs.add( COMPLETED );
    fs.add( CANCELLED );
    fs.add( ABORTED );
    FINAL_STATES = Collections.unmodifiableSet( fs );
  }
  // ...
}

into a trim one-liner:

import static StaticUtils.sset;
import static StateMachine.State.*;

public class StateMachine
{
  enum State { STARTED, COMPLETED, CANCELLED, ABORTED }

  static final List<State> FINAL_STATES = sset( COMPLETED, CANCELLED, ABORTED );
  // ...
}

Here's a code for a static list creator:

public final class StaticUtils
{
  public static final <T> Set<T> sset( T...objects )
  {
    HashSet<T> al = new HashSet<T>( objects.length );
    for ( T t : objects ) al.add( t );
    return Collections.unmodifiableSet( al );
  }
}

As one person pointed out, you can achieve the same effect for Lists via:

import static java.util.Arrays.asList;
import static StateMachine.State.*;

public class StateMachine
{
  enum State { STARTED, COMPLETED, CANCELLED, ABORTED }

  static final List<State> FINAL_STATES = asList( COMPLETED, CANCELLED, ABORTED );
  // ...
}

Slight modifications can be made for handling maps:

public final class StaticUtils
{
  // ...
  public static final <K, V> Map<K, V> smap( Pair<K, V>...pairs )
  {
    HashMap<K, V> hm = new HashMap<K, V>();
    for ( Pair<K, V> p : pairs ) hm.put( p.k, p.v );
    return Collections.unmodifiableMap( hm );
  }

  public static class Pair<K, V>
  {
    private Pair(K k, V v) { this.k = k; this.v = v; }
    private K k;
    private V v;
  }

  public static final Pair<K, V> p( K k, V v )
  {
    return new Pair<K, V>(k, v);
  }
}

Usage is similar, but bundles keys/values into a pair, the creation of which is shortcutted by the static "p" method.

import static StaticUtils.smap;
import static StaticUtils.p;
import static StateMachine.States.*;

public class StateMachine
{
  enum State { STARTED, COMPLETED, CANCELLED, ABORTED }

  static final Map<State, String> INCOMPLETE = smap( p(CANCELLED,"cancel"), p(ABORTED,"abort") );
  // ...
}

The Flywheel and the Dead Sea

2008-06-18T09:55:00.028-05:00

First thing's first. Some definitions:

The Flywheel Effect

Now picture a huge, heavy flywheel. It's a massive, metal disk mounted horizontally on an axle. It's about 100 feet in diameter, 10 feet thick, and it weighs about 25 tons. That flywheel is your company. Your job is to get that flywheel to move as fast as possible, because momentum -- mass times velocity -- is what will generate superior economic results over time.

Right now, the flywheel is at a standstill. To get it moving, you make a tremendous effort. You push with all of your might, and finally, you get the flywheel to inch forward. After two or three days of sustained effort, you get the flywheel to complete one entire turn. You keep pushing, and the flywheel begins to move a bit faster. It takes a lot of work, but at last the flywheel makes a second rotation. You keep pushing steadily. It makes three turns, four turns, five, six. With each turn, it moves faster, and then -- at some point, you can't say exactly when -- you break through. The momentum of the heavy wheel kicks in your favor. It spins faster and faster, with its own weight propelling it. You aren't pushing any harder, but the flywheel is accelerating, its momentum building, its speed increasing.

- Jim Collins in Good to Great

Now consider the above analogy concerning your software development team. Is it descriptive? Do they move via internal momentum, or need cattle-prodded to get work done? Moreover, consider this in terms of hiring software engineers. Good people want to work together - and they want to be valued. This is my positive analogy concerning the makeup of a development organization: if your group is filled with smart people it will attract other smart people, if they are empowered it will attract others looking to make a difference. Keep momentum high, and those incapable of keeping up will be flung off by the sheer force of the flywheel... what an attractive metaphor! Keep this in mind while reading the next one.

The Dead Sea Effect

The Dead Sea, of course, is a large body of water between Israel and Jordan, located well below sea level. The Jordan River empties into it; water leaves only by evaporation, which means that over the eons, the Dead Sea has become very salty (e.g., 8x saltier than the ocean). As such, it is generally unable to support life, except when spring floods temporarily lower the salinity.

Many large corporate/government IT shops — and not a few small ones — work like the Dead Sea...what happens is that the more talented and effective IT engineers are the ones most likely to leave — to evaporate, if you will. They are the ones least likely to put up with the frequent stupidities and workplace problems that plague large organizations; they are also the ones most likely to have other opportunities that they can readily move to.

What tends to remain behind is the ‘residue’ — the least talented and effective IT engineers.

- Bruce Webster

This is another common disease that plagues, not just IT shops, but companies that make their money on software. It tends to perpetuate when upper management has no real development experience, but I've seen it happen when those in charge are former developers themselves - the latter has always puzzled me, but considering middle-management's tendency to mask organizational problems that occur below them, I like to think that upper management is merely blissfully ignorant of the problem. but I digress.

Minding the Gap
What is at issue here is that, surprisingly, most development teams I encounter gravitate toward either end of the spectrum. The center of gravity is bimodal. Where one might conclude there is a bell curve associated with any particular company's talent pool, real productivity distribution is a little more lopsided. Certainly, overall talent may land somewhere around the middle, but a company entirely consisted of mid-range developers is still a dead sea. A pool of mediocre developers won't give you average results - you'll just get a decent maintenance staff with occasional updates - all the while lagging behind your more agile competitors. You must be a flywheel to move forward. It's like trying to break the Earth's gravitational field in a rocket... anything less than the threshold velocity and you'll tumble back to Earth. You must have a top developer to advance, and you must have a field of them to attract and retain others, and to keep each other motivated.

Attraction
Google started out with top notch talent. The two founders were computer science PHDs, and naturally that helped bootstrap their organization. They were able to hire the top talent, because they recognized them. Only talent can recognize talent (Corollary 2 of the Dunning-Kruger Effect).

But this is not all you require. You need to have a reason for talent to want to work with you. Remember: at this point in time, skilled developers are at a premium. They are not desperate for your job. Anyone who is, you don't want. This has a couple corollaries:

You must pay competitive wages for today. This should go unsaid - but it can't be. Don't be cheap. Equity sharing is also good. Give your people a reason to care. Don't fall into the trap of believing because you hired your first crop of developers three years ago at 60k, that a 5% raise is sufficient to match today. Don't wait for your top developers to put in their notice to start offering the raises. You must pay competitive wages to keep talent.

Money is ok, but the most attractive thing you can offer great developers is the opportunity to work with other smart, motivated people. Along with that, from personal experience, the ability to work with new technologies is also a plus (though not always necessary or possible - if you can offer it, make this known).

Attracting good developers is only part of the issue - then you must retain them. Attraction without retention of talent creates a dead sea.

Retention
ATG started out great. Also founded by two PHDs they invented JHTML, later licensed by Sun and converted to JSP. But anymore - well - how many great developers have heard of ATG? This may sound like an unfair comparison, but I don't think so. They both started out attracting great talent, but not anymore. Having worked rather closely with ATG, as well as others who have worked in and with the company, there is nothing attractive about working there. Innovation is stifled, and that knowledge floats around the aether.

Retaining developers is easy - it's all about empowerment. Beyond that, assuming you are not attracting new developers by lying (about their situation, benefits, etc.), simply allowing talented people to be valued is often enough.

Retention of talent without motivation stops a flywheel.

Motivation
Motivation is, of course, the force that causes the flywheel to turn. Keep the bar high, give talent ownership in the outcome. If you can attract top talent, and keep them motivated, retention handles itself. If you can keep talented people empowered, they will motivate themselves. And this is precisely the point. You don't need to do anything - hence the flywheel. Once it's up and running, it moves on it's own - the momentum makes it an unstoppable force.

Starting the Flywheel
Up to here, I've been pretty vague - purposefully brushing with large strokes. I want to drive home to idea that attraction, retention, and motivation are all required to create a self-sustaining flywheel of talent. So what if you're a company leaking talent, or are already a dead sea, what specifically can you do?

Step 1: Pinpoint internal talent. Ask around. Don't just ask the bosses, ask the developers' peers. All developers will know who the badasses of your organization are. Leveraging your developers allows them to have a voice in the business process - it's a great motivator, which is always great for retention. But what if you are new company, or don't have any top talent? This is more difficult - since as stated above, I believe only talent can spot talent. The next best thing to do is ask around, multiple sources, and rely on reputation - though this is the worst of the two options. It's like trying to find a good auto mechanic: you either be an expert yourself, or rely on reputation. Obviously the former is much better - and you'll probably go through several mechanics before you find a good one anyway. Reputations can be lies.

Step 2: Once you've found some top developers to meet with, leverage that talent to attract new people. Smart people tend to know other smart people. Despite the rumors, great developers are social - there are good ol' boys clubs everywhere. Chances are, your top developers are involved in other projects - either small side projects with others, or top open source projects. You never can tell. Find out. This is how you will attract the best.

Step 3: Generate mindshare. It's well known that Google Engineers are alloted 20% of their time to work on pet projects. That's where gmail, maps, movies, and other projects came from. Why let Google have all the fun? Sure, you may not be able to spare a whole day a week - but I don't care what you do, or how tight your deadlines are, every business can spare 4 hours a week for their developers to write open source tools, corporate blog, manage an external corporate bulletin board.

Make your place a fun and interesting place to work for developers - and great developers like to brag (who knows why... it must be written in our awesome DNA). Create an open source arm and release the code of internal tools your developers have written to solve problems. There are other benefits to doing this as well. Beyond the great "we have an open source arm" press you get a free army of bugfixers (mindshare may drag in new customers too). This gets your name out there, which attracts new top talent. This lets your developers run something, which keeps them motivated in other areas (for example, the work you pay them to do).

That's just one possible path to creating a self-retaining flywheel of talented, motivated developers, one of many paths, but the goal is the same. Internal motivation. You can't buy it, and you can't force it, but you certainly can build it. It will take some effort and pain on the front end, but by the time that wheel is moving, you'll wonder how you ever survived before.

The Worst of Crimes

2008-06-11T12:48:00.023-05:00

There is no sin so great for a programmer as lack of curiosity. A sufficiently curious programmer can overcome a whole range of deficiencies, including laziness. Traits detrimental to normal jobs such as lack of: intelligence, social skills, or hygiene. Some of the greatest coders I know are lazy, smelly, and not necessarily "smart" in the common meaning. But they are curious about how things work - which is why they're great. These are the people that run into defects, and rather than sit on their hands, find ways to fix them. Curiosity drives research, and from self-driven research comes experience - which can even close the gap to their more intelligent peers.

Industrious coders will stay at work all night because it is their duty to do so. Curious coders will stay at work all night because they can't sleep without knowing why the database is running so damn slowly. Which would you rather hire?

I bring this up, because at my new job we're looking to hire, and I want to ensure that we aren't just going to hire someone with lots of education, or intelligence (both important attributes) but someone who has a general curiosity about how the world works.

Question: What activities do you do outside of work?
I know it's a cliche to say "I want a well-rounded person", but in this case, I do. Not because I particularly care if my co-workers like to play street-hockey, or are into beat poetry, but because curious people tend to have a lot of interests. It's all part and parcel to wanting to know about the world. Even the biggest dullard on Earth has at least one interest. If the best they can say is "I like to watch movie", move on...

Question: What non-school- or non-work- related programming project are you most proud of?
Of course, lots of people have varied interests. This doesn't mean they are necessarily relevant to the job. My buddy Jim mentioned this good question. It's the python paradox in another form: the best programmers do it outside of work. I think it's a misnomer to claim that good programmers program for fun, because I don't care if you have fun or not. I didn't learn Ruby for fun, I learned it because I needed a good scripting language to write code generators for crappy J2EE/EJB 1.0 projects. I wasn't having fun, but I was driven by a curiosity to know how Ruby worked. And that's enough. Again, no answer equals no employment...

Specify a real problem and try and gauge interest.
This isn't a question, but a strategy - and one that I prefer. Body language will tell you when someone is really interested in a problem - they'll sit up straighter, perk up a little, and give an obvious effort to solve the problem. Remember, in theory this person wants to work for you - so trial by fire. OK, at least mediation by lighter. The point is, try explain a specific technical problem you face, just minus the domain-specific jargon. For example: we run Hibernate on top of Oracle and some of the queries are going very slow. It's a good technical question, but don't - as an interviewer - get caught in the trap of just listening for technical correctness. Curious people will ask questions before working out a possible formulation - dullards will either have no answer (take a hike!), or ramble off a list of template answers (indexing, data model, bad query generation). I look for questions, even if it's as simple as "What version of Hibernate are running?". If someone can't get excited about solving a real problem (which involves attempting to understand it) during an interview, how excited will they be after 6 months of 8 [ahem, or 10... or 12] hour days?

Games aside, good technical interview questions and strategies can be useful - but never foolproof. I assume you are a curious person, and are merely seeking to find others of our kind. You only meet with a person for a short time, so like Bram Cohen (creator of BitTorrent) says: "In an interview you can tell if a person is a pleasant conversationalist, and you can give some technical questions to rule out the truly inept, but beyond that you might as well be rolling dice." That's true, but if you look for specific personality aspect - which I suggest curiosity is a must - you can at least get a good feel for this characteristic.

Good to be back!