haruska.com :

Mashery: Post-Mortem

Tue, 16 Jun 2009 20:52:00 -0500

I work for a small VC-backed company in the mobile social media space. Something we want to offer is a public API to open up our community to development of 3rd party tools. Although we have the technical capabilities to build an infrastructure in-house, I had a look at Mashery to outsource some of the ongoing work. We were looking for:

proxy front end that can scale up as our API use grows and have a caching strategy to reduce the load on our servers
ability to throttle on a per app (or developer key) basis to anticipate server load
developer documentation that is easy to update and publish, skinned to look like our site
ability for developer feedback
analytics on the usage of the API
OAuth support

Looking at Mashery's offering, they seemed like a good fit. That was a month ago. Today, none of those things are fully baked. Nothing of their offering is self-service so everything requires customization. Most tasks run serially and take a minimum of 3-5 business days.

We should have known at the beginning. After the website erred out on purchase attempt, I was assigned an account rep who wanted to set up a conference call. After attempting to explain their cost as a "request per second per developer key", I counted them out. That type of pricing model wouldn't work with us mainly because all of our installs of, say, an iPhone app would have the same developer key. That prompted this tweet.

To be fair, the CEO got right back to me (as a side note, I guess it proves Twitter is a good way to get feedback to a company.) He seemed as confused as I was and explained they were changing their pricing model that day. Long story short, we signed up on a monthly basis based on the new, more straightforward pricing scheme (they've yet to publish as of this writing.)

One point of contention was OAuth was not a part of their standard package. It was considered an advanced custom feature. It was odd because OAuth is quickly becoming the best practice for accessing user generated content. I would expect a market leader to be driving best practices, not behind on them.

However, OAuth alone would double the monthly fee and require a custom $2500 setup. We decided to implement that ourselves. Again, it isn't that we couldn't, we just didn't want to spend the time. This also complicated matters. Now we'd have to pass valid consumer keys to Mashery so they could still perform their throttling function. This never happened...we weren't able to get to this discussion before aborting the relationship.

Next pain point was on the skinned development documentation. It took them over a week to "provision" us a portal. And when they did, it wasn't skinned. After pointing back to the contract, our rep agreed to skin the portal. We sent over a screenshot and was told it would be skinned in 7-10 business days. In short, don't expect public facing documentation for three weeks after your first monthly payment.

The next hurdle was reporting. Our rep attempted to explain to us their wacked-out URI-based reporting scheme. You had to define which parts of your URL path was important to your calls. Long story short, they don't support Ruby on Rails RESTful type urls. For example, they couldn't figure out how to make these different calls for reporting:

/pictures.xml
/pictures/39290.xml
/pictures/39290/comments.xml

The first one is a collection of all pictures. The second is retrieving the single picture with ID 39290. The last one is retrieving the comments attached to the picture 39290. I guess the problem was with ID wildcarding.

Because of this "unusual" URL scheme, we were given a statement of work for $1000 and five business days to implement what amounts to a regular expression in your code. We were told repeatedly that *no* customer has asked for anything so strange in reporting.

So, we refused to pay the ransom and asked for a refund. They've refused the refund. Apparently, they consider us "2 days from success". We're out a couple of thousand dollars and a month's headache for the effort. At least we figured it out early. I would suggest having them prove themselves to you before plunking down any money. At the least, ask for a 30-day guarantee. Don't let them develop their 1/2 baked product on your dime.

Update 6/18/09

After much back and forth with Mashery on twitter and email they have agreed to refund our money. They were gracious and took the time to understand our frustrations. In the end we have different opinions of how the process should work but I think we both have a better understanding of where each other are coming from.

Creating a git repo for a private project

Fri, 10 Apr 2009 07:40:00 -0500

I seem to be doing this more frequently whenever hacking on a new project. It's easy to set up a master repo on a remote server that you have ssh access to, I just always forget the commands. Here they are for reference:

Create a new local repo:

mkdir myproject
cd myproject
git init
touch README
git add .
git commit -m"first checkin"
cd ..

Create a bare tree

git clone --bare myproject myproject.git
touch myproject.git/git-daemon-export-ok

Upload to remote machine (in this case a "/src" folder already on the server)

scp -r myproject.git server.domain.com:/src/

Check back out from remote repo

rm -rf myproject
git clone server.domain.com:/src/myproject.git

You should be all set. Your "master" is now on your remote server. You can branch, push, pull, etc the same as you would on github or similar.

ubuntu postfix setup on verizon fios

Fri, 30 Jan 2009 16:20:00 -0600

It took me more googling than it should have to figure out how to setup a home server to relay email through verizon's smtp server. Here's what I did.

First, you need a fully qualified domain name (FQDN). This can be set in ubuntu by:

sudo /bin/hostname myserver.domain.com

Next, install postfix and mailutils:

sudo apt-get install postfix mailutils

Set up your /etc/postfix/main.cf. Here's what mine looks like:

smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu)
biff = no

# appending .domain is the MUA's job.
append_dot_mydomain = no

readme_directory = no

# TLS parameters
smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key
smtpd_use_tls=yes
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache

# See /usr/share/doc/postfix/TLS_README.gz in the postfix-doc package for
# information on enabling SSL in the smtp client.

myhostname = myserver.domain.com
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
myorigin = /etc/mailname
mydestination = myserver.domain.com, myserver, localhost.localdomain, localhost

relayhost = [outgoing.verizon.net]
smtp_connection_cache_destinations = outgoing.verizon.net
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = static:username@verizon.net:password
smtp_sasl_security_options = noanonymous
default_destination_concurrency_limit = 4
soft_bounce = yes

mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
mailbox_command = procmail -a "$EXTENSION"
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all

restart postfix and you should be all set

/etc/init.d/postfix restart

to test from the command line, use mail

myserver> mail email@isp.com
Cc:
Subject: Testing postfix
test
^D

A More Mindful Approach to Apple

Mon, 05 Jan 2009 16:19:00 -0600

It has been two years since my migration from linux to os x. Today, I'm starting the migration back.

Two years ago I was bit by the Apple bug. I had been using Linux for the ten years previous and always liked it. I was never a zealot about it the way others in the community were, rather I found it to be good for 80-90% of what I do on a daily basis. Plus, I grew to really like using open source. I was able to contribute where I could and hopefully solve someone else's issue in exchange for all this great software being maintained.

The nagging part in linux is that other 10% -- handling of media and unsupported by vendors. None of it is the fault of Linux developers. Indeed, that is why I care so deeply about creative commons and anti-DCMA issues. However, it is still an issue the average Linux user needs to deal with daily.

Then came the MacBook. It did that other 10% beautifully. Apple actually had deals with the media companies and the promise of all of it "just working" was too much to resist. Besides, it is based on BSD and had a bash shell ... maybe it really was the best of both worlds.

Problem is, it doesn't "just work". No consumer hardware does, especially laptops. Any vendor telling you otherwise is being dishonest. After joining the Mac community, I began hearing whispers about hardware issues. It seems the biggest problem isn't that the hardware or software fails, it's proving to Apple that there is an issue.

Coming from a community with open bug tracking software, open code, and at least knowing someone else is having the issue, I found Apple's process extremely disturbing. Every issue I've had it was like Apple trying to pull a Jedi Mind Trick on me - "You have no issue" they would tell me. The customer is forced to prove beyond a shadow of a doubt that there is, in fact, a problem with their perfect hardware and software. Only then, will they admit there is a problem... but not if they're actually working on it.

It's maddening. File a bug, it'll get closed as a duplicate of another bug you can't see. Hope you were the first to file! Having hardware issues? Prove it for a "genius". Don't want to get charged the $200 repair for the video card? You better mention you know about the recall from nvidia.

And that's my issue. I know no software is perfect. I wish Apple would stop pretending their software is. Give your users more information on their problem, maybe even an ETA on when it will be fixed. Doesn't it bother anyone else that their "critical security updates" notes just say "various fixes to improve stability" ?

So the new tag-line for Apple is "It just works, until it doesn't. Then, good luck"

To be fair, Linux has its share of problems. Actually, probably a lot more than Apple. The difference is how the community investigates and fixes the problems. That's what I'm more interested in. So I'll take the additional issues as long as I know they're being worked on. At worst, I could work on it myself.

Today, I begin the migration back to Linux. It won't be easy. In the past two years, I've bought into the Apple products hook, line, and sinker. Thanks to my great company, I currently have in my possession a MacBook, MacBook Pro, AppleTV, and two iPhones.

The reason I have two laptops is the MacBook Pro's video card keeps dying. Or it wakes from sleep in my bag and melts. I haven't had the time to go back to the genius bar and get it fixed again.

I'm pretty sure I'm not Apple's target market, so this shift shouldn't bother them much. What should bother them is the new MacBook Pros came out and nobody in the office wanted one. That's a first. It should scare the hell out of them.

Moving forward - the other MacBook will dual boot OS X and Ubuntu. The AppleTV just got boxee installed on it, making that hardware actually useful. And the iPhone? Well, it looks like I'll have to jailbreak it. It seems linux can wirelessly sync to that piece of hardware.

For my next hardware purchases when all this Apple hardware dies: Dell laptop with Ubuntu pre-installed, blackberry (or google phone) and a popcorn hour box.

For software, there are two things I need to figure out how to replace - Aperture and iChat screen sharing. I think both can be resolved with some research. We've really liked the iChat sharing for pair programming at work and I wouldn't want to lose that feature.

So, I'm off. Something feels right about it. I'll probably receive some ridicule at the all-Mac office but it's the right choice for me. I hope other linux refugees realize they aren't bound to one vendor for their hardware and software soon.

modifying snippets of erb during processing

Wed, 26 Nov 2008 19:25:00 -0600

Today we ran into an interesting problem that turned out to have an elegant solution. We had a list of links that we wanted displayed as comma separated. Long story short, the resulting output contained spaces before the commas (i.e. Dick , Mary , and Jane). We wanted to take that output and run a regex replace on it.

Turns out this is pretty easy to do in rails. We added the following to our helper file

def suppress_comma_spacing(&block)
  res = capture(&block)
  concat(res.gsub(/\s+,/m, ','), block.binding)
end

then wrapped the code whose output needed modification in a code block in the erb file (trivial example below)

<% suppress_comma_spacing do %>
  <%= link_to "Dick", user_path(dick) %>
  ,
  <%= link_to "Mary", user_path(mary) %>
  , and
  <%= link_to "Jane", user_path(jane) %>
<% end %>

The result is Dick, Mary, and Jane with the spaces before the commas stripped. Capture allows you to get the processed erb from executing the block. Pretty useful.

ninja-decorators

Tue, 25 Nov 2008 16:14:00 -0600

A couple of us at SnapMyLife were a bit surprised you couldn't easily use around_filters outside of rails for general ruby programs, so we implemented them and put it up on github. We called the gem ninja-decorators, mostly in a mocking sense that everything seems to be rockstars and cool_fu these days. To install the gem:

gem sources -a http://gems.github.com
sudo gem install haruska-ninja-decorators

Here is a simple example:

require 'rubygems'
require 'ninja_decorators'

class NinjaClass
  include NinjaDecorators

  attr_accessor :ret

  around_filter :common_around, [:foo, :bar]

  def foo
    @ret += "foo"
  end

  def bar
    @ret += "bar"
  end

  private

  def common_around
    @ret = "common "
    yield
    @ret += " around"
  end
end

produces

irb(main):076:0> n = NinjaClass.new
=> #<NinjaClass:0x220decc>
irb(main):077:0> n.bar
=> "common bar around"
irb(main):078:0> n.foo
=> "common foo around"

We'll add before_filter and after_filter soon.

fb2twit taken down

Thu, 11 Sep 2008 16:02:00 -0500

This little experiment started with me having a software problem, as most good projects do. There isn't a good way to have Facebook status updates get copied to twitter but not the other way around.

I've found out that open source software works, however open source services do not. I cannot believe the number of people demanding I fix or improve things on the project.

Also, it was costing me real money. I had to host a server to ping Facebook for status updates of the 500+ users and update twitter every 5 minutes. It's only $120 / year but I couldn't get anyone to step up and sponsor it (Twitter, hello?)

But I think the biggest factor is I don't really update my status on Facebook anymore. I use twitter daily, but that's because I have a core group of people there that expect me to "bother" them with the little ideas that pop into my head 120 characters at a time.

So, in the end I was maintaining a code base and a web service that I wasn't even a customer for anymore. Time to shut it down.

Of course, the code to fb2twit is up on github (it always has been.) All any of the lechers have to do is pay to host it and put it up there.

git makes open source webapps easy

Sat, 28 Jun 2008 16:00:00 -0500

I actively try and open source as much of my code as possible. I've recently found I'm writing more and more mini-webapps in rails. With subversion these were very hard to share with the public.

It mostly comes down to secrets. You don't want to share your database.yml file, etc. But how do you share the project, without screwing up deployments and without giving up secrets? How many hoops do you really want to jump through to give away your code?

Well, git removes all of those hoops. With multiple remote repositories, it's trivial to have a remote repo that is open source and another that is private. Just point your production server to the private git repo for its code and you're good to go.

fb2twit, a small facebook app that posts status updates to twitter when entered in facebook, is open sourced on github.

Reflections on RailsConf 2008

Sun, 01 Jun 2008 15:58:00 -0500

Portland has been overrun with ruby and rails geeks this week. This was my first RailsConf (out of the three) and I'm really glad I took the six hour flight from Boston.

Most of the attendees are still truly interested in perfecting their craft and finding cool code to make their projects easier. I'm glad there were pure ruby talks, including a very good talk on meta-programming.

Sure, you could take the time to catch up on RSS feeds and new projects around the rails community from your basement but it isn't the same as being able to sit face-to-face with fellow developers.

Lots of us work in small startups where you may have one or two developers to bounce ideas off of. At RailsConf, just watching people live code is insightful on areas you can improve. I've been inspired (or compelled) to revive some small plugins and clean them up on github.

If you were at RailsConf but not following the twitter feed on summize, you were missing out. Instantaneous feedback was provided on all the talks (the good, bad, and ugly) along with where people were meeting up etc.

Also, interesting to note everyone uses Macs and iPhones (myself included). This is an interesting observation on a bunch of geek early adopters. Hmmm, what personal platform should you be considering?

I love conferences where it doesn't matter what you're wearing, how you present, etc. If you can code, you're accepted, if not hang out and learn... put up or shut up.

Some of the best presentations were actually the unorganized lightning talks. This is where you give people a chance to signup and present for 5 minutes the latest thing their working on. Matt gave one on rubber, along with some pretty interesting ideas on everything from testing, deploying, to hooking into SMS / Jabber, etc.

I've actually met a lot of Boston rubists out here as well... I guess it sometimes takes a plane ride to the west coast to talk to your local geeks.

So, overall a smashing success. The keynotes were hit-and-miss and some of the rooms were over crowded; but, the essence of geeks hanging out and talking passionately about code was preserved.

Excellent work to everyone who made the gathering possible.

polymorphic_include and multi_smtp on GitHub

Thu, 15 May 2008 15:55:00 -0500

I finally took the time to wrap up two useful pieces of code into Rails plugins.

The first - polymorphic_include - I've mentioned before. It allows you to eager load polymorphic associations. I've cleaned up the code, including fixes for some nasty eval memory leaks.

The second plugin - multi_smtp - came about out of using gMail as our SMTP server for sending notifications on SnapMyLife. Google limits the number of messages per account and recommends you set up multiple accounts if you have to send more. This plugin allows you to define multiple accounts that Rails will spread the email load around.

Hope others find the plugins useful. These are my first experiments with git and GitHub, so I'm also seeing how easy it is for others to submit their ideas. I by no means want to be "keeper of the code" and like git's SCM style.

To start using git on OS X, download the dmg package from the git google code page.