Tim Riley / timriley.info

Tim in open source, September 2024

2024-10-05T02:22:00+00:00

Hello there, friends! It’s been a couple of months since my last open source update, so what have I been up to? A bunch of things, all culminating in a new Hanami 2.2 beta!

RedDotRubyConf returns!

First came RedDotRubyConf 2024 in Singapore, back again after seven years! It was great: excellent mix of talks, a small but engaged crowd, and as always, amazing hospitality. Thank you to Ted and all the organisers for bringing it back! I hope it can continue into the future!

RedDotRubyConf is a special event for me: it was my first opportunity to speak at a Ruby conference, as well as my first chance to give a workshop. Before ths one, I’d attended three different editions over the years, dating all the way back to Andy’s last event in 2012. My talk this year was a nice bookend to how I started. All the way back in 2013, I presented my stack of dry-rb, ROM and Roda as a vision for a new generation of Ruby apps. This year I presented that vision brought to its complete and streamlined conclusion in Hanami. For those of us cultivating Ruby on the side, some fruit takes time to bear.

Speaking of time: after I returned from Singapore, things were slow for a while. I had made a big push to release 2.2.0.beta1 before the conference, and that pace was not something I could maintain. This, alongside attending a Buildkite off-site, meant things were pretty quiet for a few weeks.

Hanami actions, meet validation contracts

It wasn’t long before I found a productive groove again. One of the things that helped here was some work that Krzysztof Piotrowski did to explore using full dry-validation contracts as the means of params validation in Hanami actions.

Contracts in actions has been a much-demanded feature! In fact, some work began on it all the way back at the RubyConf hack day in November 2023 (thanks to Dan Healy). While that effort petered out, Krzysztof progressed things far enough to arrive at a fully functional implementation, ready for review. He’d also been making some great contributions to Hanami lately, so I didn’t want to keep him waiting.

After reviewing Krzysztof’s work, I managed to put together another iteration of the feature that I was happy to see go in. And thanks to some helpful feedback from Adam Lassek, I came back and did one more thing, making it so that straight-up Dry::Validation::Contract classes could be used for the validation, rather than the Hanami::Params subclasses that were required in the older, 1.x-era behaviour that we had inherited for this feature.

The result is a useful spectrum of options. You can start with the simplest approach, embedding a contract directly in your action. Now instead of the params blocks you could use before (which exposed the dry-schema features only), you can use contract:

class Create < MyApp::Action
  contract do
    required(:title).filled(:string)
    required(:slug).filled(:string)
  end
end

If you have a contract that you want to share across actions, you can also reference its class directly:

class Create < MyApp::Action
  contract Posts::Contract
end

One of the internal adjustments I made was to defer the initialization of contracts until the time the action is itself initialized. This allows you to take advantage of one of dry-validation’s most powerful features: contracts that can interact with the rest of your app via external dependencies. For example:

module MyApp
  module Posts
    class Contract < Dry::Validation::Contract
      include Deps["repos.post_repo"]

      params do
        required(:title).filled(:string)
        required(:slug).filled(:string)
      end

      rule(:slug) do
        unless post_repo.unique_slug?(values[:slug])
          key.failure("must be unique")
        end
      end
    end
  end
end

With external dependencies, your validation contracts can leverage business logic anywhere in your app, while still allowing for that logic to reside in, well, a logical place. And when you use validation contracts in a Hanami app, our Deps mixin makes this as easy as can be.

(Why exactly is deferring initialization of the contract required for this? It’s because the default dependencies you specify with Deps are resolved at the time of calling .new on the contract. We can’t call that too early, like in the class body of an action, because otherwise we’d run into all sorts of load ordering troubles.)

In fact, there’s one last little dependency-related treat for you in this feature. You saw above how contracts could take their dependencies via Deps? Well now you can do exactly the same with actions, with the contract itself as a dep!

class Create < MyApp::Action
  include Deps["posts.contract"]
end

This is especially nice if you’re sharing your contracts between actions as well as other kinds of classes in your app, because it means you can use Deps as a consistent approach for using them across all places:

class CreatePost < MyApp::Operation
  # Another class, same Dep!
  include Deps["posts.contract"]
end

I started this post with a little Ruby reminiscence, so why not do a little more. Back in its early days, dry-validation was one of the most important gems in spurring dry-rb adoption. Input validation is something that every app needs, and there are few solutions out there as complete and portable as dry-validation (try it in your Rails app, really!). Today, dry-validation is as relevant as ever, and with it now fully integrated into Hanami, it’s also easier to use than ever.

In fact, I think the code snippets above serve as a great example of the kind of vision we’re building towards with Hanami: actions as standalone classes, input validation as an first-class concern itself provided by standalone classes, and a simple and universal dependencies mixin to bring them together where required. Small, focused components, each with its place, and a clear strategy for connecting them. These are not the Ruby apps you’re used to. We’re bringing something new. The little integration you see above is in many ways the the culmination of 10 years of multiple streams of volunteer OSS work.

I think that enabling development approaches like this is vital part of fostering a vibrant and diverse Ruby ecosystem. If this resonates with you, we’d love your support.

What gives me heart is that after all these years, we’re still finding new champions and contributors. If it wasn’t for Krzysztof taking on the challenge to bring contracts to actions, it would not be shipping in 2.2. Thank you, Krzysztof!

Hello again, MySQL

I really didn’t intend this post to become a treatise on input validation and its meaning for the greater Ruby community, so let’s keep things moving!

Here’s something much more straighforward: when we introduced our new database layer in beta1, we included SQLite and Postgres support. There was one major database missing: MySQL. Now it’s here.

The experience is as you’d expect: hanami new my_app --database=mysql will give you everything you need to get started with Hanami and MySQL, and after that, all the hanami db commands will work with your MySQL database as required. This was one of the last big outstanding items on our to-do list for 2.2, and now it’s done!

What’s more, we also had another new contributor come in and help make our database layer just that little bit nicer. Thanks to Kyle Plump, now if for any reason your Gemfile doesn’t contain the right gem(s) for your configured database(s), we’ll give you a helpful warning. Thank you Kyle! It’s been tremendous to work with you these last couple months.

New ways to go multi-database

The astute among you will have noticed my use of “gem(s)” for “database(s)” in the last paragraph. There’s reason for this, even putting aside my predilection for syntactical whimsy: in this last month, I introduced a whole new way to work with multiple databases in Hanami!

Since beta1, we’ve supported multiple databases along one axis: while Hanami slices may all share a single database, each may also have its own. This is as easy to configure as prefixing your database URL env var with a slice name: MY_SLICE__DATABASE_URL. Hanami takes care of the rest. (Of course, you can also choose to configure slice databases explicitly where you need greater control.)

This arrangement was how I was intending to leave things. Shipping a new fully-featured database layer for Hanami already felt ambitious enough. But friend-of-the-framework Phil Arndt needed something more: to work with multiple databases within a single slice.

Speifically, Phil needed to set up multiple ROM gateways. Gateways are ROM’s abstraction for a connection to a specific data source (remember: ROM works with more than just SQL, it also has adapters for things like HTTP and CSV and YAML and more!).

I hate to disappoint Phil, so I rolled up my sleeves, and now Hanami has native support for multiple gateways within each slice!

What I love about this is that I was able to maintain the same zero-config approach we’ve had for our database layer so far. So now your ENV vars can take you even further than before. You can start with a single database (let’s use MySQL here, to stick with the theme of this ~~dissertation~~ blog post):

DATABASE_URL=mysql2://localhost/my_app

Then from there, you can go multi-database within an app by appending a suffix for each gateway:

DATABASE_URL=mysql2://localhost/my_app
DATABASE_URL__ARTIFACTS=mysql2://localhost/my_app_artifacts

There’s no limit. You can have as many gateways as you like:

DATABASE_URL=mysql2://localhost/my_app
DATABASE_URL__ARTIFACTS=mysql2://localhost/my_app_artifacts
DATABASE_URL__WEBHOOKS=mysql2://localhost/my_app_webhooks

Slices can come to the party too. Combine slice prefixes with gateway suffixes for the ultimate in code/database granularity:

# Configure multiple databases for an `Artifacts` slice
ARTIFACTS__DATABASE_URL=postgres://localhost/my_app_artifacts
ARTIFACTS__DATABASE_URL__LEGACY=postgres://localhost/my_app_artifacts_legacy

Zero-config is the start but not the end. Like many of the features in Hanami 2, progressive disclosure is at the core of our design for gateways. If you need more than the basics, there’s another layer waiting for you. So aside from the environment variables, you can also configure gateways directly in your :db provider:

Hanami.app.configure_provider :db do
  config.gateway :extra do |gw|
    # If not given, this will still be filled from `ENV["DATABASE_URL__EXTRA"]`
    gw.database_url = "..."

    # Specify an adapter to use by name (more on this later)
    gw.adapter :yaml

    # Or configure an adapter explicitly
    gw.adapter :yaml do |a|
      # You can call `a.plugin` here
      # Or also `a.extension` if this is an `:sql` adapter
    end
  end

  # Multiple gateways can be configured
  config.gateway :another do |gw|
    # ...
  end
end

If you have a provider with a complex multi-gateway setup, then you can also configure adapters separately, and they’ll be used across all relevant gateways:

Hanami.app.configure_provider :db do
  # This adapter config will apply to all sql gateways
  config.adapter :sql do |a|
    a.extension :is_distinct_from
  end

  config.gateway :extra do |gw|
    # ...
  end

  # More gateways here...
end

Did someone say dry-cli?

Yes! Someone did say say dry-cli, and it was Benoit Tigeot, who a while back proposed a couple of very nice enhancements, then went and implemented them both!

Thanks to Benoit’s work, you can now hide certain commands from the CLI’s standard usage output:

register "completion", Commands::Completion, hidden: true

As well as receive useful suggestions if you invoke the CLI with a typo:

$ ./my-cli comma
I don't know how to 'comma'. Did you mean: 'command' ?

Commands:
  my-cli command         # This is a command

I have to make an apology here. Benoit shipped these PRs a couple of months ago. But since that time I’ve been pretty much single-mindedly working through all the things above. It took a friendly nudge from Benoit on Mastodon (I love Mastodon — let’s be friends) for these to get back on my radar again. I was frankly a little embarrassed at how I’d let down a smart new contributor by this long delay.

It’s not easy dong Hanami maintenance on nights and weekends. I wrote about needing to find balance in my last update. Once Hanami 2.2 is out, I hope will become easier to stay on top of contributions like this, because I won’t have the spectre of large, uncompleted features (like a whole database layer) weighing me down.

Anyway, I’ll be making sure we cut a new dry-cli release soon, with both of these included, as well as documentable command namespaces.

Just before: a release!

Speaking of releases, the one big goal I had after after all of the above was getting them packaged up and out into people’s hands for testing. I did this last week, releasing Hanami 2.2.0.beta2. Step by step, we’re getting there.

I’d really love it if you could check out the announcement, then run a gem install hanami --pre and kick the tyres on all the new features. This is how we get things in truly tip top shape.

Right now: a break!

The release came when it did, because as of this last week, I’m away on a three week holiday with my family.

After all the work on Hanami things (not to mention Buildkite things), I’m hoping to truly disconnect and recharge. If you’re waiting on issue/PR feedback from me, please forgive the brief interruption. I’ll be back again in late October.

Up next: 2.2 by RubyConf!

I’m looking forward to attending my second RubyConf in Chicago later this year! Last year in sunny San Diego I got to reintroduce Hanami to America through my talk and at the Hack Day.

In the leadup to RubyConf 2023, we worked very hard to get 2.1 out. Alas, it didn’t quite work out that way, but that’s no reason not to try again! So this year, we’re working very hard to get 2.2 out.

We have a couple of very good motivators for this:

Sean Collins is on the programme, giving a Hanami workshop!
I’ll be back again represeting Hanami at the Hack Day, ready to help anyone wanting to contribute.

I really want everyone coming into contact with Hanami over RubyConf to be able to gem install hanami and receive in return our full vision for maintainable, layered, database-backed Ruby apps.

We’re so close to getting this done. We’ve been executing against a well-understood plan and integrating mature, production-tested systems, so I expect no surprises this time around. As I write this, our project board only has dozen small issues left.

If we succeed, I’ll look forward to showing and talking Hanami 2.2 to many of you in Chicago! Let’s do this!

Thank you (again!) to Hanami’s contributors

There’s a certain lovely vibe to this post: I’m not working alone. So many of the improvements above were spurred on by Hanami contributors, new and old. So let me thank all of you one more time:

Thank you Krzysztof Piotrowski, for spurring on the work on contracts in actions
Thank you Kyle Plump, for taking on all number of small issues and helping us with polish (it didn’t get a mention above, but I love where we landed with the .keep files!)
Thank you Phil Arndt and Adam Lassek, for your feedback on this last month or so of work
Thank you Benoit Tigeot, for dry-cli fixes. I really hope we can work together more in the future!
Thank you Gustavo Ribeiro, for your PR to dry-cli.
Thank you Sean Collins, for everything you’re doing to prepare for the RubyConf workshop!

And I know this isn’t everyone. For example, a few weeks ago Damian Rossney and Kyle Plump got together to pair on a fix to this gnarly hanami-router issue. I love to see this! I already replied in the issue, but reviewing this is my number one thing to do when I get back from my break.

A cheeky question

Whew! What an update! If you made it this far, thank you for reading!

Since you’re here, you dedicated few, let me ask something of you. I’ve been writing these open source updates since March 2020. This is my 31st edition! I’m not the fastest writer, so each one takes some hours, typically a whole night of writing, provided I start early enough. So tell me: should I keep writing these updates? What do you take from these? Is there anything you think I could do differently? Or on the flip side, if any of you are regular long-form journallers: how do you keep it up?

Tim in open source, July 2024

2024-07-24T02:38:00+00:00

It’s been a hot minute since my last open source status update. The fact is, I’ve been too busy working on open source to write about my work on open source.

One thing I realise, however, is that work not proclaimed is work not noticed, so let me tell you what I’ve been doing.

In the nine months since September, I:

Worked on Hanami 2.1
Presented on Hanami at RubyConf (San Diego) and RubyConf Taiwan
Took a blessed short summer break
Released Hanami 2.1
Accepted the leadership of the Hanami project
Organised a Ruby unconference in Sydney, a companion event to RubyConf AU
Meticulously planned Hanami 2.2’s database layer
Built Hanami 2.2’s database layer
Released a beta of Hanami 2.2
Prepared to present at RedDotRubyConf in Singapore

That’s a lot! And herein was a pattern, repeated twice:

Plan to announce a major new release at a conference
Push hard to get it ready (oh, and prepare the talk too)
End up not quite making it
Go do the conference anyway (always fun, at least!)
Then come home to yet another big push to finally finish everything off

In the first instance, it was Hanami 2.1 and RubyConf. We were so close to release, but discovered a deal-breaking limitation our assets handling. After frantic hours attempting workarounds from California/Rome/Christchurch, we pulled the plug. There was no easy fix. So after a short break, I overhauled that part of assets system (the right decision: it’s now more flexible and better fits Hanami architecture!) and finally shipped Hanami 2.1 in February.

In the second case, it’s been Hanami 2.2 and RedDotRubyConf (I’m flying to Singapore as I write this!). This time around, we didn’t get so close to a full release, but far enough to put out a beta. Once I’m back, I expect another month or two of concerted work to get everything finished.

So, a pattern of near misses, but real progress nonetheless! I’m also encouraged by some promising signs in this latter instance.

This time around, we have some new active contributors, without whom we couldn’t have shipped the beta. Thank you Adam, Sean, and Marc! This time around, we went from release plan to a quite-complete beta in less than 60 days, instead of the 15 months between the previous major releases. This time around, we’re mere steps away from finishing the full stack vision for Hanami 2, from providing the streamlined experience we envisioned more than five years ago.

Here’s the rub: everything I’ve done with Hanami has been a fully “nights and weekends” deal for me. For the case of Hanami 2.2, it’s meant practically every night and weekend for two straight months. It’s not sustainable. This is why I couldn’t write you those monthly updates. Speaking of patterns: I’ve done this now for too many years, missed too much family time already, and I need to break this cycle.

I’m optimistic, though. Change is coming. The above was the old era. We’re about the enter the new: Hanami 2.2 is on the way! This is something to be excited for! It’s exciting for you, because with just a few commands you’ll have yourself a whole new way of building modular, maintainable, database-backed apps of all shapes in Ruby. It’s also exciting for me, because it means I can at last look up to the horizon and start planning all the great ways we can promote and build upon this new foundation.

It also means I plan to figure out ways to make this whole endeavour sustainable for me. This is the only way we can serve the Hanami and Ruby communities long into the future. Hanami turns 10 this year, and I want it to live for decades more. If you’re experienced with funding OSS, I’d love to chat with you about this.

So there we are, you’re all caught up on a productive few months for Hanami! I’m looking forward to sharing my next update with you, where I hope we can celebrate the release of 2.2 and the beginning of our new era!

2023 in review

2024-01-08T19:55:00+00:00

2023: back in action, in more ways than one.

Five conferences

After a three year break, I spoke at five conferences in the last twelve months:

RubyConf Thailand — a wonderful second edition of this event, the chance to share Hanami 2 on the stage for the first time, and first-time in person hangs with another Buildkiter!
RubyConf AU — a joyous reunion for my home community. Good to be back, and positive signs for the future. Met many Buildkiters in person here!
Brighton Ruby — I’ve admired Andy and this event for the longest time. To attend and contribute a talk was a dream come true.
RubyConf — my first time at the US-based RubyConf, and a brilliant time all around: made new friends, had many great conversations, and got to work with some first-time Hanami contributors during the hack day!
RubyConf Taiwan — What a revelation! A whole thriving Ruby community I had no idea about, and a conference was chock-a-block with interesting talks. Being able to keynote opposite Matz was a real honour.

I’d commend any of these conferences to you, and I hope to get back to each of them in the future.

Being able to talk at so many events was definitely not my plan at the outset of the year! But one thing led to another, and I just rolled with it. It was a pleasure to share Hanami with people in so many places.

One thing I appreciated over this period was the chance to refine my method of introducing Hanami, and moreover, how I deliver conference talks in general. I’m very happy with how this ended up. I ended the year giving two very different presentations, each in their own way imbuing the audience (I hope!) with a sense of both whimsy and possibility: one involved song and dance, and the other, a surprise costume reveal!

A near-release of Hanami

Back at the end of 2021, we made a big push to get Hanami 2.0 released before I made it to RubyConf Thailand, and we succeeded!

I then spent all of 2022 working towards Hanami 2.1, which would introduce our view layer and a completely new approach to handling assets. As RubyConf approached in November, we attempted to do the same thing, to use the conference as motivation and make a big push to get the release out. I spent 2+ months working every night and weekend towards this, and we got so close, but didn’t quite make it.

Just two days before my talk, while I was already at the conference, we discovered what turned out to a release-blocking issue with our front end assets compilation. After a scramble at various stopgap fixes, we decided nothing would quite cut the mustard, and deferred the release. This was disappointing, but was the best choice for the project. After a break over Christmas, I’m now ready to take a final pass at this and make the right choices for the future.

Given this, I hope for 2024 to be a big two-release year for Hanami, with 2.1 happening next month, followed by 2.2 whenever it’s ready.

Family trips!

Once the Brighton Ruby opportunity came up, we decided to make the most of it and turn it into a family trip to Europe, our first overseas trip in the post-2020 era. It was an excellent time. We got to spend a bonus summer across Brighton, London, Paris, Amsterdam, Rotterdam and Brussels. The kids travelled very well, and we all can’t wait to do it again.

We also took a few nice road trips over the year, visiting Orange, Wagga Wagga, and ending the year with a relaxing week over Christmas in a house at Bawley Point.

Work at Buildkite

I had a good year at Buildkite. I spent the year in the same team I joined in 2022. Early in the year we hired some folks and brought the team to its full complement. Everyone is lovely, and we put in some great work towards four different quarterly releases.

I reached my first “bikkiversary” milestone in July, I joined an active on-call roster for the first time, I helped establish some clear direction for our future in front end development, and ended the year acting as an engineering manager, just to help keep things steady while my existing manager took some leave. Every one of these has been a pleasure. I’m continually humbled by the talent that surrounds me at Buildkite, and I’m looking forward to another year of learning.

Best of all, we gathered for our BIPOP (Buildkite In-Person On-site Party; yes, we love acronyms), our whole-company event in Cairns. This was a blast.

Reading

I read 22 books. In order:

Distress, Greg Egan
Sea of Tranquility, Emily St. John Mandel
The Water Knife, Paolo Bacigalupi
Shards of Earth, Adrian Tchaikovsky
Eyes of the Void, Adrian Tchaikovsky
The Thousand Earths, Stephen Baxter
Eversion, Alastair Reynolds
Lords of Uncreation, Adrian Tchaikovsky
Fractal Noise, Christopher Paolini
Me, Ricky Martin
Ancillary Justice, Ann Leckie
Ancillary Sword, Ann Leckie
Ancillary Mercy, Ann Leckie
Provenance, Ann Leckie
Translation State, Ann Leckie
Creation Node, Stephen Baxter
The Spare Man, Mary Robinette Kowal
Some Desperate Glory, Emily Tesh
Fugitive Telemetry, Martha Wells
System Collapse, Martha Wells
Fourth Wing, Rebecca Yarros
Iron Flame, Rebecca Yarros

I enjoyed them all! Adrian Tchaikovsky’s Final Architecture trilogy was thrilling, and discovering Ann Leckie’s Imperial Radch world was a true wonder: I’m so glad I got to binge all five stories together. It was a joy to revisit Murderbot with the most recent two books, and I don’t think I’ve read anything faster than I did Fourth Wing while hanging around a coast house at Christmas.

This year I also figured out my ideal e-reading situation. It’s the iPad mini, and Apple’s Books app in particular. Nothing beats its responsiveness and ease of use. And the iPad mini helps with the occasional tech book I read too. And thanks to Calibre and the Obok plugin, I can also buy books from Kobo and get them into Apple Books in short order.

Assorted things

We ended the year by giving the kids (9 & 7) each their own bedrooms. My desk/office is once again located next to my bed. Given I did this for three years already in our previous apartment, I expect this to work out well enough.
I started using the Retro app to share photos with a small group of friends. It’s brilliant! It has me sharing photos much more regularly, and helped me end the year with a nice collection of memories.
Thanks to several long plane rides, I did a decent amount of movie watching. Notables: No Time to Die (sob), Shotgun Wedding (fun), Jules (surprising), and Indy 5 (a triumph). Best soundtracks? Tetris, The Thomas Crown Affair.
I continue to pay attention to the little computer on my wrist telling me to exercise, though this unfortunately wavered towards the end of the year during my crunch on Hanami and conference talks.

Open source status update, September 2023

2023-10-20T13:01:00+00:00

With the two big PRs introducing our next generation of asset support merged (here and here), September was a month for rapid iteration and working towards getting assets out in a 2.1 beta release.

The pace was lively! Towards the end of the month, Luca and I were trading PRs and code reviews on almost a daily basis. Thanks our opposing timezones, Hanami was being written nearly 24h a day!

Assorted small things

Most of the work was fairly minor: an error logging fix, some test updates for the new assets, error handling around asset manifests, and a bit of zeitwerkin’.

Making our better errors better

There was one interesting piece though. Earlier in this release cycle (back in June!), I overhauled our user-facing error handling. I added a middleware to catch errors and render static error pages intended display in production. As part of this change, I adjusted our router to raise exceptions for not found routes: doing this would allow the error to be caught and a proper 404 page displayed. So that was production sorted. For development, we integrated the venerable better_errors, wrapped by our own hanami-webconsole gem.

It was only some months later that we realised 404s in development were being returned as 500s. This turned out to be because better_errors defaults to a 500 response code at all times. In its middleware:

status_code = 500
# ...
response = Rack::Response.new(content, status_code, headers)

Well, maybe not quite at all times. The lines right beneath status_code = 500:

status_code = 500
if defined?(ActionDispatch::ExceptionWrapper) && exception
  status_code = ActionDispatch::ExceptionWrapper.new(env, exception).status_code
end

Looks like Ruby on Rails gets its own little exception carved out here, via some hard-coded constant checks that reach deep inside Rails internals. This will allow better_errors to return a 404 for a not found error in Rails, but not in any other Ruby framework.

This is not a new change. It arrived over ten years ago, and I can hardly blame the authors for wanting a way to make this work nicely with the predominant Ruby application framework of the day.

Today, however, is a different day! We’re here to change the Ruby framework balance. 😎 So we needed a way to make this work for Hanami. What didn’t feel feasible at this point was a significant change to better_errors: our time was limited and at best we had the appetite only for a minor tactical fix.

Our resulting fix in webconsole (along with this counterpart in hanami) does monkey patch better_errors, but I was very pleased with how gently we could do it. The patch is tiny:

module BetterErrorsExtension
  # The BetterErrors middleware always returns a 500 status when rescuing an exception
  # (outside of Rails). This is not not always appropriate, such as for a
  # `Hanami::Router::NotFoundError`, which should be a 404.
  #
  # To account for this, gently patch `BetterErrors::Middleware#show_error_page` (which is
  # called only when an exception has been rescued) to pass that rescued exception to a proc
  # we inject into the rack env here in our own middleware. This allows our middleware to know
  # the about exception class and provide the correct status code after BetterErrors is done
  # with its job.
  #
  # @see Webconsole::Middleware#call
  def show_error_page(env, exception = nil)
    if (capture_proc = env[CAPTURE_EXCEPTION_PROC_KEY])
      capture_proc.call(exception)
    end

    super
  end
end
BetterErrors::Middleware.prepend(BetterErrorsExtension)

In order to know which response code to use for the page, we need access to the exception that better_error is catching. Right now it provides no hooks to expose that. So instead we prepend some behaviour in front of their #show_error_page, which is only called by the time an error is to be rendered. We look for a proc on the rack env, and if one is there, we pass the exception to it, and then let better_errors get on with the rest of its normal work.

Then, in our own webconsole middleware, we set that proc to capture the exception, using Ruby closure semantics to assign that exception directly to a local variable:

def call(env)
  rescued_exception = nil
  env[CAPTURE_EXCEPTION_PROC_KEY] = -> ex { rescued_exception = ex }

  # ...
end

After that, we call the better_errors middleware, letting it do its own thing:

def call(env)
  rescued_exception = nil
  env[CAPTURE_EXCEPTION_PROC_KEY] = -> ex { rescued_exception = ex }

  status, headers, body = @better_errors.call(env)
end

And then once that is done, we can use the exception (if we have one) to fetch an appropriate response code from the Hanami app config, and then override better_errors’ response code with our own:

def call(env)
  rescued_exception = nil
  env[CAPTURE_EXCEPTION_PROC_KEY] = -> ex { rescued_exception = ex }

  status, headers, body = @better_errors.call(env)

  # Replace the BetterErrors status with a properly configured one for the Hanami app
  if rescued_exception
    status = Rack::Utils.status_code(
      @config.render_error_responses[rescued_exception.class.name]
    )
  end

  [status, headers, body]
end

That’s it! Given how light touch this is, and how stable better_errors is, I’m confident this will serve our purposes quite well for now.

We don’t want to live with this forever, however. In our future I see a fit for purpose developer errors reporter that is fully integrated with Hanami’s developer experience. Given current timelines, this will probably won’t come for at least 12 months, so if this is something you’re interested in helping with, please reach out!

Kickstarting dry-operation!

While the work on Hanami continued, I also helped kickstart work on a new dry-rb gem: dry-operation! Serving as the successor to dry-transaction, with dry-operation we’ll introduce significant new flexibility to modelling composable business operations, while still keeping a high-level API that presents their key flows in an easy to follow way.

Much of the month was spent ideating on various approaches with Marc Busqué and Brooke Kuhlmann, and then by the end of the month, Marc was already underway with the development work. Go check out Marc’s September update for a little more of the background on this.

I’m excited we’re finally providing a bridge to the future for dry-transaction, and at the same time building one of the final pieces of the puzzle for full stack Hanami apps. This is an interesting one for me personally, too, since I’m acting more as a “product manager” for this effort, with Marc doing most of the direct development work. Marc’s been in the dry-rb/Hanami orbit for a while now, and I’m excited for this opportunity for him to step up his contributions. More on this in the future!

Releasing Hanami 2.1.0.beta2!

After all of this, we capped the month off with the release of Hanami 2.1.0.beta2! This was a big step: our first beta to include both views and assets together. In the time since this release we’ve already learnt a ton and found way to take things to another level… but more on that next month. 😉 See you then!

Open source status update, August 2023

2023-09-10T13:30:00+00:00

After last month’s omnibus update, I’m back again, so soon!

August turned out to bring a lot of forward motion for our work on Hanami’s front end assets support. While Luca was taking his summer break, I carried on his work preparing hanami-assets 2.1 and its integration into the Hanami framework. Last week we caught up for a quick chat about these, and now both are merged!

Personally, I think this was an exciting evolution of how Luca and I work together. While previously we each took care of fairly distinct lines of work (there was enough to do, after all!), here we literally worked in tandem on one specific area, and it came out great!

Luca and I also hopped on another video call during August, this time with Seb Wilgosz of Hanami Mastery to record a special core team interview for the site’s 50th episode! I really enjoyed the chance to answer community questions about Hanami, and personally, it was a moment of reassurance that we’re still on the right track and are delivering useful things to people.

The episode isn’t published yet, but one thing that did arise from the episode is a new Hanami 2.1 GitHub project that I put together for tracking our remaining work for the release. Previously, this was in Trello, and with the move to GitHub I hope it will make not our remaining work move visible, but also create clearer opportunities for potential contributors.

Now, with those big two PRs merged and our remaining work more clearly listed, the pace is picking up! We’re now at the point where we can focus on the direct user experience of working with assets within a full Hanami app. I expect a lot will shake out from this in quick order. But more on that next month!

Open source status update, October 2022–July 2023

2023-07-23T07:45:00+00:00

It’s been a hot minute since my last open source status update! Let’s get caught up, and hopefully we can resume the monthly cadence from here.

Released Hanami 2.0

In Novemver we released Hanami 2.0.0! This was a huge milestone! Both for the Hanami project and the Ruby communuity, but also for us as a development team: we’d spent a long time in the wilderness.

All of this took some doing. It was a mad scramble to get here. The team and I worked non-stop over the preceding couple of months to get this release ready (including me during the mornings of a family trip to Perth).

Anyway, if you’ve followed me here for a while, most of the Hanami 2 features should hopefully feel familiar to you, but if you’d like a refresher, check out the Highlights of Hanami 2.0 that I wrote to accompany the release announcement.

Spoke at RubyConf Thailand

Just two weeks after the 2.0 release, I spoke at RubyConf Thailand 2022!

Given I was 100% focused on Hanami dev work until the release, this is probably the least amount of time I’ve had for conference talk preparation, but I was happy with the result. I found a good hook (“new framework, new you”, given the new year approaching) and put together a streamlined introduction to Hanami that fit within the ~20 minutes alotted to the talks (in this case, it was a boon that we hadn’t yet released our view or persistence layers 😆).

Check it out here:

Overhauled hanami-view internals and improved performance

With the 2.0 release done, we decided to release our view and persistence layers progressively, as 2.1 and 2.2 respectively. This would allow us to keep our focus on one thing at a time and improve the timeliness of the upcoming releases.

So over the Christmas break (including several nights on a family trip to the coast), I started work on the first big blocker for our view layer: hanami-view performance. We were slower than Rails, and that just doesn’t cut the mustard for a framework that advertises itself as fast and light.

Finding the right approach here took several goes, and it was finally ready for this pull request at the end of February. I managed to find a >2x performance boost while simplifying our internals, improving the ergonomics of Hanami::View::Context and our part and scope builders, and still retaining all existing features.

Spoke at RubyConf Australia

Also in February, I spoke at RubyConf Australia 2023! After a 3 year hiatus, this was a wonderful reunion for the Ruby community across Australia and New Zealand. It looked like we lost no appetite for these events, so I’m encouraged for next year and beyond.

To fit the homecoming theme, I brought a strong tinge of Australiana to my talk, and expanded it to include a preview of the upcoming view and persistence layers. Check it out:

Created Hanami::View::ERB, a new ERB engine

After performance, the next big issue for hanami-view was having our particular needs met by our template rendering engines, as well as making auto-escaping the default for our “first party supported” engines (ERB, Haml, Slim) that output HTML.

ERB support was an interesting combination of all these issues. For hanami-view, we don’t expect any rendering engine to require explicit capturing of block content. This is what allows methods on parts and scopes simply to yield and have the returned value match content provided to the block from within the template.

To support this with ERB, we previously had to require our users install and use the erbse gem, a little-used and incomplete ERB implementation that provided this implicit block capturing behaviour by default (but did not support auto-escaping of HTML-unsafe values). For a long while we also had to require users use hamlit-block for the same reasons, and as such we had to build a compatibility check between ourselves and Tilt to ensure the right engines were available. This arrangement was awkward and untenable for the kind of developer experience we want for Hanami 2.

So to fix all of this, I wrote our own ERB engine! This provides everything we need from ERB (implicit block capture as well as auto-escaping) and also allows for hanami-view to be used out of the box without requiring manual installation of other gems.

Meanwhile, in the years since my formative work on hanami-view (aka dry-view), Haml and Slim evolved to both use Temple and provide configuration hooks for all the behaviour we require, so this allowed me to drop our template engine compatibility checks and instead just automatically configure Haml or Slim to match our needs if they’re installed.

To support our auto-escaping of HTML-unsafe values, we’ve adopted the Object and String #html_safe? patches that are prevalent across relevant libraries in the Ruby ecosystem. This gives us the broadest possible compatibility, as well as a streamlined and unsurprising user experience. While you might see folks decry monkey patches in general, this is one example where it makes sense for Hanami to take a pragmatic approach, and I’m very pleased with the outcome.

Implemented helpers for hanami-view

After performance and rendering/HTML safety, the last remaining pre-release item for hanami-view was support for helpers. This needed a bit of thinking to sort out, since the new hanami-view provides a significantly different set of view abstractions compared to the 1.x edition.

Here’s how I managed to sort it out:

In hanami-view, add configurable part_class and scope_class settings to hanami-view
In hanami, configure slice-specific part and scope classes
In hanami, include first-party helper modules into these classes
By this point, all the wiring is in place for first-party helpers
So in hanami-view, port all relevant helpers from the 1.x hanami-helpers gem, including a new, simpler TagHelper for generating HTML tags.
And over in hanami, introduce a FormHelper that integrates with the full app, including pulling params from the request
Then seamlessly provide all first-party as well as user-defined helpers to view scopes and classes

After this, all helpers should appear whereer you need them in your views, whether in templates, part classes or scope classes. Each slice will also generate a Views::Helpers module to serve as the starting point for your own collection of helpers, too.

With hanami-view providing parts and scopes, the idea is that you can and should use available-everywhere helpers less than before, but they can still be valuable from time to time, and with their introduction, now you have every possible option available for building your views.

Added friendly error pages

While focused on views, I also took the chance to make our error views friendly too. Now we:

Render nice error pages in production mode (this is also configurable so you can add custom handling of specific error types)
Render detailed error pages in development mode, including full stack trace and interactive console, courtesy of better_errors

Worked on integrating hanami-assets

Alongside all of this, Luca has been working hard on our support for front end assets via an esbuild plugin and its integration with the framework. This has been nothing short of heroic: he’s been beset by numerous roadblocks but overcome each one, and now we’re getting really close.

Back in June, Luca and I had our first ever pairing session on this work! We got a long way in just a couple of hours. I’m looking forward to pitching in with this as my next focus.

Prepared the Hanami 2.1.0.beta1 release

With all the views work largely squared away, I figured it was time to make a beta release and get this stuff out there for people to test, so we released it as 2.1.0.beta1 at the end of June.

Spoke at Brighton Ruby!

Also at the end of June I spoke at Brighton Ruby! I’ve wanted to attend this event for the longest time, and it did not disappoint. I had a wonderful day at the conference and enjoyed meeting a bunch of new Ruby friends.

For my talk I further evolved the content from the previous iterations, and this time included a look at how we might grow a Hanami app into a more real thing, as well as reflections on what Hanami 2’s release might mean for the Ruby community. I also experimented with a fun new theme and narrative device, which you shall be able to see once the video is out 😜

Thank you so much to Andy for the invitation and the support. ❤️

Took a holiday

After all of that, I took a break! You might’ve noticed my mentions of all the Hanami work I was doing while ostensibly on family trips. Well, after Brighton Ruby, I was all the way in Europe with the family, and made sure to have a good proper 4 weeks of (bonus summer) holiday. It was fanastic, and I didn’t look at Ruby code one bit.

What’s next

Now that I’m back, I’ll focus on doing whatever is necessary to complete our front end assets integration and get that out as a 2.1 beta2 release. Our new assets stuff is the completely new, so some time for testing and bug fixing will be useful.

Over the rest of the beta period I hope to complete a few smaller general framework improvements and fixes, and from there we can head towards 2.1.0 final.

I suspect it will take at least one more OSS status updates before that all happens, so I can check in with you about it all then!

Open source status update, September 2022

2022-10-10T13:10:00+00:00

Hello there, friends! This is going to be a short update from me because I’m deep in the throes of Hanami 2.0 release preparation right now. Even still, I didn’t want to let September pass without an update, so let’s take a look.

A story about Hanami::Action memory usage

September started and ended with me looking at the r10k memory usage charts for hanami-controller versus Rails. The results were surprising!

We’d been running some of these checks as part of our 2.0 release prep, the idea being that it’d help us shake out any obvious performance improvements we’d need to make. And it certainly did in this case! Hanami (just like its dry-rb underpinnings) is meant to be the smaller and lighter framework; why were we being outperformced by Rails?

To address this I wrote a simple memory profile script for Hanami::Action inheritance (now checked in here) and started digging.

Here were there initial results:

Total allocated: 184912288 bytes (1360036 objects)
Total retained:  104910880 bytes (780031 objects)

allocated memory by gem
-----------------------------------
  56242240  concurrent-ruby-1.1.10
  53282480  dry-configurable-0.15.0
  34120000  utils-8585be837309
  30547488  other
  10720080  controller/lib

That’s 185MB allocated for 10k subclasses, with concurrent-ruby, dry-configurable and hanami-utils being the top three gems allocating memory.

This led me straight to dry-configurable, and after a couple of weeks of work, I arrived at this PR, separating our storage of setting definitions from their configured values, among other things. This change allows us to copy less data at the moment of class inheritance, and in the case of a dry-configurable-focused memory profile, cut the allocated memory by more than half.

From there, I moved back into hanami-controller and updated it to use dry-configurable for all of its inheritable attributes (some were handled separately), also taking advantage the support for custom config classes that Piotr added so we could preserve Hanami::Action’s existing configuration API.

This considerably improved our benchmark! Behold:

Total allocated: 32766232 bytes (90004 objects)
Total retained:  32766232 bytes (90004 objects)

allocated memory by gem
-----------------------------------
  21486072  other
  10880120  dry-configurable-0.16.1
    400040  3.1.2/lib

Yes, we brought 185MB allocated memory down to 33MB! This also brought us on par with Rails in the extreme end of the r10k memory usage benchmark:

Here’s a thing though: the way r10k generates actions for its Rails benchmark is to create a single controller class with a method per action. So for the point on the far right of that chart, that’s a single class with 10k methods. Hardly realistic.

So I made a quick tweak to see how things would look if the r10k Rails benchmark generated a class per endpoint like we do with Hanami::Action:

That’s more like it. This is another extreme, however: more realistically, we’d see Rails apps with somewhere between 5-10 actions per controller class, which would lower its dot a little in that graph. In my opinion this would be a useful thing to upstream into r10k. It’s already a contrived benchmark, yes, but it’d be more useful if it at least mimicked realistic application structures.

Either way, we finished the month much more confident that we’ll be delivering on our promise of Hanami as the lighter, faster framework alternative. A good outcome!

Along the way, however, things did feel bleak at times. I wasn’t confident that I’d be able to make things right, and it didn’t feel great to think we might’ve spent years putting somethign together that wasn’t going to be able to deliver on some of those core promises. Luckily, I found all the wins we needed, and learnt a few things along the way.

Hanami 2.0, here we come

What else happened in September? Possibly the biggst thing is that we organised ourselves for the runway towards the final Hanami 2.0.0 release.

We want to do everything possible to make sure the release happens this year, so I spent some time organising the remaining tasks on our Trello board into date-based lists, aiming for a release towards the end of November. It looked achievable! The three of us in the core team re-committed ourselves to doing everything we could to complete these tasks in our estimated timeframes.

So far, things have gone very well!

We’ve all been working tremendously hard, and so far, this has let us keep everything to the schedule. I’ll have a lot to share about our work across October, but that’s all for next month’s update. So in the meantime, I have to put my head back down and get back to shipping a framework. See you all again soon!

Open source status update, August 2022

2022-09-17T21:30:00+00:00

August’s OSS work landed one of the last big Hanami features, saw another Hanami release out the door, began some thinking about memory usage, and kicked off a fun little personal initiative. Let’s dive in!

Conditional slice loading in Hanami

At the beginning of the month I merged support for conditional slice loading in Hanami. I’d wanted this feature for a long time, and in fact I’d hacked in workarounds to achieve the same more than 2 years ago, so I was very pleased to finally get this done, and for the implementation work to be as smooth as it was.

The feature provides a new config.slices setting on your app class, which you can configure like so:

module MyApp
  class App < Hanami::App
    config.slices = %w[admin]
  end
end

For an app consisting of both Admin and Main slices and for the config above, when the app is booted, only the Admin slice will be loaded:

require "hanami/prepare"

Hanami.app.slices.keys # => [:admin]

Admin::Slice # exists, as expected
Main         # raises NameError, since it was never loaded

As we see from Main above, slices absent from this list will not have their namespace defined, nor their slice class loaded, nor any of their Ruby source files. Within that Ruby process, they effectively do not exist.

Specifying slices to load can be very helpful to improve boot time and minimize memory usage for specific deployed workloads of your app.

Imagine you have a subset of background jobs that run via a dedicated job runner, but whose logic is otherwise unneeded for the rest of your app to function. In this case, you could organize those jobs into their own slice, and then load only that slice for the job runner’s process. This arrangement would see the job runner boot as quickly as possible (no extraneous code to load) as well as save all the memory otherwise needed by all those classes. You could also do the invserse for your main deployed process: specify all slices except this jobs slice, and you gain savings there too.

Organising code into slices to promote operational efficiency like this also gives you the benefit of greater clarity in the separation of responsibilities between those slices: when a single slice of code is loaded and the rest of your app is made to disappear, that will quickly surface any insidious dependencies from that slice to the rest of your code (they’ll be raised as exceptions!). Cleaning these up will help ensure your slices remain useful as abstractions for reasoning about and maintaining your app.

To make it easy to tune the list of slices to load, I also introduced a new HANAMI_SLICES env var that sets this config without you having to write code inside your app class. In this way, you could use them in your Procfile or other similar deployment code:

web: HANAMI_SLICES=main,admin bundle exec puma -C config/puma.rb
feed_worker: HANAMI_SLICES=feed bundle exec rake jobs:work

This effort was also another example of why I’m so happy to be working alongside the Hanami core team. After initially proposing a more complex arrangement including separate lists for including or excluding slices, Luca jumped in and help me dial this back to the much simpler arrangement of the single list only. For an Hanami release in which we’re going to be introducing so many new ideas, the more we can keep simple around them, the better, and I’m glad to have people who can remind me of this.

Fixed how slice config is applied to component classes

Our action and view integration code relies on their classes detecting when they’re defined inside a slice’s namespace, then applying relevant config from the slice to their own class-level config object. It turned out our code for doing this broke a little when we adjusted our default class hierarchies. Thanks to some of our wonderful early adopters, we picked this up quickly and I fixed it. Now things just work like you expect however you choose to configure your action classes, whether through the app-level config.actions object, or by directly updating config in a base action class.

In doing this work, I became convinced we need an API on dry-configurable to determine whether any config value has been assigned or mutated by the user, since it would help so much in reliably detecting whether or not we should ignore config values at particular levels. For now, we could work around it, but I hope to bring this to dry-configurable at some point in the future.

Released Hanami 2.0.0.beta2

Another month passed, so it was time for another release! With my European colleagues mostly enjoying some breaks over their summer, I hunkered down in chilly Canberra and took care of the 2.0.0.beta2 release. Along with the improvements above, this release also included slice and action generators (hanami generate slice and hanami generate action, thank you Luca!), plus a very handle CLI middlewares inspector (thank you Marc!):

$ hanami middlewares

/    Dry::Monitor::Rack::Middleware (instance)
/    Rack::Session::Cookie

The list of things to do over the beta phase is getting smaller. I don’t expect we’ll need too many more of these releases!

Created memory usage benchmarks for dry-configurable

As the final 2.0 release gets closer, we’ve been doing various performance tests just to make sure the house is in order. One thing we discovered is that Hanami::Action is not as memory efficient as we’d like it to be. One of the biggest opportunities to improve this looked to be in dry-configurable, since that’s what is used to manage the per-class action configuration.

I suspected any effort here would turn out to be involved (and no surprise, it turned out to be involved 😆), so I thought it would be useful as a first step to establish a memory benchmark to revisit over the course of any work. This was also a great way to get my head in this space, which turned out to take over most of my September (but more on that next month).

Quietly relaunched Decaf Sucks

Decaf Sucks was once a thriving little independent online café review community, with its own web site (starting from humble beginnings as a Rails Rumble entry in 2009) and even native iOS app (two iterations, in fact).

I was immensitely proud of what Decaf Sucks became, and for the collaboration with Max Wheeler in building it.

Unfortunately, as various internet APIs changed, the site atrophied, eventually became disfunctional, and we had to take it down. I still have the database, however, and I want to bring it back!

This time around, my plan is to do it as a fully open source Hanami 2 example application. Max is even on board to bring back all the UI goodness. For now, you can follow along with the early steps on GitHub. Right now the app is little more than the basic Hanami skeleton with added database integration and a CI setup (Hello Buildkite!), but I plan to grow it bit by bit. Perhaps I’ll try to have something small that I can share with each of these monthly OSS updates.

After Hanami 2 ships, hopefully this will serve as a useful resource for people wanting to see how it plays out in a real working app. And beyond that, I look forward to it serving once again as a place for me to commemorate my coffee travels!

Open source status update, May–July 2022

2022-08-08T05:20:00+00:00

Hi there friends, it’s certainly been a while, and a lot has happened across May, June and July: I left my job, took some time off, and started a new job. I also managed to get a good deal of open source work done, so let’s take a look at that!

Released Hanami 2.0.0.alpha8

Since we’d skipped a month in our releases, I helped get Hanami 2.0.0.alpha8 out the door in May. The biggest change here was that we’d finished relocating the action and view integration code into the hanami gem itself, wrapped up in distinct “application” classes, like Hanami::Application::Action. In the end, this particular naming scheme turned out to be somewhat short lived! Read on for more :)

Resurrected work using dry-effects within hanami-view

As part of an effort to make it easy to use our conventional view “helpers” in all parts of our view layer, I resurrected my work from September 2020(!) on using dry-effects within hanami-view. The idea here was to achieve two things:

To ensure we keep only a single context object for the entire view rendering, allowing its state to be preserved and accessed by all view components (i.e. allowing both templates, partials and parts all to access the very same context object)
To enable access to the current template/partial’s #locals from within the context, which might help make our helpers feel a little more streamlined through implicit access to those locals

I got both of those working (here’s my work in progress), but I discovered the performance had worsened due to the cost of using an effect to access the locals. I took a few extra passes at this, reducing the number of effects to one, and memoziing it, leaving us with improved performance over the main branch, but with a slightly different stance: the single effect is for accessing the context object only, so any helpers, instead of expecting access to locals, will instead only have access to that context. The job from here will be to make sure that the context object we build for Hanami’s views has everything we need for an ergonomic experience working with our helpers. I’m feeling positive about the direction here, but it’ll be a little while before I get back to it. Read on for more on this (again!).

Unified application and slice

The biggest thing I did over this period was to unify Hanami’s Application and Slice. This one took some doing, and I was glad that I had a solid stretch of time to work on it between jobs.

I already wrote about this back in April’s update, noting that I’d settled on the approach of having a composed slice inside the Hanami::Application class to providing slice-like functionality at the application level. This was the approach I continued with, and as I went, I was able to move more and more functionality out of Hanami::Application and into Hanami::Slice, with that composed “application slice” being the thing that preserved the existing application behaviour. At some point, a pattern emerged: the application is a slice, and we could achieve everything we wanted (and more) by turning class Hanami::Application into class Hanami::Application < Hanami::Slice.

Turning the application into a slice sublcass is indeed how I finished the work, and I’m extremely pleased with how it turned out. It’s made slices so much more powerful. Now, each slice can have its own config, its own dedicated settings and routes, can be run on its own as a Rack application, and can even have its own set of child slices.

As a user of Hanami you won’t be required to use all of this per-slice power features, but they’ll be there if or when you want them. This is a great example of progressive disclosure, a principle I follow as much as possible when designing Hanami’s features: a user should be able to work with Hanami in a simple, straightforward way, and then as their needs grow, they can then find additional capabilities waiting to serve them.

Let’s explore this with a concrete example. If you’re building a simple Hanami app, you can start with a single top-level config/settings.rb that defines all of the app’s own settings. This settings object is made available as a "settings" component registration in both the app as well as all its slices. As the app grows and you add a slice or two, you start to add more slice-specific settings to this component. At this point you start to feel a little uncomfortable that settings specific to SliceA are also available inside SliceB and elsewhere. So you wonder, could you go into slices/slice_a/ and drop a dedicated config/settings.rb there? The answer to that is now yes! Create a config/settings.rb inside any slice directory and it will now become a dedicated settings component for that slice alone. This isn’t a detail you had to burden yourself with in order to get started, but it was ready for you when you needed it.

Another big benefit of this code reorganisation is that the particular responsibilities of Hanami::Application are much clearer: its job is to provide the single entrypoint to the app and coordinate the overall boot process; everything else comes as part of it also being a slice. This distinction is made clear through the number of public methods that exist across the two classes: Application now has only 2 distinct public methods, whereas Slice currently brings 27.

There’s plenty more detail over in the pull request: go check it out!

The work here also led to changes across the ecosystem:

Because we’re now importing components from the app into other slices in the root key namespace (rather than prefixed by "application." as before), we needed to make that possible in dry-system.
But with it being possible for each slice to have its own dedicated version of certain components (like "settings" in the example above), we also needed to make dry-system prefer local components when importing.
This itself needed a tweak to dry-container such that we could reverse the order of its Container#merge via a block parameter.
To make certain components conditional within the slice containers, we’re using dry-system’s class-based provider sources, which needing tweaking to support deeper class hierarchies.

This is one the reasons I’m excited about Hanami’s use of the dry-rb gems: it’s pushing them in directions no one has had to take them before. The result is not only the streamlined experience we want for Hanami, but also vastly more powerful underpinnings.

Devised a slimmed down core app structure

While I had my head down working on internal changes like the above, Luca had been thinking about Hanami 2 adoption and the first run user experience. As we had opted for a slices-only approach for the duration of our alpha releases, it meant a fairly bulky overall app structure: every slice came with multiple deeply nested files. This might be overwhelming to new users, as well as feeling like overkill for apps that are intended to start small and stay small.

To this end, we agreed upon a stripped back starter structure. Here’s how it looks at its core (ignoring tests and other general Ruby files):

├── app/
│   ├── action.rb
│   └── actions/
├── config/
│   ├── app.rb
│   ├── routes.rb
│   └── settings.rb
├── config.ru
└── lib/
    ├── my_app/
    │   └── types.rb
    └── tasks/

That’s it! Much more lightweight. This approach takes advantage of the Hanami app itself becoming a fully-featured slice, with app/ now as its source directory.

In fact, I took this opportunity to unify the code loading rules for both the app and slices, which makes for a much more intuitive experience. You can now drop any ruby source file into app/ or a slices/[slice_name]/ slice dir and it will be loaded in the same way: starting at the root of each directory, classes defined therein are expected to inhabit the namespace that the app or slice represents, so app/some_class.rb would be MyApp::SomeClass and slices/my_slice/some_class would be MySlice::SomeClass. Hat tip to me of September 2021 for implementing the dry-system namespaces feature that enabled this! 😜

(Yet another little dry-system tweak came out of preparing this too, with Component#file_name now exposed for auto-registration rules).

This new initial structure for starter Hanami 2.0 apps is another example of progressive disclosure in our design. You can start with a simple all-in-one approach, everything inside an app/ directory, and then as various distinct concerns present themselves, you can extract them into dedicated slices as required.

Along with this, some of our names have become shorter! Yes, “application” has become “app” (and Hanami::Application has become Hanami::App, and so on). These shorter names are easier to type, as well as more reflective of the words we tend to use when verbally describing these structures.

We also tweaked our actions and views integration code so that it is automatically available when you inherit directly from Hanami::Action, so it will no longer be necessary to have the verbose Hanami::Application::Action as the superclass for the app’s actions. We also ditched that namespace for both routes and settings too, so now you can just inherit from Hanami::Settings and the like.

Devised a slimmed down release strategy

Any of you following my updates would know by now that the Hanami 2.0 release has been a long time coming. We have ambitious goals, we’re doing our best, and everything is slowly coming together. But as hard as it might’ve been for folks who’re waiting, it’s been doubly so for us, feeling the weight of both the work along with everyone’s expectations.

So to make sure we can focus our efforts and get something out the door sooner rather than later, we decided to stagger our 2.0 release. We’ll start off with an initial 2.0 release centred around hanami, hanami-cli, hanami-controller, and hanami-router (enough to write some very useful API applications, for example), then follow up with a “full stack” 2.1 release including database persistence, views, helpers, assets and everything else.

I’m already feeling empowered by this strategy: 2.0 feels actually achievable now! And all of the other release-related work like updated docs and a migration guide will become correspondingly easier too.

Released Hanami 2.0.0.beta1!

With greater release clarity as well as all the above improvements under our belt, it was time to usher in a new phase of Hanami 2.0 development, so we released 2.0.0.beta1 in July! This new version suffix represents just how close we feel we are to our final vision for 2.0. This is an exciting moment!

And a bunch more

This update is getting rather long, so let me list a bunch of other Hanami improvements I managed to get done:

You can now autoload your app’s constants from within your settings file, which is useful if you’re referring to a types module to type check your settings. This turned out to be a larger change than expected, but it’s further bolstered our slice capabilities, because it led to each slice getting its own Zeitwerk autoloader instance, meaning each slice fully owns its code loading lifecycle.
.env file loading is now a much clearer step upon subclassing Hanami::App, rather than it being a side effect of loading the settings. This is useful in case you want to directly access the ENV for any other purpose.
The .prepare_container escape hatch allowing for last-minute manual adjustments to a slice’s internal dry-system container is now run after all possible framework setup has been applied
In the few places we rescue from LoadError, we now ensure we don’t swallow any unintended errors
Hanami now respects RACK_ENV if HANAMI_ENV is not set, which is useful for Rack-first apps that happen to be migrating to Hanami 2.0 (these do exist!)
I removed the settings allowing for customisation of the routes and settings file paths. The further I go with batteries included framework development, the better sense I think I’m developing for what should’t be configurable. This is one such case: we can’t provide reasonable documentation or support if these files can live in arbitrary locations.
Last but not least, I built conditional slice loading! But since this isn’t merged yet, I’ll go into details on this in next month’s update ;)

Outside my Hanami development, a new job and a new computer meant I also took the change to reboot my dotfiles, which are now powered by chezmoi. I can’t speak highly enough of chezmoi, it’s an extremely powerful tool and I’m loving the flexibility it affords!

That’s it from me for now. I’ll come back to you all in another month!

Joining Buildkite, and sticking with Ruby

2022-06-14T06:50:00+00:00

Last week I finished up at Culture Amp, and I’m excited to announce that I’ll be joining Buildkite as an engineer!

My time at Culture Amp was special. It was my first role after a decade of running Icelab with Max and Michael. Culture Amp hired everyone at Icelab after we decided to close the business, providing both a smooth transition and new opportunities to a singular group. I built a great working relationship with my manager, I was trusted to do big things, and I relished the chance to work with and learn from a large group of engineers. I’m deeply thankful for all of this.

Towards the end, I was serving as Culture Amp’s Director of Back End Engineering, and moving into engineering management. However, as any astute reader of this blog might attest, I am deeply motivated by hands on programming work, and all the learning and collaboration opportunities that go with it. I realised it was not the time to draw that chapter to a close (it might never!), and through that consideration I connected with Buildkite.

I’m excited to join Buildkite for many reasons! It’s a great Australian company with heart and personality. It brims with people I’ve long dreamt of working with. Developer tooling is an area close to my heart. And they’re growing a (majestic) Ruby app at the core of their tech. I can’t wait to dig in.

For me, this is also an intentional decision to stick with Ruby. The work I’m doing in Ruby OSS right now might be one of the biggest “dents in the universe” I get to make. I want to see this effort through, to complete our vision for Hanami 2.0, then learn from how it’s adopted by our community.

I have some time off between jobs, which I’ll use to give our Hanami work a real boost: I’ll be commiting nearly 6 weeks of full-time work to Hanami! Based on previous experience, this should see me get through what otherwise might have taken 6 months of part-time effort. I’m hoping this will get us significantly closer to 2.0. I’ll likely start another tweet thread of my efforts, so find me on Twitter if you’d like to follow along!

Open source status update, April 2022

2022-05-15T12:20:00+00:00

April was a pretty decent month for my OSS work! Got some things wrapped up, kept a few things moving, and opened up a promising thing for investigation. What are these things, you say? Let’s take a look!

Finished centralisation of Hanami action and view integrations

I wrote about the need to centralise these integrations last month, and in April, I finally got the work done!

This was a relief to get out. As a task, while necessary, it felt like drudge work – I’d been on it since early March, after all! I was also conscious that this was also blocking Luca’s work on helpers all the while.

My prolonged work on this (in part, among other things like Easter holidays and other such Real Life matters) contributed to us missing April’s Hanami release. The good thing is that it’s done now, and I’m hopeful we can have this released via another Hanami alpha sometime very soon.

In terms of the change to Hanami apps, the biggest change from this is that your apps should use a new superclass for actions and views:

require "hanami/application/action"

module Main
  module Action
    # Used to inherit from Hanami::Action
    class Base < Hanami::Application::Action
    end
  end
end

Aside from the benefit to us as maintainers of having this integration code kept together, this distinct superclass should also help make it clearer where to look when learning about how actions and views work within full Hanami apps.

Enabled proper access to full `locals` in view templates

I wound up doing a little more work in actions and views this month. The first was a quickie to unblock some more of Luca’s helpers work: making access to the locals hash within templates work like we always expected it would.

This turned out to be a fun one. For a bit of background, the context for every template rendering in hanami-view (i.e. what self is for any given template) is an Hanami::View::Scope instance. This instance contains the template’s locals, makes the full locals hash available as #locals (and #_locals, for various reasons), and uses #method_missing to make also make each local directly available via its own name.

Luca found, however, that calling locals within the template didn’t work at all! After I took a look, it seemed that while locals didn’t work, self.locals or just plain _locals would work. Strange!

Turns out, this all came down to implementation details in Tilt, which we use as our low-level template renderer. The way Tilt works is that it will compile a template down into a single Ruby method that receives a locals param:

def compile_template_method(local_keys, scope_class=nil)
  source, offset = precompiled(local_keys)
  local_code = local_extraction(local_keys)

  # <...snip...>

  method_source << <<-RUBY
    TOPOBJECT.class_eval do
      def #{method_name}(locals)
        #{local_code}
  RUBY

Because of this, locals is actually a local variable in the context of that method execution, which will override any other methods also available on the scope object that Tilt turns into self for the rendering.

Here is how we were originally rendering with Tilt:

tilt(path).render(scope, &block)

My first instinct was simply to pass our locals hash as the (optional) second argument to Tilt’s #render:

tilt(path).render(scope, scope._locals)

But even that didn’t work! Because in generating that local_code above, Tilt will actually take the locals and explode it out into individual variable assignments:

def local_extraction(local_keys)
  local_keys.map do |k|
    if k.to_s =~ /\A[a-z_][a-zA-Z_0-9]*\z/
      "#{k} = locals[#{k.inspect}]"
    else
      raise "invalid locals key: #{k.inspect} (keys must be variable names)"
    end
  end.join("\n")
end

But we don’t need this at all, since hanami-view’s scope object is already making those locals available individually, and we want to ensure access to those locals continues to run through the scope object.

So the ultimate fix is to make locals of our locals. Yo dawg:

tilt(path).render(scope, {locals: scope._locals}, &block)

This gives us our desired access to the locals hash in templates (because that locals key is itself turned into a solitary local variable), while preserving the rest of our existing scope-based functionality.

It also shows me that I probably should’ve written an integration test back when I introduced access to a scope’s locals back in January 2019. 😬

Either way, I’m excited this came up and I could fix it, because it’s an encouraging sign of just how much of this view system we’ll be able to put to use in creating a streamlined and powerful view layer for our future Hanami users!

Merged a fix to stop unwanted view rendering of halted requests

Thanks to our extensive use of Hanami at Culture Amp, my friend and colleague Andrew discovered and fixed a bug with our automatic rendering of views within actions, which I was happy to merge in.

Shipped some long awaited dry-configurable features

After keeping poor ojab waiting way too long, I also merged a couple of nice enhancements he made to dry-configurable:

Make Config#finalize! finalize itself recurively and (optionally) allow freezing of setting values
Accept hashes as possible setting values when using #update, which I followed up with support for anything implicitly convertible to a hash.

I then released these as dry-configurable 0.15.0.

Started work on unifying Hanami slices and application

Last but definitely not least, I started work on one of the last big efforts we need in place before 2.0: making Hanami slices act as much as possible like complete, miniature Hanami applications. I’m going to talk about this a lot more in future posts, but for now, I can point you to a few PRs:

Introducing Hanami::SliceName (a preliminary, minor refactoring to fix some slice and application name determination responsibilities that had somehow found their way into our configuration class).
A first, abandoned attempt at combining slices and applications, using a mixin for shared behaviour.
A much more promising attempt using a composed slice object within the application class, which is currently the base of my further work in this area.

Apart from opening up some really interesting possibilities around making slices fully a portable, mountable abstraction (imagine bringing in slices from gems!), even for our shorter-term needs, this work looks valuable, since I think it should provide a pathway for having application-wide settings kept on the application class, while still allowing per-slice customisation of those settings in whichever slices require them.

The overall slice structure is also something that’s barely changed since I put it in place way back in late 2019. Now it’s going to get the spit and polish it deserves. Hopefully I’ll be able to share more progress on this next month :) See you then!

Two years of open source status updates

2022-05-02T14:15:00+00:00

Back in March of 2020, I decided to take up the habit of writing monthly status updates for my open source software development. 22 updates and 25k words later, I’m happy to be celebrating two years of status updates!

As each month ticks around, I can find it hard to break away from cutting code and write my updates, but every time I get to publishing, I’m really happy to have captured my progress and thinking.

After all, these posts now help remind me I managed to do all of the following over the last two years (and these were just the highlights!):

Renamed dry-view to hanami-view and kicked off view/application integration (Mar 2020)
Received my first GitHub sponsor (Apr 2020), thank you Benny Klotz (who still sponsors me today!)
Shared my Hanami 2 application template (May 2020)
Achieved seamless view/action/application integration (May 2020)
Brought class-level configuration to Hanami::Action (Jun 2020)
Introduced application-level configuration for actions and views (Jul 2020)
Added automatic inference for an action’s paired view, along with automatic rendering (Jul 2020)
Introduced application integration for view context classes (Jul 2020)
Supported multiple boot file dirs in dry-system, allowing user-replacement of standard bootable components in Hanami (Aug 2020)
Rebuilt the Hanami Flash class (Aug 2020)
Resumed restoring hanami-controller features through automatic enabling of CSRF protection (Sep 2020)
Added automatic configuration to views (inflector, template, part namespace) (Oct 2020)
Released a non-web Hanami application template (Oct 2020)
Started the long road to Hanami/Zeitwerk integration with an autoloading loader for dry-system (Nov 2020)
Introduced dedicated “component dir” abstraction to dry-system, along with major cleanups and consistency wins (Dec 2020/Jan 2021)
Added support for dry-system component dirs with mixed namespaces (Feb/Mar/Apr 2021)
Released dry-system with all these changes, along with Hanami with working Zeitwerk integration (Mar/Apr 2021)
Ported Hanami’s app configuration to dry-configurable (May 2021),
Laid the way for dry-configurable 1.0 with some API changes (May/Jul 2021)
Returned to dry-system and added configurable constant namespaces (Jun/Jul/Aug/Sep/Oct 2021)
Introduced compact slice source dirs to Hanami, using dry-systems constant namespaces (Sep/Oct 2021)
Added fully configurable source dirs to Hanami (Nov/Dec 2021)
Shipped a huge amount of dry-system improvements over two weeks of dedicated OSS time in Jan 2022, including the overhaul of bootable components as part of their rename to providers, as well as partial container imports and exports, plus much more
Introduced concrete slice classes and other slice registration improvements to Hanami (Feb 2022)
Refactored and relocated action and view integration into the hanami gem itself, and introduced Hanami::SliceConfigurable to make it possible for similar components to integrate (Mar 2022)

This is a lot! To add some extra colour here, a big difference betwen now and pre-2020 is that I’ve been working on OSS exclusively in my personal time (nights and weekends), and I’ve also been slugging away at a single large goal (Hanami 2.0, if you hadn’t heard!), and the combination of this can make the whole thing feel a little thankless. These monthly updates are timely punctuation and a valuable reminder that I am moving forward.

They also capture a lot of in-the-moment thinking that’d otherwise be lost to the sands of time. What I’ve grown to realise with my OSS work is that it’s as much about the process as anything else. For community-driven projects like dry-rb and Hanami, the work will be done when it’s done, and there’s not particularly much we can do to hurry it. However, what we should never forget is to make that work-in-progress readily accessible to our community, to bring people along for the ride, and to share whatever lessons we discover along the way. The passing of each month is a wonderful opportunity for me to do this 😀

Finally, a huge thank you from me to anyone who reads these updates. Hearing from folks and knowing there are people out there following along is a huge encouragement to me.

So, let’s keep this going. I’m looking forward to another year of updates, and—checks calendar–writing April’s post in the next week or so!

Salubrious Ruby: Don’t mutate what you don’t own

2022-04-28T13:00:00+00:00

When we’re writing a method in Ruby and receiving objects as arguments, a helpful principle to follow is “don’t mutate what you don’t own.”

Why is this? Those arguments come from places that we as the method authors can’t know, and a well-behaved method shouldn’t alter the external environment unexpectedly.

Consider the following method, which takes an array of numbers and appends a new, incremented number:

def append_number(arr)
  last_number = arr.last || 0
  arr << last_num + 1
end

If we pass in an array, we’ll get a new number appended in the returned array:

my_arr = [1, 2]
my_new_arr = append_number(my_arr) # => [1, 2, 3]

But we’ll also quickly discover that this has been achieved by mutating our original array:

my_arr = [1, 2]
my_new_arr = append_number(arr) # => [1, 2, 3]
my_arr # => [1, 2, 3]

We can confirm by an object equality check that this is still the one same array:

my_new_arr.eql?(my_arr) # => true

This behavior is courtesy of Ruby’s Array#<< method (aka #append or #push), which appends the given object to the receiver (that is, self), before then returning that same self. This kind of self-mutating behaviour is common across both the Array and Hash classes, and while it can provide some conveniences in local use (such as a chain of #<< calls to append multiple items to the same array), it can lead to surprising results when that array or hash comes from anywhere non-local.

Imagine our example above is part of a much larger application. In this case, my_arr will most likely come from somewhere far removed and well outside the purview of append_number or whatever class contains it. As the authors of append_number, we have no idea how that original array might otherwise be used! For this reason, the courteous approach to take is not to mutate the array, but instead create and return a new copy:

def append_number(arr)
  last_number = arr.last || 0

  # There are many ways we can achieve the copy; here's just one
  arr + [last_number]
end

This way, the caller of our method can trust their original values to go unchanged, which is what they would likely expect, especially if our method doesn’t give any hint that it will mutate.

my_arr = [1, 2]
my_new_arr = append_number(arr) # => [1, 2, 3]
my_arr # => [1, 2]

This is a very simple example, but the same princple applies for all kinds of mutable objects passed to your methods.

A more telling story here comes from earlier days of Ruby, around how we handled options hashes passed to methods. We used to do things like this:

def my_method(options = {})
  some_opt = options.delete(:some_opt)
  # Do something with some_opt...
end

my_method(some_opt: "some value")

Using trailing options hashes like this was how we provided “keyword arguments” before Ruby had them as a language feature. Now the trouble with the method above is that we’re mutating that options hash by deleting the :some_opt key. So if the user of our method had code like this:

common_options = {some_opt: "some value"}

first_result = my_method(common_options)
second_result = my_method(common_options)

We’d find ourselves in trouble by the time we call my_method the second time, because at that point the common_options hash will no longer have some_opt:, since the first invocation of my_method deleted it — oops!

This is a great illustration of why modern Ruby’s keyword arguments work the way they do. When we accept a splatted keyword hash argument like **options, Ruby ensures it comes into the method as a new hash, which means that operations like options.delete(:some_opt) do in fact become local in scope, and therefore safe to use.

So now that we’ve covered both arrays and hashes now as Ruby’s most common “container” objects, what about the other kinds of application-specific structures that we might encounter in real world codebases? Objects representing domain models of various kinds, an instance of an ActiveRecord::Base subclass, even? Even in those cases, this principle still holds true. Our code is easier to understand and test when we can reduce the number of dimenstions to its behaviour, and mutating passed-in objects is a big factor in this, especially if you think about methods calling other methods and so on. There are ways we can design our applications to make this a natural approach to take, even for rich domain objects, but that is a topic for another day!

Until then, hopefully this walkthrough here can serve as a reminder to keep our methods courteous, to respect mutable values provided from outside, and wherever possible, leave them undisturbed and unmutated. Salubrious!

Bonus content! In preparing this post, I thought about whether it might be helpful to note that Ruby is a “pass by reference” language, since that’s the key underlying behavior that can result in these accidental mutations. However, my intuition here was actually back to front! Thanks to this wonderful stackoverflow answer, I was reminded that Ruby is in fact a “pass by value” language, but that all values are references.

Open source status update, March 2022

2022-04-10T12:30:00+00:00

My OSS work in March was a bit of a grind, but I made progress nonetheless. I worked mostly on relocating and refactoring the Hanami action and view integration code.

For some context, it was back in May 2020 that I first write the action/view integration code for Hanami 2.0. Back then, there were a couple of key motivators:

Reduce boilerplate to an absolute minimum, to the extent that simply inheriting from Hanami::View within a slice would give you a view class fully integrated with the Hanami application.
Locate the integration code in the non-core gems themselves (i.e. in the hanami-controller and hanami-view gems, rather than hanami), to help set an example for how alternative implementations may also integrate with the framework.

Since then, we’ve learnt a few things:

As we’ve gone about refining the core framework, we’ve wound up having to synchronize changes from time to time across the hanami, hanami-controller, and hanami-view gems all at once.
Other Hanami contributors have noted that the original integration approach was a little too “magical,” and didn’t allow users any path to opt out of the integration code.

Once I finished my work on the concrete slice classes last month, I decided that now was the time to address these concerns, to bring the action and view class integrations back into the hanami gem, and to take a different approach to activating the integration code.

The work in progress is over in this PR, and thankfully, it’s nearly done!

The impact within Hanami 2 applications will be fairly minimal: the biggest change is that your base action and view classes will now inherit from application variants:

# slices/main/lib/action/base.rb

require "hanami/application/action"

module Main
  module Action
    class Base < Hanami::Application::Action
      # Previously, this inherited from Hanami::Action
    end
  end
end

By using this explicit application superclass for actions and views, we hopefully make it easier for our users to understand and distinguish between the integrated and standalone variants of these classes. This distinct superclass should also provide us a clear place to hang extra API documentation relating to the integrated behavior of actions and views.

More importantly for the overall experience, Hanami::Application::Action and Hanami::Application::View are both now kept within the core hanami gem. While the framework heads into this final stretch of work before 2.0 final, this will allow us to keep together the aspects of the integration that tend to change together, giving us our best chance at providing a tested, reliable, streamlined actions and views experience.

This is a pragmatic move above all else — we’re a team with little time, so the more we can do to give ourselves confidence in this integrated experience working properly, like having all the code and tests together in one place, the quicker we should be able to get to the 2.0 release. Longer term, we’ll want to provide a first-class integration story for third party components, and I believe we can lead the way in how we deliver that via our actions and views, but that’s now firmly a post-2.0 concern in my mind.

In the meantime, I did take this opportunity to rethink and provide some better hooks for classes like Hanami::Application::View to integrate with the rest of the framework, chiefly via a new Hanami::SliceConfigurable module. You can see how it works by checking out the code for Hanami::Application::View itself:

# frozen_string_literal: true

require "hanami/view"
require_relative "../slice_configurable"
require_relative "view/slice_configured_view"

module Hanami
  class Application
    class View < Hanami::View
      extend Hanami::SliceConfigurable

      def self.configure_for_slice(slice)
        extend SliceConfiguredView.new(slice)
      end
    end
  end
end

Any class that extends Hanami::SliceConfigurable will have its own .configure_for_slice(slice) method called whenever it is sublcassed within a module namespace that happens to match the namespace managed by an Hanami slice. Using the slice object passed to that hook, that class can then read any slice- or application-level config to set itself up to integrate with the application.

In the example above, we extend a slice-specific instance of SliceConfiguredView, which will copy across application level view configs, as well configure the view’s part namespaces to match the slice’s namespace. The reason we build a module instance here (this module builder pattern is a whole technique that I’ll gladly go into one day, but it’s a little out of scope for these monthly updates) is so that we don’t have to keep any trace of the slice as state on the class after we’re done using it for configuration, making it so the resulting class is as standalone as possible, and not offering any way for its users to inadvertently couple themselves to the whole slice instance.

Overall, this change is feeling quite settled now. All the code has been moved in and refactored, and all that’s left is a final polishing pass before merge, which I hope I can get done this week! A huge thank you to Sean Collins for his original work in proposing an adjustment to our action integration code. It was Sean’s feedback and exploratory work here that got me off the fence here, and made it so easy to get started with these changes!

That’s it for me for now. See you all again next month, hopefully with some more continued core framework polishing.

Let the shape of the code reflect its flow

2022-03-24T11:50:00+00:00

Say you’re building a system for handling messages off some kind of queue. For each message, you need to run a series of steps: first to decode the message, next to wrap it in some common structure, and finally, to process the message based some logic provided by the users of your system.

Let’s imagine the queue subscription as provided: we’ll have a subscriber object that yields a message to us via a #handle method:

subscriber.handle do |message|
  # Here's where we need to hook our logic
end

For each of processing steps, let’s also imagine we have corresponding private methods in our class:

#decode(message)
#build_event(decoded_message) — with an “event” being that common structure I mentioned above
#process(event)

With these set up, we could wire them all together in our handler block like so:

subscriber.handle do |message|
  process(build_event(decode(message)))
end

This is hard to grok, however. There’s a lot going on in that one line, and most critically, you have to read it inside out in order to understand its flow: start with decode, then work backwards to build_event and then process.

Instead, we should strive to let the shape of our code reflect its flow. We want to make it easy for the reader of the code to quickly understand the flow of logic even with just a glance.

One step in this direction could be to use intermediate variables to hold the results of each step:

subscriber.handle do |message|
  decoded_message = decode(message)
  event = build_event(decoded_message)
  process(event)
end

This isn’t bad, but the variable names at the beginning of the line add extra noise, and they push back the most meaningful part of each step — the private method names — into a less prominent location.

What I would recommend here is that we take advantave of Ruby’s Object#then to turn this into something that actually looks like a pipeline, since that’s the flow that we’re actually creating via these methods: the steps run in sequence, and the output of one step feeds into the next.

subscriber.handle do |message|
  message
    .then { |message| decode(message) }
    .then { |decoded| build_event(decoded) }
    .then { |event| process(event) }
end

This makes it much clearer that this is a pipeline of three distinct steps, with message as its starting point. Through the shape of those blocks, and the pipe separators distinguishing the block argument from the block body, it also brings greater prominence to the name of the method that we’re calling for each step.

Most importantly, we’ve made this code much more scannable. We’re giving the eye of the reader hooks to latch onto, via the repeated “thens” stacked on top of each other, in addition to their corresponding blocks. The shape of the code embodies its flow, and in doing so, we’ve created a table-of-contents-like structure that both summarises the behaviour, and can serve as a jumping off point for further exploration if required.

To further reduce noise here, we could try Ruby’s new numbered implicit block arguments:

subscriber.handle do |message|
  message
    .then { decode(_1) }
    .then { build_event(_1) }
    .then { process(_1) }
end

However, I’d consider this a step too far, since it takes away what is otherwise a helpful signal, with the block argument name previously serving as a hint to the type of value that we’re dealing with at each point in the pipeline.

By taking the time to consider the flow of our logic, and finding a way for the shape of code to embody that flow, we’ve made our code easier to understand, easier to maintain, and — why not say it? — truer to itself. This is a method I’d walk away from feeling very satisfied having written. Salubrious!

Salubrious Ruby

2022-03-24T09:40:00+00:00

I’ve been writing Ruby for over 21 years now — over half my life! — and in this time, I’ve learnt so much about both the language as well as software development in general. You might have noticed some of the products of that learning in my open source contributions, chiefly the dry-rb and Hanami projects.

I honestly feel blessed to have found a language that could accompany me on a learning journey over so many years.

For much of my time with Ruby, I worked with a tight-knit group of people: the developers at Icelab, and my OSS collaborators. Over the last few years, however, I’ve had the good fortune of working with a much larger group of Rubyists at Culture Amp, and have had many opportunities to share what goes into making a good Ruby application.

There’s little I enjoy more than talking quality code, but lately I’ve been so focused on shipping said code, that I haven’t taken much time to step back and acknowledge what I’ve learnt along the way.

With this announcement, I’m hoping to change this, and create a little accountability for myself along the way! So with no further ado, let me announce a new series here on this little blog, which I’m calling Salubrious Ruby.

I hope you’ll join me while I share the things both big and small that go into making a healthy, wholesome Ruby application!

As I go, I’ll keep pointers to all the articles below:

Open source status update, 🇺🇦 February 2022

2022-03-19T12:55:00+00:00

After the huge month of January, February was naturally a little quieter, but I did help get a couple of nice things in place.

Stand with Ukraine 💙💛

This is my first monthly OSS update since Russia began its brutal, senseless war on Ukraine. Though I was able to ship some work this month, there are millions of people whose lives and homeland have been torn to pieces. For a perspective from one of our Ruby friends in Ukraine, read this piece and this update from Victor Shepelev, aka zverok.

Let’s all continue to support Ukraine, and help the international community continue doing the same.

Concrete slice classes

For this month I focused mostly on getting concrete slice classes in place for Hanami applications. As I described in the alpha7 release announcement, concrete slice classes give you a nice place for any slice-specific configuration.

They live in config/slices/, and look like this:

# config/slices/main.rb:

module Main
  class Slice < Hanami::Slice
    # Slice config goes here...
  end
end

As of this moment, you can use the slice classes to configure your slice imports:

# config/slices/main.rb:

module Main
  class Slice < Hanami::Slice
    # Import all exported components from "search" slice
    import from: :search
  end
end

As well as particular components to export:

# config/slices/search.rb:

module Search
  class Slice < Hanami::Slice
    # Export the "index_entity" component only
    export ["index_entity"]
  end
end

Later on, I’ll look to expand this config and find a way for a subset of application-level settings to be configured on individual slices, too. I imagine that configuring source_dirs on a per-slice basis may be useful, for example, if you want particular source dirs to be used on one slice, but not others.

The other thing you can currently do on these slice classes is configure their container instance:

# config/slices/search.rb:

module Search
  class Slice < Hanami::Slice
    prepare_container do |container|
      # `container` (a Dry::System::Container subclass) is available here with
      # slice-specific configuration already applied
    end
  end
end

This is an advanced feature and not something we expect typical Hanami users to need. However, I wanted this in place so I could continue providing “escape valves” across the framework, to allow Hanami users to reach below the framework layer and manually tweak the lower-level parts without having to eject entirely from the framework and all the other niceties it provides.

Slice registration refactors

As part of implementing the concrete slice classes, I was able to make some quite nice refactors around the way we handle slices within the Hanami application:

All responsibility for slice loading and registration has now away from Application (which is already doing a lot of other work!) into a new SliceRegistrar.
The .prepare methods inside both Application and Slice are now roughly identical in structure, with their many constituent setup steps extracted into their own well-named methods (for example). This will make this phase of the boot process much easier to understand and maintain, and I also think it hints at a future in which we have an extensible boot process, wherein other gems may register their own steps as part of the overall sequence that is run when you .prepare and application or slice.

The single-file-app dream lives on

One nice outcome of the concrete slice work is the fact that these classes are not actually required in your Hanami application for it to boot and do its job. It will still look for directories under slices/ and dynamically create those classes if they don’t already exist in config/slices/. What’s even better, however, is that I made this behaviour more easily user-invokable via a public Application.register_slice method. This means you can choose to explicitly register a slice, in cases where the framework may not otherwise detect it:

module MyApp
  class Application < Hanami::Application
    # That's all! This will define a `Main::Slice` class for you.
    register_slice :main
  end
end

But that’s not all! Since these slice classes will now be the place for slice-specific configuration, we may need to provide this when explicitly registering a slice too. For this, you can provide a block that is then evaluated within the context of the generated slice class:

module MyApp
  # Defines `Main::Slice` class and instance_evals the given block
  class Application < Hanami::Application
    register_slice(:main) do
      import from: :search
    end
  end
end

And lastly, you can also provide your own concrete slice class at this point, too:

module MyApp
  class Application < Hanami::Application
  end
end

module Main
  class Slice < Hanami::Slice
  end
end

MyApp::Application.register_slice :main, Main::Slice

One of the guiding forces behind this level of flexibility (apart from it just feeling like the Right Thing To Do) is that I want to keep open the option for single-file Hanami applications. While the framework will always be designed primarily for fully-fledged applications, with their components spread across many source files, sometimes a single file app is still the right tool for the job, and I want Hanami to work here too. As I put the final polish on the core application and slice structures over the coming couple of months, I’ll be keeping this firmly in mind, and will look to share a nice example of this in a future blog post :)

Removed some unusued (and unlikely to be used) flexibility

While I like to try and keep the Hanami framework flexible — and we’ve already looked at several approaches to this just above — I’m also conscious of the cost of this flexibility, and how in certain cases, those costs are just not worth it. One example of this was the removal of the configurable key separator in dry-system earlier this year. In this case, keeping the key separator configurable meant not only significant internal complexity, but also the fact that we could never write documentation that we could be fully confident would work for all people. To boot, we hadn’t heard of a single user wanting to change that separating over the whole of dry-system’s existence.

As part of my work this month, I removed a couple of similar settings from Hanami:

I removed the config.slices_namespace setting, which existed in theory to allow slices to also live inside the application’s own module namespace if a user desired (e.g. MyApp::Main instead of just ::Main). In reality, I think that extra level of nesting will be too invoncenient for users to want. More importantly, I think that having our slices always mapping to single top-level modules will be important for our documentation (and generators, and many other things I’m sure) to be clearer.
I also remove the config.slices_dir setting, for much the same reasons. Hanami will be far easier to document and support if slices are always loaded from slices/ and nowhere else.

Made `Application.shutdown` complete

Did you know that you can both boot and shut down an Hanami application? The latter will call stop on any registered providers, which can be useful if you need to actively disconnect from any external resources, such as database connections.

You can shutdown an Hanami application via Application.shutdown, but the implementation was only partially complete. As of this PR, shutdown now works for both slices (and their providers) and when shutting down an application, it will shutdown all the slices in turn.

Simplified configuration by permitting `env` to be provided just once

Another little one: the application configuration depends on knowing the current Hanami env (i.e. Hanami.env) in several ways, such as knowing when to set env-specific defaults, or apply user-provided env-specific config. Until now, it’s been theoretically possible to re-set the env even after the configuration has loaded, which makes the env-specific behaviour much harder to reason about. With this change, the env is now set just once (based on the HANAMI_ENV env var) when the configuration is initialized, allowing us to much more confidently address the env across all facets of the configuration behavior.

(This and the shutdown work together in a single evening session. For many reasons, I was feeling down, and this was a nice little bit of therapy for me. So much of what I’ve been doing here lately spans multiple days and weeks, and having a task I could complete in an hour was a refreshing change.)

Worked to define a consistent action and view class structure

This effort was mostly driven by Luca, but we worked together to arrive at a consistent structure for the action and view classes to be generated in Hanami applications.

For actions, for example, the following classes will be generated:

A single application-level base class, e.g. MyApp::Action::Base in lib/my_app/action/base.rb. This is where you would put any logic or configuration that should apply to every action across all slices within the application.
A base class for each slice, e.g. Main::Action::Base in slices/main/lib/action/base.rb, inheriting from the application-level base class. This is where you would put anything that should apply to all the actions only in the particular slice.
Every individual action class would then go into the actions/ directory within the slice, e.g. Main::Actions::Articles::Index in slices/main/actions/articles/index.rb.

For views, the structure is much the same, with MyApp::View::Base and Main::View::Base classes located within an identical structure.

The rationale for this structure is that it provides a clear place for any code to live that serves as supporting “infrastructure” for your application’s actions and views: it can go right alongside those Base classes, in their own directories, clearly separated from the rest of your concrete actions and views.

This isn’t an imagined requirement: in a standard Hanami 2 application, we’ll already be generating additional classes for the view layer, such as a view context class (e.g. Main::View::Context) and a base view part class (e.g. Main::View::Part).

This structure is intended to serve as a hint that your own application-level action and view behavior can and should be composed of their own single-responsibility classes as much as possible. This is one of the many ways in which Hanami as a framework can help our users make better design choices, and build this up as a muscle that they can apply to all facets of their application.

Released alpha7

Last but not least, I cut the release of Hanami 2.0.0.alpha7 and shared it with the world.

What’s next?

My next focus has been on a mostly internal refactor to move a bunch of framework integration code from hanami-controller and hanami-view back into the hanami gem itself, since a lot of that is interdependent and important to maintain in sync in order to provide a cohesive, integrated experience for people building full stack Hanami applications. This should hopefully be ready by the next alpha, and will then free me up to move back onto application/slice polish.

Open source status update, December 2021 and January 2022

2022-02-13T13:20:00+00:00

I’ve been on an absolute tear with my OSS work lately. I’ve been feeling such good momentum that every time I got a moment at the computer, I just had to keep pushing forward with code rather than writing one of these updates. Now that there’s been a couple of Hanami releases and a big dry-system one since the my last update, it’s time for a roundup.

The best thing about the last two months was that I took the first two week in January off from work, and dedicated myself 9-5 to OSS while everyone was still on holidays (the reality was probably more like 10am-4pm then 9pm-12am, but I was having too much fun to stop). This allowed me to get through some really chunky efforts that would’ve been really hard to string along across post-work nighttime efforts alone.

All of this has meant I’ve got a lot to get through here, so I’m going to revert to dot list form with at most a comment or two, otherwise I’ll never get this post done (and you probably won’t want to read it all anyway). Here goes:

Back in December I started the new source_dirs configuration for Hanami, along with some corresponding broadening of the dry-system component dir config API
I also updated the Hanami router to initialize itself lazily, which will allow for for the Hanami applications to scale down better to Lambda/FaaS-size deployments, since you can now keep the application non-booted and still access the router for e.g. a subset of your routes. It’s also an important step towards faster application reloading in development.
In January (happy new year!) I took care of the Hanami 2.0.0.alpha5 release in January (featuring the source_dirs config, the lazy router, and more).
I then moved onto dry-system for the rest of the month, kicking things off by finishing a comprehensive glossary for dry-system, intended to go into our docs as a quick reference to alo important terms, but I also used it as a trial run for some naming changes that I’ve been wanting to make for a while.
That was received well so I went ahead and renamed “bootable components” to “providers” (and the “init” provider lifecycle step to “prepare”), as well as the renaming the “component provider” concept to “external provider sources.” This change has made so many parts of dry-system easier to explain more precisely, since we avoid a range of confusing overlaps. It feels really good.
With the naming change done, I overhauled the provider implementation as well, making it so that providers were backed by distinct classes and corresponding instances, and also clarifying just how providers can access their target container. This one took my four different tries to get right (and ate at my mind over a whole weekend), but I think it turned out really well. The new class-backed providers open up a lot of possibilities for better handling complex provider code, while still preserving ease of use for simpler providers.
In more provider tidy up, I removed the use method from the provider API, since the target container can now be consistently accessed, and can be used to start other provider’s via its own standard .prepare and .start methods.
I updated our built-in “settings” provider source to use Dotenv rather than maintain our own .env* file loading and parsing code. Goodbye extra code, I will not miss you.
I scratched one of my biggest itches and removed an overload of dry-configurable’s setting method inside dry-system. This one proved to be a bit of a challenge to sort out, since it required adjustments to class-level config access inside dry-configurable, and dry-configurable’s test suite is so exacting it often doesn’t leave much wriggle room. In this case I was able to thread the needle and even bring some clarity to previous unspecified behaviours.
Another little thing, I removed support for a configurable key namespace separator in dry-system, which really never made sense given how much dry-system needs to manipulate these keys, and given how important it is for our docs to make sense with "." used as the key separator everywhere.
I expended dry-system’s API for its container configuration, formalising container configuration as a distinct phase that can be completed with its own configured! method. This allows dry-system’s config object to be configured directly rather than forcing everyone to go through the configure do |config| block-based API, which was a quirk specific to dry-system, and not in any other usage of dry-configurable throughout our ecosystem. This not only makes things easier for our direct users, but will make Hanami’s integration of dry-system more straightforward, since it’s (fairly lengthy) config and setup code no longer needs to descend into that extra block.
In another glossary-driven change, I renamed the “manual registrar” to “manifest registrar”, since it loads registrations from manifest files, also freeing the term “manual registration“ for any individual act of calling .register on the container.
This one is big! I added support for partial container exports and imports, which is only something I’ve been thinking about for three years or more. And it turned out to be really straightforward! This is going to be a huge part of dry-system usage going forward, and really reinforces how it can help you better organise your code both in the small and in the large. With dry-system containers able to hide most of their implementation away and just export a few “public” components, they’ll begin to work much better as representations of key high-level concerns withing your applications.
I tidied dry-system’s dependency graph plugin, and ensured it supported all injector strategies.
I fixed a 2-year-old bug and made it so that registrations in providers preserve all their options when moving into the target container.
I fixed a 2.5-year-old bug and ensured that providers can’t inadvertently re-start themselves while already in the process of starting and cause an infinite loop
I restored support for a configurable instance proc on component dir config.
I added an option to providers to support conditional loading, making it so that you can have a provider enabled only in particular circumstances, and have it not be loaded (saving time and memory, and improving clarity) when it’s not needed.
I added a couple more useful predicates to Dry::System::Identifier
And finally, I got Ian Ker-Seymer’s work on a dry-system Zeitwerk plugin across the line, making it so that dry-system users can use an integrated Zeitwerk setup with just a single line.

Last week I released all of that dry-system work in version 0.23.0, officially our biggest release ever (go read the release notes!). We’re now looking really close to a 1.0 release for dry-system, with just a few things left to go.

I then updated Hanami to use this latest dry-system (including all the updated terminology) and support partial slice imports and exports, which we then released as Hanami 2.0.0.alpha6 just a few days ago.

Getting to this point was a lot of work, and it represents a big milestone in our Hanami 2.0 journey. I’m extremely grateful that I could make this my sole focus for a little while.

Thank you to my sponsors ❤️

My work in Ruby OSS is kindly supported by my GitHub sponsors.

Thank you in particular to Jason Charnes and now also Seb Wilgosz (of HanamiMastery) for your support as my upper tier sponsors!

The 22 dot points in this post show that Hanami 2 is truly getting closer, but there are many dots left to go. I’d love for your support too in helping make this happen.

Open source status update, November 2021

2021-12-13T12:35:00+00:00

One of the things I find hardest about writing these open source status updates is my opening sentence. So I’m just going to leave this one in here and get on with journalling another month of steady progress!

We shipped the first of our monthly Hanami alphas!

After our last alpha release and our promise to ship monthly, we delivered the next alpha right on time, with Hanami 2.0.0.alpha4 going out the door on the 7th of December.

The release was a very orderly process this time around, especially compared to my late night marathons of the last two. Luca was our release manager (we’re alternating duties), and a few days before the release, we already had the gems lined up and release announcement prepared. I’ve a feeling this will make for a really helpful cadence for us. Anyway, go check out the announcement post for the changes, a few of which I’ll cover in detail below.

Access to routes helpers inside all actions

Way back in July, Marc Busqué prepared a class to provide access to helper methods returning URLs for all of the named routes in an application. This languished for a while, but we got it merged in preparation for the alpha3 release. This made the routes helpers accessible via a component registered with the "routes" container key. This is all fine if you want to inject this object explicitly, but we want to make it easier still in the common usage contexts, like action and view classes.

For alpha4 we’ve made it automatically available in both those places!

Making it available in the actions was a fairly straightforward extension of the existing set of dependencies we automatically provide. There was a little wrinkle here, however: the “resolver” that we have the Hanami application provide to the router (for resolving action objects from string container keys) was resolving those objects eagerly, i.e. directly fetching them from the container for every declared route. This led to an infinite loop, since the first action would auto-inject the router helper, which in turn required the router, which then tried to load each again again, and so on. After a small change to make the router resolver lazy, we were back in business and the routes helper was available. The lazy router resolver is something we would have wanted to build before 2.0.0 final anyway (a big principle behind 2.0.0 is to make it easy to load only subsets of your application), so I’m glad we had this as a prompt to do it now.

The end result is that you can now have routes like this:

module TestApp
  class Routes < Hanami::Application::Routes
    define do
      slice :main, at: "/" do
        get "test", to: "test"
        get "examples", to: "examples.index", as: :examples
      end
    end
  end
end

And then access your named routes inside an action with no extra work:

module Main
  module Actions
    class Test < Hanami::Action
      def handle(req, res)
        res.redirect routes.path(:examples)
      end
    end
  end
end

Thanks to Marc’s ongoing work in this space, we’ll have the same arrangement set up for views by this time next month, too.

Improvements to the Hanami console

I’ve been preparing another Hanami app at work lately, and noticed we’d experienced some regressions in the console experience compared to the arrangement we had in place back for our original Hanami 2 application template. Notably, slice helpers weren’t available, nor was there a shortcut to access the application class. I fixed these here, and now this wonderful experience is available again:

$ ./bin/hanami console

test_app[development]> app
=> RecognitionService::Application

test_app[development]> main
=> #<Hanami::Slice:0x00007fdc97ab47a0
 @application=RecognitionService::Application,
 @container=Main::Container,
 @name=:main,
 @namespace=Main,
 @namespace_path="main",
 @root=#<Pathname:/Users/tim/Source/cultureamp/recognition-service/slices/main>>

Having such quick and easy access to the application and its slices is a big part of what makes the Hanami console experience so pleasurable! And this is in addition to its constant start time, no matter how large your app (remember my note above about having the framework support loading subsets of the application?), because it only loads the most minimal subset of the app, and then brings in the rest of your components lazily, only as you require them.

While I was in CLI land, I also decided to finally figure out why our IRB-based console wasn’t offering a custom prompt, and managed to get that one fixed too. IRB shows its age in this regard: Pry offers a much nicer setup and customisation experience.

Preparing for flexible source dir configuration

In the remainder of my OSS time this month, I was slowly moving towards the next evolution of the configurable Hanami component dirs I shipped last month. The plan here is to make it support “source dir” configuration in general, not just for component dirs. This should allow a user to configure component dirs when they want source dirs to be auto-registered (along with full access to the dry-system component dir configuration), as well as ordinary source dirs, when they want a directory to be autoloaded by Zeitwerk, but not populate the container.

The beginning of my work on the Hanami side is available here, and the approach I’m taking is to leverage dry-system’s own Config::ComponentDirs class for most of the component dir heavy lifting. This means we’ll need to add a range of new behaviours to that class, which is in progress here.

I’m hoping to have this one ready by the end of the year. More on this in next month’s update.

Open source status update, October 2021

2021-11-15T00:30:00+00:00

It’s one of my favourite kinds of monthly updates, folks, because I have another Hanami alpha release to share! Let’s dive straight into it.

We shipped dry-system 0.21.0!

Remember all that good stuff I described about dry-system’s component dir namespaces last month? It’s now properly released! You can upgrade to dry-system 0.21.0 to give it a try.

You can also check out the new component dirs page of our user documentation that I created for this release, which outlines every configurable aspect of dry-system’s component dirs. There’s a lot more we can do to improve dry-system’s documentation, but this is at least a start.

We shipped Hanami 2.0.0.alpha3!

And here’s the big one! After 6 months, we shipped another Hanami 2.0 alpha release! Go read the announcement post to start with – I’ll wait! – then we can look a bit more behind the scenes.

So what was this actually like for me? Shipping this alpha turned out almost exactly like the last one: a few nights straight trying to get everything ready, followed by another in which I decide we’re shipping come hell or high water. And exactly like last time, it ended with me in a daze at 1:30am having just finished the blog post, waiting for the Hanami website static generation to to complete so I could finally hit the sack 😅

A notable milestone for me was that this was the first time I pushed out the releases of all the individual Hanami gems. I’m looking forward to this becoming a regular thing in the future.

Streamlined slice directories

Given the months I spent getting dry-system ready for week, of course this is favourite aspect of this release. We now support this as a standard source directory structure for each of your application’s slices:

└── slices
    └── main
        ├── actions/
        ├── lib/
        ├── repositories/
        └── views/

lib/ is intended to hold your slice’s core business logic. Inside lib/, every file is expected to define a class inside your slice’s Ruby namespace, in this case Main. So lib/my_class.rb would define Main::MyClass, and would be registered in the slice as "my_class".

Previously, this had to be lib/main/my_class.rb, and removing that one redundant main/ from the path was the driver behind our months of work in this space. I’m thankful now there’s one less possible papercut for our users, but I’m also thankful we did the work right, because it opened up the possibility for those other source directories in the slice!

actions/, views/, and repositories/ are these new entrants here. The intention for these directories is to hold special categories of classes for your slice. In this case actions/ and views/ are key entrypoints, in that their job is to provide an external interface, then and mostly coordinate with other objects from the slice to invoke the necessary business logic. repositories/ are intended to be the key interface between your slice and its persistence layer, and in many cases may provide cross-cutting behaviour within the slice.

For each of these non-lib/ directories, there are some different rules: they’re expected to hold files defining classes in a matching namespace. So in this case, actions/posts/show.rb would define Main::Actions::Posts::Show, and would be registered under a matching namespace, at "actions.posts.show".

You can add your own additional source directories too, too. To start with, we’ve made them configurable via config.component_dir_paths inside your application class, which defaults to ["actions", "repositories", "views"]. I plan to make this a much richer configuration object in the coming months, allowing you to add directories for autoloading only (i.e. not for component auto-registration), as well as manually configure the namespace rules for those dirs that are auto-registering components.

New loading rules for the top-level lib/

Along with the changes to the source directories in the slices, we made a big change to how we handle the top-level lib/. Previously, we would auto-register components from lib/[application_namespace]/. These would end up in the application container, and then in turn be imported into every slice container.

For example, given a lib/my_app/my_shared_class.rb, the application container would have a "my_shared_class" component, and every slice would then have an imported "application.my_shared_class" component. This general approach is one I’ve worked with for a while now, including in the in-house Icelab framework that preceded Hanami 2 development, and it’s worked reasonably well. The application container has been a good place to put components that are common to other slices. For a moment in the lead up to this release, I even toyed with further enshrining this approach, and loading such components from a special app/ directory. However, this led to some great feedback and a follow-up discussion with Luca, in which he noted that directories like these are ripe for unintended misuse, in that they create undesirable coupling between the application and all of its slices (not to mention the possible confusion with the Rails-like app/ directory naming).

So to address this concern and encourage a healthier separation of concerns within Hanami 2 apps, we’ve changed the code loading rules for the top-level lib/:

lib/[application_namespace]/ will still be autoloaded, so you won’t need to manage any manual require statements to bring in your own classes
However, lib/[application_namespace] will no longer be a component directory, so files here will no longer auto-register in the application container

With these changes, you can still use lib/[application_namespace] to keep your own shared base classes and other manually invokable code, but if you would like to share components with your slices, we’re encouraging you to create your own dedicated slices to house them. This act of slice creation and naming will hopefully encourage you to take the opportunity to draw more appropriate boundaries between these extra slices, giving them better cohesion and allowing other slices to import only the parts they need.

For example, if you previously had clusters of components in your app container related to (a) password encryption as well as (b) exchange rate calcuation, both as common concerns, then the approach to take here would be to extract them into well-bounded slices of their own, at slices/password_encryption and slices/exchange_rates, and then import those from only the other slices that need them, e.g.

module MyApp
  class Application < Hanami::Application
    config.slice :admin do
      # Importing a common "password_encryption" slice into the admin slice; all components from
      # the imported slice will be accessible with keys prefixed by "password_encryption."
      import :password_encryption
    end
  end
end

This is a much more intentional arrangement of concerns, and should result in a better factored and more maintainable modular application.

Of course, if you really do need to register components in the application container, you can still do this via a file in config/boot/, e.g.

Hanami.application.register_bootable :my_component do |container|
  start do
    require "some_global_component"

    register "my_component", SomeGlobalComponent.new
  end
end

Hopefully this provides just the right amount of friction (compared to the ease of creating a new slice with its own auto-registered classes) that this path is only taken for the very small number of components that truly need to be global.

Lastly, after all of these changes, one thing we haven’t messed with is the fact that the top-level lib/ directory is still added to Ruby’s standard $LOAD_PATH, which means you have a place to put any files outside of the application namespace (e.g. if you’re incubating future-gems inside your application) and still require them explicitly.

All in on Zeitwerk

One thing is common to all the changes above: they all depend on the Zeitwerk autoloader. Our experience with Zeitwerk has been fantastic: it’s well documented, has been configurable wherever we need it, and then utterly predictable once we’ve set it up.

These changes have further embedded the importance of Zeitwerk for Hanami 2: it’s become well and truly a “load bearing” part of the framework. So we’ve decided to go all in on Zeitwerk for Hanami 2.

Initially, the autoloader was configurable, with it being possible to opt out and fall back to managing your own standard require statements everywhere. As of alpha3, however, the autoloader is now always on, an expected part of the framework. As a former “require what you require” kind of Rubyist, now that I’ve had some experience with Zeitwerk, there’s no way I’m going back. I expect every Hanami 2 user to feel the same, and with Zeitwerk in place, we give ourselves the best chances for further streamlining our users’ experience in the future.

Thank you Xavier Noria for your sterling work in this space 🙏🏼

Coming up: monthly alphas!

Getting to the point of releasing alpha3 was a lot of work, and hopefully the last of our big foundational overhauls. Now we’re on the other side of this, we want to really build our momentum towards a 2.0.0 final release. So from here we’ll be releasing monthly alphas, collecting up any of the work that’s happened over the previous month. We’ve already set a date for the next one: look out for something on or around December 7th!

I’m really excited about this next phase. If you’ve followed along with my previous OSS updates, you would’ve picked up how much of a drag it has been for working on months on end with no reward or punctuation of any kind. I hope the monthly cadence will help us keep our changes short and sharp, and things overall moving at a much faster clip.

Thank you to my sponsors ❤️

My work in Ruby OSS is kindly supported by my GitHub sponsors.

Thank you in particular to Jason Charnes for your continued support as my sole level 3 sponsor. Jason, you’re an absolute mensch.

If you want to help make Hanami 2 a reality, I’d love it if you could join my sponsors too.

Open source status update, September 2021

2021-10-11T00:45:00+00:00

After the last few months of seemingly slow progress (and some corresponding malaise), September was a real watershed for my OSS work! It featured seven gem releases and one giant PR being made ready for merge. Let’s take a look.

dry-configurable 0.13.0 is out, at last!

I started the month resolved to finish the work on dry-configurable 0.13.0 that I detailed in the last episode.

I started my updating and merging any of the pending changes across dry-configurable’s dependent dry-rb gems: dry-container, dry-monitor, dry-schema, and dry-validation. Fortunately, thanks to the extra compatibility work I’d done in dry-configurable, all of these changes were straightforward.

By this time, there was nothing left to do but release! So on the evening of the 12th of September, I decided I get this done. At 9:55pm, I shipped dry-configurable 0.13.0, at long last!

And then I shipped the following:

dry-container 0.9.0
dry-system 0.20.0
dry-monitor 0.50
dry-schema 1.8.0
dry-validation 1.7.0
And for good measure, even though it turned out not to need dry-configurable compatibility changes, dry-effects 0.2.0

By 11:30pm, this was all done and I happily sent out the announcement tweet.

In the time since, we haven’t seen or heard of any issues with the changes, so I think I can consider this change a success!

Despite it taking as long as it did, I’m glad we made this change to move dry-configurable to a clearer API. A lesson I’m taking away from this is to think again before mixing optional positional parameters with keyword args splats in Ruby methods; though this is largely a non-issue for Ruby 3.0, moving away from this while retaining backwards compatibility did cause some grief for Ruby 2.7, and on top of that, I value the extra clarity that keyword arguments bring for anything but an immediately-obvious and required singular positional argument.

Testing compact slice lib paths in Hanami

With the dry-configurable taken care of, it was time to do the same for the saga of dry-system namespaces.

Before I committed to polishing off the implementation in dry-system, I wanted to double check that it’d do the job we needed in Hanami: to elide the redundant lib directory in e.g. slices/main/lib/main/, turning it into slices/main/lib/ only, with all the classes defined therein still retaining their Main Ruby constant namespace. As hoped, it did exactly that! As I shared with the other Hanami contributors:

Like all of my favourite PRs, it was 3 months of dry-system work, followed by a 3-line change in Hanami 😄

A fun quip, but I think this is an important aspect of the work we’re doing in Hanami 2. We’re not putting in this amount of effort just to arrive at a toughly coupled framework that can deliver only for a small subset of users. Rather, we’re trying to establish powerful, flexible, well-factored building blocks that deliver not only the default Hanami experience, but also serve as useful tools unto themselves. The idea with this approach is that it should allow an Hanami user to “eject” themselves from any particular aspect of the framework’s defaults whenever their needs require it: they can dive deeper and use/configure constituent parts directly, while still using the rest of the framework for the value it provides.

dry-system component dir namespaces are done, at last!

Confident about the use case in Hanami, I used the rest of the month (and a little bit of October, shh!) to finish off the dry-system namespaces. Here’s how they look:

class AdminContainer < Dry::System::Container
  configure do |config|
    config.root = __dir__

    config.component_dirs.add "lib" do |dir|
      dir.namespaces.add "admin", key: nil
    end
  end
end

This example configures a single namespace for lib/admin/ that ensures the components have top-level identifiers (e.g. "foo.bar" rather than "admin.foo.bar").

Namespaces take care of more than just container keys. If you wanted to mimic what we’re doing in Hanami and expect all classes in lib/ to use the Admin const namespace, you could do the following:

config.component_dirs.add "lib" do |dir|
  dir.namespaces.root const: "admin"
end

You’re not limited to just a single namespace, either:

config.component_dirs.add "lib" do |dir|
  dir.namespaces.add "admin/system_adapters", key: nil, const: nil
  dir.namespaces.add "admin", key: nil
  dir.namespaces.add "elsewhere", key: "stuff.and.things"
end

If you want to learn more, go read the overview in the PR: there’s around 3,000 words explaining the history, rationale, and full details of feature, as well as a whole bunch of implementation notes.

Getting this ready to merge took around three weeks of really concerted work (almost every night!), but I’m super glad to finally have it done. Component dir namespaces represent another huge leap for dry-system. With namespaces, dry-system can load and manage code in almost any conceivable structure. With this change giving us support for “friendlier” source directory structures like the one we’ll use in Hanami, I hope it means that dry-system will also become as approachable as it already is powerful.

Figured out a strategy for upcoming Hanami alpha releases

A final highlight of the month was getting together for another in-person chat with Luca! (Can you believe we’ve only done this four or five times at most?) Among other things, we figured out a strategy for the next and subsequent Hanami 2.0.0.alpha releases.

Here’s the plan:

As soon as the dry-system namespaces are released and the new slice lib paths configured in Hanami, we’ll ship alpha3. A code loading change is a big change and we want to get it into people’s hands for testing ASAP.
Our focus will continue to be on stripping as much boilerplate as possible away from the generated application code, and now that I’m done with all the big efforts from the last month, my contributions here should be a lot more iterative
Going forward, we’ll ship a new alpha release every month, collecting up all the changes from that month

So this means you should here about a new Hanami release by the time I’m writing my next set of notes here!

Thank you to my sponsors ❤️

My work in Ruby OSS is kindly supported by my GitHub sponsors.

Thank you in particular to Jason Charnes for your unwavering support as my sole level 3 sponsor (it was great to chat to you this month as well!). Thanks also to Sebastian Wilgosz of HanamiMastery for upgrading his sponsorship!

If you’d like to support my work in bringing Hanami 2 to life, I’d love for you to join my sponsors too.

See you all next month!

Open source status update, July and August 2021

2021-09-06T13:00:00+00:00

Hello again, open source fans! After two months, we’re overdue for an update! July and August were rather quiet months for me on the OSS front, but I’ve been steadily beavering away at two fairly large things.

Preparing for the dry-configurable 0.13.0 release

Back in May (May!) I shared details of a couple of major changes we wanted to make to dry-configurable’s API before 1.0:

The default value for the setting must be supplied as default: rather than a second positional argument
A setting’s constructor (or ”processor”) can no longer be supplied as a block, instead it should be a proc object passed to constructor:

Those changes have long since merged, but the release didn’t immediately follow, for a couple of reasons:

Since dry-configurable is so widely used across the dry-rb and related ecosystems, we needed to double check that every dependent gem would continue to work undisturbed
In some early checks for this, we found some small incompatibilities that would need to be addressed

Do you ever have a chore on your to-do list that you just feel… ugh about? Well this was one such thing for me, which is why it didn’t go anywhere for a little while.

But an unglamorous task like this (along with an even stronger aversion to too much open WIP) turned out to be just what I needed to ease myself back into programming in August, after a few weeks of struggling to get anything done.

So, I started by drawing up an enormous matrix (seriously, check it, it’s enormous!) of dependent gems to test against both present and future of dry-configurable, and then slowly got to work on it.

To start with, I cracked the nut that was likely to give us as-good-as-possible backwards API compatibility. This was one of those fun Ruby 2.7/3.0 keyword argument incompatibilities. Check out the diff from that PR for probably one of the longer comment blocks I’ve written in a while.

Then, I discovered and fixed a minor issue with dry-system’s delegation of keyword arguments to a dry-configurable method. Released that as dry-system 0.18.2 and 0.19.2. That in turns fixed some issues with a few other gems relying on both dry-configurable and dry-system together.

Next, I made a tiny change to dry-schema to ensure compatibility with another aspect of the upcoming dry-configurable. Released that as dry-schema 1.7.1.

Then lastly, I added some flags to allow users to manually disable the deprecation notices if they’re not yet able to upgrade their app or their dependent gems to use the latest API.

And after all of that, we’re truly just around the corner from shipping this new dry-configurable! Here’s what’s left:

I need to write some good release notes for the changes
Then merge each of the still-open API compatibility PRs for each of the dependent gems (there’s about 4 to go)
Then release dry-configurable 0.13.0
Then release minor version bumps for all of the dependent gems (6 in total)

I’m hoping to get this done in the next ~week or so. It’ll be a relief to have this done!

Continued work on first-class dry-system namespace support

The other large thing I’ve continued to work on is the expanded support for namespaces in dry-system that I detailed in my last update. I turned to this one when I wanted something a bit more “fun” while I was avoiding the dry-configurable work back in July. It’s now at the point where I’m really quite happy with the new abstractions I was able to identify along the way.

Once the dry-configurable work is done, I plan to come back to this one, and before I spend any more time tidying it for release, I want to actually exercise the new feature from Hanami’s side to make sure it can achieve the elided source directory structure that inspired this work in the first place. All things going well, I should be able to show you it in action in next month’s post!

Thank you to my sponsors (new and old!) ❤️

Thank you to Thomas Carr for joining my GitHub sponsors in July!

And of course, a continued thanks to Jason Charnes and the rest of my sponsors for your ongoing support!

If you too would like to support my work, you can sponsor me on GitHub.

See you all next month! With these two things close to done, we might just be cresting a hill.

Open source status update, June 2021

2021-07-11T13:00:00+00:00

June felt like a little bit of a down month for me in OSS. For various reasons, I struggled to find decent blocks of time to make headway into all the loops I opened back in May. But in the end, I found a way to nudge some other important things forward!

Beginning work on first-class namespace support for dry-system

Rather than push against the grain at a low ebb, I used what time I had in June to do some more nourishing ”fun” work, to get back into dry-system again and prepare the way for Hanami to have a flatter Ruby source file directory structure.

What we want is instead of Main::HelloWorld being defined in this file:

slices/main/lib/main/hello_world.rb

We want too remove that second redundant main/ and have the file like so:

slices/main/lib/hello_world.rb

The idea here is to make the deeper hierarchies imposed by each namespaced Hanami slice a little less ”in your face,” and in doing so, make this whole framework just that much more approachable to a wider range of users.

Zeitwerk already supports part of what we’d like to do here, through its support for custom root namespaces. Using this, we’d set up that elided source directory from above like this:

loader = Zeitwerk::Loader.new
loader.push_dir "slices/main/lib", namespace: Main

However, for Hanami to use both Zeitwerk and dry-system together, more work is required. This is also made more challenging by the fact that dry-system needs to map names in two directions, from identifier to source file/constant (when lazy loading a single registration) and from source file to constant/identifier (when finalizing a container and crawling its managed directories for all source files), whereas Zeitwerk only needs to go in one direction, from constant to source file.

Until now, the only support dry-system has had for namespacing was its default_namespace setting on its component dirs. This worked very simplistically. Given a setting like the following:

config.component_dirs.add "lib" do |dir|
  dir.default_namespace = "main"
end

If dry-system encountered a file like main/hello_world.rb inside that component dir, then it would strip off the leading "main" when forming its identifier, leaving it as "hello_world". And when the container is lazy loading, it would first try prepending the default_namespace to find a file for a given identifier, before falling back to searching without the namespace.

This has worked fine for most use cases, but it won’t cut it for our Hanami plans.

So to address this, I’m adding namespaces as a first-class concept to dry-system component dirs. A replacement for the default_namespace config above would look like this:

config.component_dirs.add "lib" do |dir|
  dir.namespaces.add "main"
end

To add a namespace, we specify the path within the component dir, along with an identifier and/or const to use within that path.

The example above is actually shorthand for the following:

config.component_dirs.add "lib" do |dir|
  dir.namespaces.add "main", identifier: nil, const: "main"
end

Playing this out, this means: ”within the main/ subdirectory of this component dir, don’t add anything to the beginning of the component’s identifiers, but expect them all to be within the Main object namespace.”

Following on from this, we should be able to configure some wildly divergent component loading rules. For example:

dir.namespaces.add "main", identifier: "bits_and_bobs", const: "stuff_and_things"

With this config, we’d expect files in main/ to define classes inside the StuffAndThings namespace, and have their identifiers all prefixed by "bits_and_bobs".

As you might’ve noticed, the namespaces.add method means we can configure multiple namespaces:

dir.namespaces.add "main"
dir.namespaces.root # special method for component dir root
dir.namespaces.add "admin"

Namespaces will be referenced in order when resolving components or loading files into the container, too, which improves what was otherwise a poorly defined part of dry-system’s behaviour. So with the configuration above, if you had a main/component.rb and an admin/component.rb, the file inside main/ would be favoured in all cases and would always appear in the container as the "component" registration, because its namespace was added first.

So, after all of this, we now have the necessary tools to support our elided directory structure from the top, which would look something like this:

config.component_dirs.add "lib" do |dir|
  dir.namespaces.root const: "main"
end

I have a messy work in progress PR underway to support all of this, so feel free to check it out if you’d like to see the code equivalent of the under construction gif. This will end up being yet another fairly big overhaul of dry-system internals, so I think it’ll probably take me the better part of July to get it finished. Iterating over the namespaces as part of component loading will also have some performance implications, so I’ll want to do some benchmarking of that process too.

That said, I’m really excited to find another opportunity to make dry-system more powerful and better structured than before! And of course, to then employ that to create a nicer experience for future Hanami users :)

Thank you to my sponsors (new and old!) ❤️

A huge thank you to Bohdan V. for joining my GitHub sponsors during June!

Thanks also to Jason Charnes and the rest of my sponsors for your continued support! If you’d like to join them in supporting my work, you too can sponsor me on GitHub.

See you all next month, folks!

Open source status update, May 2021

2021-06-08T13:55:00+00:00

Well didn’t May go by quickly! Here’s what I got up to in OSS over the month.

For starters, do you remember the big Hanami 2.0.0.alpha2 I mentioned last month? Yep, that happened. And it was a little cheeky of me to sneak it into my March/April update, because it happened into the first week of May!

So after a short while recuperating from that big push, there wasn’t a whole lotta time left in the month! And apart from this, it was kind of a funny month, because it brought a different kind of work to the sort I’d been doing for a while.

Welcoming Marc Busqué to the team!

The real highlight of the month was Marc Busqué getting on board with Hanami development! Marc’s brought a huge amount of fresh energy to the team and got right into productive development work. It’s great be to working with you, Marc!

Preparing dry-configurable’s API for 1.0

One of the first things Marc did was make some adjustments to dry-configurable’s setting API, to make it more consistent as one of the final steps before we can release 1.0 of that gem.

setting will now take only a single positional argument, for the name of the setting. Everything else must be provided via keyword arguments for improved consistency and clarity, plus easier wrapping by other gems. This means:

The default value for the setting must be supplied as default: rather than a second positional argument
A setting’s constructor (or ”processor”) can no longer be supplied as a block, instead it should be a proc object passed to constructor:

We merged these in these PRs, which include a deprecation pathway, so the previous usage will (largely) continue to work. While we were in dry-configurable, I also made a fix for it to work with preexisting #initialize methods accepting keyword args when its module is included, as well as removing implicit hash conversion which can result in unexpected destructuring when passing a configurable object to a method accepting a keyword args splat.

These dry-configurable changes haven’t yet been released, but hopefully we can make it happen sometime in June. The reason is that they were in service of a couple of larger efforts, both of which are still in flight (read on below for more detail!).

In the meantime, after these API changes, I did a sweep of the dry-rb ecosystem to bring things up to date, which led to PRs in dry-system, dry-container, dry-effects, dry-monitor, dry-rails, dry-schema, dry-validation, and hanami-view. Phew! It just goes to show how load-bearing this little gem is for our overall ecosystem (and how wide-ranging the impact of API changes can be). I haven’t merged these yet either, but will hope to do so in the next week or so, once we’ve ensured we have compatibility with both current and future dry-configurable APIs for the range of dry-rb gems that are past their respective 1.0 releases.

Porting Hanami::Configuration to dry-configurable

One thing that led to a couple of those dry-configurable fixes was my work in updating Hanami::Configuration to use dry-configurable. This class had gotten pretty sprawling with its manual handling of reading/writing a wide range setting values, which is squarely in dry-configurable’s wheelhouse, and the result is much tidier (and now consistent with how we’re handling configuration in both hanami-controller and hanami-view). This one again isn’t quite ready to merge (are you sensing a theme?), but it’s probably just an hour away from being done. I’ll look forward to having this one ticked off!

Updating Hanami’s application settings to use dry-configurable (and more)

Hanami’s application settings (the ones you define for yourself in config/settings.rb) have been very dry-configurable-like since their inception, but backed by custom code instead. It’s been on my to-do list for a long time to switch this over to dry-configurable, but with Marc joining the team, we’ve finally got some traction here! You can the original PR and an current, in-progress PR as well.

This one was a lot of collaborative fun. Marc made the broad initial steps, I jumped in to poke around and explore the design possibilities, and then he took my direction and ran with it, adding some other nice improvements along the way, like introducing a ”settings store” abstraction, which in our default implementation will continue to rely on dotenv.

This one is also close to being done. Watch this space (and all the other spaces I’ve mentioned so far, if you’re keen).

Providing a default types module to application settings (or not), and probably turning the whole thing into a regular class

Marc pivoted quickly from the above work to another long-standing to-do of ours: making a types module automatically available to the application settings. Having type-safe settings is one of the nicest features of the way we’re handling them, and I’d like this to be as smooth as possible for our users!

This turned out to be a bit of a rabbit hole, as evidenced by this sprawling PR discussion, but I think it’s led us to a good place.

Currently, the application settings must be defined in a block provided to the Hanami::Application.settings:

Hanami.application.settings do
  setting :sentry_dsn
end

Due to the combination of Ruby’s use of the lexical scope for constant lookups within blocks and dry-types’ standard reliance upon types collections as modules, with custom types defined as constants, it was nigh on impossible to auto-generate and provide an ergonomic, idiomatic types module for use within a block like that (see the linked PR discussion for details).

So this led us to the decision to move the application settings definition to a good ol’ ordinary Ruby class:

module MyApp
  class Settings < Hanami::Application::Settings
    setting :sentry_dsn
  end
end

This will still be looked up and loaded by the framework automatically, but because we’re using a regular class, we can rely on all the regular Ruby techniques for referring to a types module. This means we could choose to access a types module that the user has already created for themselves, e.g. MyApp::Types:

require "my_app/types"

module MyApp
  class Settings < Hanami::Application::Settings
    setting :sentry_dsn, MyApp::Types::String
  end
end

Or even create our own localised types module right within the class:

require "dry/types"

module MyApp
  Types = Dry.Types

  class Settings < Hanami::Application::Settings
    setting :sentry_dsn, Types::String
  end
end

This is much simpler and less likely to confuse! Better still, because we have a regular class at our disposal, users can now add their own custom behavior to their settings:

require "dry/types"

module MyApp
  Types = Dry.Types

  class Settings < Hanami::Application::Settings
    setting :sentry_dsn, Types::String.optional

    def sentry_enabled?
      !sentry_dsn.nil?
    end
  end
end

So I think this is a positive direction to be heading in. Plus, it reinforces the Hanami philosophy of ”a place for everything and everything in it’s place,” with this settings class being a great exemplar of a single-responsibility class, even for something that’s a special part of the framework boot process.

Plans for June

Well, I think that about brings you all up to speed for now. My plan for the rest of June is to make sure I can help merge all of those PRs! And then I’ll be getting back into Zeitwerk-land hand looking for ways to simplify the Ruby source file structures that we have inside our application and slice directories.

Thank you to my sponsors (including NEW SPONSORS!!) ❤️

May turned out to be hugely encouraging month for my GitHub sponsorships!

Thank you to Jason Charnes for upgrading your sponsorship! And thank you to Janko Marohnić and Aldis Berjoza for beginning new sponsorships! 🥰 Thanks also to Sebastian Wilgosz who began a periodic sponsorship based on a portion of his Hanami Mastery project sponsorships.

Little things like this this really do mean a lot, so folks, thanks again! 🙏🏼

If you’d like to support my ongoing OSS work, I’d love it if you could join my cadre of intelligent and very good looking sponsors on GitHub. And as ever, thank you to my existing sponsors for your ongoing support!

See you next month!

Open source status update, March and April 2021

2021-05-10T12:45:00+00:00

Hello again, open source friends. It’s been a little while, so today you get a double update for March and April, and what a couple of months they were!

For most of March I had a break from OSS, because we moved apartments! The move itself was a culmination of a multi-year purchase and saving process, so I definitely enjoyed the opportunity to savour the moment. We’re well settled now, and in matters relating to this here blog, I now have a dedicated room with a nice big desk (and new display!) from which to do all my computering, so I suppose you can all expect some more expansive thoughts in the future.

As for April, it was momentous for several other reasons. Let’s run through them…

Added system-wide defaults for dry-system component_dirs

Wrapping up this pull request was the first thing I did to get back into swing of things. I had taken care of most of it before the house move, but I needed to polish it a little before it was ready to merge.

With this now in place, it’s possible to configure settings in a dry-system container that apply to all of its subsequently added component dirs:

class MyContainer < Dry::System::Container
  configure do |config|
    # ...

    # These can be configured within each component_dir, but you can now
    # provide defaults for all dirs here
    config.component_dirs.loader = MyCustomLoader
    config.component_dirs.default_namespace = "my_app"

    # These added component_dirs will have the above settings as defaults
    config.component_dirs.add "lib"

    config.component_dirs.add "another_dir" do |dir|
      # You can still override the defaults within any given dir
      dir.loader = AnotherLoader
    end
  end
end

Apart from wanting this just to ensure the new dry-system component_dirs feature felt fully rounded, I also felt it was important for dry-system’s autoloader support to be ergonomic, allowing it to become just a one or two-liner.

# Configure autoloader support in one place, instead of once for every
# added component_dir
config.component_dirs.loader = Dry::System::Loader::Autoloading
config.component_dirs.add_to_load_path = false

The review for this PR raised an interesting nugget. The change resulted in two places with the exact same collection of settings defined:

The top-level config.component_dirs (Dry::System::Config::ComponentDirs)
The individual component dir configuration object yielded to each call of config.component_dirs.add (Dry::System::Config::ComponentDir)

To keep these in sync, I wanted to have the settings defined in one place, and copied over into the other. My initial implementation used some deep knowledge of dry-configurable internals:

module Dry
  module System
    module Config
      class ComponentDirs
        # ...

        # Settings from ComponentDir can be configured here to apply all added dirs as
        # defaults
        @_settings = ComponentDir._settings.dup

In the review, Piotr aked why I was using this ivar, which led to a helpful discussion. I had used the ivar because it offered a one-liner to import a copy of the settings all at once, but in talking it through, I realised I’d been relying on private aspects of dry-configurable that I wouldn’t recommend to others. Despite both of these gems inhabiting the same dry-rb ecosystem, we should really be using public APIs like any other gem user. So this is really an opportunity for a public API to emerge and provide this same outcome, something like this:

_settings.replace(ComponentDir._settings.dup)

Until then (see this issue if you want to help!), I’ve avoided the direct access of the ivar through this slightly wordier but just as effective approach:

# Settings from ComponentDir are configured here as defaults for all added dirs
ComponentDir._settings.each do |setting|
  _settings << setting.dup
end

Released dry-system 0.19.0

With that last PR merged, and the changelog prepared (so big it needed its own PR!), we were ready to release dry-system 0.19.0! This release was big enough that it merited its own announcement post. Check it out for the headline features that this release brings. For all the juicy details, you always the previous four months of my open source status updates.

As part of preparing for the release, I also teed up a dry-system 1.0 issues milestone on GitHub. Please check them out if you’d like to pitch in! Most of the issues are much more targeted than the sweeping refactors I did over the last while, so they should be approachable.

(Sidenote: I also prepared a similar 1.0 issues milestone for dry-configurable, the other key gem we’d like to bring to 1.0 sometime this year)

Finished Hanami Zeitwerk integration

With the dry-system release done, I turned my attention back to Hanami, and quite promptly finished the Zeitwerk integration. This turned out to be a breeze after all the foundational work on dry-system. Zeitwerk author and all around mensch Xavier ”fxn” Noria also took a look, and this comment of his left me chuffed:

That is all? Man, I do not know the details but this patch tells me the work behind this integration is really good 💯.

Thank you for the encouragement, fxn 😊

As for how the integration works, there’s nothing really particular to dive into (unusual for me, I know!). It all… just works. There’s a new config.autloader setting on the Hanami application class that you can use to configure the Zeitwerk instance directly, or set to false or nil if you want to disable autloading. In most cases, though, you shouldn’t have to worry, everything will just work.

As part of testing this out, I ported one of my work’s Hanami 2.0 applications to the new Zeitwerk support, and in the process was able to remove more than 150 individual require lines from the app. This is going to make building Hanami apps a much nicer experience!

Released Hanami 2.0.0.alpha2!

Finally, all of the ducks were in a row to cut the long-awaited next alpha release of Hanami 2.0.0. The team banded together to get all the final pieces sorted over a furious few days of activity, leading to the release and my official announcement on 4th May (pushed at about 1am here in Canberra!).

Please go check out the announcement, I think it’s a great summary of everything we’ve been working towards for these last two years, and it will hopefully get you excited for what Hanami can offer the next generation of Ruby apps (apps of all kinds, too, because as of this last-moment PR, we’re explicitly no longer a web-only framework).

As for my feelings about all of this, I think my tweet at the time says it all:

Hanami 2.0.0.alpha2!

I’m incredibly proud of what we’ve assembled so far, and can’t wait to take this vision to full fruition.

In the meantime, what a milestone! I’m so happy 😭

Thanks to @jodosha, @solnic and the whole Hanami team for such a meaningful couple years of work.

If you’d like to kick the tyres on this new alpha release, go check out the application template that we’ve brought up to date and moved into the Hanami GitHub org.

What’s next?

Our goal for the next alpha is to reduce the boilerplate from the app template as much as possible. Hop onto this discussion thread if you’d like to share your thoughts!

There’s a lot of further work to the framework internals I’d like to do (i.e. much of what’s on the Trello board), but having a spick and span template is a great crystalising goal for the next release, getting us as quickly as possible to our desired generated application, leaving time to work on the internals afterwards.

Thank you to my sponsors ❤️

It’s been a huge effort to get to where we are today, and I certainly enjoyed reaching the milestone, but there’s still plenty left to do. I’d love it if you could sponsor me on GitHub to help sustain my efforts over all the Hanami releases to come. And as ever, thank you to my existing sponsors for your ongoing support!

See you next month!

Open source status update, February 2021

2021-03-09T11:20:00+00:00

Well hey there, Ruby open source fans! February for me was all about consolidating the dry-system breakthroughs I made last month.

I started off by testing the work on a real app I made, and happily, all was fine! Things all looking good, I wrote myself a list and shared it with my Hanami colleagues:

[timriley] So what’s left for me to do here:

Release dry-configurable with the new cloneable values support

Apply @Nikita Shilnikov’s excellent feedback to https://github.com/dry-rb/dry-system/pull/157

Merge https://github.com/dry-rb/dry-system/pull/155

Merge https://github.com/dry-rb/dry-system/pull/155

Merge https://github.com/hanami/controller/pull/341

Merge https://github.com/hanami/hanami/pull/1093

In a new PR, configure Zeitwerk for Hanami, and enable the autoloading loader for Hanami’s container component_dirs

Everything started well: by the 15th of Feb I released dry-configurable 0.12.1 with the new cloneable option for custom setting values. And a mere hour later, I merged the two dry-system PRs! Woo, we’re on the home straight!

At that point, I gave Nikita the go-ahead to test all the dry-system changes on some of the apps that he manages: he’s absolutely brazen about running our bleeding edge code, and I love it. In this case, it was very helpful, because it revealed a little wrinkle in my heretofore best laid plans: if you configure a dry-system container with a component dir and a default namespace, and some of the component files sit outside that namespace, then they would fail to load. This was a valid use case missing from our test suite, and something I must’ve broke in my major changes last month.

This turned out to be relatively simple for me to hack in a fix, but at the same time I noticed an opportunity for yet another improvement: spread across multiple parts of dry-system was a bunch of (often repeated) string manipulation code working on container identifiers, doing things like removing a leading namespace, or converting a delimited identifier to a file path. It felt like there was a Dry::System::Identifier abstraction just waiting to be let out.

And so I did it! In this omnibus PR, I introduced Dry::System::Identifier, refactored component building once more, and, not to be forgotten, fixed the bug Nikita found.

I’m really happy with both of the refactorings. Let’s start with Identifier: now we have just a single place for all the logic dealing with identifier string manipulations, but we also get to provide a new, rich aspect of our component API for users of dry-system. For example, as of my work last month, it’s now possible to configure per-component behaviour around auto-registration, and we can now use the component’s identifier like so:

config.component_dirs.add "lib" do |dir|
  dir.default_namespace = "my_app"

  dir.auto_register = lambda do |component|
    !component.identifier.start_with?("entities")
  end
end

Isn’t that neat? The Identifier began life as an internal-only improvement, but here we get to make our user’s life easier too, with namespace-aware methods like #start_with? (which will return true only if the "entities" is complete leading namespace, like "entities.user", and not "entities_abc.user"). I’d like to add a range of similar conveniences to Identifier before we release 1.0. Please let me know what you’d like to see!

The other benefit of Identifier is that it’s vastly simplified how we load components. Check out how we use it in ComponentDir#component_for_path, which is used when the container is finalizing, and the auto-registrar crawls a directory to register a corresponding component for each file (comments added below for the sake of explanation):

def component_for_path(path)
  separator = container.config.namespace_separator

  # 1. Convert a path, like "my_app/articles/operations/create.rb"
  #    to an identifier key, "my_app.articles.operations.create"
  key = Pathname(path).relative_path_from(full_path).to_s
    .sub(RB_EXT, EMPTY_STRING)
    .scan(WORD_REGEX)
    .join(separator)

  # 2. Create the Identifier using the key, but without any namespace attached
  identifier = Identifier.new(key, separator: separator)

  # 3. If the identifier is part of the component dir's configured default
  #    namespace, then strip the namespace from the front of the key and
  #    rebuild the identifier with the namespace attached
  if identifier.start_with?(default_namespace)
    identifier = identifier.dequalified(default_namespace, namespace: default_namespace)
  end

  # 4. By this point, the identifier will be appropriate for both default
  #    namespaced components as well as non-namespaced components, so we can go
  #    ahead and use it to build our component!
  build_component(identifier, path)
end

Even without the comments, this method is concise and easy to follow thanks to the high-level Identifier API. What you’re also witnessing above is the very fix for the bug Nikita found! Getting to this point was a perfect example of Kent Beck’s “for each desired change, make the change easy (warning: this may be hard), then make the easy change” process. And you bet, it felt good!

In this change I actually introduced a pair of methods on ComponentDir: #component_for_path (as we saw above), which is used when finalizing a container, as well as #component_for_identifier, which is used when lazy-loading components on a non-finalized container. Previously, these two methods were both class-level ”factory” methods on Component itself. By moving them to ComponentDir, not only are they much closer to the details that are important for building the component, they provide a nice symmatry which will help ensure we don’t miss either case when making changes to component loading in future. Component winds up being a much simpler class, too, which is nice.

After all of this, Nikita gave me a happy thumbs up and we were good to merge this PR and resume preparation for a major dry-system release!

But not so fast, I also:

Spent a full evening adding API docs to the code I introduced/changed in this PR
Got the thing merged, then swept through the user docs to make sure they reflected the recent changes
And started working on a mammoth CHANGELOG entry upcoming release

It was in writing the CHANGELOG entry that I realised I needed to make one last change before I can really consider this release done: I want to create a way to configure default values for all component dirs. This will be helpful for when we eventually add a one-liner use :zeitwerk plugin to dry-system, which will need to ensure that dir.loader = Dry::Systen::Loader::Autoloading and dir.add_to_load_path = false are set for all subsequent user-configured component dirs. Given the amount of breaking changes we’ll be making with this release, I’d hate to see yet any unnecessary extra churn arise from this work. So that’s my first task for the month of March.

In the meantime, I have Hanami ready and waiting for this new dry-system release! As soon as our ducks are finally in a row, I’ll be able to merge this and begin the long-anticipated work on configuring Zeitwerk within Hanami.

Looking back on this month, I spent most of it feeling frustrated that I was still working on dry-system after all this time, when I just wanted to get back and make that very last change in Hanami before we could ship the next alpha release. This was exacerbated by a series of late nights that I pulled trying to get that bugfix and related Identifier changes working in a way I was happy with. I finished the month feeling pretty drained and the slightest bit cranky.

I’m indeed happy with the outcome and glad I put in the work. And yes, I got to the point where I could laugh about it. From the PR description:

As part of this work, I also continued my dry-system refactoring journey, because this is more or less my life now 😆.

What I’m taking away from this experience is a reminder that I need to be patient. When working on nights-and-weekends open source, things will only happen when they can happen, and the best we can do is the best we can do. I’ll keep on pushing hard, but February has reminded me to make sure I take care of my (physical and mental) health too.

We’re already a good week into March by the time I’m writing this, and later this month I expect to be moving apartments (yay!), so March may be a slightly less full month from me on the OSS front. If I can squeeze in that last dry-system fix and get myself in a position to begin some early experiments of Zeitwerk in Hanami, I’ll be happy. That’ll put us in a good place to ship the next Hanami alpha early in April.

Thank you to my sponsors ❤️

If you want to give me a boost in all of these efforts, I’d love for you to sponsor me on GitHub. Thank you to my sponsors for your ongoing support!

See you next month!

Open source status update, January 2021

2021-02-01T11:50:00+00:00

I had a very satisfying January in Ruby OSS work! This month was all about overhauling the dry-system internals. That I’ve written about this in both November and December just goes to show (a) how long things actually take when you’re doing this on the side (and I’m not lazing about, I spend at least 3 nights a week working on OSS), and (b) just how much there was going on inside of dry-system.

So to set the scene, here’s the circuitous path I took in adding rich component directory configuration to dry-system. This is the commit history before I tidied it:

2020-11-24 Add some early WIP
2020-11-25 Accept pre-configured component dirs
2020-11-25 Configure with block as part of initialize
2020-11-25 Provide path when initializing ComponentDir
2020-12-01 Add Rubocop rule
2020-12-01 Allow ComponentDirs to be cloned
2020-12-01 Clarify names
2020-12-01 Fix wording of spec
2020-12-01 Fixup naming
2020-12-01 Start getting component_dirs in place
2020-12-01 Update auto-registrar to use component_dirs
2020-12-01 Update specs for component_dirs
2020-12-03 Total messy WIP
2020-12-23 Get some WIP laid down on Booter#find_component
2020-12-23 Remove some WIP comments
2020-12-23 Tidy
2020-12-23 Update file_exists? behavior
2021-01-04 Add error
2021-01-04 Get things closer
2021-01-04 Provide custom dry-configurable cloneable value
2021-01-04 Remove unused settings
2021-01-04 Use a Concurrent::Map
2021-01-05 Add FIXME about avoiding config.default_namespace
2021-01-05 Introduce ComponentDir with behaviour separate to config
2021-01-05 Remove note, now that Loader#call is doing a require!, we’re fine
2021-01-05 Remove top-level default_namespace config
2021-01-05 Tidy Component
2021-01-05 Update FIXME
2021-01-11 Add docs
2021-01-11 Add docs for file exists
2021-01-11 Document Booter#boot_files method as public
2021-01-11 Don’t preserve file_path when namespacing component
2021-01-11 Expand docs
2021-01-11 Fix
2021-01-11 Flesh out ComponentDir
2021-01-11 Remove TODO
2021-01-11 Remove unused method
2021-01-11 Rip out the Component cache
2021-01-11 Stop setting file attribute on Component
2021-01-11 Tidy up Component file_path attr
2021-01-11 Tweak names
2021-01-11 Use a faster way of building namespaces path
2021-01-12 Use cloneable option for component_dirs setting
2021-01-14 Do not load components with auto_register: false
2021-01-14 Initialize components directly from AutoRegistrar
2021-01-14 Remove stale comment
2021-01-14 Remove unneeded requiring of component
2021-01-14 Scan file for magic comment options when locating
2021-01-14 Use dry-configurable master
2021-01-15 Add spec (and adjust approach) for skipping lazy loading of auto-register-disabled components
2021-01-15 Add unit tests for ComponentDir
2021-01-15 Tidy AutoRegistrar
2021-01-16 Add extra attributes to component equalizer
2021-01-16 Add unit tests for Component.locate
2021-01-16 Make load_component easier to understand
2021-01-18 Use base Dry::Container missing component error

Yep. 56 commits and just under two calendar months of work, with the break in early December being my sting doing Advent of Code. Luckily, that “Total messy WIP” left me with a passing test suite after some heavy refactoring, but it did take a day or two to figure out just what I was doing again! Note to self: leave more notes to self.

Rich, independent component directory configuration for dry-system

The (tidied) pull request for this change has a lengthy description, focused on implementation. If you’re interested in the details, please have a read!

Here’s the long and the short of it, though: previously, dry-system would let you configure a top-level auto_register setting, which would contain an array of string paths within the container root, which the system would use to populate the container. This would often be used alongside another top-level setting, default_namespace, which would strip away a common namespace prefix from the container identifiers, and a call to .load_paths! for each directory being auto-registered, to ensure the sources files within those directories could be properly required:

class MyApp::Container < Dry::System::Container
  configure do |config|
    config.root = __dir__
    config.auto_register = ["lib"]
    config.default_namespace = "my_app"
  end

  load_paths! "lib"
end

Those are three different things you would need to know how to use just right in order to set up a properly working dry-system container. Luckily, most users could copy a working example and then tweak from there. Also, users would typically only set up a single directory for auto-registration, so those three elements would only need to apply to that one directory only. If you ever tried to do more (for example, now that we have an autoloading loader, configure one directory to use the autoloder and another not to), things would fall apart.

Things brings us to the rich component directory configuration, and indeed the introduction of a ”Component Directory” as a first-class concept within dry-system. Here’s how a container setup would look now:

class MyApp::Container < Dry::System::Container
  configure do |config|
    config.root = __dir__

    config.component_dirs.add "lib" do |dir|
      dir.auto_register = proc do |component|
        !component.path.match?(%r{/entities/})
      end
      dir.add_to_load_path = false
      dir.loader = Dry::System::Loader::Autoloading
      dir.default_namespace = "my_app"

      # Also available, `dir.memoize`, accepting a boolean or proc
    end
  end
end

Now the behavior for handling a given component directory can be configured on that directory and that directory alone. In the above example, another component diretory could be added with diametrically opposed settings to the first, and everything will still be dandy!

As you can also see, the degree of configurability has also increased greatly over the released versions of dry-system. Now you can opt into or out of auto-registration for specific components by passing a proc to the auto_register setting. Memoization of registered components can also be enabled, disabled, or configured specifically with the memoize setting.

(While you’re here, also check out the dry-configurable change I made to allow cloneable setting values, without which we couldn’t have provided this rich nested API for configuring particular directories)

Consistent component loading behavior, including magic comments!

With the changes above in place, I could remove the .auto_register! container class method (done in this pull request, also with its own lengthy description), which leaves the component_dirs setting as the only way to tell dry-system how to load components from source files.

Not only does this make for an easier to configure container, it also supports a more consistent component loading experience! Now, every configurable aspect of component loading is respected in the container’s two methods of auto-registering components: either via finalizing the container (which loads everything up front and freezes the container) or via lazy-loading (which loads components just in time, and is useful for keeping container load time down when running unit tests or using an interactive console, among other things).

It also means that magic comments within source files are respected in all cases, where previously, only a subset of comments were considered, and only when finalizing a container, not during lazy-loading.

This means you can now have a source file like this:

# auto_register: false

class MyEntity
end

And MyEntity will never find its way into your container.

Or you can have a source file like this:

# memoize: true

class MySpecialComponent
end

And when the component is registered, it will be memoized automatically.

Magic comments for dry-system are great, I use them all the time, and now they’re even more powerful!

More consistent, easier to understand dry-system internals

I’ve worked in the dry-system codebase quite regularly over the last few years, and certain parts have always felt a little too complicated, often leaving me confused, or at least afraid to change them. This is no discredit everyone who worked on dry-system previously! Its an amazing innovation, and its features just grew organically over the years to make it the capable, powerful system it is today!

However, given I was going to be deep in the code again to implement the changes I wanted, I took the chance to refactor as much as I could. And I’m just delighted in the outcome! For example, check out how .load_component and .load_local_component (which are used for lazy-loading components) used to look:

def load_component(key, &block)
  return self if registered?(key)

  component(key).tap do |component|
    if component.bootable?
      booter.start(component)
    else
      root_key = component.root_key

      if (root_bootable = component(root_key)).bootable?
        booter.start(root_bootable)
      elsif importer.key?(root_key)
        load_imported_component(component.namespaced(root_key))
      end

      load_local_component(component, &block) unless registered?(key)
    end
  end

  self
end

def load_local_component(component, default_namespace_fallback = false, &block)
  if booter.bootable?(component) || component.file_exists?(component_paths)
    booter.boot_dependency(component) unless finalized?

    require_component(component) do
      register(component.identifier) { component.instance }
    end
  elsif !default_namespace_fallback
    load_local_component(component.prepend(config.default_namespace), true, &block)
  elsif manual_registrar.file_exists?(component)
    manual_registrar.(component)
  elsif block_given?
    yield
  else
    raise ComponentLoadError, component
  end
end

And here’s how they look now:

def load_component(key)
  return self if registered?(key)

  component = component(key)

  if component.bootable?
    booter.start(component)
    return self
  end

  booter.boot_dependency(component)
  return self if registered?(key)

  if component.file_exists?
    load_local_component(component)
  elsif manual_registrar.file_exists?(component)
    manual_registrar.(component)
  elsif importer.key?(component.root_key)
    load_imported_component(component.namespaced(component.root_key))
  end

  self
end

def load_local_component(component)
  if component.auto_register?
    register(component.identifier, memoize: component.memoize?) { component.instance }
  end
end

Just look at that improvement! We went from a pair of methods that always confused me (with their mixed responsibilities, multiple conditionals and levels of nesting) to a simple top-to-bottom flow in .load_component, and .load_local_component reduced to a simple 3-liner with just a single job.

Weeks later, I’m still marvelling at this. I think it’s one of the best refactorings I’ve ever done.

These improvements didn’t come on their own. As you might notice there, component is carrying a lot more of its own weight. This includes a new set of methods for finding and loading components from within component directories (namely Dry::System::Component.locate and .new_from_component_dir), and indeed the new Dry::System::ComponentDir abstraction itself, which together provide the consistent component loading behavior I described above.

Dry::System::Loader converted to a class interface

One thing I noticed during the work on component loading is that a new Dry::System::Loader would be instantiated for every component, even through it carried no other state apart from the component itself, so I turned it into a stateless, class-level interface, and hey presto, we save an object allocation for every component we load.

This is a breaking change, but hey, so is everything else I’ve described so far! I figure this is the right time to sort these things out before we look to a dry-system 1.0 release sometime in the next few months (which is seeming much more attractive after this round of work!).

I appreciated being appreciated 🥺

Given how significant my plans were for all these changes, I made sure to keep Piotr and the other maintainers in the loop over those couple of months of work.

Then, when Piotr reviewed my first finished pull request for this work, he left me the most amazing comment. I want to repeat it here in full (that is, to take it straight in the pool room):

@timriley thanks for this very detailed description, it made much more sense for me to carefully read it and understand the changes rather than to examine the diff. Since what you did here, conceptually, makes perfect sense to me, AND it resulted in simplified implementation which at the same time makes the library much more powerful, I have nothing to complain about 😆 FWIW - I’ve read the diff and nothing stands out as potentially problematic. I reckon seeing it work in real-world apps will be a much better verification process, it’s a huge change after all.

This is clearly a huge milestone and I honestly didn’t expect that the lib will be so greatly improved prior 1.0.0, so thank you for this huge effort, really ❤️

One thing I’ll probably experiment with would be a backward-compatibility shim so that previous methods (that you removed) could still work. This should make it simpler to upgrade, but please tell me if this is a stupid idea.

I will also upgrade dry-rails!

Tim, seriously, again, this is an epic refactoring, I’m so happy. You’re taking dry-system to the next level. Thank you! 🚀 🎉 🙇🏻

After a full year of labouring away at this stuff, often to uncertain or, frankly, even unknowable ends, a comment like this has just given me the fuel to go another year more. Thank you, Piotr ♥️

If there’s someone out there in OSS land whose work you appreciate, please take the time to tell them! It might mean more than you think.

Next steps with dry-system, Hanami, and Zeitwerk, and the alpha2 release

Now that the bulk of the dry-system work is done, here’s what I’m looking to get done next:

Run some final tests using the dry-system branches within a real application
Work with Piotr to coordinate related changes to dry-rails (which configures dry-system auto-registration)
Updating Hanami to configure component_dirs within its own dry-system containers
Then work out how to enable Zeitwork within Hanami and use the autoloading loader for its component directories by default, while still providing a clean way for application authors to opt out if they’d rather use traditional require-based code loading

I expect this will take most of February. Once this is done, we’ll finally be in the clear for a 2.0.0.alpha2 release of Hanami. Focusing on Zeitwerk and dry-system has pushed it back a couple of months, but I hope everyone will agree it was worth the wait!

Thank you to my sponsors! 🙌🏼

Thanks to my GitHub sponsorts for your continuing support! If you’re reading this and would like to support my work in making Hanami 2.0 a reality, I’d really appreciate your support.

See you all next month!

Open source status update, December 2020

2021-01-06T11:50:00+00:00

Happy new year! Before we get too far through January, here’s the recap of my December in OSS.

Advent of Code 2020 (in Go!)

This month started off a little differently to usual. After spending some time book-learning about Go, I decided to try the Advent of Code for the first time as a way to build some muscle memory for a new language. And gosh, it was a lot of fun! Turns out I like programming and problem-solving, go figure. After ~11 days straight, however, I decided to put the effort on hold. I could tell the pace wasn’t going to be sustainable for me (it was a lot of late nights), and I’d already begun to feel pretty comfortable with various aspects of Go, so that’s where I left it for now.

Rich dry-system component_dir configuration (and cleanups!)

Returning to my regular Ruby business, December was a good month for dry-system. After the work in November to prepare the way for Zeitwerk, I moved onto introducing a new component_dirs setting, which permits the addition of any number of component directories (i.e. where dry-system should look for your Ruby class files), each with their own specific configurations:

class MyApp::Container < Dry::System::Container
  configure do |config|
    config.root = __dir__

    config.component_dirs.add "lib" do |dir|
      dir.auto_register = true    # defaults to true
      dir.add_to_load_path = true # defaults to true
      dir.default_namespace = "my_app"
    end
  end
end

Along with this, I’m removing the following from Dry::System::Container:

The top-level default_namespace and auto_register settings
The .add_to_load_path! and .auto_register! methods

Together, this means there’ll be only a single place to configure the behaviour related to the loading of components from directories: the singular component_dirs setting.

This has been a rationalization I’ve been wanting to make for a long time, and happily, it’s proving to be a positive one: as I’ve been working through the changes, it’s allowed me to simplify some of the gnarlier parts of the gem.

What all of this provides is the right set of hooks for Hanami to specify the component directories for your app, as well as configure each one to work nicely with Zeitwerk. That’s the end goal, and I suspect we’ll arrive there in late January or February, but in the meantime, I’ve enjoyed the chance to tidy up the internals of this critical part of the Hanami 2.0 underpinnings.

You can follow my work in progress over in this PR.

Helpers for hanami-view 2.0

Towards the end of the month I had a call with Luca (the second in as many months, what a treat!), in which we discussed how we might bringing about full support for view helpers into hanami-view 2.0.

Of course, these won’t be ”helpers” in quite the same shape you’d expect from Rails or any of the Ruby static site generators, because if you’ve ever heard me talk about dry-view or hanami-view 2.0 (here’s a refresher), one of its main goals is to help move you from a gross, global soup of unrelated helpers towards view behaviour modelled as focused, testable, well-factored object oriented code.

In this case, we finished the discussion with a plan, and Luca turned it around within a matter of days, with a quickfire set of PRs!

First he introduced the concept of custom anonymous scopes for any view. A scope in dry-view/hanami-view parlance is the object that provides the total set of methods available to use within the template. For a while we’ve supported defining custom scope classes to add behavior for a view that doesn’t belong on any one of its particular exposures, but this requires a fair bit of boilerplate, especially if it’s just for a method or two:

class ArticleViewScope < Hanami::View::Scope
  def custom_method
    # Custom behavior here, can access all scope facilities, e.g. `locals` or `context`
  end
end

class ArticleView < Hanami::View
  config.scope = ArticleViewScope

  expose :article do |slug:|
    # logic to load article here
  end
end

So to make this eaiser, we now we have this new class-level block:

class ArticleView < Hanami::View
  expose :article do |slug:|
    # logic to load article here
  end

  # New scope block!
  scope do
    def custom_method
      # Custom behavior here, can access all scope facilities, e.g. `locals` or `context`
    end
  end
end

So nice! Also nice? That it was a literal 3-line change to the hanami-view code 😎 Also, also nice? You can still “upgrade” to a fully fledged class if the code ever requires it.

Along with this, Luca also began adapting the existing range of global helpers for use in hanami-view 2.0. I may dislike the idea of helpers in general, but truly stateless things like html builders, etc. I’m generally happy to see around, and with the improvements to template rendering we have over hanami-view 1.x, we’ll be able to make these a lot more expressive for Hanami view developers. This PR is just the first step, but I expect we’ll be able to make some quick strides once this is in place.

Thank you to my sponsors! 🙌🏼

Thank you to my six GitHub sponsorts for your continuing support! If you’re reading this and would like to chip in and help push forward the Ruby web application ecosystem for 2021, I’d really appreciate your support.

See you all next month!

2020 in review

2021-01-04T11:55:00+00:00

2020, hey? What a time to finally get back on my ”year in review” horse. It was a calamitous year for many of us, but there’s a lot I’m thankful for from 2020.

Work

In January I started a new job with Culture Amp, as part of the Icelab closing and the whole team moving across. I couldn’t have found a better place to work: the people are inspiring, I’ve wound up with a great mentor/manager, and the projects are nourishing. There’s so much I can contribute to here, and I know I’m still only scratching the surface.

A new workplace with it’s own tech ecosystem meant I did a lot of learning for work this year. Among other things, I started working with event sourcing, distributed systems, AWS CDK, and even a little Go as the year came to an end.

I’m full-time remote with Culture Amp. Under ordinary circumstances (which went out the window before long), this would mean semi-regular visits to the Melbourne office, and I was lucky enough to do that a couple of times in January and February before travel became unworkable. Aside from that, I’ve enjoyed many hours over Zoom working with all my new colleagues.

And just to tie a bow in the a big year of work-related things, Michael, Max, and I worked through to December putting the finishing touches on the very last Icelab project, Navigate Senate Committees.

I’m deeply grateful to be where I am now, to have smoothly closed one chapter of work while opening another that I’m excited to inhabit for many years to come.

OSS

This year in OSS has been all about Hanami 2.0 development. By the end of 2019, I’d already made the broad strokes required to make dry-system the new core of the framework. 2020 was all about smoothing around the edges. I worked pretty consistently at this throughout the year, focusing on a new technique for frictionless, zero-boilerplate integrated components, view/action integration, a revised approach for configuration, and lately, support for code loading with Zeitwerk.

Towards the beginning of the year, I decided to follow Piotr’s good example and write my own monthly OSS status updates. I published these monthly since, making for 9 in the year (and 10 once I do December’s, once this post is done!). I’m really glad I established this habit. It captures so much of the thinking I put into my work that would otherwise be lost with time, and in the case of the Hanami project, it’s a way for the community to follow along with progress. And I won’t lie, the thought of the upcoming post motivates me to squeeze just a little bit more into each month!

This year I shared my Hanami 2 application template as a way to try the framework while it’s still in development. We’re using it for three production services at work and it’s running well.

Helping to rewrite Hanami for 2.0 has been the biggest OSS project I’ve undertaken, and work on this was a slog at times, but I’m happy I managed to keep a steady pace. I also rounded out the year by being able to catch up with Luca and Piotr for a couple of face to face discussions, which was a delight after so many months of text-only collaboration.

On the conference side of things, given the travel restrictions, there was a lot less than normal, but I did have a great time at the one event I did attend, RubyConf AU back in February (which seems so long ago now). Sandwiched between the Australian summer bushfires and the onset of the coronavirus, the community here was amazingly lucky with the timing. Aside from this, the increase of virtual events meant I got to share a talk with Philly.rb and appear on GitHub’s open source Friday livestream series.

I also joined the GitHub sponsors program in May. I only have a small group of sponsors (I’d love more!), but receiving notice of each one was a true joy.

And in a first for me, I spent some time working on this year’s Advent of Code! I used it as an opportunity to develop some familiarity with Go. It was great fun! Code is here if you’re interested.

Home & family

The ”stay at home” theme of 2020 was a blessing in many ways, because it meant more time with all the people I love. This year I got to join in on Clover learning to read, write, ride a bike, so many things! Iris is as gregarious as ever and definitely keen to begin her school journey this coming year.

As part of making the most of being home, I also started working out at home in March (thanks, PE with Joe!), which I managed to keep up at 5 days/week ever since! I haven’t felt this good in years.

And to close out, a few other notables:

Started visiting the local library more, I enjoyed reading paper books again
Some particularly good reads from the year: Stephen Baxter’s Northland trilogy, Cixin Liu’s The Supernova Era, Stephen Baxter’s The Medusa Chronicles, Roger Levy’s The Rig, Kim Stanley Robinson’s Red Moon, Mary Robinette Kowal’s The Relentless Moon, Kylie Maslen’s Show Me Where it Hurts, and Hugh Howey’s Sand
Ted Lasso on Apple TV+ was just great
Got a Nintendo Switch and played through Breath of the Wild, a truly spectacular experience
I think that’s all for now, see you next year!

Open source status update, November 2020

2020-12-07T10:50:00+00:00

Hello again, dear OSS enthusiasts. November was quite a fun month for me. Not only did I merge all the PRs I outlined in October’s status update, I also got to begin work on an area I’d been dreaming about for months: integrating Hanami/dry-system with Zeitwerk!

Added an autoloading loader to dry-system

Zeitwerk is a configurable autoloader for Ruby applications and gems. The “auto” in autoloader means that, once configured, you should never have to manually require before referring to the classes defined in the directories managed by Zeitwerk.

dry-system, on the other hand, was requiring literally every file it encountered, by design! The challenge here was to allow it to work with or without an auto-loader, making either mode a configurable option, ideally without major disruption to the library.

Fortunately, many of the core Dry::System::Container behaviours are already separate into individually configurable components, and in the end, all we needed was a new Loader subclass implementing a 2-line method:

module Dry
  module System
    class Loader
      # Component loader for autoloading-enabled applications
      #
      # This behaves like the default loader, except instead of requiring the given path,
      # it loads the respective constant, allowing the autoloader to load the
      # corresponding file per its own configuration.
      #
      # @see Loader
      # @api public
      class Autoloading < Loader
        def require!
          constant
          self
        end
      end
    end
  end
end

This can be enabled for your container like so:

require "dry/system/loader/autoloading"

class MyContainer < Dry::System::Container
  configure do |config|
    config.loader = Dry::System::Loader::Autoloading
    # ...
  end
end

Truth is, it did take a fair bit of doing to arrive at this simple outcome. Check out the pull request for more detail. The biggest underlying change was moving the responsibility for requiring files out of Container itself and into the Loader (which is called via each Component in the container). While I was in there, I took the chance to tweak a few other things too:

Clarified the Container.load_paths! method by renaming it to add_to_load_path! (since it is modifying Ruby’s $LOAD_PATH)
Stopped automatically adding the system_dir to the load path, since with Zeitwerk support, it’s now reasonable to run dry-system without any of its managed directories being on the load path
Added a new component_dirs setting, defaulting to ["lib"], which is used to verify whether a given component is ”local” to the container. This check was previously done using the directories previously passed to load_paths!, which we can’t rely upon now that we’re supporting autoloaders
Added a new add_component_dirs_to_load_path setting, defaulting to true, which will automatically add the configured component_dirs to the load path in an after-configure hook. This will help ease the transition from the previous behaviour, and make dry-system still work nicely when not using an autoloader

With all of this in place, a full working example with Zeitwerk looks like this. First, the container:

require "dry/system/container"
require "dry/system/loader/autoloading"

module Test
  class Container < Dry::System::Container
    config.root = Pathname(__dir__).join("..").realpath
    config.add_component_dirs_to_load_path = false
    config.loader = Dry::System::Loader::Autoloading
    config.default_namespace = "test"
  end
end

Then Zeitwerk setup:

loader = Zeitwerk::Loader.new
loader.push_dir Test::Container.config.root.join("lib").realpath
loader.setup

Then, given a component “foo_builder”, at lib/test/foo_builder.rb:

module Test
  class FooBuilder
    def call
      # We can now referencing this constant without a require!
      Entities::Foo.new
    end
  end
end

With this in place, we can resolve Test::Container["foo_builder"], receive an instance of Test::FooBuilder as expected, then .call it to receive our instance Test::Foo. Tada!

I’m very happy with how all this came together.

Next steps with dry-system

Apart from cracking the Zeitwerk nut, this project also gave me the chance to dive into the guts of dry-system after quite a while. There’s quite a bit of tidying up I’d still like to do, which is my plan for the next month or so. I plan to:

Make it possible to configure all aspects of each component_dir via a single block passed to the container’s config
Remove the default_namespace top-level container setting (since this will now be configured per-component_dir)
Remove the .auto_register! method, since our component-loading behaviour requires component dirs to be configured, and this method bypasses that step (until now, it’s only really worked by happenstance)
Make Zeitwork usable without additional config by providing a plugin that can be activated by a simple use :zeitwerk

Once these are done, I’ll hop up into the Hanami framework layer and get to work on passing the necessary configuration through to its own dry-system container so that it can also work with Zeitwerk out of the box.

Hanami core team meeting

This month I also had the (rare!) pleasure of catching up with Luca and Piotr in person to discuss our next steps for Hanami 2 development. Read my notes to learn more. If you’re at all interested in Hanami development (and if you’ve reached this point in my 9th straight monthly update, I assume you are), then this is well worth a read!

Of particular relevance to the topics above, we’ve decided to defer the next Hanami 2 alpha release until the Zeitwerk integration is in place. This will ensure we have a smooth transition across releases in terms of code loading behaviour (if we released sooner, we’d need to document a particular set of rules for alpha2 but then half of those out the window for alpha3, which is just too disruptive).

Thank you to my sponsors!

After all this time, I’m still so appreciative of my tiny band of GitHub sponsors. This stuff is hard work, so I’d really appreciate your support.

See you all again next month, by which point we’ll all have a Ruby 3.0 release!

Open source status update, October 2020

2020-11-03T11:30:00+00:00

October was the month! I finally got through the remaining tasks standing between me and an Hanami 2.0.0.alpha2 release. Let’s work through the list now!

Application views configured with application inflector

Now when you subclass Hanami::View inside an Hanami app, it will now use the application’s configured inflector automatically. This is important because hanami-view uses the inflector to determine the class names for your view parts, and it’s just plain table stakes for an framework to apply inflections consistently (especially if you’ve configured custom inflection rules).

The implementation within hanami-view was quite interesting, because it was the first time I had to adjust an ApplicationConfiguration (this one being exposed as config.views on the Hanami::Application subclass) to hide one of its base settings. In this case, it hides the inflector setting because we know it will be configured with the application’s inflector as part of the ApplicationView behaviour (to refresh your memory, ApplicationView is a module that’s mixed in whenever Hanami::View is subclassed within a namespace managed by a full Hanami application).

Ordinarily, I’m all in favour of exposing as many settings as possible, but in this case, it didn’t make sense for a view-specific inflector to be independently configurable right alongside the application inflector itself.

Rest assured, you don’t lose access to this setting entirely, so if you ever have reason to give your views a different inflector, you can go right ahead and directly assign it in a view class:

module Main
  class View < Hanami::View
    # By default, the application inflector is configured

    # But you can also override it:
    config.inflector = MyCustomInflector
  end
end

There was a counterpart hanami PR for this change, and I was quite happy to see it all done, because it means we now have consistent handling of both action and view settings: each gem provides their own ApplicationConfiguration class, which is made accessible via config.actions and config.views respectively. This consistency should make it easier to maintain both of these imported configurations going forward (and, one day, to devise a system for any third party gem to register application-level settings).

Application views have their template configured always

One aspect of the ApplicationView behaviour is to automatically configure a template name on each view class. For example, a Main::Views::Articles::Index would have its template configured as "articles/index".

This is great, but there was an missing piece from the implementation. It assumed that your view hierarchy would always include an abstract base class defined within the application:

module Main
  # Abstract base view
  class View < Hanami::View
  end

  module Views
    module Articles
      # Concrete view
      class Index < View
      end
    end
  end
end

Under this assumption, the base view would never have its template automatically configured. That makes sense in the above arrangement, but if you ever wanted to directly inherit from Hanami::View for a single concrete view (and I can imagine cases where this would make sense), you’d lose the nice template name inference!

With this PR, this limitation is no more: every ApplicationView has a template configured in all circumstances.

Application views are configured with a Part namespace

Keeping with the theme of improving hanami-view integration, another gap I’d noticed was that application views are not automatically configured with a part namespace. This meant another wart if you wanted to use this feature:

require "main/views/parts"

module Main
  class View < Hanami::View
    # Ugh, I have to _type_ all of this out, _by hand?_
    config.part_namespace = Views::Parts
  end
end

Not any more! As of this PR, we now have a config.views.parts_path application-level setting, with a default value of "views/parts". When an ApplicationView is activated, it will take this value, convert it into a module (relative to the view’s application or slice namespace), and assign it as the view’s part_namespace. This would see any view defined in Main having Main::Views::Parts automatically set as its part namespace. Slick!

Security-related default headers restored

Sticking with configuration, but moving over to hanami-controller, Hanami::Action subclasses within an Hanami app (that is, any ApplicationAction) now have these security-related headers configured out of the box:

X-Frame-Options: DENY
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Content-Security-Policy: it’s long, just go check the code

These are set on the config.actions.default_headers application-level setting, which you can also tweak to suit your requirements.

Previously, these were part of a bespoke one-setting-per-header arrangement in the config.security application-level setting namespace, but I think this new arrangement is both easier to understand and much more maintainable, so I was happy to drop that whole class from hanami as part of rounding this out this work.

Automatic cookie support based on configuration

The last change I made to hanami-controller was to move the config.cookies application-level setting, which was defined in the hanami gem, directly into the config.actions namespace, which is defined inside hamami-controller, much closer to the related behaviour.

We now also automatically include the Hanami::Action::Cookies module into any ApplicationAction if cookies are enabled. This removes yet another implmentation detail and piece of boilerplace that users would otherwise need to consider when building their actions. I’m really happy with how the ApplicationAction idea is enabling this kind of integration in such a clean way.

Check out the finer details in the PR to hanami-controller and witness the corresponding code removal from hanami itself.

Released a minimal application template

It’s been a while now since I released my original Hanami 2 application template, which still serves as a helpful base for traditional all-in-one web applications.

But this isn’t the only good use for Hanami 2! I think it can serve as a helpful base for any kind of application. When I had a colleague ask me on the viability of Hanami to manage a long-running system service, I wanted to demonstrate how it could look, so I’ve now released an Hanami 2 minimal application template. This one is fully stripped back: nothing webby at all, just a good old lib/ and a bin/app to demonstrate an entry point. I think it really underscores the kind of versatility I want to achieve with Hanami 2. Go check it out!

Gave dry-types a nice require-time performance boost

Last but not least, one evening I was investigating just how many files were required as one of my applications booted. I noticed an unusually high number of concurrent-ruby files being required. Turns out this was an unintended consequence of requiring dry-types. One single-line PR later and now a require "dry/types" will load 242 fewer files!

Savouring this moment

It’s taken quite some doing to get to this moment, where an Hanami 2.0.0.alpha2 release finally feels feasible. As you’d detect from my previous posts, it’s felt tantalisingly close for every one of the last few months. As you’d also detect from this post, the final stretch has involed a lot of focused, fiddly, and let’s face it, not all that exciting work. But these are just the kind of details we need to get right for an excellent framework experience, and I’m glad I could continue for long enough to get these done.

I’m keenly aware that there’ll be much, much more of this kind of work ahead of us, but for the time being, I’m savouring this interstice.

In fact, I’ve even given myself a treat: I’ve already started some early explorations of how we could adapt dry-system to fit with zeitwerk so that we can reliable autoloading a part of the core Hanami 2 experience. But more on that later ;)

Thank you to my sponsors!

I now have a sponsors page on this here site, which contains a small list of people to whom I am very thankful. I’d really love for you to join their numbers and sustain my open source work.

As for the next month, new horizons await: I’ll start working out some alpha2 release notes (can you believe it’s been nearly 2 years of work?), as well as continuing on the zeitwerk experiment.

See you all again, same place, same time!

Open source status update, September 2020

2020-10-06T11:32:00+00:00

Well, didn’t September just fly by? Last month I predicted I’d get through the remaining tasks standing in the way of an Hanami 2.0.0.alpha2 release, and while I made some inroads, I didn’t quite get there. At this point I’ve realised that after many consecutive months of really strong productivity on OSS work (which for me right now is done entirely on nights and weekends), a downtick of a couple of months was inevitable.

Anyway, let’s take a look at what I did manage to achieve!

Reintroduced CSRF protection module to hanami-controller

Sometime during the upheaval that was hanami and hanami-controller’s initial rewrite for 2.0.0, we lost the important CSRFProtection module. I’ve brought it back now, this time locating it within hanami-controller instead of hanami, so it can live alongside the action classes that are meant to include it.

For now, you can manually include it in your action classes:

require "hanami/action"
require "hanami/action/csrf_protection"

class MyAction < Hanami::Action
  include Hanami::Action::CSRFProtection
end

And if you need to manually opt out of the protections for any reason, you can implement this method in any one of your action classes:

def verify_csrf_token?(req, res)
  false
end

Either way, I encourage you to check out the code; it’s a simple module and very readable.

Started on automatic enabling of CSRF protection

For a batteries included experience, having to manually include the CSRFProtection module isn’t ideal. So I’m currently working to make it so the module is automatically included when the Hanami application has sessions enabled. This is close to being done already, in this hanami-controller PR and this counterpart hanami PR. I’m also taking this an an opportunity to move all session-related config away from hanami and into hanami-controller, which I think is a more rational location both in terms of end-user understandability and future maintainability.

We’ll see this one fully wrapped up in next month’s update :)

Improving preservation of state in dry/hanami-view context objects

This one was a doozy. It started with my fixing a bug in my site to do with missing page titles, and then realising that it only partially fixed the problem. I wasn’t doing anything particularly strange in my site, just following a pattern of setting page-specific titles in individual templates:

- page_title "Writing"

h1 Writing
  / ... rest of page

And then rendering the title within the layout:

html
  head
    title = page_title

Both of these page_title invocations called a single method on my view context object:

def page_title(new_title = Undefined)
  if new_title == Undefined
    [@page_title, settings.site_title].compact.join(" | ")
  else
    @page_title = new_title
  end
end

Pretty straightforward, right? However, because the context is reinitialized from a base object for each different rendering environment (first the template, and then the layout), that @page_title we set in the template never goes anywhere else, so it’s not available afterwards in the layout.

This baffled me for a quite a while, because I’ve written similar content_for-style helpers in context classes and they’ve always worked without a hitch. Well, it turns out I got kinda lucky in those cases, because I was using a hash (instead of a direct instance variable) to hold the provided pieces of content, and since hashes (like most objects in Ruby) are passed by reference, that just so happened to permit the same bits of content to be seen from all view context instances.

Once I made this relisation, I first committed this egregious hack just to get my site properly showing titles again, and then I mulled over a couple of options for properly fixing this inside hanami-view.

One option would be to acknowledge this particular use case and adjust the underlying gem to support it, ensuring that the template context is used to initialize the layout context. This works, and it’s certainly the smallest possible fix, but I think it papers over the fundamental issue here: the the creation of multiple context instances is a low-level implementation detail and should not be something the user needs to think about. I think a user should feel free to set an ivar in a context instance and reasonably expect that it’ll be available at all points of the rendering cycle.

So how do we fix this? The obvious way would be to ensure we create only a single context object, and have it work as required for rendering the both the template and the layout. The challenge here is that we require a different RenderEnvironment for each of those, so the correct partials can be looked up, whether they’re called from within templates, or within part or scope classes. This is why we took the approach of creating those multiple context objects in the first place, so each one could have an appropriate RenderEnvironment provided.

So how do we keep a single context instance but somehow swap around the underlying environment? Well, as a matter of fact, there’s a gem for that. After discovering this bug, I was inspired and stayed up to midnight spiking on an approach that relies upon dry-effects and a reader effect to provide the differing render_environment to a single context object.

(The other effect I felt was the extreme tiredness the next day, I’m not the spritely youth I used to be!)

Anyway, if you haven’t checked out dry-effects, I encourage you to do so: it may help you to discover some novel approaches to certain design challenges. In this case, all we need to do is include the effect module in our context class:

module Hanami
  class View
    class Context
      # Instance methods can now expect a `render_env` to be available
      include Dry::Effects.Reader(:render_env)
    end
  end
end

And ensure we’re wrapping a handler around any code expected to throw the effect:

module Hanami
  class View
    module StandaloneView
      # This provides `with_render_env`, used below
      include Dry::Effects::Handler.Reader(:render_env)

      def call(format: config.default_format, context: config.default_context, **input)
        # ...

        render_env = self.class.render_env(format: format, context: context)
        template_env = render_env.chdir(config.template)

        # Anything including Dry::Effects.Reader(:render_env) will have access to the
        # provided `template_env` inside this handler block
        output = with_render_env(template_env) {
          render_env.template(config.template, template_env.scope(config.scope, locals))
        }

        # ...
      end
    end
  end
end

With this in place, we have a design that allows us to use a single context object only for entirety of the render lifecycle. For the simplicity to the user, I think this is a very worthwhile change, and I plan to spend time assessing it in detail this coming month. As Nikita (the author of dry-effects) points out, there’s a performance aspect to consider: although we’re saving ourselves some object allocations here, we now have to dispatch to the handler every time we throw the reader effect for the render_env. Still, it feels like a very promising direction.

Filed issues arising from production Hanami 2 applications

Over the month at work, we put the finishing touches on two brand new services built with Hanami 2. This helped us to identify a bunch of rough edges that will need addressing before we’re done with the release. I filed them on our public Trello board:

This goes to show how critical it is for frameworks like Hanami to have real-world testing, even at these very early stages of new release development. I’m glad I can also serve in this role, and grateful for keenness and patience of our teams in working with cutting edge software!

Fixed accidental memoization of dry-configurable setting values

Last but not least, I fixed this bug in dry-configurable that arose from an earlier change I made to have it evaluate settings immediately if a value was provided.

This was a wonderful little bug to fix, and the perfect encapsulation of why I love programming: we started off with two potentially conflicting use cases, represented as two different test cases (one failing), and had to find a way to satisfy them both while still upholding the integrity of the gem’s overall design. I’m really happy with how this one turned out.

🙌🏼 Thanks to my sponsors!

This month I was honoured to have a new sponsor come on board. Thank you Sven Schwyn for your support! If you’d like to give a boost to my open source work, please consider sponsoring me on GitHub.

See you all next month!

Open source status update, August 2020

2020-08-31T12:25:00+00:00

Oh, hello there, has it been another month already? After my bumper month in July, August was a little more subdued (I had to devote more energy towards a work project), but I still managed to get a few nice things done.

Hanami session configuration back in action

In a nice little surprise, I realised that all the building blocks had fallen into place for Hanami’s standard session configuration to begin working again.

So with a couple of lines of config uncommented, Luca’s “soundeck” demo app has working cookie sessions again. Anyone pulling from my Hanami 2 application template will see the same config enabled after this commit, too.

Container auto-registration respects application inflector

Another small config-related changed I made was to [pass the Hanami 2 application inflector]((https://github.com/hanami/hanami/pull/1069) through to to the dry-system container handling component auto-registration.

With this in place, if you configure a custom inflection for your app, e.g.

module MyApp
  class Application < Hanami::Application
    config.inflector do |inflections|
      inflections.acronym "NBA"
    end
  end
end

Then it will be respected when your components are auto-registerd, so you can use your custom inflections as part of your module namespacing.

With the setup above, if I had a file called lib/my_app/nba_jam/cheat_codes.rb, the container would rightly expect it to define MyApp::NBAJam::CheatCodes.

I’m delighed to see this in place. Having to deal with awkward namespaces (e.g. SomeApi instead of SomeAPI) purely because the framework wasn’t up to the task of handling it has long been an annoyance to me (these details matter!), and I’m really glad that Hanami 2 will make this a piece of cake.

This outcome is also a testament to the design approach we’ve taken for all the underpinning dry-rb gems. By ensuring important elements like an inflector were represented by a dedicated abstraction - and a configurable one at that - it was so easy for Hanami to provide its own inflector and see it used wherever necessary.

Customisable standard application components

Every Hanami 2 application will come with a few standard components, like a logger, inflector, and your settings. These are made available as registrations in your application container, e.g. Hanami.application["logger"], to make them easy to auto-inject into your other application components as required.

While it was my intention for these standard components to be replaceable by your own custom versions, what we learnt this month is that this was practically impossible! There was just no way to register your own replacements early enough for them to be seen during the application boot process.

After spending a morning trying to get this to work, I decided that this situation was in fact pointing to a missing feature in dry-system. So I went ahead and added support for multiple boot file directories in dry-system. Now you can configure an array of directories on this new bootable_dirs setting:

class MyContainer < Dry::System::Container
  config.bootable_dirs = [
    "config/boot/custom_components",
    "config/boot/standard_components"
  ]
end

When the container locates a bootable component, it will work with these bootable_dirs just like you’d expect your shell to work with its $PATH: it will search the directories, in order, and the first found instance of your component will be used.

With this in place, I updated Hanami to to configure its own bootable_dirs and use its own directory for defining its standard components. The default directory is secondary to the directory specified for the application’s own bootable components, so this means if you want to replace Hanami’s standard logger, you can just create a config/boot/logger.rb and you’ll be golden!

Started rationalising flash

Last month when I was digging into some session-related details of the framework, I realised that the flash we inherited from Hanami 1 was pretty hard to work with. It didn’t seem to behave in the same way we expect a flash to work, e.g. to automatically preserve added messages and make them available to the next request. The code was also too complex. This is a solved problem, so I looked around and started rationalising the Hanami 2 flash system based on code from Roda’s flash plugin. I haven’t had the chance to finish this yet, but it’ll be first cab off the rank in September.

Plans for September

With a concerted effort, I think I could make September the month I knock off all my remaining tasks for a 2.0.0.alpha2 release. It’s been tantalisingly close for a while, but I think it could really happen!

Time to get stuck into it.

🙌🏼 Thanks to my sponsors!

Lastly, my continued thanks to my little posse of GitHub sponsors for your continued support, especially Benjamin Klotz.

I’d really love for you to join the gang. If you care about a healthy, diverse future for Ruby application developers, please consider sponsoring my open source work!

Open source status update, July 2020

2020-08-03T12:30:00+00:00

July was a great month for my work on Hanami!

After a feeling like I stalled a little in June, this time around I was able to get to the very end of my initial plans for application/action/view integration, as well as improve clarity around comes next for our 2.0 efforts overall.

Getting closer on extensible component configuration

Whenever I’ve worked on integrating the configuration of the standalone Hanami components (like hanami-controller or view) into the application core, I’ve asked myself, “if the app author chose not to use this component, would the application-level configuration still make sense?” I wanted to avoid baking in too many assumptions about hanami-controller or hanami-view particulars into the config that you can access on Hanami.application.config.

In the long term, I hope we can build a clean extensions API so that component gems can cleanly register themselves with the framework and expose their configuration that way. In the meantime, however, we need to take a practical, balanced approach, to make it easy for hanami-controller and hanami-view to do their job while still honouring that longer-term goal in spirit.

I’m happy to report that I think I’ve found a pretty good arrangement for all of this! You can see it in action with we we load the application.config.actions configuration:

module Hanami
  class Configuration
    attr_reader :actions

    def initialize(env:)
      # ...

      @actions = begin
        require_path = "hanami/action/application_configuration"
        require require_path
        Hanami::Action::ApplicationConfiguration.new
      rescue LoadError => e
        raise e unless e.path == require_path
        Object.new
      end
    end
  end
end

With this approach, if the hanami-controller gem is available, then we’ll make its own ApplicationConfiguration available as application.config.actions. This means the hanami gem itself doesn’t need to know anything else about how action configuration should be handled at the application level. This kind of detail makes much more sense to live in the hanami-controller gem, where those settings will actually be used.

Let’s take a look at that:

module Hanami
  class Action
    class ApplicationConfiguration
      include Dry::Configurable

      # Define settings that are _specific_ to application integration
      setting :name_inference_base, "actions"
      setting :view_context_identifier, "view.context"
      setting :view_name_inferrer, ViewNameInferrer
      setting :view_name_inference_base, "views"

      # Then clone all the standard settings from Action::Configuration
      Configuration._settings.each do |action_setting|
        _settings << action_setting.dup
      end

      def initialize(*)
        super

        # Apply defaults to standard settings for use within an app
        config.default_request_format = :html
        config.default_response_format = :html
      end

      # ...
    end
  end
end

This configuration class:

(a) Defines settings specifically for the Hanami::Action behaviour activated only when used within a full Hanami app
(b) Clones the standard settings from Hanami::Action::Configuration (which are there for standalone use) and makes them available
(c) Then tweaks some of the default values of those standard settings, to make them fit better with the full application experience

This feels like an ideal arrangement. It keeps the ApplicationConfiguration close to the code in ApplicationAction, which uses those new settings. It means that all the application integration code can live together and evolve in sync.

Further, because Hanami::Action::ApplicationConfiguration exposes a superset of the base Hanami::Action::Configuration settings, we can make it so any ApplicationAction (i.e. any action defined within an Hanami app) automatically configures every aspect of itself based on whatever settings are available on the application!

So for the application author, the result of all this groundwork should be a blessedly unsurprising experience: if they’re using hanami-controller, then they can go and tweak whatever settings they want right there on Hanami.application.config.actions, both the basic action settings as well as the integration-specific settings (though most of the time, I hope the defaults should be fine!).

When we do eventually implement an extensions API, we can at that point just remove the small piece special-case code from Hanami::Application::Configuration and replace it with hanami-controller reigstering itself and making its settings available.

If you’re interested in following these changes in more detail, check out hanami/hanami#1068 for the change from the framework side, and then hanami/controller#321 for the ApplicationConfiguration and hanami/controller#320 for the self-configuring application actions. (I also took an initial pass at this in hanami/hanami#1065, but that was surpassed by all the changes linked previously - I took small steps, and learnt along the way!)

I also made matching changes to view configuration. All the same ideas apply: if you have hanami-view loaded, you’ll find an Hanami.application.config.views with all the view settings you need, and then application views will self-configure themselves based on those values! Check out hanami/hanami#1066 and hanami/view#176 for the implementation.

Fixed `handle_exception` inside actions

One of the settings on Hanami::Action classes is its array of config.handled_exceptions, which you can also supply one-by-one through the config.handle_exception convenience method.

It turns out another handle_exception still existed as a class method, clearly an overhang of the previous action behaviour. I took care of removing that, so now there should be no confusion whenever action authors configure this behaviour (especially since the old class-level method didn’t work with inheritence).

Automatically infer paired views for actions

Believe it or not, the work so far only took me about half-way through the month! This left enough time to roll through all my remaining “minimum viable action/view integration” tasks!

First up was inferring paired views for actions. The idea here is that if you’re building an Hanami 2 app and following the sensible convention of matching your view and action names, then the framework can take care of auto-injecting an action’s view for you.

So if you had an action class like this:

class Main
  module Actions
    module Articles
      class Index < Main::Action
        include Deps[view: "views.articles.index"]

        def handle(request, response)
          response.render view
        end
      end
    end
  end
end

Now, you can drop that include Deps[view: "…"] line. A matching view will now automatically be available as the view for the action!

This works even for RESTful-style actions too. For example, an Actions::Articles::Create action would have an instance of Views::Articles::New injected, since that’s the view you’d want to re-render in the case of presenting a form with errors.

If you need it, you can also configure your own custom view inference by providing your own Hanami.application.config.actions.view_name_inferrer object.

To learn more about the implementation, check out the PR and then this follow-up fix (in which I learnt I should always write integration tests that exercise at least two levels of inheritance).

Automatically render an action’s view

With the paired view inference above, our action class is now looking like this:

class Main
  module Actions
    module Articles
      class Index < Main::Action
        def handle(request, response)
          response.render view
        end
      end
    end
  end
end

But we can do better. For simple actions like this, we shouldn’t have to write that “please render your own view” boilerplate.

So how about just this?

class Main
  module Actions
    module Articles
      class Index < Main::Action
      end
    end
  end
end

Now, any call to this action will automatically render its paired view, passing through all request params for the view to handle as required.

And the beauty of this change was that, after all the groundwork laid so far, it was only a single line of code!

As Kent Beck has said, “for each desired change, make the change easy (warning: this may be hard), then make the easy change.” The easy change indeed. Moments like these are why I love being a programmer :)

Integrated, application-aware view context, and some helpers

Let’s keep going! This month I also gave the “automatic application integration” treatment to Hanami::View::Context. Now when you inherit from this within your application, it’ll be all set up to accept the request/response details that Hanami::Action is already passing through whenever you render a view from within an action.

With these in place, we’re now providing useful methods like session and flash for use within your view-related classes and templates. If you want to add additional behaviour, you can now access request and response on your application’s view context class, too.

While I was doing this, I also took the opportunity to hash out some initial steps towards a standard library of view context helpers with an Hanami::View::ContextHelpers::ContentHelpers module. If you mix this into your app’s view context class, you’ll also have a convenient content_for method that works like you’d expect. Longer term, I’ll look to move this into the hanami-helpers gem and update the existing helpers to work with the new views, including providing a nice way to opt in to whatever specific helpers you want your application to expose.

In the meantime, check out all this fresh view context goodness here.

Hanami 2.0 application template is up to date

After all of this, I took a moment to update my Hanami 2 application template. If you create an app from this template today, all the features I’ve described above will be in place and ready for you to try. I also enabled rack session middleware in the app, because this is a requirement for the flash and session objects as well as CSRF protection.

Hanami 2.0 Trello board

Last but not least, as I was finally seeing some clear air ahead, I took a chance to bring our Hanami 2.0 Trello board up to date!

As it currently stands, I have just 7-8 items left before I think we’ll be ready for the long-awaited Hanami 2.0.0.alpha2 release.

Beyond that, I hope the board will help everyone coordinate the remainder of our work in preparing 2.0.0. At very least, I’m already feeling much better knowing we’re a little more oranized, with a single, up-to-date place where it’s easy to see what’s next as well as add new items whenever we think of them (I’ve no doubt that plenty more little things will crop up).

Plans for August

So that was July. What. A. Month.

I tell you, I was exceedingly happy to have finally completed my “get views and actions properly working together for the first time” list, which turns out to have taken the better past of five months.

For August, I plan to knock out as many of my remaining 2.0.0.alpha2 tasks. Some of them are pretty minor, but one or two are looming a little larger. We’ll see how many I can get through. One thing I’m accepting more and more is that when open sourcing across nights and weekends, patience is a virtue.

Thanks for sticking with me through this journey so far!

🙌🏼 Thanks to my sponsors… could you be the next?

I’ve been working really hard on preparing a truly powerful, flexible Ruby application framework. I’m in this for the long haul, but it’s not easy.

If you’d like to help all of this come to fruition, I’d love for you to sponsor my open source work.

Thanks especially to Benjamin Klotz for your continued support.

Philly.rb talk on hanami-view 2.0

2020-07-14T10:35:00+00:00

Last month I had the honour of speaking at Philly.rb’s first remote meetup!

I took the opportunity to revise my year-ago talk about dry-view and update it for our current plans for hanami-view 2.0.

You can watch along here:

Doing a talk like this for a fully-remote audience was tricky. With its narrative structure and dramatic premise, I rely on audience cues to make sure I’m hitting the mark. Here, I just had to passionately project into the void… and hope it landed!

Fortunately, the Philly.rb crew was great, and the “after show” was where the remote format really excelled. A bunch of great questions came up, and I was able to jump into screen-sharing mode and do an interactive session on how both view rendering and the broader framework will hang together for Hanami 2.0.

All up, I really enjoyed the session. Thank you Ernesto for facilitating it! With our limited travel possibilities right now, I’d love to do this a little more: please reach out if you’d be interested in me contributing to your remote conference or meet-up.

Open source status update, June 2020

2020-06-28T11:05:00+00:00

After last month’s breakthroughs with the Hanami view rendering, I was looking forward to “rolling downhill… and collecting a bunch of quick wins.” I was unfortunately a little over-optimistic there, but in June I did manage to get a few nice things done. Let’s take a look.

Hanami application actions with deeper class hierarchies

To start off the month, I upgraded one of my work applications to use last month’s Hanami::Action improvements. And it turns out, this revealed a shortcoming! In this application, we have multiple tiers of “base actions”, with one for the whole application, and then one for each slice:

module MyApp
  # Application base action
  class Action < Hanami::Action
  end
end

module Main
  # Slice base action, inheriting from application base
  class Action < MyApp::Action
  end
end

My first pass at the “application action” work didn’t account for this, and it resulted in the slice’s base action (Main::Action in the example above) not receiving the proper application-specific behaviour.

This is a great example of how important it is to “dogfood” in-development frameworks and libraries like this, and it’s why I made a conscious decision to be both a ”super user” and core developer of all the tools we’ve been developing in dry-rb, rom-rb, and now Hanami. The feedback you get from really using them is invaluable, and the ensuring feedback cycle means that we can fold in related improvements really quickly.

In this case, I needed to make this adjustment to hanami-controller and this one over here to hanami. And with that, this application can now continue to run on the cutting edge :)

Class-configurable Hanami actions

With those changes done, I turned my attention to my major focus for the month. I adjusted Hanami::Action so that it is now class-configurable, just like Hanami::View.

Here’s how you can now configure an action:

class MyAction < Hanami::Action
  config.default_response_format = :json
end

With this in place, when you instantiate the action, the configuration from its class will automatically apply:

action = MyAction.new
action.(rack_env) # Default response format will be JSON

What’s useful about this approach is that the configuration is inherited, which creates the opportunity for a base action to hold common configuration for all its subclasses:

class BaseAction < Hanami::Action
  config.default_response_format = :json
end

class AnotherAction < BaseAction
end

action = AnotherAction.new
action.(rack_env) # Default response format will be JSON

If you’ve been following along with my recent updates, you’ll see where this is going. With this inheritable class-based configuration in place, we’ll be able to leverage the “Application Action” behavior that is seamlessly added to Hanami::Action subclasses when they’re defined within a full Hanami application, and in this case, apply all the necessary action configuration from the framework.

That’s the plan for July. We’ll see how far we can go. There’s still a few things to work out, like how we can allow any Hanami::Action setting to be configured for the whole application, without having to duplicate every setting into the application-level Hanami::Configuration and the related classes.

The pull request for this work is now done and should hopefully merge soon. One thing I was quite happy with was how I managed to make the switch from configuration being applied as an externally injected object to the class-based configuration without having to upend the entire test suite. I made this possible by keeping the injected configuration object, and making the class-based configuration its default parameter:

module Hanami
  class Action
    module StandaloneAction
      def new(configuration: self.configuration.dup, **args)
        allocate.tap do |obj|
          obj.instance_variable_set(:@configuration, configuration.finalize!)

          # Other details snipped out... let me know if you'd like to hear the
          # story behind ths whole new/allocate dance :)
        end
      end
    end
  end
end

Not only did this allow me to sidestep an overhaul of the test suite, but it also retained a wonderful flexibility: if for any reason you need a particular action instance to behave differently from its class’ default configuration, you can still pass in your own configuration object. I call that a win!

An aside on component ”fit” within dry-system and Hanami 2

A big theme of my work so far with Hanami 2 has been making Hanami’s own components fit just right with dry-system, which manages the application and slice containers. With dry-system as it currently stands, this means:

Each source file should be entirely self-contained. A single require for that file should bring in enough of the outside world for the class defined therein to be fully functional.

Each class should work with a simple .new. Instead of requiring the container’s coponent loader to somehow satisfy a whole range of various initializers, each class should provide sensible defaults, such that a simple .new is enough to get a working instance. This means a couple of things:

Firstly, all injected dependencies should have working defaults. This is how dry-auto_inject (i.e. the include Deps["some_dep"] we’ll see inside Hanami application components) works: the specified dependencies are resolved from the container and effectively become the default arguments for the class’ #initialize parameters.

Next, any standard configuration should be already in place, without any additional argument passing. This is exactly why we moved the Hanami::Action config onto the class, so that SomeAction.new can already have the configuration it needs.

If you’re already working with dry-system, or designing components to fit well with dry-system or eventually Hanami 2, these characteristics would be good to keep in mind.

dry-configurable settings eagerly evaluate their value

To preserve the behavior of various Hanami action settings, I needed to make a small change to dry-configurable.

It’s been a few months now since Piotr entirely rewrote dry-configurable, and in my view, the effort has been a smashing success: I think the code is far easier to understand and work with. Thanks Piotr!

Since the rewrite, we’ve had to make a few little adjustments as we’ve discovered additional use cases out in the wild, and this was one of them. We needed to make it a setting’s constructor would run immediately when provided a value, ensure we could provide immediate feedback in the case of an invalid value.

So given this setting:

setting :default_response_format do |format|
  Utils::Kernel.Symbol(format) unless format.nil?
end

We can now expect an error to be raised if the provided value cannot be symbolized:

# Will raise an exception!
config.default_response_format = 123

Thanks to the well-factored code of the rewrite, the required change was very small, and now we have the best of both worlds when it comes to dry-configurable evaluating its settings: when a value is provided, the value will run through the setting’s constructor immediately, which provides the early feedback we want in situations like the above. When a value is not yet provided, the constructor doesn’t run, waiting until a value is later provided or until the whole configuration is finalized, which is useful behavior for when a configuration object takes some time to be fully prepared.

Thanks to my sponsors 🙌🏼

As of this week, I now have five GitHub sponsors! Thank you, sponsors: I’m ever grateful for all your support.

If you’d like to pitch in and support my open source work, you can sponsor me here.

Thanks especially to Benjamin Klotz for your continued support.

Thanks for reading!

The winter solstice has now passed here in Australia, so while the nights are getting shorter, I’ll still be pushing hard through the evenings to try and reach this critical ”minimum viable actions/views” milestone for Hanami 2. See you all at the end of July! 👋🏼

Rubyists, we must do better

2020-06-08T05:05:00+00:00

Ruby: come for the language, stay for the community. That’s the story I tell people, anyway.

Yesterday, however, we saw some of the worst in our community.

An issue was raised with the RuboCop team about changing the name in light of the race-related police brutality issues (and more) being tackled in the USA right now.

Bozhidar Batsov, the project author and lead maintainer, declined the request and closed the issue, noting how disruptive it would be to rename something so widely used, and claiming that he wanted to keep “politics out of software.” A few days later, several follow-up comments were made, taking various angles and urging him to reconsider his position. A couple of forks were made without notice. Everything argued until this point at least looked to be in good faith.

Now, all of those recent events happened while Bozhidar was asleep (he’s in Bulgaria, while most of the commenters looked to be in the US). That’s a lot to wake up to, I’m sure (and a snarky tweet or two likely stung the most). I don’t know Bozhidar, though I’m sure it’s safe to say we both love Ruby, and we also both wear sunglasses in our social media avatars, which I appreciate. What I do know, however, is that he’s poured hundreds, maybe thousands, of hours of free labour into this project, so I can fully appreciate how shocking this must have felt to him. He shared this reaction on the GitHub issue.

What happened next was truly horrifying: a vile, toxic pile-on, ruthlessly villifying the folks who were in good faith trying to make the software more welcoming (I won’t link to the comments in the issue, nor other comments on social media, many of which are equally irredeemable).

This was online bullying, pure and simple. And online bullying kills. There is no place for this in our community.

Bozhidar eventually locked the issue, but as RuboCop lead, he’s yet to make any statement about the behaviour of some of those who spoke in “support” of him (I’m sure he has a lot going on, and I’ve reached out to him about this privately).

Update: Bozhidar has now published his thoughts on the matter, and I’m grateful for his conscientious leadership here.

As for me, let me also speak this, as one representative of the Ruby community: There is no place for this vitriol here. This behaviour is NOT welcome in ANY form.

There are many ills that plague the world right now, and whether or not you think software naming changes have a role to play in improving things — I do — it should at least be clear that one easy way to make a better world is to bring our best selves into every situation. And if you feel you can’t do this, consider just giving it five minutes.

I notice that RuboCop, as of about 18 hours ago, now has a Code of Conduct:

This document provides a few simple community guidelines for a safe, respectful, productive, and collaborative place for any person who is willing to contribute to the RuboCop community. It applies to all “collaborative spaces”, which are defined as community communications channels (such as mailing lists, submitted patches, commit comments, etc.).

Participants will be tolerant of opposing views.

Participants must ensure that their language and actions are free of personal attacks and disparaging personal remarks.

When interpreting the words and actions of others, participants should always assume good intentions.

Behaviour which can be reasonably considered harassment will not be tolerated.

Unforunately, there’s no clear recourse specified for those who do not abide by these guidelines, which I feel is a critical element for a Code of Conduct to ensure a truly welcoming space. However, even in the absence of this, I still sincerely hope that it brings about materially healthier interactions within the RuboCop community in future.

To Bozhidar: thank you for everything you’ve contributed to the Ruby and other open source communities. You’ve personally made our community a better place. I’m sorry many aspects of this were hard for you, but I also hope you can help to visibly steer things back in a positive direction from here.

To everyone else in the Ruby community: “Matz is nice so we are nice” is not a pithy aphorism. It’s a bare minimum, a mere starting point. We can and should aim for better.

Open source status update, May 2020

2020-06-01T06:30:00+00:00

May was a breakthrough month in terms of the integration of the standalone components into Hanami 2. Let’s dig right in.

Seamless Hanami view integration

Last month I wrote about my first pass at integrating Hanami view classes with application they exist within. It looked like this:

module Main
  # "Base" view class for `main` slice
  class View < Hanami::View[:main]
  end

  module Views
    class Articles
      # Class for a specific view, inheriting from base
      class Index < Main::View
      end
    end
  end
end

In this approach, inheriting from Hanami::View[:main] would tell the subclass to apply its configuration using details from the main slice. It worked fine, but there’s still some redundancy there:

module Main                         # <- We're clearly in the `main` slice
  class View < Hanami::View[:main]  # <- So why do we have to repeat it here?
  end
end

In trying to get this stuff done for the next alpha release, Luca has definitely been encourating a pragmatic approach to getting things in place (“perfect is the enemy of good,” or in this case “shipped”). It’s the right way to go, but even still, this nagged at me, especially given our goal of reducing boilerplate as much as possible.

After a while I realised that the application itself could provide a facility to help us out in this situation. Given a class like Main::View, and given that each slice ”owns” a specific namespace, there is enough information in the class name alone to infer which slice it belongs to. So now we have this:

Hanami.application.component_provider(Main::View)
#=> #<Hanami::Slice:0x00007fc2ae074568
# @application=Soundeck::Application,
# @booted=true,
# @container=Main::Container,
# @name=:main,
# @namespace=Main,
# @namespace_path="main",
# @root=#<Pathname:/Users/tim/Source/hanami/soundeck/slices/main>>

Pass in a class or instance, and get its slice. Easy. With this in place, we can use it from the .inherited hook of Hanami::View to get all the information we need for truly seamless integration:

module Main
  class View < Hanami::View
  end
end

Look, zero boilerplate!

But this is only half of the seamless view integration story: now that we can infer the slice that provides a given view, how do we add the slice-specific behaviour, especially given we’re still inheriting directly from Hanami::View, which still needs to be able to provide the behaviour for standalone (non-integrated) views?

Our original approach for this was also pragmatic: when we need a subclass of a given Hanami component (like Hanami::View or Hanami::Action) to behave differently within an Hanami application versus when used standalone, we would just monkey patch the application-specific behaviour. Now, anyone who knows me would know this isn’t approach I would not tolerate for long. 😉 Even still, I was willing to do it for the sake of expedience. You can see the approach (and all my misgivings about it) in lib/hanami/action/extensions/application_action.rb in my proof of concept action integration PR.

But as is the theme of this section, it nagged at me. What we really needed here was for the patched methods providing the integration specialisations to be able to call super to get to the standalone behaviour wherever they needed. With a monkey patch, this isn’t possible because you end up completely replacing the methods (or having to resort to hacky “alias method chain”-style approaches).

One way to solve this would be to have a deeper inheritance chain (Hanami::ApplicationView < Hanami::View) and using different superclasses for integrating views versus standalone views, but that bifurcates view usage in an unfriendly way, and more likely than not would make one of those two use cases more awkward than the other.

At this point I realised Ruby gives us another option for this: modules! What we needed here were two different modules in the ancestor chain for a given view class, with the “nearest” one providing the application integration behaviour (e.g. [Main::Views::Articles::Index, Main::View, ApplicationView, StandaloneView]), and the next one back providing the standard standalone behaviour. This way, the application integration module can add only the specialisations it requires, and can call super whenever it needs.

The final piece to this puzzle is to make it so that the ApplicationView module can provide behaviour that’s specific to a given slice. This is where the module builder pattern comes in. Instead of this ApplicationView module being a plain old static module, we can initialize it with the slice object that we get when we’re subclassing Hanami::View in the first place.

So with this in place, here’s what Hanami::View and its .inherited roughly look like:

require_relative "view/application_view"
require_relative "view/standalone_view"
# ...

module Hanami
  class View
    include StandaloneView

    # ...

    def self.inherited(subclass)
      super

      # If inheriting directly from Hanami::View within an Hanami app, configure
      # the view for the application
      if subclass.superclass == View && (provider = application_provider(subclass))
        subclass.include ApplicationView.new(provider)
      end
    end

    def self.application_provider(subclass)
      if Hanami.respond_to?(:application?) && Hanami.application?
        Hanami.application.component_provider(subclass)
      end
    end
    private_class_method :application_provider
  end
end

And the resulting ancestors for an actual view class:

Main::Views::Articles::Index.ancestors
# => [
#   Main::Views::Home::Index,
#   Main::View,
#   #<Hanami::View::ApplicationView[main]>,
#   Hanami::View,
#   Hanami::View::StandaloneView::InstanceMethods,
#   Hanami::View::StandaloneView,
#   # ...
# ]

These are exactly the number of different places we need to neatly slot in all the behaviour for our truly seamless view integration!

The resulting arrangement has some other nice benefits, too, because the integration logic has now moved out from the hanami gem and over into the hanami-view gem itself:

It makes hanami-view easier to maintain, because the the standalone and integration view behaviours can be seen side by side and kept in sync.
It also represents some first steps towards ensuring there isn’t just a single “blessed” view system, by making it clear that components should integrate themselves with the application framework, rather than the other way around.

I was chuffed with how this all worked out, and I’m much, much happier with the overall arrangement now. Hats off to Ruby for being such a flexible language! Check out the full hanai-view PR for this new integration approach, as well as the corresponding integration hooks (and reduction in view integration code!) inside the hanami gem.

New `Hanami.application?` check

A small subtle thing you might have noticed above was that check for Hanami.application?. This is another hook I added to make it easier for components to integrate (or not) with an Hanami application. Because many of the Hanami components can be used on their own (hanami-view, hanami-router, and hanami-controller in particular), the Hanami namespace will definitely exist, but not necessarily a full Hanami.application. This Hanami.application? check provides a safe way to determine if an application has been defined before activating any integration code.

Right now this is defined directly on the Hanami module by the hanami gem, but we’ll also be adding it to hanami-utils so you can safely use it without having to require the full application gem.

Seamless view rendering within Hanami actions

All the polishing of the view integration was a warm-up for the main game this month: properly implementing the integration of view rendering into Hanami actions.

This was the approach we agreed upon after our experiments last month:

class Index < Main::Action
  include Deps[view: "views.articles.index"]

  def handle(req, res)
    # Views are rendered by the response
    res.render view
  end
end

And as of a few days ago, the work is complete!

That single res.render view belies a lot of underlying logic. What we do with this integration is provide the view with all the request-specific data that it might need to render itself, things like the current session, flash messages, CSRF token, etc. This is all set up automatically for you as soon as you inherit from Hanami::Action within an existing Hanami application.

Sound familiar? That’s because we follow the exact same integration approach for actions as we do for views. Hanami::Action now has a StandaloneAction module providing the basic functionality, and an ApplicationAction module that is initialized with the action’s slice, so it can pick up whatever details it needs from the slice or application to provide the view rendering integration.

The crux of the integration is the action setting up a view context object with the request/response pair created when the action is called. This view context is automatically passed to the view when res.render is called. Having the request/response pair available to the context means that the context object can provide methods to make those details like the flash available for use within the view templates, scopes, and parts (these are links to dry-view documentation, since at this point, hanami-view unstable and dry-view 0.7 are effectively the same).

This integration is eminently flexible. There are multiple points at which an application author can customise it. The first would be to add their own methods to the view context class within their application (I’ll show examples of this this next month). The next would be to override any of these methods within their base action class:

#view_options, to pass additional options to every view as it is rendered
#view_context_options, to pass additional options to the view context
#build_response, to customize the response object that is prepared for passing to the action’s #handle method (it’s unlikely this will need to be customised, but it’s nice to keep all options open)

I think these methods are a perfect example of the approach we’re taking with Hanami 2 development: conveniences by default, but every possible measure available to adjust things when you need to diverge from the defaults.

Next steps for actions and views

This month was all about laying the proper groundwork for action and view integration. With this done, my plan for June is to roll through all these steps to round it out:

Making Hanami::Action configuration class-based, like we have for view configuration, so we can auto-configure actions based on their slice
Making actions automatically infer a paired view (think a “views.article.index” view for an Actions::Articles::Index) so you don’t need to explicitly auto-inject them
Then going a step further, making it so that actions automatically render their paired view too, meaning you don’t not even need to implement #handle for basic render-only actions
Creating an Hanami::View::ApplicationContext context class with a default set of helpful methods for use within all Hanami views, including those that require access to the request (as described above)
Making the view context container identifier configurable

With Hanami 2, while we’re building upon many years worth of open source efforts in terms of the existing libraries we’re pulling together, it’s become very clear to me through this process that doing good integration work is just as much effort all over again. Thanks for your patience while we work through this as best we can.

Thanks to my sponsors 🙌🏼

Earlier this month I announced that I joined GitHub sponsors, and since then, two more kind people began sponsoring me. Thank you so much Samuel Williams and Thomas Klemm!

And a big shout out to Benjamin Klotz for your continuing support 😄

Thanks for reading!

This post turned out to be a big one! The fact that I had so much to say speaks to just how pivotal a month this was. I’m looking forward to the next few weeks of rolling downhill from here and collecting a bunch of quick wins. See you all again at the end of June! 👋🏼

Sponsor me on GitHub

2020-05-21T13:40:00+00:00

I’ve joined GitHub’s sponsors program.

Here’s my pitch:

I love Ruby. I’ve worked with Ruby for nearly 20 years, and even after all this time, it still brings me joy!

My mission in open source is to ensure the joy of Ruby extends to the experience of writing real, maintainable applications, for teams of all sizes.

For me, this comes from following a hybrid FP/OOP approach to writing Ruby, and for the last 6+ years, I’ve been helping to build tools to make this easy for application developers.

I’m a co-founder and core team member of the dry-rb project, and a core team member of the rom-rb and Hanami projects.

Right now, I’m heavily involved in the effort to build Hanami 2.0, which represents a new model for Ruby open source collaboration, and will provide a true framework of the future for Ruby application developers.

I do the majority of my open source work as my “second job,” on nights and weekends. Your kind support will help sustain me in these efforts.

Thank you! 🙏🏼

I love contributing to the Ruby OSS ecosystem. But a labour of love is still a labour. Right now I‘m working hard towards building Hanami 2, and have come to realise this is the single biggest effort I’ve made in OSS so far. There’s a lot to do, and it’s taking many hours over evenings and weekends.

This is where GitHub sponsorships come in. My plan for these is not to take my OSS work full-time or anything like that. I love the job I already have and I’ll be continuing my OSS contributions regardless. What the sponsorships will provide are small, tangible gestures of thanks that will go a long way towards making my OSS efforts feel sustainable.

I’m delighted to report that these really do make a difference, because I already feel it from my first two sponsors! Thank you so much, Benjamin and Jason!

As an added means of giving back, I’m using this a motivation to write more here, both summarising the work I’m doing in OSS each month as well as delving deeper into some of the software design ideas I’ve been kicking around all these years.

If any of this sounds good to you, I’d appreciate your support. Thank you very much!

Interviewed on GitHub’s “Open Source Friday” video series

2020-05-21T12:45:00+00:00

Last week I appeared on GitHub’s fledgeling “Open Source Friday” Twitch stream, where I joined Brian Douglas to discuss the dry-rb project. It was wide-ranging conversation: we discussed dry-rb in general, why I think it helps you write better code, and how to get started as a user and contributor. We also touched on the role dry-rb will play for Hanami 2, Ruby open source in general, and the many positive aspects of Ruby and its community that continue to bring me joy.

It was a fun chat, and I think it gives a good sense about why we do everything that we do. Go have a listen!

Sharing my Hanami 2 application template

2020-05-07T11:40:00+00:00

We’ve been hard at work building Hanami 2 for a while now, but the truth is, it’ll be a while still until it’s truly ready.

In the meantime, I’ve put together an Hanami 2 application template that reflects how I’m currently using various Hanami 2 components within the applications I develop day-to-day.

It currently consists of:

The Hanami 2 application core
Hanami router
Hanami controller and view manually integrated into the app (along with some helpers for rendering views within actions)
rom-rb configured to use a Postgres database
A bin/run CLI with some extra commands to help you work with the database (db create, db migrate, db create_migration, etc.)
A little static assets manager built using Webpack, along with an assets helper object for use within views
A fully configured RSpec setup

If you want to begin exploring the ideas and features of Hanami 2, all while the framework is still under active development, this template is a great way to get started! As the framework improves, I’ll keep this template updated alongside it.

Fair warning: this template comes with no documentation and no support, but hopefully the code can serve as a helpful guide for those willing to dig around.

Please go ahead and check out the template! I’m excited to share this early look into the Hanami 2 app development experience. Even at this early stage, it’s a framework that gives me great joy to use every day. 🌸

Open source status update, April 2020

2020-04-30T12:50:00+00:00

Hello again after another month! I’m excited to turn these monthly updates into a habit, and I’m equally as excited to share my progress with you!

Hanami basic view integration

I started off the month by merging the beginnings of my Hanami 2 view integration work.

This makes it possible for views to configure themselves based on the application and slice in which they’re situated, and the result is zero boilerplate view classes:

module Main
  class View < Hanami::View[:main]
  end

  module Views
    class Articles
      class Index < Main::View
      end
    end
  end
end

In this example, the Views::Articles::Index view will have a matching template name configured ("articles/index"), as well as the appropriate paths to find that template within the Main slice’s directory.

This is a great start, but it’s not the end of the view integration story for Hanami 2!

View integration with Hanami actions

The next step in this journey is making it nice to actually go ahead and render these views within Hanami actions, as well as making sure that various request-specific details (like CSRF tokens, flash messages, and even just the request path) are available to views that require them.

This was a tough nut to crack, but I think it also demonstrates just how grateful I am to be collaborating with Luca on all of this.

I started off this effort by copying over the very light-touch view/action integration that I’d already established within the Hanami 2 apps I’ve built so far. This consisted of a few extra helper methods inside the application’s base action, allowing you to render a view like this:

module Main
  module Actions
    module Articles
      class Index < Main::Action
        include Deps[view: "views.articles.index"]

        def handle(req, res)
          render req, res, page: req.params[:page]
        end
      end
    end
  end
end

This worked okay, but it felt awkward in a couple of ways. Having to pass the extra req, res to every render could quickly become laborious. It also didn’t fit well in PUT or POST actions, where you’re typically either redirecting (via res.redirect_to "...") or rendering (in this case via render) based on the success of your action. Having to operate on the res in one case and then via a helper directly on self in the other just didn’t feel balanced.

Luca encouraged me to find a better approach. After various circuitious discussions squeezed into our paltry ~1 hour Canberra/Rome daily overlapping time window, we decided it best to sketch out some complete-ish working implementations in code. This helped us land on this:

class Create < Main::Action
  include Deps[view: "views.articles.index"]

  def handle(req, res)
    if create_article_somehow(req.params[:article])
      res.redirect_to "/admin/articles"
    else
      res.render view, validation: errors_go_here
    end
  end
end

This gives us a much nicer symmetry, with both redirecting and view rendering being a capability of the response (internally, the rendering just boils down to calling the view object and assigning its output to the its body), and a little less line-noise, since we also established a way to ensure the res object has all the request-specific details that a view may need.

What I also like about this implementation is that action classes still very much retain their ”what you see is what you get” quality. They are very much coordinators of application domain services, existing simply to handle the HTTP-related aspects of the web interface, and passing control to the other services as required. Views become just another element in this mix, handled like any other dependency of an action.

It’s still very early days for this view/action integration, but we’re happy enough with the direction such that I can now begin finessing it and finishing it, hopefully over the month of May.

The work so far (which is messy, be warned!) is spread across PRs in both hanami/controller and hanami/hanami. I’ve also made corresponding adjustments to our soundeck demo app to show this rendering approach in action, as well as various ways you can test actions that render views.

Various Hanami things: CLI env handling, dotenv improvements, middleware fix

Last week I also made some small fixes and tweaks to the nascent Hanami 2 CLI: I made it so the application boot process triggered via CLI actions was deferred late enough such that the -e flag (to set the HANAMI_ENV) is properly respected (important if you want to migrate your actual test database!). Related, I expanded the range of files we load via Dotenv for application settings, so you can have your .env.test and ~~eat it too~~ have it actually respected when running CLI commands with -e test.

I also stumbled across and fixed a bug that prevented Rack middlewares from being mounted in the Hanami router if their #initialize accepted only a single app arg. This is why it’s good to have a real everyday app running on in-progress frameworks like this; such things get noticed and fixed much more quickly!

Bonus OSS: development environment tooling

One more thing: I was glad to have the opportunity to bring several elements of Icelab’s former shared development environment into our new Culture Amp team:

Our shared dotfiles
Our Homebrew commands tap, which includes bootstrap-developer-system to install the development environment, including the dotfiles, and bootstrap-asdf, which takes care of installing the range of asdf-managed tools we use (including the finnicky verification of the nodejs package signatures)
And our scripts to rule them all, which implements the normalized script pattern for every one of our apps, making it possible for the app to install

After a decade of trying various shared development environments, and now working with this approach for the last couple of years, I’m still incredibly happy with this combo: it’s very light touch and the components are all very simple, making the whole thing a breeze to maintain. I’m happy it can live on and continue serve our team within a new company!

Did I mention I also gave my own dotfiles a refresh recently too?

Shout out to my GitHub sponsor!!

I’ll write about this in more detail soon, but here’s the short story: it dawned on me recently that helping to bring Hanami 2 into fruition will be a long slog, and that I could do with all the encouragement I could get. So I signed up for GitHub sponsors.

Within a week I already have one sponsor: thank you so much to Benjamin Klotz for your support! Receiving that first “You have a sponsor” email was truly a joy 😄.

See you next month 👋🏼

Well, that’s it for April! I’ll be spending May focused on getting that Hanami view/action integration done. It’s going to take some doing, but once it’s in place it’ll represent a major step forward for the 2.0 effort.

See you back here again next month!

Open source status update, March 2020

2020-03-27T11:48:00+00:00

Inspired by Piotr’s open source status update, I thought I’d begin a similar practice (and get back into the swing of blogging, since this year’s effectively a write-off for in-person meetings).

I hope to do this monthly, but since this is the first one, let me recap the whole of the last year or so! Here are the highlights:

dry-view 0.6

In late 2018 and early 2019 I built and released dry-view 0.6.0, which contained all the abstractions I thought it needed for 1.0. Then I gave a talk about it and my philsophy on view layer design in general.

Not much happened after that. I needed a bit of time to recover from my usual intense talk preparation and the blur of conference-driven-development that preceded it. Still, for the rest of the year I was a very satisfied user of dry-view 0.6 within my Icelab applications.

Hanami core application/slices structure

Later in 2019 I spent a while refining Snowflakes, the ersatz framework we developed at Icelab for our own dry-rb/rom-rb/roda-based applications. Most of my work was about enshrining an application/sub-application structure and reducing the amount of boilerplate required for dry-system to provide this.

This turned out to be a prescient effort, because I was able to pivot this approach into the application and slices structure that will form the core of the Hanami framework for its upcoming 2.0 release.

Working on this was a supremely rewarding experience, because it represented a true meeting of the minds between Luca, me, and the other dry-rb team members. We each brought our own experiences to bear on this problem, and through a good amount of back and forth and give and take, we found a way to make what I think will both be a convenient, low-ceremony framework structure that at the same time remains extremely powerful, flexible, and cleanly structured.

We’re still working on it, and time will tell, but I’m feeling good about it!

Hanami application settings

Andrew Croome and I got together at Rails Camp Kyneton in November last year and plotted how to add first-class support for application settings to Hanami. Andrew made the broad strokes that weekend, and then I followed up with some adjustments, and I merged our co-authored PR a couple of weeks ago.

Around that same time, solnic released an amazingly simplified complete rewrite of dry-configurable, and we realised it would serve as a useful underpinning for the Hanami application settings and could also open up some new possibilities like nested settings. I’ve been working since then to prepare the way for this, roughing the changes into Hanami and fixing a couple of things with dry-configurable along the way. There’s a few more things to do here, and we’re expecting these last changes to culminate with a release of dry-configurable 1.0.

{dry,hanami}-view renaming and application integration

I’ve temporarily put these dry-configurable adjustments on the backburner, however, so I can focus on some other things we more pressingly need for the next Hanami alpha release: integration of views and actions with the larger framework.

For 2.0, we’ve always intended that dry-view would serve as the view layer for Hanami. The big news here is that we’re now renaming dry-view to hanami-view! Yes, just as hanami-cli became dry-cli to improve the coherence of the gems that make up our two organisations, dry-view will be moving up into the Hanami org to sit alongside hanami-router and hanami-controller as the three main tentpoles of the web application stack.

These renaming decisions have been very easily made, which I think goes to show just how aligned we all are on building an understandable and complementary ecosystem of next-gen Ruby OSS tools :)

I’ve done the renaming and the new code is up in the master branch of hanami/view, with the default branch for the project being renamed to “1.x-master” until we’re done with the 2.0 development.

Now I’m working on making it so that views can sit within an Hanami application with zero boilerplace. This is my current work-in-progress effort. The goal is to have it so that your view can be as simple as this:

module Main
  module Views
    class MyView < Main::View
    end
  end
end

And work out of the box with all appropriate conventions and any required configurations pulled from its slice and the broader Hanami application.

And like any well-behaved component within a dry-system application, it works with dry-system’s auto-registration, which means the view will be fully functional and behave consistently when resolved directly from the container:

Main::Slice["views.my_view"]

Or auto-injected into an action:

module Main
  module Actions
    class MyAction < Main::Action
      include Deps["views.my_view"]
    end
  end
end

Or even instantiated directly, like you might do in a test:

RSpec.describe Main::Views::MyView do
  subject(:my_view) { described_class.new }
end

As you can see from the PR, there’s not actually all that much required to get the views integrated nicely. What’s left for me a this point is to tidy it up, consider what view configuration might need to exist at the application-level, and then make sure we’re doing everything we can to ensure hanami-view isn’t inextricably tied to the framework: I want to make sure that hanami-view could be switched out for some other view renderer and have the experience feel just as smooth.

Once this is done, the approach for action integration will be very similar, so I hope I can roll through that one relatively quickly!

See you next month 👋🏼

Well, I think this has been a pretty good first effort at a monthly update! I hope it’s helpful for anyone wanting to know a little more about what’s going on in my little corner of the Ruby OSS world.

I’m very excited by where things are going, and after a topsy turvy last 6 months for me personally (closing Icelab, staring a new job, etc.), I’m feeling well situated to resume regular work in the OSS sphere, so I’m hoping I’ll have a good few things to report on next month, too.

See you then!

Speaking at RubyConf Thailand

2019-03-19T10:00:00+00:00

I have some very exciting news: I’ll be speaking at RubyConf Thailand in September! This will be the first ever Ruby conference in Bangkok, and it’s an honour to be part of it.

It’s been a thrill every time I could join up with the Ruby community in the Asia Pacific region. With events like RedDotRubyConf currently taking a break, I’m relishing another chance to head back to warmer climes, to make new friends, and to learn about all the interesting work that people are doing.

With Michael Kohl and crew running the conference, I have no doubt it will be a fantastic event. I hope to see you there! (Hint, the CFP is open until June)

Podcast interview: Remote Ruby

2019-03-18T10:55:00+00:00

A few weeks back I had the pleasure of participating in my first podcast interview! I joined Jason and Chris for Episode 30 of Remote Ruby. We talked about my getting started with programming, then some of the origins and motivations behind Icelab, dry-rb, and rom-rb. It was a lot of fun!

In fact, while you’re there, I’d recommend just going ahead and subscribing to the whole Remote Ruby podcast. Jason and Chris are gracious hosts, enthusiastic about Ruby and its future, and have a knack for getting interesting guests: every time a new episode comes out, it goes straight to the top of my queue.

I first met Jason when I visited Nashville for Southeast Ruby in 2017. Even in the midst of his final-day conference preparations (and he was organizing this thing solo!), he took a couple of hours to hang out with me and nerd out on all things Ruby. This exemplifies the kind of person he is, and I’m really glad that he and Chris are bringing this same brand of hospitality to the community at large through their podcast.

Talking “Views, from the top”

2019-03-17T10:55:00+00:00

dry-view 0.6.0, an introductory talk, and plans for 1.0

2019-02-12T09:00:00+00:00

Speaking at RubyConf AU 2019

2018-11-22T10:00:00+00:00

I’m thrilled to be speaking at next year’s RubyConf AU in Melbourne!

I’ll be addressing the humble server-rendered view, in a talk entitled Views, from the top:

/Veni, vidi, view-ci/ – I came, I saw, I left the views in a mess.

With server-rendered HTML still delivering most of the web, our views deserve more than a grab bag of helpers.

Come along and learn the tools and techniques to conquer the design challenges that a complex view layer presents.

This topic has been brewing in my mind (and in various codebases) for a good two to three years now, so I’m especially excited to share it. I hope to present a compelling case for better architected views, and offer a good reminder that this is a layer in our web apps that is still ripe for improvements.

The conference’s speaker lineup is still emerging, but it’s already packed with amazing-sounding talks. I hope to see you there!

Talking functional architecture at RedDotRubyConf

2017-06-30T00:00:00+00:00

Icelab talks at RubyConf AU 2017

2017-03-06T00:00:00+00:00

A Rodakase retrospective: 1 year of dry-rb in production

2017-02-23T00:00:00+00:00

2016 in review

2017-01-08T21:41:28+00:00

I had a good run of year in review posts, but fell off the bandwagon lately. It’s time to change that. Before I dive into 2016, here’s a recap of the intervening years:

2013: Around the world tickets in hand, Misch and I worked, volunteered, and played Japan, Vietnam, Hong Kong, the USA, Finland, Germany, the UK, Spain, and Italy. An amazing time! Attended my first (and only) WWDC and had a blast. Started working on Decaf Sucks 2.0 (you’ll hear more about that much later).

2014: Settling back in Canberra and realising we could live for a long time in our (large by world standards) apartment, we renovated a little: new floors, paint, curtains. Made it feel like a whole new place. Misch and I gave birth to Clover, our best and most satisfying team effort yet.

2015: Took our first with-kid overseas trip, and cruised through to Clover’s first birthday and our first parenting anniversary (which we celebrated with a giant bánh mì party). Icelab gathered for FarmLab, and we discussed alternatives-to-Rails for the first time. Our grandmothers both passed away, and we spent time with the extended family. Jojo and I held Rails Camp in Canberra in December, where we got to eat cake for Rails’ 10th birthday and watch Star Wars with 70 of our friends. Misch and I got pregnant again but sadly lost the little baby at 7 weeks.

Phew. That was some time. Now onto 2016.

At home with the (expanded!) family

Losing a baby at the end of 2016 was a big thing, but thankfully it came at a time when work and other demands scale back, so Misch and I spent some good quality time together and could regroup.

We got a couple of big things done in the beginning of the year. First up, we bought a car! After two years of mostly car-free life, it was time for another way to get around the place. Our little Škoda Fabia does just that, and is fun to drive.

Next, we renovated our bathroom! Knowing we’ll be living here for many years to come, this was a big and worthwhile upgrade to our home amenity. We splashed out and got a Toto washlet, too. I regret nothing.

And in the last big thing for 2016, we became pregnant again and gave birth to baby Iris Persephone in October. This time around, the room at the Birth Centre was brimming with family. We wanted Clover there, so along came Misch’s parents too. Clover’s excited cry of “Baby!” upon seeing Iris come into the world is something I’ll always remember. Iris’ arrival brought another 6 weeks of time at home, which I enjoyed even more now that we’re a family of four.

Decaf Sucks 2.0

With Misch’s encouragement, I returned to my long stalled effort to release our all-new 2.0 version of Decaf Sucks. Turned out it didn’t need all that much; with just a couple of weeks of effort, Max and I got everything wrapped up and released it to the world. It was a weight from my shoulders and I’m happy to finally have it out there.

dry-rb

2016 brought a seismic shift in how I write Ruby applications. After some experimentations with rom-rb and Piotr Solnica’s rodakase experiment late in 2015, I knew this was my future. So I dove in and contributed as much as I could to the fledgling set of libraries now known as dry-rb. And we got a lot done. We released a whole bunch of gems, made things “official” with the launch of a website and discussion forum, and expanded the core team of developers to 5.

Along with sharing code, I wanted to start sharing some of the thinking behind the dry-rb style of Ruby app development, so I set about blogging, and managed to publish once a week for a good few months. This culminated with an introductory talk I gave at RedDotRubyConf in Singapore. This was my first conference talk and I relished the opportunity to really polish a particular message. Luckily, I was able to build upon this a repeat performance at Rails Camp in Adelaide and at a Ruby community workshop over in Perth. No doubt, you can expect to hear plenty more from me about dry-rb in 2017 :)

Icelab

Icelab kicked off 2016 by celebrating our 10th birthday! I think we’ve built a remarkable little company and work-home to many good people, and I think the next 10 years will be even better.

For me, most of 2016 at Icelab was spent getting us settled onto dry-rb and rom-rb as our preferred stack for server-side applications. We shipped our first production app with these all the way back in February, launched our new website as an open source example app in June, and we have several more big sites that’ll see the light of day early in 2017. It took a little while to get over the knowledge and productivity hump, but I feel we’ve hit a good rhythm with the stack now, and given we’re the long-term maintainers of most of the things we ship, it’ll be something that I expect will pay dividends for many years to come.

Open source was another big theme for the year. Along with our ongoing contributions to dry-rb, we took an “open source first” approach to any other standalone, reusable pieces of code we wrote. This small shift was a big help in making better design choices right from the beginning. You’ll be able to see some of this bear fruit when we take our advanced form builder to 1.0 next year. It’s already been an incredibly useful tool across our client projects.

I’m also proud that Icelab began contributing to the open source infrastructure that powers Ruby apps everywhere through our contributions to Ruby Together, which we joined in 2016 as Australia’s first Emerald member.

And all the rest

And now I’ll collect everything else I could think of into a few broadly categorised lists:

Computer life:

I’ve removed Twitter apps from all my platforms. It’s helped me focus.
Said goodbye to gentlyremind.me, the little Rails app I’ve been running to email me my Twitter favourites. Now that IFTTT can do that same thing, I’m happy to have one less running thing I have to worry about.
Sometime in June I surpassed 250,000 all time Icelab chat messages.
Mulled many times with my co-workers on how we could run a better kind of tech meet-up in Canberra. Maybe this year!

Software development:

Continued my love/hate relationship with Docker, but I think now I’ve managed to find the right place for it in our development life: standardised production environments, and local dev only when we have to run something unusual.
After uncountable years, I’m finally looking away from Heroku as our production environment of choice.
Shipped a production iOS app built using Turbolinks for iOS, and it turned out rather nicely. I’d be happy to play with it some more.
We settled on Attache as a standard handler for all our file uploads. I feel it is a smart architectural choice (and I was happy to meet its affable creator Choon Keat in Singapore!)
We started to build Danger into our CI builds. It’s already helpful, and I think we’re just scratching the surface.
time_math2 is a great little Ruby library and a wonderful archetype for how “expressive” Ruby libraries can be made without Rails-style monkey patches.

Physical things:

The Mizudashi cold brew coffee pot I picked up to celebrate the launch of Decaf Sucks 2.0 makes amazing coffee and I’ve been putting it to good use ever since the weather warmed up. I’m aiming for 100% uptime of cold brew all summer long.
The Minaal daily shipped from their Kickstarter campaign and it immediately became my every day carry. A great companion to their larger carry-on bag.

Apps:

After trying innumerable things and never settling, I’ve finally found a home for all my writing (pieces long, short, random or otherwise): it’s Ulysses. What a great app.
Castro 2 came out with an ingenious new mechanic and I’m very happy to continue using it. It’s helped my jump onto a few new podcasts without the worry of managing them.
CarPlay is great. I’ll readily admit this was a deciding factor in our new car choice and I wasn’t disappointed.
Paw is now my one-stop shop for all my HTTP requestin’. Super polished.
I’m back on good old Mail.app and happy to ignore all the we’ll-host-your-mail-and-your-passwords offerings that continue to swirl around.

Books, film, TV, etc.

Ripped through quite a bit of fiction as I waited for Clover to sleep (happily now she does this on her own). Highlights: Seveneves, Proxima & Ultima, The Prefect, Aurora and the Wool trilogy.
I look at my Letterboxd profile and once again resolve to watch more cinema. Anyway, 2016’s highlights were The Big Short, Hunt for the Wilderpeople, Easy A, Arrival, Crazy, Stupid Love and of course Rogue One.
Subscribing to Netflix has been great. And fits perfectly well with our no-TV household.

Random:

Cooked all the Filipino food I could think up. It was great to have this as a motivating theme behind all my cooking.
And I tried toast and yoghurt for breakfast for the first time. Guess there’s always time for new firsts ¯_(ツ)_/¯.

Announcing Decaf Sucks 2.0

2016-09-07T00:00:00+00:00

Next-generation Ruby web apps with dry-rb, rom-rb, and Roda: RedDotRubyConf 2016

2016-06-28T00:00:00+00:00

Conversational rom-rb, part 2: types, associations, and update commands

2016-06-10T00:00:00+00:00

A conversational introduction to rom-rb

2016-06-03T00:00:00+00:00

Put HTTP in its place with Roda

2016-05-24T00:00:00+00:00

A change-positive Ruby web application architecture

2016-05-19T00:00:00+00:00

Better code with an inversion of control container

2016-05-10T00:00:00+00:00

Effective Ruby dependency injection at scale

2016-05-03T00:00:00+00:00

Functional command objects in Ruby

2016-04-26T00:00:00+00:00

Inactive records: the value objects your app deserves

2016-04-19T00:00:00+00:00

My past and future Ruby

2016-04-14T00:00:00+00:00

New podcasts

2014-11-05T21:36:52+00:00

I’ve enjoyed a couple of new podcasts lately:

Slate’s Working podcast finds people in interesting jobs and interviews them about their workdays. It’s brand new. The first episode – with Stephen Colbert – was fantastic. The show is short and dense. David Plotz as the host (along with some helpful editing, I’m sure) gets the guest talking (they usually have a lot to say) and then gets out of the way. I’ve appreciated finding another non-tech podcast to keep in my roster.

I still have plenty of room for good tech podcasts, though. Thoughtbot have just launched another new podcast, called The Bike Shed, covering their general experiences in web development. This looks like it will be a discussion show with regular hosts Sean Griffin and Derek Prior. They seem humble and grounded, and the first show on Sandi Metz’ rules was thoughtful, and directly applicable to my work as a web developer. I’m still thinking over what they shared. I’m also appreciative they’ve kept the show to under 30 minutes. This makes it easy to cover on a walk into work!

Our Problem with Boxen

2014-05-18T06:05:05+00:00

Over a year ago, we started using Boxen at Icelab to manage our team’s development environments. It’s worked as advertised, but not well enough. I’m looking for something else.

Our problem with Boxen is that it contains hundreds of different components, all of which are changing, all the time. It needs constant tending. By the time a team is big enough to need something like Boxen, it’s paradoxically too small to look after it properly: to have someone whose job it is to be “Boxenmaster,” someone who knows how these intricate parts all interact and can run the regular clean installs needed to make sure that the experience is smooth for new people (we’ve found it typical for a functional Boxen setup to stay working over repeated updates, while clean installs end up failing). We need a tool that we can set up once for a project and then have it still work reliably 6 months later when we revisit the project for further work.

Boxen can be a trojan horse. If you don’t look after it properly, you risk creating two classes of developers within your team: those who were around to get Boxen working when you first rolled it out, now enjoying their nice functional development environments, and those who get started later, whose first experience in the team is one of frustration, non-productivity and a disheartening, unapproachable web of interconnected packages. And no one is safe: as soon as you want to upgrade that version of PHP, you could slip down there too.

So while Boxen can indeed work as manager of personal development environments, and while Puppet is a serious and capable configuration manager, my recommendation is not to commit to Boxen half-heartedly. In fact, you may not even need something so rigorous and complex if your team is still small. I’m looking into such alternatives right now. I’m not settled on anything yet, but it’ll likely involve virtual machines, and perhaps use Docker as a way to keep it lightweight while working on multiple projects at a time. Fig seems like it could be a good fit. I’ll keep you posted.

Decaf Sucks 2: A New Old Design

2013-08-23T23:18:33+00:00

Decaf Sucks 2: Starting Over

2013-08-10T16:21:41+00:00

Decaf Sucks 2 Is Coming, 2 Hours at a Time

2013-08-05T14:14:44+00:00

Sneaking Into the Everyday

2013-07-18T16:04:35+00:00

Levelling Up With AngularJS: Building a Reusable Click to Edit Directive

2013-07-01T05:36:19+00:00

Two i's

2013-06-17T01:26:34+00:00

I think it’s always good to be working on two things: The next most important thing, and the next most interesting thing.

– Jason Fried

Things Get Fuzzy Fast: Examine Your Day So You Can Learn From It

2013-06-04T03:01:55+00:00

A Manageable Multi-Database Redis Development Setup

2013-04-16T01:32:39+00:00

There is No Lab

2013-03-12T21:30:00+00:00

Announcing Decaf Sucks 1.1

2013-01-10T08:00:00+00:00

It’s been a while since we’ve talked about Decaf Sucks on the iPhone. Our first release has run smoothly for a year and a half, helping many thousands of iPhone owners find better coffee. Thanks to our well caffeinated contributors, nearly half of our reviews are now submitted from the app.

On this note, we’re very excited to announce the release of Decaf Sucks version 1.1 in the App Store. This release makes it even easier to contribute reviews with the addition of Facebook login support (now available wherever Decaf Sucks exists).

We’ve also added support for the iOS 6 and the larger iPhone 5 display, as well as giving the app some extra polish all around. In particular, our overseas friends can rest easy, because the distance to your favourite cafes is now reported in miles or kilometres based on the device’s region format setting.

Decaf Sucks 1.1 is still free and waiting for you in the App Store. Get it now and find some great coffee near you!

Read on for a full list of all the things we’ve changed.

New features:

Login with Facebook to write reviews
Support for the larger iPhone 5 display
Distance units are localised into miles or kilometres based on the device’s settings
iOS 6 compatibility updates

Other improvements:

When setting a cafe’s location, show a callout above the map pin to make it clear that it can be dragged around.
When setting a cafe’s location, ensure that the pin is actually draggable on the first tap.
When setting a cafe’s location, always show a proper full address after dragging the pin around.
Show a proper non-retina image for the back button’s pressed state.
Fix a bug that sometimes caused the map view to start very zoomed out.
When a review is completely empty, make sure the “Post” button stays disabled.
Ensure the map for a single cafe encompasses the current location, if it is nearby.
Fix a bug that led to some reviews being set to Hyderabad, India. Yes, this was really weird.

Decaf Sucks 1.1 is available in the App Store now.

My Boring Adventure

2013-01-06T03:30:00+00:00

Austin Kleon’s Steal Like An Artist, his manifesto for creativity in the digital age, is full of useful tips, with one that stands out for me:

Be boring. (It’s the only way to get work done.)

While an inspired spark may be source of creative work, it’s the drudgery of thorough execution that’s necessary to bring it to completion. Even the smallest projects are an accumulation of many tiny details, each requiring time to get right. To do this, you need to show up and work, day after day. Being boring enables this.

But you can be boring and have adventure too. The way I’ve done this is to combine work and travel. Not the frenetic “I’ve got to squeeze as much as possible into my annual leave” kind of travel, but months at a time in a new place. This allows you the time to develop the routine needed for consistent, productive work, alongside many opportunities to explore your new home at a comfortable pace. Every lunch break affords you a chance to see something new, every outing for coffee the chance to become a regular in a new neighbourhood. Andy Warwick puts it excellently in this Quora thread: “the mundane becomes an adventure when you live in a foreign land.”

And when you wrap up your work for the night or for the week, there are hours and days ahead of you for adventure and discovery, which can inform and fuel your otherwise perfectly boring working life.

Written from Adelaide, Australia, where I’m living for two months.

2012 in Review

2013-01-01T05:20:00+00:00

A life abroad

This was a year marked by two new worlds: Garmisch and I spent six months living in the Philippines and two in Hong Kong. We experienced the full stretch of the urban spectrum; from the wide blue skies, leisurely pace (and mild chaos) of a provincial tropical island, through to Hong Kong’s unending bustle and its dense, concrete verticality.

It was our first considerable stretch of time overseas. Living in the relative isolation did bring some challenges, but we overcame them, and in doing so we became a closer couple and hardier humans. And we’ve left the experience with a new perspective on what’s important, and on how we run our lives from here (Hint: there’s still a little more exploring to do!).

This wasn’t the year’s only travel, either. The long stretches abroad were punctuated by visits elsewhere: Wellington in February for the magnificent Webstock, Australia in April for a group hug with my workmates, Singapore in May for RedDotRubyConf, and Tasmania in November for Railscamp 12. And outside all of this, with our own place rented out, we spent some quality time staying with our parents, firstly in Canberra and now in Adelaide.

A bigger, better Icelab

It was a big year for Icelab. In January, we had five people: our Canberra office and Max on his own in Melbourne. In March, we merged with Inventive Labs, welcomed Narinda, Toby and Ally to the team, and assumed their office space in The Ironmongers on Brunswick Street. In August, Melissa joined us in Melbourne as a project manager. In September, David joined us in Canberra as a developer. In December, we moved out of our (then very crowded) office in Canberra into a bigger, much more comfortable space just a few blocks away. We finished the year with nine amazing people and permanent offices in both Canberra and Melbourne. And all of this while I was mostly overseas. A distributed team can really work.

The larger team meant we put out more work than ever. Plenty to be proud of, and plenty I wish I could have been more involved in. That said, I did get to ship a few large projects this year:

PDF e-ticket support for Ticketscout
The Thousands, with Max Wheeler & Anthony Kolber
Youcamp, with Toby Allder and Michael Honey, along with contributions from almost everyone in the lab

Some other work endeavours

I had less time than usual this year to work on side-projects, but I did manage to ship gentlyremind.me in February. It sends you a daily email of your recently favourited tweets. I use it everyday and it’s nice to see my friends do the same.

I also found a little time to take a look into RubyMotion for building iOS apps. I put together an introductory talk and presented at a Canberra Ruby meeting and at Railscamp 12, where I also ported the bulk of the Decaf Sucks iOS app to RubyMotion in just a day or so. I expect I’ll spend a lot more time with this in the future.

Some new apps

A notable trend that emerged this year was my increased use of activity-tracking apps. Films I watched went in the lovely Letterboxd. My Decaf Sucks page continued to grow as I explored new cities café-by-café. In Hong Kong, I started using Foursquare so I could record the eateries and other interesting places I visited, and it’s stuck with me since then. I tried to take note of my general activities using Day One on both the Mac and iOS. It hasn’t quite become habitual yet, but anything I record there is a bonus. Finally, Rdio has become a daily source of musical wonder; my iTunes library is long since deleted.

Getting to the point

One of the freshest, foremost things in my mind about this year is a hard slog in the last six months; Icelab was growing, and there were was just a big backlog of work that had to be done. To help, I bore down and worked harder and longer than ever before. We got past it, but it came at some personal cost. I lost opportunities for spending time with my wife, for exercise, exploring my new locations, and my own creative work.

After all of that, though, I feel we’ve arrived in a good position to create a more sustainable workload in the future, and I haven’t lost any of my passion for creating things. If I still feel like this now, I know it won’t change, and I know that I’ll continue to put lots of time into it. What I have learnt is how it feels to do it out of some kind of obligation, and I’ll do my best to ensure I don’t fall into that position again.

But the biggest story, and the story worth remembering, is that I could do all of this — the hard work included — while still sharing wildly new experiences and building a stronger relationship with Garmisch. I couldn’t think of a better friend and travel companion, and I’m ever thankful for her company and support.

I’ve finished the year with a stronger sense than ever of empowerment and direction. 2013 will be big!

Out My Window

2012-09-16T00:30:00+00:00

The view out the window in my previous and current overseas offices: Talisay City (Negros Occidental, Philippines) and Sheung Wan (Hong Kong).

From a Good Idea and Persistence Came Sneakers

2012-09-10T21:30:00+00:00

Stephen Tobolowsky shares some behind the scenes memories of Sneakers, an all-time favourite film of mine, and finishes with his thoughts on scriptwriter Phil Alden Robinson:

Years later I ran into Phil at the symphony. I asked him I how he was able to come up with such a great script. He blushed and said he had worked on it for nine years. I know spending a long time writing something doesn’t guarantee success. But not giving up on a good idea almost always does.

Lost Cities for iOS

2012-08-28T21:30:00+00:00

Lost Cities is one of my favourite card games, and over the weekend I was surprised to learn about the release of a version for iOS. The app was released by TheCodingMonkeys, who previously built the Carcassonne app. I’m happy to report that Lost Cities is adapted just as delightfully for the iPhone.

Lost Cities a two-player game, and while the mechanics don’t allow for the pass-and-play experience that Carcassonne offers, it’s still fun and responsive to play against friends, whether they’re in the same room or across the internet. The move to iOS also enhances the core experience of the game: that the scorekeeping and card counting is done for you allows you to focus solely on your tactics and execution. The single player challenges also encourage you to try different techniques and gameplay styles. This is a digital adaptation done right.

The graphical polish on the app is amazing, and the music is exquisite. Together, they really do create a feeling of daring and adventure, which is quite remarkable for a game that otherwise consists of stacks of cards. You want to try this game, if only to see how a card or board game should be done on iOS.

I was also interested to learn that TheCodingMonkeys were also the developers behind SubEthaEdit, the comparatively nerdy and (and definitely less game-like) networked collaborative text editor. Nice to see a company take their experience from one area of software development and use it to find success with some completely different products.

Fast Downscaling of Retina OS X Screenshots

2012-08-22T21:30:00+00:00

Screenshots from the retina MacBook Pro look comically large when I share them with my @1x teammates. This kind of fidelity is especially unnecessary when casually sharing annotated snapshots back and forth during the app development process.

Automator makes it easy to build an app that will halve the dimensions of the images. Open Automator, choose to build a new Application, then add these steps:

Save it to /Applications and you’re done. Then, dragging an image onto the app’s icon will immediately copy it to your desktop and rescale it. Put it in your Dock for even easy access, or if you use LaunchBar, you can do it even faster with InstantSend.

Saying Goodbye, Keeping Your Team

2012-08-21T12:30:00+00:00

Noah Stokes shares his decision to leave San Francisco, and demonstrates that you can both move and take your colleagues with you:

Bold now shares a virtual office. Garrett is still in San Francisco, while Sam has moved to Austin, Texas and Charlie remains in Raleigh, North Carolina. It’s bittersweet to be virtual, but at the same time we all work to live, not live to work. So why not love where you live.

I’ve spent 9 of the past 12 months working remotely with my team at Icelab, and in this time, my wife and I have experienced life and adventures together in some amazing and personally significant parts of the world. It’s been a unique and precious period in our lives, and my work’s been a critical enabler for it.

And when I am at work, we’ve been as productive as ever. The time collaborating remotely has actually strengthened my understanding of my teammates. It takes a certain degree of thoroughness to explain yourself in text and over Skype. You can’t take shortcuts in your communication. It’s true that this has led to frustration at times, but it’s ultimately led to a more acute knowledge of their workings and motivations.

Anyone who builds things knows that worthwhile creations take time to mature. One of the most worthwhile things you can work on is your team. If you’ve found a good one, do your best to keep it, no matter where you live.

Moved to Hong Kong

2012-08-17T08:15:00+00:00

As I write this, I look over Hollywood Road and watch the antique stores close down for the day. A stream of red-topped taxis buzz by. There’s still a waft of incense in the air from the nearby Man Mo temple. In a sweep of the eye, I can see hundreds of tightly packed residences stacked above each other. That’s right, I’m in Hong Kong!

Time has really flown by since my wife and I finished in the Philippines and came back to chilly Australia at the end of June. We spent a whirlwind seven weeks catching up with friends and work, and now we’ve already started the next chapter of our overseas adventures: two months in Hong Kong.

Why Hong Kong? We’ve visited a couple of times and have really enjoyed it here, and after eight months in the provincial Philippines, it would be fun to experience the opposite end of the urban spectrum. Working remotely for Icelab has been no trouble, so we can go where our fancies take us.

We’re staying in Sheung Wan, a fantastic neighbourhood. It’s an older area and still largely residential, so it’s quiet and charming, but just 10 minutes by foot to the bustle of Central. There are antique shops, galleries, boutiques and a bunch of decent cafés. Monocle has a great video introduction to the neighbourhood.

Smarter the second time

Compared to the initial effort of preparing for the Philippines, moving here has been a breeze. All our affairs were already in order, so getting out again was merely a shuffling of what was in the suitcases. If you’ve made the effort to relocate for a significant period of time, you almost owe it to yourself to back it up with another trip.

We’ve learnt from our first experience and have done a few things differently this time. Firstly, we’re staying in an apartment we found via Airbnb. The place is very comfortable and already well set up. Within a day of arriving, I was able to put in a productive day at work. For a shorter stay, it’s great not to worry about finding and establishing a new place to live.

We also packed far lighter. Here’s my packing list for 2 months:

Essentials

6 t-shirts
1 pair of shorts
1 pair of jeans
1 pair of shoes (My trusty New Balance 574’s)
1 pair of thongs
Assorted underwear and socks (much less than the Philippines)
Toiletries

Gear

MacBook Pro
iPad (mostly for reading)
Jawbone Jambox
4 external HDDs (2 x backup, 2 x media)
Camera

All up, 13kg in one still quite empty suitcase.

Say hello

We’ve a little over 6 weeks left here in Hong Kong. I’m hoping to make it out to a couple of tech meet-ups, but would love to meet anyone for lunch of coffee. Say hello!

RubyMotion & Rails Responders at the Canberra Ruby Crew

2012-06-20T19:30:00+00:00

This week the Canberra Ruby Crew meetings kicked off again after a long hiatus. Both Hugh and I gave presentations. Hugh spoke about our use of Rails responders for tidily offering layout-less content intended for display in modal popup windows. You won’t see his slides online just yet, but his use of Impress.js with the demos embedded in iframes was ingenious.

I gave an introduction to RubyMotion, the impressive new Ruby implementation for developing iOS apps:

I’ve played with all the other half-arsed attempts at bridging different languages or web tech with native iOS deployment, and none of them has left me convinced. RubyMotion is different. It feels first-class, and in its brief couple of months since release, so many libraries have sprung up that look to make iOS development much more pleasant. I’m convinced enough that I’ll be porting Decaf Sucks across. I’ll let you know how it goes.

If you’re interested in learning RubyMotion, here are some resources worth checking out:

Free screencast from The Pragmatic Studio
The repository of example apps
The examples from iOS Programming Big Nerd Ranch 3rd ed, ported to RubyMotion
A long list of open source RubyMotion projects (and another here)

Many thanks to Hugh and Matthew for organising this month’s meet. If you’re interested in coming along next time, join the mailing list and follow @rubyaustralia on Twitter. See you then!

Icelab Singapore and RedDotRubyConf 2012

2012-06-03T22:30:00+00:00

Wrapping Rack Middleware to Exclude Certain URLs (For Rails Streaming Responses)

2012-05-21T02:35:00+00:00

The Darker Side of Software Development

2012-05-08T22:30:00+00:00

Jim Whimpey read my last post and shared exactly how he feels when consumed with a software problem:

After hours without progress you go to bed thinking about it. Thinking about it makes you angry and frustrated but you can’t stop thinking about it. You spend so long at the computer you lose all perspective and follow paths that take you further from the solution. With each minute you spend banging your head against the problem you think less clearly and the possibility of solving the problem decreases. It is frustrating.

This is exactly what happened with me. I think there’s only so many times you can work like this before you burn out. It’s important to learn from those moments and develop a sense for when they begin to arise, so you know when it’s time to step back and short-circuit the cycle.

Money, Stress and The Cloud

2012-05-06T20:30:00+00:00

Did you in-app purchase anything from the minibar, sir?

2012-03-29T22:30:00+00:00

A Feeling of Lightness

2012-03-22T23:55:00+00:00

In a recent Back to Work episode, Dan shared a line from Fight Club that powerfully represents my feelings about clutter:

As Tyler Durden said it best, “The things you own own you, man.”

Back to Work #56, One Giant Beholder Dinosaur, at 45:00.

I’ve removed clutter from my life in a big way. I did it by moving overseas.

What have I been doing here? Working, writing, exploring, drinking coffee, enjoying the company of my wife. Living life as usual. Nearly everything I do is in the 13 inches of my MacBook Pro. Everything I read is in my iPad. I have only a small collection of clothes (it’s the Philippines, after all). Effectively, I’ve been living with a single suitcase worth of things.

Yes, we have bought some furniture and other things to make ourselves comfortable here, but I consider these entirely temporary. At the end of our sojourn here, I’ll not think twice about about parting with them.

The result: an incredible feeling of lightness. A feeling of agility, and possibility. Our next steps are unhindered by our possessions. We’ve been thinking about where in the world we want to visit. About what places we want to get to know better. The future is exciting and ours to wield.

This opening of the mind has extended far beyond geography. We’ve started to think more broadly than ever before about our ambitions for our work, what we create, and how we want to live. This experience will shape every aspect of our lives for years to come. (And at the very least, I’ll certainly have a different mindset about how to populate a home, when we do settle down again)

You might consider packing up and moving overseas a rather extreme way to remove clutter from your life. I agree that it’s extreme, but it has the singular advantage that it facilitates a true mental decluttering along with the physical one. I think this makes it entirely worthwhile.

Perhaps the change doesn’t need to be as extreme as ours. Maybe a house swap would get you away from most of your posessions long enough to give you a different perspective. Or maybe you could extend an overseas holiday and conduct some of your work there, so you can experience ordinary life in another place. Or perhaps you really just need to throw some stuff out.

Whatever you do, taking some concrete steps now may too bring you unexpected new feelings of lightness, feelings that only fold back into further positive action. It’s worth a try.

Derek Sivers on Doing Business

2012-03-18T04:30:00+00:00

Derek Sivers answers “Are business decisions for love or for money?” and summarises everything I want to do in business:

For me? Love. The purpose of money is to trade for things that make you happy. So if you can bypass money and get directly to the happy, you’ve saved a lot of trouble. And it makes others happier, too, when you organize your business around non-monetary things.

If this jibes with you, then his book Anything You Want is well worth a read.

For the Love of Fika

2012-03-16T05:30:00+00:00

In Sweden, they have a custom I can definitely get into:

While it’s true that many other countries also take coffee breaks, they aren’t appreciated in the same manner as they are here, nor are they mandated by law.

This is from the Nordic Coffee Culture Blog, which is a mainstay on my Flipboard home screen, along with Dear Coffee, I Love You.

Export Trajectory Stories to CSV

2012-03-14T12:30:00+00:00

Icelab gets inventive, quadruples Melbourne team

2012-03-13T10:30:00+00:00

Roxette in Hong Kong

2012-03-13T10:20:00+00:00

I saw Roxette play live in Hong Kong last week, something I thought would never happen, given Marie’s brain tumor, the band’s extended hiatus, and my location in the southern hemisphere. So it was a pleasure and privilege to see this childhood favourite band of mine in person, along with my eager wife.

It was a great show. We were treated to new renditions of their songs (like a rock version of Stars, Per singing Crash! Boom! Bang!) and some masterful guitar work (like the epic extended solos in 7Twenty7 and The Look). But what stood out the most to me was the unadulterated passion and energy they all expressed. It was truly inspiring; I hope to feel and act the same way after the next 25 years of doing what I do.

Shawn Blanc on Benjamin Franklin’s Daily Schedule

2012-03-13T09:55:00+00:00

Why me must tend to how we spend our time:

I regularly come back to my own daily schedule to re-evaluate it and see if it is serving me as well as it should be. Because schedules, like finances, make excellent slaves but horrible masters.

Shawn also nicely summarises my feelings about time and the creative process:

One big myth about creativity is that it cannot be harnessed. It is silly to believe that a creative person should live without routine or accountability or discipline.

Swipe to Go Back

2012-03-02T05:30:00+00:00

A Precious Hour

2012-03-01T23:30:00+00:00

Rands on why being busy is no substitute for being creative. He’s building the habit of taking an hour each day to create and is already seeing the results. I did the same last year it was one of the major reasons that Decaf Sucks for iPhone actually became a thing.

A Webstock 2012 Recap

2012-02-29T23:30:00+00:00

Announcing gentlyremind.me

2012-02-27T22:30:00+00:00

Be More Than Your NDA

2012-02-26T20:30:00+00:00

This kind of conversation has happened to me a few times at various tech meetups.

Me:

Great to meet you! So what kind of things do you work on?

Them:

Oh, I can’t tell you.

Me:

…

Where can you go from here? The weather? Sports?

At meetups like this, the work is what binds us together. I understand that you can’t talk about your work if your employers impose an NDA, but that doesn’t have to mean the end of the conversation. How about a side project? Have you started one of those? It may only take a few hours on a weekend to get something running, and then you have something you can freely use to better connect with people. If not a side project, maybe there are a few libraries or tools you’ve been using that are interesting to you. Maybe you could talk about them.

Participating in a tech meetup may initially be about hearing the presentations, but it doesn’t end there. You’ll want to give something back too. This means having something to talk about with your peers. Your NDA doesn’t have to stop that.

500 Words before 8am

2012-02-24T05:30:00+00:00

Wise advice:

I make sure to start every day as a producer, not a consumer.

Which reminds me of this, from _why the lucky stiff:

when you don’t create things, you become defined by your tastes rather than ability. your tastes only narrow & exclude people. so create.

Customising ActiveRecord's attribute formatting on inspect

2012-02-02T10:30:00+00:00

Render Single-Line Markdown Text with Redcarpet

2012-01-24T13:05:00+00:00

New Relic, Heroku and Rails Streaming Responses

2012-01-20T16:45:00+00:00

2011 in Review

2012-01-05T10:30:00+00:00

Each time I come to these posts, the year just past seems more activity-filled than ever. 2011 has been no exception. Another big year, with quite a few notable firsts for me:

A stake

In July I became a director and equal partner at Icelab, along with Michael and Max. My time working at the lab so far has been immensely satisfying. I am privileged to work with such a great team, and look forward to having Icelab as the foundation of many exciting future endeavours.

An app

For years I’d toyed with the idea of building an iOS app. It’s a platform I love to use, and one that will play an important role in the future of computing, and I wanted to be a part of it. Finally, with my attendance at Swipe Conference as motivation, I learnt Objective-C and Cocoa, and built and released an app (also thanks to the tireless design work of Max). Better still, it was an app for my favorite place on the internet, Decaf Sucks.

We released the app in August, just after Decaf Sucks’ second birthday. Now, with a few months behind it, it looks like the release has been been quite successful. Personally, it’s found me a bunch of new coffee to try, and helped to take me well past 250 reviews. I hope it is a positive contribution for the coffee lovers of the world.

With this big first release under my belt, I feel empowered to create a whole bunch of fun new experiences on iOS. Hopefully there’ll be a lot more to share on this front in 2012.

A talk

In September, the whole Icelab crew travelled up to Sydney for Web Directions South, where Michael and I spoke about smart choices for smartphone apps (i.e., web or native, and tell me without the dogma).

This was my first major conference presentation and I was honored to be part of such an inspirational event. I found the whole experience to be really rewarding. I enjoyed the singular focus that it encouraged for a good few weeks of thought, and the extra depths that it encouraged me to plumb in order to give a comprehensive overview of the topic. Its an experience I’d definitely like to repeat. Conferences of the world, who would like to have me in 2012? ;-)

A move

If all this wasn’t enough, my wife and I capped off our year by packing up our lives in Australia and moving to the Philippines! I must say a little more about my amazing wife at this point: not only did she put up with me, look after me and keep me sane throughout my work on all of the above, she also shouldered most of the load for planning this move. I couldn’t have done half as much this year without her, and I’m lucky to be her husband.

Anyway, as I write this now, I look out our front window onto the lush green of a small park, surrounded by palm trees, goats grazing, roosters crowing, and the kids of the neighbourhood running around. I’m definitely in a whole new place, and I look forward to the many adventures I will have here, away from the computer screen.

[caption id=”” align=“alignnone” width=“1024.0”] Approaching Sugar Beach, near Sipalay, Negros Occidenal[/caption]

And more

In 2011, Icelab grew from three to five people. Hugh and Andy are good humans to see everyday and are invaluable as members of the team. I got to witness a lot of great stuff come out of the lab this year, much of which I wish I could have worked on myself. I think this is a sign of a good workplace.

I did get to make one big release. In November we launched the new Corner Hotel site, along with a completely new ticket sales and box office management system, a Rails app that was practically my sole responsibility for the whole year. It’s one of Icelab’s biggest projects to date, and happily, everything’s gone pretty well since launch.

Aside from shipping new projects, we also kept-on-shipping with our coffee subscription service, Dispatch. It’s now over a year in the running, and we’ve had a lot of fun sending coffee around the country.

In the middle of 2012, my wife and I started eating paleo. It was a terrific decision. I’ve never felt healthier in my life, nor eaten better food. While it’s harder to follow closely here in the Philippines, we do our best, and I know that it will be an eating habit that I’ll keep for the long term.

And in one final notable event, in April we were lucky to witness the wedding of our good friends Hugh & Tomomi. Since moving to Canberra a few years ago, the friends I’ve made there really make it feel like home. We’ll be back before too long.

So that’s 2011. I finished the year feeling close to burnt out, but I’m proud of what I was able to achieve and I’ve now learnt the critical (if not already obvious) lesson that it’s best not to have many work commitments overlapping with an international house move. Now that we’ve settled here and had a short break over Christmas, I’m ready to go again and looking forward to a productive & venturesome 2012!

Decaf Sucks Launch Countdown: A Restrospective

2011-12-23T09:30:00+00:00

Dispatch's First Birthday

2011-12-09T22:30:00+00:00

Moving to the Philippines

2011-11-03T11:20:00+00:00

This Saturday, I am moving to the Philippines!

For a long time, I’ve wanted to spend an extended time overseas, to learn a new place, culture and language, and tackle all the interesting challenges that go along with it. Now, it’s actually going to happen, thanks to my equally adventurous wife and flexible workplace.

I’ll be living in Bacolod, Negros Occidental and continuing my work for Icelab as usual. I’m really looking forward to getting involved with the web/mobile development community over there, both in the Philippines and in SE Asia in general. So if you’re in the area, please say hi!

This also means that Icelab is much more easily available for hire in SE Asia. If you’re after some finely crafted web or mobile apps, get in touch.

It’s going to be a great adventure, and I can’t wait to get started. I’ll no doubt have plenty of interesting things to share, so be sure to follow me on Tumblr and Flickr, where I’ll be documenting my travel experiences.

Announcing Decaf Sucks for iPhone

2011-08-30T00:25:00+00:00

Decaf Sucks Launch Countdown: Development Complete!

2011-08-23T02:25:00+00:00

Decaf Sucks Launch Countdown: Un-Designing the App

2011-08-06T06:50:00+00:00

Decaf Sucks Launch Countdown: Finishing the API

2011-07-30T06:45:00+00:00

Decaf Sucks Launch Countdown: Getting to Work

2011-07-23T04:50:00+00:00

Decaf Sucks Launch Countdown: Starting Again, Again

2011-07-15T00:05:00+00:00

RailsCamp 9

2011-06-14T00:45:00+00:00

Run Your Own Piece of Heroku with Foreman

2011-06-03T02:50:00+00:00

What's New With Decaf Sucks

2011-05-26T00:45:00+00:00

Canberra's Best Coffee

2011-05-10T22:30:00+00:00

A feature article on HerCanberra, with the lowdown on the Canberra coffee scene.

Whether it’s your morning pick-me-up, social lubricant, or lifelong passion, few things are more evocative than coffee. And as the Canberra days turn colder, the city’s cafés will see a growing flock of people come through their doors in search of a warming espresso beverage. Luckily for Canberrans, there’s no better time than now to find a great coffee and café experience in our city.

Two years ago I created Decaf Sucks along with the rest of the caffeine-loving crowd at Icelab as a place for us all to record our coffee drinking journeys. It has since become a vibrant and considerably useful resource for location-based café and coffee reviews. It is a place for us all to help each other find the good coffee and avoid the bad.

Since it is homegrown here in Canberra, it also has the internet’s best coverage of cafés in our city, with 66 people having contributed over 335 reviews for 174 different cafés. Today I’d like to take you on a tour of some of Canberra’s standout cafés, based on the opinions of the Decaf Sucks reviewers.

The hottest café in Canberra right now would have to be Lonsdale Street Roasters in Braddon. They’ve hit on a mix of great in-house roasted coffee, simple and well made food from a wood oven, and an always-bustling friendly atmosphere. Heading into Civic, we have Bean in the City, a hole in the wall that is big on coffee, as well as selling the best portuguese tarts in the city. Over in the west of the city, head into Harvest (the newest member of the Kingston Grind, Group 7 and Tonic family) and give one of their syphon or cold press coffees a try.

If you’re in the south of the city, be sure to try Lava in Weston, for what could be the best coffee south of the lake. Or if you’re after a meal and a place to hang out a while, head to A Bite To Eat, A Drink As Well, where you can enjoy the expansive space filled with raw brick, comfy lounges and 1970s-kitsch furniture, or soak up the sun in the courtyard out the back. It’s also a good place for a beer if your visit lingers into the afternoon.

You needn’t despair if you ever find yourself hankering for a caffeine hit in Fyshwick, since you can just head to Ona Coffee on Wollongong St, where they are serious about good coffee. Heading back north, the Belconnen Fresh Food Markets offers a couple of good coffee options among the fruit and veg, in Beppe’s Tuscan Kitchen and As Nature Intended.

Finally, there’s no dearth of decent coffee if you’re heading for a drive outside Canberra. Try Cork Street Gallery Cafe in Gundaroo for a rustic outdoor dining area, where you can enjoy some delicious pizza with your coffee. In Bungendore, The Provincial Pantry has received much high praise from the Decaf Sucks reviewers, and The Albion in Braidwood looks to be the pick of a town that is already bursting with tasty food and drink options.

While that should certainly keep you busy for a while, we’ve only just scratched the surface. There’s plenty of other good coffee to discover in Canberra, and Decaf Sucks can help you do it. Visit the site in your desktop browser and you can enter a location to see all the reviews for cafés in that area. If you’re out and about, visit Decaf Sucks on your iPhone for an optimised experience that can show you the cafés around your precise location.

We’d love to welcome you into the Decaf Sucks family! We made Decaf Sucks as simple as possible for people to contribute, so please consider writing some reviews. All you need is an opinion to share and a star rating, nothing more. Better still, logging in is easy through connecting your Twitter or Facebook accounts. Your reviews will not only serve as a handy coffee diary, but also could be instrumental in helping introduce other people to fantastic new café experiences, whether in Canberra or further afield.

For all those times when you’re happy to stay at home, we can also help you to drink great coffee. Last year we launched our coffee bean subscription service: Dispatch, from Decaf Sucks. We’ve teamed up with the Lonsdale Street Roasters to deliver a fresh and tasty bag of coffee beans to your door every month. We introduce different beans every month, making Dispatch a great way to discover new coffee flavours. It’s an easy way to keep your cupboard stocked with fresh beans, and also makes for excellent gift idea. Check it out.

This month we have a special offer for the readers of HerCanberra. Write at least one review on Decaf Sucks, and you could win a free month’s delivery of fresh Dispatch coffee beans. Just email us at hello(at)decafsucks.com to let us know and you’ll in the running. We look forward to reading your reviews!

Tim Riley is a web developer who thinks almost as much about coffee as he does about building great experiences for the web and mobile devices. He lives in Braddon with his wife, and they love being just a stone’s throw from many great cafés. Tim works at Icelab and blogs at openmonkey.com. His coffee of choice is the macchiato.

Put Your Mac to Sleep With a Backup

2011-05-01T06:10:00+00:00

One aspect of my backup strategy is a full clone of my MacBook Pro’s hard drive using SuperDuper. This is very useful, since it means that I have a complete, bootable duplicate of my hard drive that I can use to get back up and running quickly in the event of a hardware failure. The downside of backing up this way, however, is that it can become a “whenever I remember” approach, which is a recipe for quickly falling behind or forgetting altogether.~

This is especially the case if you use a portable computer that isn’t always hooked in and running at a particular location (I carry mine to and from the lab each day).

Here’s the solution that has worked for me: configure a SuperDuper backup schedule that commences whenever your external disk is connected, and finishes by putting your Mac to sleep:

My MacBook usually gets opened over the course of a normal evening at home. My rule is that the only way I can put it to sleep for the night is by connecting my backup drive and triggering the SuperDuper backup. It’s fast and simple; connecting a single cable to the USB port is no bother at all. The backup runs and then my Mac goes to sleep, and it means I can sleep easier too, with my data better protected.

nb. If you adopt this approach, it’s useful to know the control+shift+eject keyboard shortcut, which you can use to turn off your Mac’s display once you’ve attached the backup drive.

The Benefits of Waking Up Early

2011-04-18T04:45:00+00:00

Useful Heroku-Friendly Rewrites with rack-rewrite

2011-04-06T06:45:00+00:00

New site

2011-03-27T21:00:00+00:00

Here I am again, with a new blog platform and site design. This time around, I have some interesting things to say about it. The reason for this particular rewrite was to unify my writing. Since I joined Icelab, I’ve published the bulk of my writing onto the blog over there. This is entirely my intention, since it all helps to raise awareness of the good work that we’re doing, but it does leave my personal site looking rather unloved. Now I mave made it so all my posts are available in a single place, right here. Everything is available in snippet form on the home page, as well as in the archives and the feed. When you choose to read a particular article, you will be taken back to the Icelab site for the articles originating from there, while my personal articles you will see in full here.

This was all remarkably easy to achieve using the small but powerful toto Ruby blog engine. Eschewing the traditional database, Toto reads your articles from files on disk, and is happy to parse Markdown for you. Since we also use Markdown on the Icelab site, I can simply paste in copies of those articles verbatim, and toto is good to go. Then, if I add a url attribute to the article’s metadata, Toto will link to that URL for viewing the article in full. For example:

title: Embrace the Metaclass and Extend Your ActiveModels
published_at: 2011-03-25 16:47:00
location: Canberra, Australia
url: http://icelab.com.au/notes/embrace-the-metaclass-and-extend-your-activemodels/

Part of the challenge in building [RentMonkey](http://rentmonkey.com.au/) is dealing with...

This is possible through one minor extension to toto:

module Toto
  class Article
    alias_method :local_path, :path
    def path
      self[:url] ? self[:url] : local_path
    end

    alias_method :local_url, :url
    def url
      self[:url] ? self[:url] : local_url
    end
  end
end

And that is all the Ruby hacking I had to do. Rather nice!

Also of interest is the new design. I’m using CSS media queries to create a responsive layout that works well at all window sizes and across different mobile devices. I’m also using Typekit to clothe the whole site in the lovely FF Tisa Web Pro web font. I’m also finally hosting this site on Heroku. The simple everything-is-in-git philosophy of Toto makes this a breeze. No comments here for now, but it’s easy to contact me, and I’m always open to amending articles based on feedback.

The design is my own doing, so naturally it is minimal (and contains approximately two colours), but I hope it gives the words (and code samples) room to breathe. I now have a much more prominent sidebar than ever before, because I have more to tell you than ever before. You know about Decaf Sucks and Dispatch, right? Or that we’re aiming to change the real-estate scene with RentMonkey? Hopefully my sidebar can do a little to help tell this story.

As for the rest of the story, now it’s up to me to write it.

Embrace the Metaclass and Extend Your ActiveModels

2011-03-25T06:15:00+00:00

FactoryGirl and has_many associations

2011-03-23T06:35:00+00:00

Taking the pain out of save_and_open_page

2011-03-23T00:15:00+00:00

TaskPaper and To-Do List Bankruptcy Without the Trauma

2011-02-02T00:05:00+00:00

Let’s face it: you are an engaged, curious person, and you accumulate to-do list items like <your preferred simile here>: book and film recommendations, app and open source ideas, blog post ideas, restaurants and cafés to visit, things to see or do, online and off. These items are especially easy to accumulate if you use an app like TaskPaper, whose over-the-air sync enables universal capture of to-do items, whether at your Mac or while out and about on your iPhone.

Before long, you’ll have more items on your list than you could conceivably do (even with all your ambition). These are great to keep around for reference or for that quiet weekend that needs filling, but in the end, they do little more than weigh you down and get in the way of the the important real-world things you need to do. Your to-do list shouldn’t make you anxious.

Fortunately, TaskPaper offers another feature that will help you manage this problem: its simple single-file, plain-text data format. With TaskPaper, you can spread your to-do lists across multiple files, and have them open them concurrently. This is largely its reason for being, and what sets it apart from the heavier duty task management apps like OmniFocus. The benefit of this single text file is that you can easily move it and rename it, enabling you to shift everything aside and declare to-do list bankruptcy.

I’ve just done this myself, moving my Things.taskpaper file to Archive 2010.taskpaper. Bam: instant clean slate. Create a brand new Things file in its place, copy across a select few important tasks, and you’re ready to go. You can even keep your archived to-do lists in your synced SimpleText folder, so they remain accessible for reference on your iPhone as well as your Mac (There’s nothing wrong with a security blanket if it doesn’t get in the way).

There’s much benefit to keeping your to-do list short. It helps you focus on the tasks that matter and actually achieve them. TaskPaper helps you balance these two by allowing you to contain and group your items by project and tags.[1] Sometimes, though, it helps to have the less important stuff completely out of the way. Declaring to-do list bankruptcy is a simple, pain-free way to get started on this path. It’s helping me to keep on top of the important things, and with publishing this post, I’ll have one less item in the list. Now, onto the next thing!

[1] Another way would be to use a completely separate file for the “Someday” or “Maybe” tasks. A little more friction, but cleaner separation.

Dispatch #2: Your Coffee, Faster

2011-01-26T23:30:00+00:00

2010 in Review

2011-01-05T00:05:00+00:00

2010 was one of my most exciting and productive years yet, working with great people to build some products that I’m truly proud of. When I look back, I hope that this summary will help remind me that it was the start of something amazing.

I Shipped

This year I was able to enjoy a developer’s greatest pleasure, many times over: shipping.

First came the year’s most significant milestone for me: after 18 months of work on weekends and evenings, we finally launched RentMonkey. Some called it “the best form on the internet,” others said it is “easily the most gorgeous and usable rental property leasing app you’ve ever seen!”, we think it is a great way to take back control of your biggest investments. We’re proud of what we’ve done so far and very excited to continue to build it in 2011. This is just the beginning; we’re in this for the long haul.

With the first RentMonkey release behind me, I threw myself some long-overdue updates for our other baby, the café and coffee review site Decaf Sucks. Not long after it’s first birthday, we released a new design and a whole swag of improvements, including Facebook logins and a lovely infinite scroll. This helped increase interest in the site and now we’re just a touch shy of 1000 reviews!

This wasn’t the only news on the internet-coffee-obsession front. Just before Christmas we embarked on a whole new business venture, coffee subscriptions. Dispatch, from Decaf Sucks was conceived in one week and launched the very next (I wrote more here about that particular whirlwind). It’s been received really well and we’ve had a lot of fun sharing some coffee love around Australia.

Work at Icelab

Much of the energy and drive behind all of the above has risen from the inspirational team and environment at Icelab, where I’ve just finished my first full year of work. We saw our fair of client project launches, with the most notable for me being Exploring Democracy for Old Parliament House, which was the lab’s first big Rails app, and the Water Storage iPhone app for the Bureau of Meteorology, which saw me get my hands dirty with Objective C for the first time. Icelab has been the perfect place for me to grow into a the well-rounded developer that I want to be.

A few months ago we also released a new Icelab site, designed and built as beautifully as ever by Max. It’s great to have a better platform for sharing the things that we get up to. I’ve taken to blogging there with vigour (which explains the quietness here), and we’ve all been sure to keep our portfolio nice and fresh.

Open Source

Right before the 2010 Rails Rumble, I was able to extract the OmniAuth-based Twitter and Facebook logins from Decaf Sucks and release it as an open source Rails 3 engine called OmniSocial. I blogged about it and was lucky to have it featured on The Ruby Show and Ruby5. This little publicity boost at its release has actually seen it adopted in a few apps, which has been fantastic to see.

In 2011 I’d like to do a bit more of this: identifying useful patterns from my apps and extracting them into useful open source tools. John Nunemaker is my inspiration in this regard.

Ruby Community

More on the Ruby front, I continued to run the Canberra Ruby Crew meetings in 2010, moving them to the more centrally-located Icelab. We had good attendance and a bunch of new faces, not to mention the usual pizza, beer and good times.

Canberra hosted Australia’s seventh RailsCamp in April, where I was able to continue working on my nebulous iOS development skills (and have Happy Birthday sung to me by 100 drunken Rubyists). I also made a bunch of short talks at both the Canberra and Sydney Ruby groups, as well as at Canberra’s BarCamp:

Building your Capistrano Recipe Book
An Introduction to Sinatra
Using Titanium to Build Mobile Apps with JavaScript (a lightning talk with Max, here’s an audio recording)
JavaScript Testing with Capybara and Cucumber
Facebook Authentication with OAuth
Google Maps JavaScript API v3

Lastly, I ventured to Japan with a glut of Aussie Rubyists for my second RubyKaigi!

More Highlights

In short form: I wrote my first Rails 3 application, as well as more JavaScript than ever (made easy via CoffeeScript). I surpassed 100 café reviews on Decaf Sucks, averaging more than two reviews per week. I started a tumblelog about heavy metal with my brother-in-law, and it has been some of the most challenging writing I’ve done to date, but also a great way to discover new music. I got an iPad and iPhone 4, fantastic, life-changing devices both. I was motivated and inspired every week by the amazing stories and topics on the 5by5 podcasts. I started playing Ultimate Frisbee and working out three mornings per week. My laptop is finally backed up regularly. I rediscovered the fun of photo-sharing through Instagram.

Last, but never least, 2010 was a thoroughly enjoyable first year of marriage to my amazing wife. She has been the rock and support that allowed me to do all of the above. We had plenty of fun times together, including a weekend in the world’s biggest spa bath, a 3-week adventure across Japan, and a lovely first anniversary getaway.

That’s it for a big year. In 2011 I will continue to get excited and do things. Watch this space.

Wishing You a Happy, Caffeinated Christmas

2010-12-20T23:30:00+00:00

Announcing Dispatch, From Decaf Sucks

2010-12-13T23:30:00+00:00

Welcome to the OmniSocial

2010-10-10T23:30:00+00:00

Decaf Sucks First Birthday Updates

2010-10-06T23:30:00+00:00

Decaf Sucks, our coffee and café review web site, recently turned 1 year old. For a site created over a single weekend for Rails Rumble, Decaf Sucks has been a great success. Over the last year, 143 different contributing reviewers have written 610 reviews for 502 cafés in 18 different countries. Outstanding!

Here’s how we’ve travelled over time. The blue line is our cumulative review count, and the red is the number added each day.

The site has worked amazingly well in its first incarnation. However, with the number of reviews growing, it has become time to make some changes. So, as a first birthday celebration, we recently pushed up a whole swag of improvements to the site. Here is the lowdown:

New design: Wider and shinier, the new design puts more information in front of you without sacrificing cleanliness.
Infinite scroll: No more pagination! Scroll to the bottom of a café or review list, and another set of reviews will appear for you.
Reviews grouped by cafe: We’re starting to see 3 or 4 reviews for popular cafés, so it’s time to organise things better. Now we group together all the reviews for a cafe and present that cafe to you with the average rating across all its reviews. This reduces the clutter and gives you a better idea of which cafes are truly good and bad. It also makes the map of nearby cafes far more useful - no more duplicate entries!
Facebook logins: This should help a much wider group of people share their experiences.
People pages: Meet our reviewers and follow their cafe adventures via convenient ATOM feed.
Performance improvements: We’ve started minifying and bundling all our CSS and JavaScript, which should help with performance particularly on the iPhone. We’ve also eliminated a bunch of jQuery plugins by using the Google Maps API directly.

Go check it out and leave a review!

Easy server backups to Amazon S3 with duplicity

2010-09-22T23:30:00+00:00

JavaScript Testing with Cucumber and Capybara

2010-04-09T04:55:00+00:00

Capybara is a Ruby DSL for easily writing integration tests for Rack applications. It is an alternative to Webrat and can easily replace it as a backend for your Cucumber features. Its power and utility lies in that it comes bundled with several different browser simulators, equipping you with a flexible toolkit for testing all parts of your application, from the simplest page to the most complex, JavaScript-heavy page.

[caption id=”” align=“alignnone” width=“500.0”] A photo of real capybaras.[/caption]

Using Capybara means you can now confidently write your Cucumber features as the first thing to drive the development of your application, regardless of how you plan to implement the feature. Another win for testing!

Using Capybara

Capybara, much like Webrat before it, provides a broad set of steps you can use to test your application in Cucumber features:

Scenario: Creating an article
  Given I go to the new article page
  And I fill in "Title" with "JavaScript Testing with Cucumber and Capybara"
  And I fill in "Body" with "Here is my article"
  And I press "Create"
  Then I should see "Article created"

By default, it uses rack-test as the simulator to run your scenarios against your application. rack-test is the fastest of all Capybara’s drivers, and should be used whenever possible to keep down the time needed to run your entire suite of scenarios. It should have no trouble handling anything like the above scenario.

You can use the rack-test driver according to one simple rule: no JavaScript, no problems. However, Cucumber does include a helper that allows you to use Capybara with rack-test to test the links with onclick JavaScript handlers that Rails uses for deleting records:

<%= link_to 'Delete', article_path(@article), :method => :destroy, :confirm => 'Are you sure?' %>

Scenario: Deleting an article
  Given an article exists
  When I go to the article's page
  And I follow "Delete"
  Then I should see "Article deleted"

Testing JavaScript with Different Browser Simulators

Many parts of your application will be simple to test with scenarios like the above, but you will invariably have other parts that depend on complex interactions powered by JavaScript or AJAX requests. While these parts have historically been the hardest to test, their complexity in fact demands the strongest test coverage! Capybara will let you write the tests that these parts of your application deserve.

Capybara comes with built-in drivers for several browser simulators that support JavaScript: Selenium, Culerity and Celerity. While Webrat has always allowed you to use Selenium as a browser simulator, it is an additional dependency that has at times been difficult to configure.

With Capybara, using Selenium is as simple as putting a @javascript tag above a scenario:

@javascript
Scenario: Endless pagination with AJAX
  Given 10 articles exist
  When I go to the articles page
  Then I should see 5 articles
  When I follow "More articles"
  Then I should see 10 articles

This tests the kind of endless pagination that you see on Twitter. Running the above scenario, you would see Firefox open up and automatically run through your steps on a working copy of your app. The AJAX request will fire and you’ll see the extra 5 articles appear on the page. Everything passes. Excellent!

If you’d like to avoid a dependency on a GUI browser for your tests, you can try Celerity as a browser simulator, via the Culerity driver. Celerity is a JRuby wrapper for HtmlUnit, a Java headless browser with JavaScript support. Culerity is a wrapper for Celerity that allows any Rails app to use Celerity for testing, without the app having to run under JRuby. In practice, this is not as convoluted as it sounds. The end result is that Culerity will fire up a separate JRuby process to run tests against your app, while your app executes in its usual Ruby runtime.

All you need to do to is put a @culerity tag above your scenario:

@culerity
Scenario: Endless pagination with AJAX
  Given 10 articles exist
  When I go to the home page
  Then I should see 5 articles
  When I click "More articles"
  Then I should see 10 articles

Running this same scenario through Culerity, you’ll see each step pass as before, just without the browser running on-screen. This is useful if you’re looking to have your integration tests work nicely on a headless CI box or something similar.

Just like you can use @culerity to force a particular driver for a scenario or feature, you can explicitly require selenium by using the @selenium tag, and rack-test with @rack_test. The @javascript tag is reserved for Capybara’s default JavaScript-supporting simulator. You can change this by creating a features/support/capybara.rb file and including the following:

# :culerity or :selenium
Capybara.javascript_driver = :culerity

You can also change the default driver for all scenarios:

Capybara.default_driver = :selenium

Here’s One I Prepared Earlier

I’ve put together a Rails app that includes all the examples covered so far, ready for you to try yourself. The only thing you need is a recent version of bundler 0.9. To clone the app and and set it up:

git clone git://github.com/timriley/capybara-demo.git capybara-demo
cd capybara-demo
bundle install

Then you can run the Cucumber features:

bundle exec rake cucumber:all

And you can also start a server to verify the features manually:

./script/server

Installing Cucumber with Capybara

Getting Cucumber and Capybara in your own app is easy.

gem install cucumber capybara

If you’re on the bundler, then just put the following in your Gemfile and run bundle install.

group(:test) do
  gem 'cucumber'
  gem 'cucumber-rails'
  gem 'capybara'
  gem 'culerity'
  gem 'celerity', :require => nil # JRuby only. Make it available but don't require it in any environment.
end

Then run the Cucumber generator with the --capybara flag:

cd /my_app
./script/generate cucumber --capybara

Running this generator will create a web_steps.rb file in features/step_definitions. This file is a mostly-compatible replacement for the webrat_steps.rb that you may have had otherwise. It contains largely the same steps, and all of the following will work as you expect:

Given I am on the home page
When I go to the home page
When I press "Submit"
When I follow "New article"
When I fill in "Address" with "5 Smith Street" within "fieldset#home-address"
When I select "ACT" from "State" within "fieldset#home-address"
When I check "Remember Me"
When I uncheck "Remember Me"
When I choose "My Radio Button"
Then I should see "Article created" within ".flash"
Then I should not see "Error creating article"
Then the "Title" field should contain "My Title"
Then the "Title" field should not contain "My Title"
Then the "Remember Me" checkbox should be checked
Then the "Remember Me" checkbox should not be checked
Then I should be on the home page
Then show me the page

While most of your features should run just fine, you may need to make some small tweaks to correct things, especially if you used some of the date selector steps that were present in webrat_steps.rb but not in web_steps.rb.

Installing Culerity Support

While the Selenium driver will work for you out of the box, you need JRuby installed in order to use Culerity & Celerity. If you are using RVM, this is also easy:

rvm install jruby

Culerity will look for a jruby binary and use it to run Celerity. With RVM, you’ll need to make a jruby symlink to the versioned binary that it installs:

cd ~/.rvm/bin
ln -s jruby-1.4.0 jruby

That’s it. You’ll note in the Gemfile example above that we install the celerity gem for culerity to use, but don’t require it in any of the app’s environments, since your app (is most likely) not running under JRuby.

Go to it!

Since I’ve started using Capybara, I’ve been enjoying writing more integration for more parts of my applications, and it’s been a lot of fun. I hope this guide helps you follow the same path!

Rails Label Helpers with Blocks

2010-03-30T03:20:00+00:00

Whenever I work closely with designers, I try to learn a trick or two. From my latest project with the inimitable Max Wheeler, one of the things I picked up was his preferred strategy for building forms: nesting the inputs along with the label text inside a label tag. Something like this:

<label>
  Name
  <input type="text" name="article[title]"/>
</label>

This structure behaves as you would expect. Clicking the “name” text on the page will focus the input element nested within the same label. The benefit is that you no longer have to worry about synchronising the input’s DOM ID with the label’s for attribute.

Doing this using the ActionView form helpers is not currently possible. Fortunately, it is easy to roll your own solution using a custom FormBuilder.

class SmartLabelFormBuilder < ActionView::Helpers::FormBuilder
  def label(method, content_or_options_with_block = nil, options = {}, &block)
    if !block_given?
      # No block, use the standard label helper.
      super(method, content_or_options_with_block, options)
    else
      # We've got a block. This is where we want to do our business.
      options = content_or_options_with_block.is_a?(Hash) ? content_or_options_with_block.stringify_keys : {}

      if errors_on?(method)
        (options['class'] = options['class'].to_s + ' error').strip!
      end

      @template.content_tag(:label, options, &block)
    end
  end

  private

  def errors_on?(method)
    @object.respond_to?(:errors) && @object.errors.respond_to?(:on) && @object.errors.on(method.to_sym)
  end
end

ActionView::Base.field_error_proc = Proc.new do |html_tag, instance_tag|
  html_tag
end

Include the above somewhere in your app (perhaps a file in lib/) and now you can start building you forms like this:

<% form_for(@article, :builder => SmartLabelFormBuilder) do |form| %>
  <% form.label(:title) do %>
    Title
    <%= form.text_field(:title) %>
  <% end %>
<% end %>

See this gist to fork or download these code examples.

The form builder will pass back to the regular label method if you’re not using a block, so you can still create standalone labels if you need.

It also takes care of displaying form errors. If a label contains a field with an error, then the helper will give it and error class. Then you can use CSS selectors like label.error and label.error input@to change the appearance of your label text and inputs for these fields. For this to work nicely, I’ve overwritten ActionView’s field_error_proc so that it does nothing by default to fields with errors.

Handling labels and inputs in this way is also totally fine for testing tools. Capybara, for example, uses a bunch of different xpaths to locate fields, including some that support labels with nested inputs. So a Cucumber step like this works exactly as you expect:

When I fill in "Title" with "Rails Label Helpers with Blocks"

You might also want to check out this patch for rails 3 that adds block support to labels. It’s in need of a few testers and “+1” comments so that it can get incorporated!

BarCamp Canberra 2010 and an Introduction to Sinatra

2010-02-14T22:00:00+00:00

Earlier this month was BarCamp Canberra 2010, this city’s third BarCamp event. Having never been to one, I thought it was high time I checked it out. Given that most of the attendees would be unfamiliar with Ruby, it was also a prime opportunity to give an introductory-level talk. I chose to talk about Sinatra, since it was small enough to allow for a good overview in the 20-minute time slot.

This was my first long-form presentation, and I’m happy with how it went. I had an interested audience with some good questions, and we were even able to fit in a live deployment of a sample app to Heroku. Hopefully I also attracted a few more people to our Canberra Ruby Crew meets!

The talk was written in showoff, Scott Chacon’s Sinatra application for building browser-based slides. I suggest you give showoff a go for your next talk; I thoroughly enjoyed the process of using it to create mine. It freed me from the burden of colours, transitions, layout, and positioning, and let me concentrate on the most important aspect of my talk, its content. It also means that you can view the slides online (thanks, Heroku!) as well as their source code.

The BarCamp Canberra organisers did a great job: everything went smoothly and the turnout was excellent. It was interesting for me to see the goings-on outside the Ruby circuit that I usually frequent. Max and I were also able to showoff our little web/iPhone hybrid app we built using Symphony and Titanium.

The proclaimed outcome of the day (according to twitter, anyway) was an increased motivation to go out and do, the program of talks was a little light on the kind of hands-on technical topics that equip people to take action and create something. In this way, I think that Ruby hackers or other web or application developers are in a position to make an important contribution to BarCamps or other similar events. It’s a great opportunity to revisit the basics (or even hit on something advanced) and share it with others.

Making Your Capistrano Recipe Book

2010-01-19T04:45:00+00:00

If you’ve deployed more than one application with Capistrano, then you’ve probably repeated yourself in most of the deploy scripts. Heck, you’ve probably copied and pasted from one to another just to get things going. Moving your common deployment logic into a gem will prevent this duplication and allow you to codify your best techniques in a single place. It’s building a toolbelt where before the tools were scattered over the garage floor.

Most importantly, removing the onerous job of writing lengthy deploy scripts means you can keep focused on the business value of your application. Basic deployment should never get in the way of that.

I gave a presentation on this topic at the January 2010 Sydney RORO meeting. Below are the slides and in this article I’ll write in more detail about the technique and why it is useful.

Making Your Capistrano Recipe Book

Background

I’ve just finished two years working in an applications development team at the Autralian Medical Council in Canberra. While I was there, we built many tightly-focused Rails apps. Currently there are eight in production and a number more under development. In order to keep things easy to manage, we did a lot of things to simplify the deployment process.

The first step we took was to automate the provision of homogenous virtual servers. I wrote a few thousand lines of bash, tied it to xen-tools, and hey presto, we could build new servers in one command and a couple of minute’s wait. Now that we had a standard server environment for each application, we saw another issue appear: lots of duplication in their Capistrano deploy.rb scripts. So the second measure was to extract all the common Capistrano tasks and configuration into a gem.

Getting Started

Here’s the kind of deploy script you might have for a simple application. First, you’ll have the standard information about your application, its repo, and the target host for deployment.

set :application, 'myapp.mycorp.com'
set :user, 'deployer'
set :deploy_to, "/home/deployer/deployments/#{application}"
set :use_sudo, false

role :app, '192.168.0.1'
role :web, '192.168.0.1'
role :db, '192.168.0.1', :primary => true

set :scm, :git
set :repository, 'git@git.mycorp.net:myapp.git'
set :branch, 'master'
set :deploy_via, :remote_cache

default_run_options[:pty] = true
set :ssh_options, { :forward_agent => true }

Then you’ll want to add some deployment tasks that work with Passenger:

namespace :deploy do
  desc "Restarting mod_rails with restart.txt"
  task :restart, :roles => :app, :except => { :no_release => true } do
    run "touch #{current_path}/tmp/restart.txt"
  end

  [:start, :stop].each do |t|
    desc "#{t} task is a no-op with mod_rails"
    task t, :roles => :app do ; end
  end
end

Finally, you might have some custom tasks and callbacks to set up your “Thinking Sphinx”:http://freelancing-god.github.com/ts/en/ installation.

before 'deploy:setup', 'sphinx:create_db_dir'
before 'deploy:setup', 'sphinx:generate_yaml'
after 'deploy:update_code', 'sphinx:symlink'

namespace :sphinx do
  desc 'Create a directory to store the sphinx indexes'
  task :create_db_dir, :roles => :app do
    run "mkdir -p #{shared_path}/sphinx"
  end

  desc 'Generate a config yaml in shared path'
  task :generate_yaml, :roles => :app do
    sphinx_yaml = <<-EOF
development: &base
  morphology: stem_en
  config_file: #{shared_path}/config/sphinx.conf
test:
  <<: *base
production:
  <<: *base
EOF
    run "mkdir -p #{shared_path}/config"
    put sphinx_yaml, "#{shared_path}/config/sphinx.yml"
  end

  desc 'Symlink the sphinx yml and config files, and the db directory for storage of indexes'
  task :symlink, :roles => :app do
    run "ln -nfs #{shared_path}/sphinx #{release_path}/db/sphinx"
    run "ln -nfs #{shared_path}/config/sphinx.yml #{release_path}/config/sphinx.yml"
    run "ln -nfs #{shared_path}/config/sphinx.conf #{release_path}/config/sphinx.conf"
  end
end

Creating a Gem Heirarchy

To extract some of the common patterns from the above deploy script, we’ll need to create a heirarchy of files that we’ll eventually turn into our gem. Here’s how it should look:

Capistrano Boilerplate

You’ll need something like the following wrapped around any Capistrano code that you use in your gem.

unless Capistrano::Configuration.respond_to?(:instance)
  abort "capistrano/ext/multistage requires Capistrano 2"
end

Capistrano::Configuration.instance.load do
  # stuff here...
end

The above will only work on Capistrano version 2 or greater (which should be no problem, now that we’re up to 2.5.13).

You could also do the following if you wanted something that works in all versions, including those before 2:

configuration = Capistrano::Configuration.respond_to?(:instance) ?
  Capistrano::Configuration.instance(:must_exist) :
  Capistrano.configuration(:must_exist)

configuration.load do
  # stuff here...
end

Extracting Deployment Code into your Gem

OK, so here’s some of the basic deploy script stuff moved into the lib/mycorp/base.rb file in the gem:

require 'capistrano/mycorp/common'

configuration = Capistrano::Configuration.respond_to?(:instance) ?
  Capistrano::Configuration.instance(:must_exist) :
  Capistrano.configuration(:must_exist)

configuration.load do

# User details
_cset :user, 'deployer'
_cset(:group) { user }

# Application details
_cset(:app_name) { abort "Please specify the short name of your application, set :app_name, 'foo'" }
set(:application) { "#{app_name}.mycorp.com" }
_cset(:runner) { user }
_cset :use_sudo, false

# SCM settings
_cset(:appdir) { "/home/#{user}/deployments/#{application}" }
_cset :scm, 'git'
set(:repository) { "git@git.mycorp.net:#{app_name}.git" }
_cset :branch, 'master'
_cset :deploy_via, 'remote_cache'
set(:deploy_to) { appdir }

# Git settings for Capistrano
default_run_options[:pty] = true # needed for git password prompts
ssh_options[:forward_agent] = true # use the keys for the person running the cap command to check out the app

end

These lines are lifted directly from the original deploy.rb, with one notable exception: many of the original @set@ calls have been replaced with _cset. _cset is a method used by the Capistrano’s internal deploy.rb file, which we have put inside common.rb:

def _cset(name, *args, &block)
  unless exists?(name)
    set(name, *args, &block)
  end
end

What _cset allows you to do is provide overrideable defaults for any Capistrano option. This is a powerful thing because it allows us to set conventions in our gem. As any Rails developer would know, conventions save time and make it easier for multiple people to work with the same tools or on the same code.

For example, take the user option from above. We’ve set it to ‘deployer’ by default. Because we’ve done it using _cset, any deploy script using this gem can choose to either accept this default or provide its own custom user details.

Note that whenever _cset (or set, for that matter) is called with the a block, it is lazily evaluated. This is important to do if you are including another value provided by a set or _cset call (like :app_dir above that uses the application setting).

Finally, take note of the @abort@ call in the block for setting app_name. You can use abort in your gem whenever there are settings that must be defined by the individual deploy scripts.

Tasks & Callbacks

The Thinking Sphinx tasks and callbacks can go straight into the gem without any modification:

configuration = Capistrano::Configuration.respond_to?(:instance) ?
  Capistrano::Configuration.instance(:must_exist) :
  Capistrano.configuration(:must_exist)

configuration.load do

_cset(:app_name) { abort "Please specify the short name of your application, set :app_name, 'foo'" }

before 'deploy:setup', 'sphinx:create_db_dir'
before 'deploy:setup', 'sphinx:generate_yaml'
after 'deploy:update_code', 'sphinx:symlink'

namespace :sphinx do
  desc 'Create a directory to store the sphinx indexes'
  task :create_db_dir, :roles => :app do
    run "mkdir -p #{shared_path}/sphinx"
  end

  desc 'Generate a config yaml in shared path'
  task :generate_yaml, :roles => :app do
    sphinx_yaml = <<-EOF
development: &main_settings
  config_file: #{shared_path}/config/sphinx.conf
  pid_file: #{shared_path}/pids/sphinx.pid
production:
  <<: *main_settings
EOF
    put sphinx_yaml, "#{shared_path}/config/sphinx.yml"
  end

  desc 'Symlink the sphinx yml and config files, and the db directory for storage of indexes'
  task :symlink, :roles => :app do
    run "ln -nfs #{shared_path}/sphinx #{release_path}/db/sphinx"
    run "ln -nfs #{shared_path}/config/sphinx.yml #{release_path}/config/sphinx.yml"
    run "ln -nfs #{shared_path}/config/sphinx.conf #{release_path}/config/sphinx.conf"
  end
end

end

In this example, I’ve included the callbacks as well as the tasks. Any deploy script that requires this file from the gem will have these sphinx tasks run automatically. This is useful to keep your deploy scripts concise and your deployment behaviour consistent, but it may not suit everyone. The alternative, to include only the tasks in the gem and require the callbacks to be in the deploy script, would make more sense if you’re planning to offer your gem to a wide audience with varying deployment requirements.

Building the Gem

Building the gem is easy with Jeweler. First, create a Rakefile with something like the following:

begin
  require 'jeweler'
  Jeweler::Tasks.new do |gemspec|
    gemspec.name = "capistrano-mycorp"
    gemspec.summary = "MyCorp recipes for Capistrano"
    gemspec.description = "MyCorp recipes for Capistrano"
    gemspec.email = "tim@openmonkey.com"
    gemspec.homepage = "http://github.com/timriley/capistrano-mycorp"
    gemspec.authors = ["Tim Riley"]
  end
rescue LoadError
  puts "Jeweler not available. Install it with: gem install jeweler"
end

Then generate a version number:

echo "0.0.1" > VERSION

And now you can run rake gemspec and rake build to generate a .gem file that you can install on your system. Jeweler also provides easy ways to get your gem onto Gemcutter or Rubyforge, so take a look at its README for more information.

The Result

Now that your gem is installed, you can take to your deploy script with a machete:

set :app_name, 'myapp'

role :app, '192.168.0.1'
role :web, '192.168.0.1'
role :db, '192.168.0.1', :primary => true

require 'capistrano/mycorp/base'
require 'capistrano/mycorp/thinking_sphinx'

That’s it! The result is a beautifully concise script that contains just the information that is pertinent to the application. It is easy to write and easy to read.

Suggested Uses

Building a gem of Capistrano recipes would be useful for:

Internal teams wanting to standardise their deployment process and remove duplicate code from multiple deploy scripts.
Agencies or freelancers wanting to reduce the amount of overhead required to start new projects.
Individuals wanting to share their Capistrano tricks with the community.
Hosting companies wanting to give their users a set of tasks for easily deploying to their servers.

Find Out More

You can see all my example code in the timriley/capistrano-mycorp repository on GitHub.

I hope this comes in handy!

2009 in a Few Lists

2010-01-07T04:50:00+00:00

First, the once-in-a-lifetimes:

Garmisch and I bought our first home and moved in. Inner-city living is great!
We organised our wedding
And got married!
Then had an exciting honeymoon in Phú Quốc, Sài Gòn and Hong Kong.

These were without doubt the highlights of the year, and together they form fond memories I will hold for years to come, and the backdrop to what I know will be an wonderful married life with Misch.

Plenty of other stuff happened. For starters, I accepted an offer to join the team at Icelab as Senior Developer. I’m excited to start to work with the talented folk there in just a few weeks! This meant that in December I wrapped up my couple of years at the AMC. I got to do some pretty neat things:

Built a payments service that has since processed over $10M and handles the credit card payments for four different public applications
Built a new redundant virtual server infrastructure to handle the AMC’s eight Rails apps, thanks to Xen, 3000 lines of custom bash scripts for automated provision (if only Chef came out earlier), haproxy and nginx for load balancers, MySQL replication, NFS shares backed by a distributed block device, and a gem chock full of shared capistrano recipes. Plenty of moving parts but things have run pretty smoothly since!
Came back fired up from RailsConf and managed to rejig (with the vital help of the team) the development methodology for my final project: pair programming all the time; tests first all the time; cucumber features first wherever possible; weekly iterations and user stories written and prioritised by the business.
Finally started building the simplest things that could first possibly work.
Put a lot of energy into helping build a positive and passionate team culture.

Let’s not forget the fun outside of work:

Went overseas for my first major web development conferences: RailsConf in Vegas and Ruby Kaigi in Tokyo.
Entered the Rails Rumble with Max and Hugh and built Decaf Sucks. It is still going strong with over 300 reviews.
Presented Decaf Sucks at WebJam 10 in Sydney.
Founded a startup with three others and got busy working on our product.
Did a little open source stuff, most notably acts-as-importable, campfire-bot, unfuddle-helpdesk and openmonkey.com.

2010 is looking like it will be a ripper!

The Atomic Shell Script

2009-11-04T06:25:00+00:00

The single file is the fundamental building block of unix and the smallest unit that you work with when configuring servers. So, when scripting your repetitive server tasks, keeping your code contained within a similarly singular, atomic file is a worthy goal for the simplicity and convenience that it brings. A way to achieve this for complex scripts may not always be immediately clear, but it usually both possible and worth the effort.

[caption id=”” align=“alignnone” width=“500.0”] Image from freakdog.[/caption]

Here’s my example. When we started using request-log-analyzer to periodically analyse the logs from one of our Rails apps, it looked like I would need to use multiple files:

A Ruby script to call request-log-analyzer with the appropriate log file and output the data we want.
Another Ruby file to contain the definition for our custom log analysis rules (this is a requirement for request-log-analyzer).
An SQLite database file for storing the parsed data from the log files.

What I initially was hoping to do in a single script had become a tenuous concert of interdependent files that would no doubt require documentation for my teammates to install and maintain. With a little creativity, I overcame the issue and reduced everything to a single script, achieving simplicity of installation and use. First, I’ll show you the script (abridged for this article), and then the explanation of my technique:

#!/usr/bin/env ruby

require 'time'
require 'fileutils'
require 'tempfile'

require 'rubygems'
require 'activesupport'
require 'sqlite3'
require 'ruport'

Tempfile.class_eval do
  # Remove the dashes that Tempfile would otherwise put in the tmpname.
  def make_tmpname(basename, n)
    case basename
    when Array
      prefix, suffix = *basename
    else
      prefix, suffix = basename, ''
    end

    t = Time.now.strftime("%Y%m%d")
    path = "#{prefix}#{t}#{$$}#{rand(0x100000000).to_s(36)}#{n}#{suffix}"
  end

  def klassify
    File.basename(self.path, '.rb').gsub(/\/(.?)/) { "::" + $1.upcase }.gsub(/(^|_)(.)/) { $2.upcase }
  end
end

LIB_FILE = Tempfile.new(['file_format', '.rb'])
DB_FILE = Tempfile.new(['questions', '.sqlite3'])
LOG_FILES = ARGV.blank? ? Dir["/var/log/rails/rails-questions-production/rails-questions-production-#{Time.now.yesterday.strftime('%Y%m%d')}*"].first : ARGV.join(' ')

LIB_FILE.puts "class #{LIB_FILE.klassify} < RequestLogAnalyzer::FileFormat::Rails\n"
LIB_FILE.puts <<'EOF'
  line_definition :current_user do |line|
    line.regexp = /Logged in as: (.+)/
    line.captures << { :name => :email, :type => :string }
  end
end
EOF
LIB_FILE.flush

unless system("request-log-analyzer -f #{LIB_FILE.path} -d #{DB_FILE.path} #{LOG_FILES} > /dev/null")
  exit "Error running request-log-analyzer"
end

db = SQLite3::Database.new(DB_FILE.path)

all_logins_query = <<EOQ
SELECT email AS user_email FROM current_user_lines GROUP BY user_email;
EOQ

all_logins_rows = db.execute2(all_logins_query)
all_logins_table = Ruport::Data::Table.new(:column_names => all_logins_rows.shift, :data => all_logins_rows)

puts
puts "All Logins"
puts all_logins_table.as(:text, :ignore_table_width => true)

# Cleanup
LIB_FILE.close!
DB_FILE.close!

The trick here is that the single ruby script creates all the auxiliary files that it needs, every time it runs! Ruby’s Tempfile is your friend, since it will take care of creating unique temporary files for you, and clean them up again when you call #close!.

The other trick is getting request-log-analyzer to accept a Tempfile for its custom file format definition. It expects a single class to be defined inside this file, with a name matching the name of the file. Nothing a little monkey patch to Tempfile couldn’t handle. I overwrote make_tempname to ensure no dashes were used in the file name (these aren’t allowed in the names of Ruby classes), and then I added a #klassify instance method to convert the filename into the kind of class name that request-log-analyzer would expect.

I’m happy with the way this his script turned out. It may be a little more involved, but the result is far more explicit, without any dependence on other moving parts. This singular, atomic script is easy to install and maintain, and requires no documentation apart from its own source code.

Complex Nested Forms with Rails and Unobtrusive jQuery

2009-10-13T00:20:00+00:00

I came across Ryan Bates’ complex-form-examples project when I needed to build a complex form recently. It’s an excellent educational reference for building these complex forms in a Rails app. You know the kind of forms, the ones where you want to add or remove an arbitrary number of child associations of a parent record. True to form, the latest version of Ryan’s example app uses Rails’ new accepts_nested_attributes_for, which lets you create, edit or delete the collection of associated child objects just by passing through the appropriate attributes to the parent record.

The example app works beautifully, but a couple of its approaches didn’t match how I wanted to build my app:

It uses inline JavaScript. The JavaScript triggers and code to dynamically add and remove parts of the form for child objects are mixed in with the rest of my page markup, not in application.js or another file dedicated to my JavaScript and behaviour.
The JavaScript function to add fields to the form takes a string argument that contains all the HTML markup for these forms. Now, I’m no expert, but this just didn’t seem as clean or efficient as possible. [caption id=”” align=“alignnone” width=“500.0”] Image from carsten_tb.[/caption]

So when I incorporated Ryan’s examples into my application, I made a couple of changes:

All the JavaScript is unobtrusive. The behaviour is kept in my application.js along with the rest of my JavaScript.
This means the page markup is cleaner. The links to add and remove child elements have no inline JS, just a certain class name for jQuery to hook onto.
The JavaScript functions that add new fields to the form don’t get given the markup for those fields in an argument. The fields are already in the form, wrapped in a div that is hidden with a display: none rule. The JavaScript function finds these fields in the DOM and then duplicates them and inserts them into the right place. In essence, the hidden fields act as a template.
For the form to work with multiple child objects, the links to add new fields include an HTML5 custom data attribute called data-association to store the appropriate association name for the child. The JavaScript uses this field to find the right hidden template fields (see above). Neat!
All the JavaScript uses jQuery, because that matched the rest of my application. No big deal.

I applied these changes to an unobtrusive-jquery branch in my fork of complex-form-examples. Please clone or fork it to take a look and make any improvements! I hope it can come in handy. Thanks to Ryan Bates for his excellent work in building the example application.

Decaf Sucks, and a Rails Rumble Redux

2009-09-02T06:35:00+00:00

Decaf Sucks

48 hours. 3 hackers (Max, Hugh and yours truly), 1 web application built from scratch. Allow me to introduce you to Decaf Sucks, our Rails Rumble entry and a site for helping you find great coffee.

We hate bad coffee. We hate the way it tastes, the way it smells, and most of all we hate paying for it. Decaf Sucks is about helping each other to find the good cafés and avoid the bad ones.

So check it out and leave a review. Also be sure to try the site on your iPhone. It uses the GPS to enable you to find the best cafés nearby, and write a review as you bask in the sweet glow of a just finished cup of coffee.

Lessons Learnt Starting Up Fast

We’ve been trying to build this app for over a year. We would have spent well in excess of 48 hours together working on our first prototype. The trouble was that we dreamt big and started to build an app that would do everything at once. Finally, it took the Rails Rumble to get us somewhere. We were forced focus on a core experience and how we could implement it quickly. I’m very pleased with the result, and would like to share the things I have learnt in competing over the last three Rumbles.

Build something you want to use. There is no stronger motivation than this.

Say no, and do it early. Define the core experience your application will provide and build that well. Save any other ideas for later.

Have a designer on your team. This is the single biggest improvement to our previous Rumble entries. It was great to have the talented Max on the team. First impression and overall polish counts for a lot. Allow your app to make the best impression!

Track your outstanding tasks in meatspace. Try a whiteboard or a large sheet of butcher’s paper. This is an easy technique to keep the group mindful of what is left to do. It’s also very satisfying to write a big tick or strike through a completed task.

Work in the same place. This is the fun part: hack together, share your meals and enjoy the weekend. This also makes it easy to discuss your app, form consensus quickly, and get back to building.

Don’t plan much for Monday. When it comes to Sunday afternoon and you realise you still have many hours left of work to do, you don’t want anything to get in the way. The last two years we have worked through Sunday night to finish our apps. Each time it has been worth it. If you are an employee and working in any kind of IT role, tell your boss that this is 40 hours of the most intense, practical and relevant training anyone could receive, and hopefully they see reason and give you the time off. (This only applies if you are in a timezone similar to GMT +10)

Have someone focus solely on user experience. This doesn’t necessarily have to be your designer.

Keep most of your site accessible without a login. You’ve worked hard. Minimise the number of hurdles required for people to admire it.

Outsource your authentication. When you do need authentication, outsource it. Use Twitter, Facebook, or OpenID. Your choice will depend on the kind of audience you’re seeking.

Use only one or two models. Key models at least. Any more and the chances are that you will have more pages to build than your weekend will allow.

Use good plugins. These will save you so much time! The heroes for us this year were twitter-auth and geokit. Both are well built and well documented, with a great out of the box experience. They offered no unpleasant surprises, meaning we could integrate them easily.

Refer to Railscasts! This site is an invaluable resource. If you’re trying something new with Rails, chances are that there is already a screencast about it.

Build your server early. Towards the end of the weekend, you’ll be focused on completing your application and deploying often. Don’t let an outstanding server build get in the way.

Build your server fast. Don’t let your server build consume too much time either. Use an automated provision tool like passenger-stack. If you predict that you will have an involved setup process, buy a slice for a week or two before the competition, build the server to your specifications, and document your process.

Something Different

Now for some things that I would try differently next time:

Implement a working grey box design first. I find it a lot easier to build an application backend if it is at least for a prototypical frontend.

Write Cucumber integration tests for the application’s “happy path.” As the last long night wore on, we were making plenty of changes that had the potential to break things. It would have been good to have some tests confirm everything still worked as expected. I don’t think it’s feasible over the weekend to build your app via TDD/BDD, but a small set of integration tests would help without being too burdensome.

Write these Cucumber features ahead of time. I don’t think this would break the competition rules and it would be useful to have these from the beginning. This would also help you and your teammates crystallise the application’s functionality before the build weekend.

Context

One of the reasons I like competing in the Rails Rumble is that it provides sweet, glorious context. If you’re working as a programmer somewhere that requires your attention in many places, frequent context shifts are a fast way to swap happy productivity for overall malaise. The Rails Rumble is a way to rejuvenate yourself. Gustavo Duarte captures the experience well:

Few things are better than spending time in a creative haze, consumed by ideas, watching your work come to life, going to bed eager to wake up quickly and go try things out. I am not suggesting that excessive hours are needed or even advisable; a sane schedule is a must except for occasional binges. The point is that programming is an intense creative pleasure, a perfect mixture of puzzles, writing, and craftsmanship.

While it is indeed one of those occasional binges, this year’s Rails Rumble was a timely personal reminder of why I play this game: to focus on creating new and helpful things in interesting ways. See you next year.

Show Me the Page!

2009-08-06T00:15:00+00:00

Get this in your Cucumber steps.

Then /^show me the page$/ do
  save_and_open_page
end

Thanks to the magic of webrat’s lesser known save_and_open_page, this will save the current page to a temporary file and open it in your browser. Be sure to gem install launchy for it to work.

Here’s a sample of show me the page in a scenario:

Scenario: Signing in
  When I go to the home page
  And I fill in "username" with "john"
  And I fill in "password" with "john"
  Then show me the page

Very useful for debugging.

Machinist and Paperclip

2009-07-25T03:55:00+00:00

Once you’ve found a comfortable fixture replacement for your tests, there is no going back. For many, that’s something like Factory Girl. For me, it is Pete Yandell’s Machinist. So when I wanted to write some Cucumber scenarios for a model using Paperclip for file attachments, I definitely needed a factory for creating it!

In Machinist, building factories (called blueprints) is easy for standard ActiveRecord attributes and associations. Check out the comprehensive README in the repository. Here is the start of a blueprints.rb file:

require 'faker'

Sham.title { Faker::Lorem.sentence }

AttachedDocument.blueprint do
  title
end

As it turns out, it is equally easy to add support for models using Paperclip. Say our model looks like this:

class AttachedDocument < ActiveRecord::Base
  has_attached_file :document
end

Then your blueprint should incorporate the attached file like this:

require 'faker'

Sham.title { Faker::Lorem.sentence }
Sham.document { Tempfile.new('the attached document') }

AttachedDocument.blueprint do
  title
  document
end

Using a Tempfile instance to represent the paperclip attachments works without a hitch, and the implementation of Tempfile will ensure there are no file duplicates. It will also mean that the other methods that paperclip sets up (such as document_file_name, document_content_type and document_file_size) are all accessible on the generated object. Now get going and test!

Fast GitHub Clone Bash Function Using the OS X Clipboard

2009-07-21T06:40:00+00:00

Bash Functions Save Time

The keen programmer never has any shortage of new and interesting open source projects to inspect, and for me, GitHub is increasingly the place where the ones I want to see are hosted.

Whenever I want to have a good close look at something, I’ll make a clone of the project to my computer and use TextMate to poke around the source. If I want to keep the source around for reference and have no intention to modify it, I’ll clone the project to ~/Code/sources/<user_name>-<repository_name>. I started using this naming convention to help me quickly identify where again to find the upstream copy of the project when I return to it.

I’ve been following this pattern for quite some months now, and in my ongoing effort to decrease keystrokes, I made a bash function to take care of it for me:

function ghclone {
  gh_url=${1:-`pbpaste`}
  co_dir=${HOME}/Code/sources/$(echo $gh_url | sed -e 's/^git:\/\/github.com\///; s/\//-/; s/\.git$//')

  if [-d $co_dir]; then
    cd $co_dir && git pull origin master
  else
    git clone "${gh_url}" "${co_dir}" && cd "${co_dir}"
  fi
}

Optional Arguments and Clipboard Access Save Even More Time

This ghclone function accepts a single argument, the URL of the repository on GitHub. If it does not exist as a local clone, it will clone it and leave you in its directory. If you’ve already cloned it before, it will take you to the clone and update it.

This single function argument is optional, thanks to Bash’s cool parameter substitution capabilities. The gh_url variable is set from the function argument if it is passed, or from OS X’s clipboard:

gh_url=${1:-`pbpaste`}

For those of us who are used to more expressive languages like Ruby, it is akin to this:

gh_url = ARGV[0] || `pbpaste`

The pbaste utility is shipped by default with Mac OS X, and returns the current value in the system clipboard. This is cool! It means that, with this shortcut function, I can clone GitHub repositories very quickly:

Click the copy button next to the repository URL on GitHub.
Open the terminal and type ghclone
Start exploring!

Throw the function into your ~/.bash_profile to get started. If pbpaste is interesting to you, be sure to check its counterpart, pbcopy, which you can use to populate the clipboard from the shell.

Thinking Sphinx RSpec Matchers

2009-07-18T23:40:00+00:00

How do you use RSpec to drive the design of your models that will use Thinking Sphinx for search? Say you’re already using Cucumber for integration tests to verify your index builds correctly and searches return the results you expect. For your models’ specs, you’ll want something that is lighter but doesn’t sacrifice your overall test coverage.

To achieve this, I wrote a couple of small RSpec matchers that inspect the Thinking Sphinx index object on your model to ensure that it contains the fields and attributes that you expect.

Spec::Matchers.define(:index) do |*field_names|
  description do
    "have a search index for #{field_names.join('.')}"
  end
  match do |model|
    all_fields = field_names.dup
    first_field = all_fields.pop

    model.sphinx_indexes.first.fields.select { |field|
      field.columns.length == 1 &&
        field.columns.first.__stack == all_fields.map { |s| s.to_sym } &&
        field.columns.first.__name == first_field.to_sym
    }.length == 1
  end
end

Spec::Matchers.define(:have_attribute) do |*attr_names|
  description do
    "have a search attribute for #{attr_names.join('.')}"
  end
  match do |model|
    all_attrs = attr_names.dup
    first_attr = all_attrs.pop

    model.sphinx_indexes.first.attributes.select { |attr|
      attr.columns.length == 1 &&
        attr.columns.first.__stack == all_attrs.map { |s| s.to_sym } &&
        attr.columns.first.__name == first_attr.to_sym
    }.length == 1
  end
end

Put these matchers in your spec_helper.rb or somewhere else handy, and then you can use them in your model specs:

describe Question do
  it { should index(:topic) }
  it { should have_attribute(:state) }
end

They read quite nicely in the single-line format above, and the matchers provide a readable description when you run the spec:

Question
- should have a search index for topic
- should have a search attribute for legacy_mastery

While these matchers work well for me, I feel that @sphinx_indexes@ is perhaps an object I should leave alone, and not something I can rely on having continued access to. Please leave a comment if you have any suggestions for doing this more cleanly!

What I did learn, however, is how simple it was to write custom matchers for RSpec. If you haven’t tried it before, I strongly suggest you give it a go! RSpec’s matcher DSL is straightforward, and the documentation has everything you need to get started.

Using RSpec Ordered Message Expectations to Tighten your Specs

2009-07-01T13:35:00+00:00

I quite enjoy the competitive undercurrent of ping pong pair programming. As the person writing the implementation code, it is fun to write something that will turn a test green, but still not necessarily do what my partner was expecting. Taking this approach has also been helpful for improving our specs. Take this example controller spec:

describe ArticlesController do
  describe "handling create" do
    before(:each) do
      @article = mock_model(Article, :save => nil)
      Article.stub(:new).and_return(@article)

      @user = mock_model(User)
      controller.stub(:current_user).and_return(@user)
    end

    it "should build a new article from posted data" do
      Article.should_receive(:new).with('title' => 'Test Post')
      post :create, :article => {:title => 'Test Post'}
    end

    it "should assign the current user as the article's author" do
      @article.should_receive(:author=).with(@user)
      post :create
    end

    it "should save the article"
      @article.should_receive(:save)
      post :create
    end
  end
end

This looks like a reasonable set of concise, clear examples, but you can easily make them all pass and without building a controller action that does what you expect:

class ArticlesController < ApplicationController
  def create
    @article = Article.new(params[:article])
    @article.save
    @article.author = current_user
  end
end

This satisfies the examples, but saving the article before assigning the current user as author isn’t what we would have intended. Enter RSpec’s ordered message expectations. These allow you to specify the order in which you expect an object to receive message calls.

describe ArticlesController do
  describe "handling create" do
    it "should save the article after assigning the current user as author"
      @article.should_receive(:author=).with(@user).ordered
      @article.should_receive(:save).ordered
      post :create
    end
  end
end

This example would fail with the above controller action, and force us to write it properly:

class ArticlesController < ApplicationController
  def create
    @article = Article.new(params[:article])
    @article.author = current_user
    @article.save
  end
end

The result is a controller that does what you expect, a stronger set of specs, and an increased capacity for true behaviour driven development. Win, win, win!

De-@wip Your Cucumber Stories

2009-06-29T06:55:00+00:00

We’ve been really hitting the Cucumber hard in the recent few iterations of our current project. Now that we’re ping pong pair programming, the Cucumber story is the first thing we write, and we revisit regularly during the red-green-refactor cycle.

To make this easy, we place a @wip work in progress tag at the top of the current stories:

Feature: Make coffee

  # This one is done.
  Scenario: First coffee of the day

  # This is the one we're working on.
  @wip
  Scenario: Afternoon perk up

Then we use cucumber -t wip to run just the stories in progress. Once the stories are green and we’re happy to move on, we often forget to remove the @wip tags before committing. I wrote a little sed command in a bash alias to make this easier:

# Remove any @wip tags from Cucumber features.
alias dewip="sed -E -i '' -e '/^[[:blank:]]*@wip$/d;s/,[[:blank:]]*@wip//g;s/@wip,[[:blank:]]*//g' features/**/*.feature"

Throw it in your .bash_profile for ease of access! Here’s a gist for your forking pleasure.

Rails Templates as the New Geek Code

2009-05-20T02:30:00+00:00

Anyone who’s been around the Internet long enough should remember The Geek Code. This meme sought to provide a - however obsbcure - succinct textual distillation of the attributes and interests of any geek.

GED/J d-- s:++>: a-- C++(++++) ULU++ P+ L++ E---- W+(-) N+++ o+ K+++ w---
  O- M+ V-- PS++>$ PE++>$ Y++ PGP++ t- 5+++ X++ R+++>$ tv+ b+ DI+++ D+++
  G+++++ e++ h r-- y++**

This geek code block represents someone trained in education and law who is tall and dresses casually, knows his way around Linux and Ultrix pretty well, hates emacs, but loves indulging in a little Dilbert. Look it up, it’s elaborate.

Today, a Rails developer can provide a template for generating new apps that can uniquely embody all their development tools and preferences in a single place.

gem 'haml'
gem 'mislav-will_paginate', :lib => 'will_paginate', :source => 'http://gems.github.com'
gem 'chriseppstein-compass', :lib => 'compass', :source => 'http://gems.github.com'
gem 'thoughtbot-paperclip', :lib => 'paperclip', :source => 'http://gems.github.com' if yes?('Paperclip gem?')
gem 'authlogic' if yes?('Authlogic gem?')

You’re a haml guy? Right on. Plus compass for CSS! You must like really things semantic.

file '.testgems',
%q{config.gem 'rspec'
config.gem 'rspec-rails'
config.gem 'notahat-machinist', :lib => 'machinist', :source => 'http://gems.github.com'
config.gem 'ianwhite-pickle', :lib => 'pickle', :source => 'http://gems.github.com'
config.gem 'webrat'
config.gem 'cucumber'
}
run 'cat .testgems >> config/environments/test.rb && rm .testgems'

RSpec along with Cucumber backed by Machinist and Pickle for test data factories. You must be Australian. But that’s cool, that’s a pretty helpful combo for tests.

git :init
git :add => '.'
git :commit => '-a -m "Initial commit from AMC Rails template"'

Building a complete Rails template has been really useful for us at the AMC. We’re often asked to bring up quick, single-purpose apps alongside the many components that we’re building in our mostly service-oriented architecture.

Our template is available on GitHub for your perusal and reuse. It does everything from setting up plugins and gem dependencies, to generating a deploy script using our custom capistrano extensions, to including a default layout and stylesheet. This is how we roll, captured in a single file.

Importing Legacy Data in Rails

2009-05-01T22:10:00+00:00

How Did we Get Here?

Our current work project is a long-overdue rebuild of a critical business system as a modern Rails web app. We’re building this thing according to agile practices to the best of our ability. Each week we provide a new, working release that is an incremental improvement on the last. We’re not looking to replace the current system in a single swoop, and expect the new system to work alongside the old one for quite a while.

This means that we’ll need to maintain the same data in both systems for the duration of their lives together. We don’t want the new Rails app to access the data directly from the old app’s database, since this would prevent us from following the conventions that makes working with Rails so pleasant. Instead, we’ve come up with a way to import the legacy data into a new database.

In doing this, I’ve built a Rails plugin to make the experience easier. It is called Acts as Importable and it is now available on GitHub. This article will show our technique for importing the legacy data and how Acts as Importable helps.

Accessing the Legacy Data

We use ActiveRecord to access the legacy data. It takes a bit of legwork to shoehorn the legacy schema into ActiveRecord models, but once that is done, we have satisfactory access to the data we want to import.

The first thing we do is provide a Legacy::Base model from which all other legacy models can inherit. This provides a single place to define access to the legacy database.

class Legacy::Base < ActiveRecord::Base
  self.abstract_class = true
  establish_connection "legacy_#{Rails.env}"

  acts_as_importable
end

You can setup the extra database connections in config/database.yml, like below. Our legacy connection is to an MS SQL server via ODBC. Tim Lucas has an excellent tutorial on how to get that set up.

legacy_development:
  adapter: sqlserver
  mode: odbc
  dsn: LEGACY
  autocommit: true
  host: localhost
  username: NTDOMAIN\username
  password: password

Now, here are a couple of the legacy models that inherit from Legacy::Base to access the legacy database. For the sake of this article, consider the code examples to come from a quiz application with models for categories, questions, and responses.

class Legacy::Question < Legacy::Base
  set_table_name 'quiz_questions'
  set_primary_key 'QuestionNumber'

  belongs_to :category,
             :class_name => 'Legacy::Category',
             :foreign_key => 'CategoryCode'
end

class Legacy::Category < Legacy::Base
  set_table_name 'quiz_categories'
  set_primary_key 'Code'
end

These are simple examples, but you get the idea. set_table_name and set_primary_key are your friends when you have table names and keys that defy Rails’ conventions. If you want to use ActiveRecord associations to access your legacy data, you will also want to become familiar with the options available for specifying things like the class, join table and column names. Check the rdocs for has_many, belongs_to and has_and_belongs_to_many and look out for options like :class_name, :foreign_key, :join_table, and :association_foreign_key.

Converting the Legacy Data

Each of the legacy models provides a to_model method that returns a new model ready to be saved into the new, non-legacy database.

# New model
class Category < ActiveRecord::Base
end

# Legacy model
class Legacy::Category < Legacy::Base
  set_table_name 'quiz_categories'
  set_primary_key 'Code'

  def to_model
    ::Category.new do |c|
      c.name = self.Description
    end
  end
end

Importing the Legacy Data

Now that the import rules are defined for all the legacy models we care about, how do we get them all into the new database? This is where Acts as Importable comes in. This plugin helps to smoothen the import of an entire system’s worth of legacy data. You saw it enabled above in the Legacy::Base class:

class Legacy::Base < ActiveRecord::Base
  self.abstract_class = true
  establish_connection "legacy_#{Rails.env}"

  acts_as_importable
end

Acts as importable will let you import your legacy models in different ways:

# Import a single question
Legacy::Question.import(123)
# or
Legacy::Question.find(123).import

# Import all questions
Legacy::Question.import_all

# Import all the questions, 1000 at a time
Legacy::Question.import_all_in_batches

These are all different ways of instantiating your legacy models, calling the #to_model method, and saving the returned object. The value of the plugin is that it adds a little bit of extra smarts to this basic premise.

belongs_to Relationships & Caching

Acts as Importable provides a #lookup class method for finding the new ID of an imported legacy model. When each legacy model is imported using the above methods, Acts as Importable saves the legacy model’s class name and ID along with the rest of the data in new model (be sure to provide legacy_class and legacy_id columns for this to work). The first time you lookup a legacy record, it uses ActiveRecord::Base#find with the appropriate values for legacy_class and legacy_id as conditions. It will save the ID in a lookup hash in memory, so that the next time you want to lookup from the same ID, the result is returned without a trip to the database.

This is really best seen in action:

# New models
class Question < ActiveRecord::Base
  belongs_to :category
  has_many :responses, :dependent => :destroy
end
class Category < ActiveRecord::Base
  has_many :questions
end

# Legacy model
class Legacy::Question < Legacy::Base
  def to_model
    ::Question.new do |q|
      # import the category association
      q.category_id = Legacy::Category.lookup(self.category.try(:id__))
    end
  end
end

Setting the id for associations directly like this is important for conserving trips to the database importing a model’s belongs_to relationships, just like the one above. Be wary when using lookups for large data sets, however, as it will likely consume quite a bit of memory to store the ID mappings.

As an aside, we use #try(:id__) on the legacy model to provide the ID for the lookup, because #id on a nil value actually returns its #object_id. If the associated object doesn’t exist, we don’t want to pass a bogus ID. Acts as Importable provides Object#try and the ActiveRecord::Base#id__ to id alias for you.

The lookups will work automatically if the class name of the model you’re importing is the same as the legacy model’s, eg. Legacy::Question to Question. If the class name of the model you’re creating is different, you can tell that to Acts as Importable so that the lookups can continue to work:

# New model
class DifferentThing < ActiveRecord::Base
end

# Legacy model
class Legacy::Thing
  acts_as_importable :to => 'DifferentThing'

  def to_model
    ::DifferentThing.new
  end
end

has_many Relationships & Building

You can import has_many relationships quite easily, using the #build method on the association proxy. Here it is in action, expanding on the Question’s #to_model method from above:

# New models
class Question < ActiveRecord::Base
  belongs_to :category
end
class Response < ActiveRecord::Base
  belongs_to :question
end

# Legacy model
class Legacy::Question < Legacy::Base
  def to_model
    ::Question.new do |q|
      q.category_id = Legacy::Category.lookup(self.category.try(:id__))

      # Build the responses
      (1..5).each do |i|
        q.responses.build(:body => self.send(:"response_#{i}"))
      end
    end
  end
end

When you save the new model, the newly built associated models are saved too.

Importing Large Sets of Legacy Records

We ran into some slowness importing legacy models with large numbers of records, or with other models with large amounts of data in particular fields. To get around this, we use #import_all_in_batches, which only retrieves 1000 models at a time for processing. This is based on Jamis Buck’s technique for faking cursors in ActiveRecord, and as such, it requires a numeric primary key for the legacy models (you’d normally expect this to be the case, but it isn’t for a few of our legacy tables).

Idempotence of Imports

As I mentioned in the introduction, the legacy app we’re replacing will remain in use as we incrementally build the new system. We will need to continue to synchronise the legacy data with the new system during this time. We’ll therefore need our import process to be idempotent, meaning that multiple imports can run and result in the same set of data at the other end. Mostly, this just means that we’ll want to avoid creation of duplicate records in the new database.

We approach this pretty simply. Each night the old records are deleted and new ones re-imported in their place. You’ll want to pick an approach that best suits your situation.

There is one complication, however, insofar as the new system is used to create some entirely new content that relates to the imported models. If models are deleted and re-imported from the legacy system every night, their IDs would be different each time. To get around this, we hard-code the ID of certain imported models to the same value as their respective legacy model’s ID. This is very simply done:

class Legacy::Question < Legacy::Base
  def to_model
    ::Question.new do |q|
      # Hard-code the ID
      q.id = self.id

      q.category_id = Legacy::Category.lookup(self.category.try(:id__))

      (1..5).each do |i|
        q.responses.build(:body => self.send(:"response_#{i}"))
      end
    end
  end
end

Final steps

There are two final pieces to our import system. Firstly, a way to control the order of the imports:

class Legacy::Importer
  def self.run
    Legacy::Category.import_all
    Legacy::Question.import_all_in_batches

    # Flush all the lookup tables
    Legacy::Category.flush_lookups!
    Legacy::Question.flush_lookups!
  end
end

It is important to control the order if your imported models are going to relate to each other. Some records will need to exist before the others can link to them.

And finally, a rake task:

namespace :legacy do
  desc "Import the legacy data."
  task :import => :environment do
    Legacy::Importer.run
  end
end

That’s it! This approach certainly works to our liking, but I would love to hear your thoughts on this issue. Please feel free to post a comment below.

Adaptive script/console Shell Alias for both Rails and Sinatra

2009-03-06T05:25:00+00:00

Like many keystroke-efficient Rails hackers, I’ve long had a line in my .bash_profile file to alias sc to script/console, along with a host of other tricks.

This shortcut was more than sufficient until recently, when I started writing Sinatra apps. The minimal framework that it is, Sinatra doesn’t provide a console script like Rails, but I found you can easily achieve the same effect by running irb -r your_sinatra_app.rb.

Not wanting my fingers to have to deviate from habits long held, I changed my sc alias into a full-blown bash function that will drop you into a Rails console, Sinatra console or just a plain irb console based on your location within the filesystem:

function sc {
  if [-x script/console]; then
    script/console
  else
    sinatra_rb=`egrep -l "^require.+sinatra.$" *.rb 2>/dev/null`
    if [-e $sinatra_rb]; then
      irb -r $sinatra_rb
    else
      irb
    fi
  fi
}

Throw it in your .bash_profile and have fun!

Cucumber steps for testing page URLs and redirects

2009-03-03T23:35:00+00:00

With the new project at work, we’re making sure we follow the best practices we know. This means building the app in weekly iterations according to user stories that we review and schedule with the business owners. It also means being strict about writing tests first, and so far we’re doing pretty well.

We’re using the shiny new Cucumber as much as possible for our high-level integration testing. Along with Cucumber’s webrat steps, it is easy to write features that, among other things, can request pages, fill in forms, and check for text on the returned pages. When we add in Ian White’s pickle, we also have integration with Rails’ named routes to keep your features readable and steps DRY.

We often use this one webrat step for visiting a page:

When /^I go to (.+)$/ do |page_name|
  visit path_to(page_name)
end

This is the only step that webrat provides relating to pages and paths. We found that we needed a couple more:

Then /^I should be on the (.+?) page$/ do |page_name|
  request.request_uri.should == send("#{page_name.downcase.gsub(' ','_')}_path")
  response.should be_success
end

Then /^I should be redirected to the (.+?) page$/ do |page_name|
  request.headers['HTTP_REFERER'].should_not be_nil
  request.headers['HTTP_REFERER'].should_not == request.request_uri

  Then "I should be on the #{page_name} page"
end

With these steps, we can properly check the URL of the page we’ve been returned, and whether or not we have been redirected during that request cycle. This was important for us to have the fullest coverage possible for our authorization features:

Feature: Users cannot access to the system without logging in
  In order to protect the system from unauthorized access
  An anonymous user
  Should not have access to the system

  Scenario: Visiting the login page
    Given an anonymous user
    When I go to the new login page
    Then I should be on the new login page

  Scenario: Redirecting to login page
    Given an anonymous user
    When I go to the home page
    Then I should be redirected to the new login page

The first scenario ensures that we don’t end up in a redirect loop for an anonymous user directly visiting the login page, and the second scenario ensures that an anonymous user accessing any page in the system is appropriately blocked from access and redirected to the login page.

Business Cards and Hacker Groups

2009-02-27T23:00:00+00:00

I’m very happy to announce that this year I’ll be attending my first software development conference, RailsConf 2009, thanks to the generous sponsorship of my employers, the Australian Medical Council. This fantastic opportunity has brought forward some thought on what I had already planned for this year: to make some business cards.

I like the idea of having my key contact details ready on a single card to hand to someone new if we’ve started a conversation that we’d like to continue later. I can see the utility particularly in situations where I meet people in a business context or in opportunities for discussing freelance work. It means we can talk as much as possible about whatever we like without needing to worry about the clumsy manual transmission of contact information.

[caption id=”” align=“alignnone” width=“500.0”] Image from cjc.[/caption]

I’m not so sure about the value of business cards in the context of this conference and a big group of hackers. I am keenly looking forward to the chance to meet many new people at RailsConf, but I do not know how to most easily preserve contact with them. For example, when I meet people at RORO events, I typically ask them if they are on the twitter, then get out my iPhone and jot their name down into the notes app. I can foresee this becoming difficult to scale to larger groups.

I’d like to ask your advice. Would it be useful to have my twitter/web contacts down on a card for giving away in these same situations in the future? Is this acceptable in this community? Or should I just take things as they come: not worry about remembering the details of everyone I meet, accept that they will take much the same approach, and throw away the idea of business cards for these situations? How do you use business cards?

Handy Shell Alias for Restarting your Rack Apps

2009-02-25T06:50:00+00:00

alias r='touch tmp/restart.txt'

Most useful if you’re using the Phusion Passenger and its Preference Pane to serve your apps through Apache on OS X.

Installing Gitosis on Dreamhost

2009-02-25T01:00:00+00:00

Hugh’s forays back into shared hosting over the last week have reminded me that I could make better use of the cheap Dreamhost account that I have had languishing for the last few years. I am going to migrate the few things I have running on my more expensive Slicehost slice. Quick way to save a bit of money each month, particularly with the equally languishing Australian Dollar.

First of these is Gitosis, which is a very handy way to securely store and share your private Git repositories. Since most accounts on Dreamhost are on shared host, you’ll need to install it inside your home directory. Turns out this is quite simply done, and Marco Borromeo provides a good tutorial.

I had to take an additional couple of steps in order to get things to work with the Python 2.4 installation on my particular host, so I have provided a complete list here. This should hopefully work on any of the Dreamhost servers. Please leave a note in the comments if you have to do anything differently.

The first step you’ll need to do is create a dedicated user account for the gitosis installation. I also chose to create a dedicated subdomain at the same time. Then, all you’ll need to do is follow these steps:

# 1. Create dirs for unpacking your source code and installing your apps
mkdir $HOME/src
mkdir $HOME/apps

# 2. Install the latest version of git
cd $HOME/src
wget http://kernel.org/pub/software/scm/git/git-1.6.1.3.tar.gz
tar zxvf git-1.6.1.3.tar.gz
cd git-1.6.1.3
./configure --prefix=$HOME/apps NO_MMAP=1
make && make install

# 3. Create dir for local python modules
mkdir -p $HOME/apps/lib/python2.4/site-packages
export PYTHONPATH=$HOME/apps/lib/python2.4/site-packages

# 4. Install setuptools python module
cd $HOME/src
wget http://peak.telecommunity.com/dist/ez_setup.py
python2.4 ez_setup.py --prefix=$HOME/apps

# 5. Install gitosis
cd $HOME/src
git clone git://eagain.net/gitosis.git
cd gitosis
python2.4 setup.py install --prefix=$HOME/apps

# 6. Add new paths to shell environment
echo 'export PATH=$HOME/apps/bin:$PATH' >> $HOME/.bashrc
echo 'export PATH=$HOME/apps/bin:$PATH' >> $HOME/.bash_profile
echo 'export PYTHONPATH=$HOME/apps/lib/python2.4/site-packages' >> $HOME/.bashrc
echo 'export PYTHONPATH=$HOME/apps/lib/python2.4/site-packages' >> $HOME/.bash_profile
. ~/.bash_profile

# 7. Paste your public SSH key into a temporary file on your server. I'll assume it to be '$HOME/id_rsa.pub'

# 8. Initialise gitosis with your public key
gitosis-init < $HOME/id_rsa.pub

Once you’re done with these, follow the second half of this excellent gitosis introduction to get started hosting your repositories on Dreamhost!

Image courtesy of Patrick Havens.

Cleaner Sinatra integration with Compass 0.4

2009-02-15T12:55:00+00:00

Chris Eppstein has been working hard on Compass lately to improve its integration with application frameworks. This gives me the pleasure of updating the code from my earlier post about integrating Sinatra and Compass:

gem 'chriseppstein-compass', '~> 0.4'
require 'compass'

configure do
  Compass.configuration do |config|
    config.project_path = File.dirname( __FILE__ )
    config.sass_dir = File.join('views', 'stylesheets')
  end
end

get "/stylesheets/screen.css" do
  content_type 'text/css'

  # Use views/stylesheets & blueprint's stylesheet dirs in the Sass load path
  sass :"stylesheets/screen", :sass => Compass.sass_engine_options
end

The above is everything you need for your Sinatra app to use Compass 0.4 to render your CSS. The biggest change in 0.4 is that Compass now comes with a configuration singleton. I set it up above in Sinatra’s configure block and tell it that I keep my Sass stylesheets in views/stylesheets inside the application directory. Keeping the Sass configuration separate from the working application code keeps your render calls short and concise, like the rest of your well-crafted Sinatra app.

Thanks to Chris for his hard work and for providing the example code for the Sinatra integration.

A Cycle Helper for Sinatra

2009-02-12T22:20:00+00:00

Sinatra’s minimalist tack encourages you to build just the number of helpers that is required for your app. In doing so, it’s also a chance to improve your Ruby fu. While the source for the helpers that come with Rails provides an excellent starting point for your particular subset, they’re often built to keep all comers happy. You can do something a lot slimmer for your narrowly focused Sinatra app.

Here’s what I came up with for a cycle helper to alternately colour table rows via cycling their CSS classes:

Uppublished_at: check the end of this post for some improved solutions!

helpers do
  def cycle
    @_cycle ||= reset_cycle
    @_cycle = [@_cycle.pop] + @_cycle
    @_cycle.first
  end

  def reset_cycle
    @_cycle = %w(even odd)
  end
end

For reference, this is the source for Rails’ equivalent set of helpers, comprising about one hundred or so lines of code. Now, here are my helpers in use:

%table.tickets{:cellpadding => 0, :cellspacing => 0}
  %thead
    %tr
      %th No.
      %th Summary
      %th Reporter
      %th Assignee
      %th Updated
  %tbody
    - reset_cycle
    - group.tickets.each do |ticket|
      %tr{:class => "#{ticket.out_of_bounds? ? 'out-of-bounds' : 'unassigned'} #{cycle}"}
        = partial('ticket_row', :locals => {:ticket => ticket})

Nothing like writing your own helpers for a nice sense of achievement! And now a question for you? Can you think of a neater way to cycle through a series of strings? Would love to hear your feedback.

Uppublished_at: Oceanic Ruby guru Lachie Cox has forked my helpers and provided some smarter versions. Thanks Lachie!

helpers do
  def cycle
    %w{even odd}[@_cycle = ((@_cycle || -1) + 1) % 2]
  end

  CYCLE = %w{even odd}
  def cycle_fully_sick
    CYCLE[@_cycle = ((@_cycle || -1) + 1) % 2]
  end
end

Caching and Expring Stylesheets and Javascripts in Sinatra

2009-02-08T21:30:00+00:00

The code examples in this article were extracted from Tim Lucas’s toolmantim.rb, a weblog app that inspired me to start playing with Sinatra. Thanks to Tim for his good work!

One of the most rewarding things I’ve found while playing with Sinatra so far it encourages me to learn more about the implementation of the underlying mechanics of a modern web application. This is a framework that doesn’t coddle you: the things that you get for free in Rails and the other larger web frameworks are nowhere to be seen. What’s left is lean & mean, and ready to be shaped into whatever form you fancy!

So let’s talk about stylesheets and javascript files. These are assets that don’t change as often as the pages in your web app. Your javascript files are more than likely to be served directly from the filesystem. Your stylesheets may also be static files, but Sinatra also provides excellent support for generating CSS from Sass templates.

If you’re using Sass, you’ll no doubt have something like this in your Sinatra app:

get "/stylesheets/screen.css" do
  content_type 'text/css'
  sass :"stylesheets/screen"
end

When someone requests the /stylesheets/screen.css file, the above action is run and the generated CSS is sent back. You won’t want the very same thing happening for the next page request. You can fix this by setting the ‘Expiry’ header in the responses you send to stylesheet requests:

get "/stylesheets/screen.css" do
  content_type 'text/css'
  response['Expires'] = (Time.now + 60*60*24*356*3).httpdate
  sass :"stylesheets/screen"
end

Setting the ‘Expires’ header in the above action will encourage client proxies or browsers to cache your generated stylesheet.

This is all pretty great, but this technique still leaves a couple of gaps. Any static files (like javascripts) are not served through Sinatra, so we can’t manually set an Expires header for them. Further, for the stylesheets that Sinatra does generate, we’ll need a way to force the clients to refresh them whenever we modify their source Sass files. We can address these gaps with a couple of Sinatra helpers:

helpers do
  def versioned_stylesheet(stylesheet)
    "/stylesheets/#{stylesheet}.css?" + File.mtime(File.join(Sinatra::Application.views, "stylesheets", "#{stylesheet}.sass")).to_i.to_s
  end
  def versioned_javascript(js)
    "/javascripts/#{js}.js?" + File.mtime(File.join(Sinatra::Application.public, "javascripts", "#{js}.js")).to_i.to_s
  end
end

Use these helpers to load your javascripts and stylesheets in the appropriate places in your layout file:

!!! Strict
%html{:xmlns =>'http://www.w3.org/1999/xhtml', 'xml:lang' => 'en', :lang => 'en'}
  %head
    %meta{'http-equiv' => 'Content-Type', :content => 'text/html; charset=utf-8'}/
    %title My Sinatra App

    %link{:href => versioned_stylesheet('screen'), :media => 'screen', :rel => 'stylesheet', :type => 'text/css'}/
    %script{:src => versioned_javascript('application'), :type => 'text/javascript'}/

These helpers will generate links with a timestamp appended, like so: stylesheets/screen.css?1233119990. This timestamp is derived from the file modification time of the source Sass files for your stylesheets (and in the case of javascripts, the static javascript files themselves), which means that a new timestamp will appear in your page source automatically after you make modifications. This will trigger the clients to request the newest stylesheets, which will in turn stay cached until the next change and minimise the total download for each request to your Sinatra application.

Truism

2009-02-05T04:45:00+00:00

designaday:

“Broken gets fixed. Shoddy lasts forever”

One of the developers I work with said this after I complained about a lingering issue in one of our products. It rings true. When deadlines are tight, and there is more work to get done than there are developers or hours in the schedule, it’s not the squeaky wheel, but the jammed one that gets the grease. The lesson, then, is to make sure it gets done right the first time. You never know when you’ll have the opportunity to revisit it.

This is all too true. I’m especially mindful of this given that I’m just now embarking on a fresh rewrite of something that was particularly broken. We’re making sure the customer understands that we’re taking the time necessary to do things right the first time (or in this case, the second time).

Using Compass for CSS in your Sinatra application

2009-01-27T11:30:00+00:00

Stage left: Sinatra, the hottest new Ruby DSL for expressive, singular web application development. Stage right: Compass, the CSS metaframework that takes the ease of use of Blueprint’s grids and Sass’ syntax, and lets you combine them in a manner semantic & modular, just how you like it.

Until now, these actors could never meet. With the release of the 0.9 series of Sinatra, however, it has become a breeze.

The key change occurred with the restructure of Sinatra’s rendering logic. When you call sass to render a Sass stylesheet, you can now pass options that go directly to the Sass engine. See this excerpt from lib/sinatra/base.rb in the codebase:

def sass(template, options={}, &block)
  require 'sass' unless defined? ::Sass
  options[:layout] = false
  render :sass, template, options
end

def render_sass(template, data, options, &block)
  engine = ::Sass::Engine.new(data, options[:sass] || {})
  engine.render
end

See that options[:sass] hash going to ::Sass::Engine.new? That’s how we’ll get Compass to work. All it needs are some extra load paths to be passed to Sass::Engine, which you can do when you render your Sass stylesheet. Compass also conveniently provides a configuration singleton to let you do this just once in the configure block of your application:

gem 'chriseppstein-compass', '~> 0.4'
require 'compass'

configure do
  Compass.configuration do |config|
    config.project_path = File.dirname( __FILE__ )
    config.sass_dir = File.join('views', 'stylesheets')
  end
end

get "/stylesheets/screen.css" do
  content_type 'text/css'

  # Use views/stylesheets & blueprint's stylesheet dirs in the Sass load path
  sass :"stylesheets/screen", :sass => Compass.sass_engine_options
end

This stylesheet action at the bottom of this example will parse your views/stylesheets/screen.sass file and your Compass inclusions and mixins will output exactly the CSS that you expect. All done!

Update (2009-02-15): The code above was updated for the improved Sass configuration support in Compass 0.4. See my post on the 0.4 release for an explanation of the improvements. Before this release, your stylesheet actions would have looked a lot more ungainly:

get "/stylesheets/screen.css" do
  content_type 'text/css'

  # Use views/stylesheets & blueprint's stylesheet dirs in the Sass load path
  sass :"stylesheets/screen", { :sass => { :load_paths => (
    [File.join(File.dirname( __FILE__ ), 'views', 'stylesheets')] +
    Compass::Frameworks::ALL.map { |f| f.stylesheets_directory })
  } }
end

Of course, you won’t want Sass loading up all the Compass files and re-rendering your CSS with every page request, so you will want to cache this generated CSS file. This is another handy Sinatra trick I’ve come across, but I’ll save that for the next entry.

Accessing Basecamp's writeboards programatically with Ruby

2009-01-08T13:25:00+00:00

Basecamp’s writeboards provide a great facility for versioned, collaborative document editing. However, any user of Basecamp will be able to tell you that they don’t fit in as neatly as the other components of the system. Going to a writeboard takes you to an interim “loading” page before displaying the writeboard outside the regular interface of Basecamp.

To the developer, the difference between writeboards and the rest of Basecamp is highlighted further because writeboards are the only part of the app without an API. The enterprising Ruby developer, however, sees this as no barrier! So without further ado, I bring you writeboard-rb, a little Ruby class that allows you to access the contents of a writeboard programatically:

wb = Basecamp::Writeboard.new(
  :url => 'http://bigcorp.updatelog.com/W1234567',
  :username => 'user',
  :password => 'pass',
  :cookie_jar => '/tmp/foo',
  :use_ssl => true
)

puts wb.contents

Behind the scenes, no fewer than five calls to the curl command line utility are made, along with a wee bit of Hpricot screen scraping to facilitate a form post otherwise made with JavaScript during the normal writeboard loading page. Using curl is necessary here because we need to preserve session cookies for both the Basecamp and Writeboard domains during the process of fetching the writeboard’s contents.

Anyway, please go check out the code on github, and feel free to fork it to make any improvements or extensions! I hope it can come in handy for you. It was a fun little challenge to take it this far.

Capistrano task to selectively update crontabs

2008-11-25T02:25:00+00:00

In a lot of our Rails projects, we need to throw a few entries into a user’s crontab, for things like scheduled rake tasks or thinking sphinx updates.

However, we don’t want to overwrite the entire crontab file whenever we want to add new entries or change the details of the existing ones.

Thus, I’ve devised a little technique to allow selective updates of the crontab. Take this demo and apply it where necessary in your capistrano deploy.rb file:

namespace :my_rake_task do
  task :add_cron_job, :roles => :app, :except => { :no_release => true } do
    tmpname = "/tmp/appname-crontab.#{Time.now.strftime('%s')}"
    # run crontab -l or echo '' instead because the crontab command will fail if the user has no pre-existing crontab file.
    # in this case, echo '' is run and the cap recipe won't fail altogether.
    run "(crontab -l || echo '') | grep -v 'rake my_rake_task' > #{tmpname}"
    run "echo '12 1,11,18 * * * cd #{current_path} && RAILS_ENV=production rake my_rake_task' >> #{tmpname}"
    run "crontab #{tmpname}"
    run "rm #{tmpname}"
  end
end

To step you through it, this task will dump the current crontab to a file, excluding the command that you care about, then add the command back to the bottom of the file, and install the crontab again, with all the other contents preserved. This will allow you to change the particular entry without clobbering the entire file. Nifty!

Using Markov Chains to provide English language seed data for your Rails application

2008-10-22T14:10:00+00:00

I spent a portion of last weekend’s Rails Rumble preparing a script that would seed our application’s database with test data. Among other things, having a well populated database is useful to fully test all the parts of the application’s interface that might not come into play when using a smaller data set. It also gives you the ability to get a true sense of how your app will look and feel after the real users start to pour in content (hopefully!).

For the Rumble project, I used the faker Ruby gem, which provides methods for generating realistic names, domains and email addresses. It also provides a text generator that pulls random strings from Lorem Ipsum. However, while the app may feel have felt fully populated, seeing the Latin everywhere didn’t make it feel right.

So for the next project, I will generate random English language strings for the seed data. Rather than pulling words ad hoc from a dictionary or something, we can use Markov Chains to generate reasonably realistically structured text for us. A bit of searching revealed a Ruby Quiz page with an implementation of a text generator using Markov Chains. It explains that the generator:

bq. read[s] some text document(s), making note of which characters commonly follow which characters or which words commonly follow other words (it works for either scale). Then, when generating text, you just select a character or word to output, based on the characters or words that came before it.

I took implementation provided on this page, and added a couple of convenience methods to easily fetch sequences of words or whole sentences:

# Courtesy of http://rubyquiz.com/quiz74.html

class MarkovChain
  def initialize(text)
    @words = Hash.new
    wordlist = text.split
    wordlist.each_with_index do |word, index|
    add(word, wordlist[index + 1]) if index <= wordlist.size - 2
   end
  end

  def add(word, next_word)
    @words[word] = Hash.new(0) if !@words[word]
    @words[word][next_word] += 1
  end

  def get(word)
    return "" if !@words[word]
    followers = @words[word]
    sum = followers.inject(0) {|sum,kv| sum += kv[1]}
    random = rand(sum)+1
    partial_sum = 0
    next_word = followers.find do |word, count|
      partial_sum += count
      partial_sum >= random
    end.first
    next_word
  end

  # Convenience methods to easily access words and sentences

  def random_word
    @words.keys.rand
  end

  def words(count = 1, start_word = nil)
    sentence = ''
    word = start_word || random_word
    count.times do
      sentence << word << ' '
      word = get(word)
    end
    sentence.strip.gsub(/[^A-Za-z\s]/, '')
  end

  def sentences(count = 1, start_word = nil)
    word = start_word || random_word
    sentences = ''
    until sentences.count('.') == count
      sentences << word << ' '
      word = get(word)
    end
    sentences
  end
end

Then, in my seed data script, I prime a MarkovChain instance with some good literature (I chose Sir Arthur Conan Doyle’s The Lost World from Project Gutenburg), and then use it to populate my records:

mc = MarkovChain.new(File.read("#{RAILS_ROOT}/db/populate_source.txt"))

# create_or_update method taken from http://railspikes.com/2008/2/1/loading-seed-data

1.upto(100) do |comment_id|
  Comment.create_or_update(
    :id => comment_id,
    :title => mc.words(3).titleize,
    :body => mc.sentences(3)
  )
end

Done! While the text isn’t the kind of fluent prose you’d expect from the real-life internet commenters, it does look much more realistic than Lorem Ipsum, and often provides cause for a bit of a chuckle. An example of the kind of stuff you’ll get from The Lost World is below. Feel free to choose your favourite text as the source :)

Luxurious voyage I had a solemnity as one would no sneer would have just after two surviving Indians in a precipice, and were accompanying to his thumb over his shoulders, the effort.

For some background on how to load seed data into your Rails app, see this page on the Rail Spikes blog for a good introduction.

For the win!

2008-10-20T10:20:00+00:00

In the last 48 hours, two friends and colleagues and I laboured over a brand new web app, for our entry in this year’s Rails Rumble. The result is something we’re proud to share with you: Ideas FTW!.

We all have plenty of ideas, most of which we never have the opportunity to act on. Ideas FTW is an idea incubator that allows you to see your ideas come to life. You can vote on ideas, post your own, take up an idea and start an effort to achieve it on your own or in a group.

The site is simple and colourful, and will only become more fun to use as more ideas get added. So, please do sign up with your OpenID and empty your overflowing brains! I’m looking forward to seeing some interesting ideas fill the site.

Of course, the app is far from perfect yet. A few bugs slipped through and there are certainly a couple more key features we would like to add. If you have any suggestions for improving Ideas FTW, please feel free to leave a comment on our effort page.

Hugh, Michael and I all had a great time working together to create this site. We learnt a great many new things over the weekend and I am sure that I can speak for the others to say the lack of sleep was worth it. If you enjoy the site, please feel free to vote for us when the tallies open in a couple of days.

Configuring god to monitor Sphinx's searchd

2008-09-10T12:55:00+00:00

God is a server monitoring tool helpful for starting your server processes and keeping them running, all the while making sure they don’t misbehave. In the Rails world, it appears to be used most commonly with packs of mongrels or thins. However, this is not to say it can’t be used to monitor other software. For one of my recent work projects, we’ve been using God to monitor the searchd process that Sphinx uses to serve results to search queries.

The configuration required for this can be based mostly on what you see around the place for thins or mongrels:

require 'yaml'

app_config = YAML.load(File.open(File.dirname( __FILE__ ) + "/config.yml"))['production']

God.watch do |w|
  w.group = "#{app_config['app_name']}-sphinx"
  w.name = w.group + "-1"

  w.interval = 30.seconds

  w.uid = app_config['user']
  w.gid = app_config['group']

  w.start = "searchd --config #{app_config['app_root']}/config/sphinx.conf"
  w.start_grace = 10.seconds
  w.stop = "searchd --config #{app_config['app_root']}/config/sphinx.conf --stop"
  w.stop_grace = 10.seconds
  w.restart = w.stop + " && " + w.start
  w.restart_grace = 15.seconds

  w.pid_file = File.join(app_config['app_root'], "tmp/pids/sphinx.pid")

  w.behavior(:clean_pid_file)

  w.start_if do |start|
    start.condition(:process_running) do |c|
      c.interval = 5.seconds
      c.running = false
    end
  end

  w.restart_if do |restart|
    restart.condition(:memory_usage) do |c|
      c.above = 100.megabytes
      c.times = [3, 5] # 3 out of 5 intervals
    end
  end

  w.lifecycle do |on|
    on.condition(:flapping) do |c|
      c.to_state = [:start, :restart]
      c.times = 5
      c.within = 5.minutes
      c.transition = :unmonitored
      c.retry_in = 10.minutes
      c.retry_times = 5
      c.retry_within = 2.hours
    end
  end
end

There are a number of other changes required for this configuration to work properly. Firstly, you will require a config/config.yml file, containing something like the following:

development: &production_settings
  app_name: myapp
  user: deployer
  group: deployer
  app_root: "/home/deployer/deployments/myapp.amc.org.au/current"
test:
  << *production_settings
production:
  << *production_settings

This config file sets the app’s short name, the user and group to run the searchd process, and the path to the rails application root. We keep these settings in a yaml file because they are also used in a number of other places besides here. If you don’t have this requirement, you can put all of these settings directly into your god config file.

The other requirement is that we make Sphinx use a configuration file that has a predictable name. The default Sphinx config file, when using Pat Allen’s awesome Thinking Sphinx plugin, has a file name contains the name of the current Rails environment. God can’t use this to launch searchd, because it is does not run within the context of the Rails environment. To set location and name of the sphinx configuration file, set up a config/sphinx.yml file that contains something like this:

development: &production_settings
  config_file: /home/deployer/deployments/myapp.amc.org.au/current/config/sphinx.conf
  pid_file: /home/deployer/deployments/myapp.amc.org.au/current/tmp/pids/sphinx.pid
test:
  << *production_settings
production:
  << *production_settings

Thinking Sphinx respects these settings and uses them when it generates the Sphinx config file.

So that’s about it. Make sure you’ve run rake thinking_sphinx:index at least once in your Rails app, to generate the Sphinx configuration and indexes, and then you are ready to start god and have your searchd automatically started and monitored!

Lessons learnt the hard way: Don't use script/console --sandbox on production apps

2008-09-10T02:05:00+00:00

Don’t use script/console -s or script/console --sandbox on your live, running Rails application. The built-in transactions that roll back on exit are nifty, but if your database’s transactions use any kind of locking, then you will get locks on the tables for any models that you use in the console. This will most likely stop your application from working.

Moral of the story? If you need to use the console on a live application, just be careful. You’re smart, you’re cautious, you don’t need the sandbox. It will only cause more trouble than it is worth.

Beijing Olympic medal tally for our Campfire bot

2008-08-10T08:00:00+00:00

I’ve recently been working on a bot framework for 37signals’ Campfire web chat system. We use Campfire at work and it’s been great for having fun together and increasing team cohesiveness.

Having a bot in the room is also good for random quotes, Chuck Norris facts and comic strips. We also plan to add a few useful work-related things, such as git & subversion commit messages and server monitoring alerts.

The bot is plugin-oriented and easy to extend. Check out the code on github. It’s still in flux and I would love to hear any feedback.

This afternoon I hacked together a little plugin to report the Beijing olympics medal tally, thanks to an idea of Sean’s. The code is highly specific, but it shows how easy (and fun!) it is to add a command to the bot and scrape data off the web using hpricot.

Here it is:

require 'open-uri'
require 'hpricot'

class BeijingTally < PluginBase

  on_command 'tally', :tally

  def tally(msg)
    output = "#{'Pos.'.rjust(6)} - #{'Country'.ljust(25)} - G - S - B - Total\n"
    rows = ((Hpricot(open('http://results.beijing2008.cn/WRM/ENG/INF/GL/95A/GL0000000.shtml'))/'//table')[1]/'tr')[2..-1]
    rows.each_with_index do |row, i|
      cells = row/'td'
      output += "#{strip_tags_or_zero(cells[0].inner_html).rjust(6)} - " # position
      output += "#{((i == rows.length - 1) ? '' : strip_tags_or_zero(cells[1].inner_html)).ljust(25)} - " # country
      output += "#{strip_tags_or_zero(cells[-5].inner_html).rjust(3)} - " # gold
      output += "#{strip_tags_or_zero(cells[-4].inner_html).rjust(3)} - " # silver
      output += "#{strip_tags_or_zero(cells[-3].inner_html).rjust(3)} - " # bronze
      output += "#{strip_tags_or_zero(cells[-2].inner_html).rjust(3)}\n" # total
    end

    paste(output)
  end

  private

  # Take away the HTML tags from the string and insert a '0' if it is empty
  def strip_tags_or_zero(str)
    (out = str.gsub(/<\/?[^>]*>/, "").strip).blank? ? '0' : out
  end
end

Mum's Savoury Mince Pockets

2008-06-28T04:00:00+00:00

These are easy to make and taste excellent, and are also good cold or reheated the next day! Thanks, Mum.

Ingredients:

1 kg good (beef) mince
2 carrots, peeled and diced
1 onion, diced
1 tsp garlic
2 tsp curry powder
1 cup frozen peas and corn
Tomato sauce
Sweet chilli sauce
2 cups cooked rice
Lebanese bread, around 3-4 pieces

Steps:

Fry the mince with the garlic and curry powder.
Once the mince is getting close to done, add the peas and corn.
Add tomato sauce and sweet chilli sauce to taste.
Add the cooked rice and mix it all up.
Cut the lebanese bread in half to form pockets and fill with the mince and rice mixture.
Put the filled pockets in an oven at 180°C for 5-10 minutes, or until crisp.

Displaying both local and HTTP remote images in Prince XML generated PDFs

2008-06-27T04:30:00+00:00

In our Rails apps, we use the awesome Prince XML to generate PDFs. We interact with the prince command line application using the Ruby library and Rails helper from the guys over at subimage interactive.

When using their helper to generate a PDF from a Rails template, all image tags have the src attribute altered so they point to paths that are relative to the local filesystem, not just the root of your application.

However, this breaks any images that you are loading from remote locations over HTTP. For us, this ended up breaking the static Google Maps that we were generating.

So here’s an updated make_pdf helper that only modifies the image paths if they are local. This lets us use both local and HTTP-hosted images on the same PDF!

# We use this chunk of controller code all over to generate PDF files.
#
# To stay DRY we placed it here instead of repeating it all over the place.
#
module PdfHelper
  require 'prince'

  private
    # Makes a pdf, returns it as data...
    def make_pdf(template_path, pdf_name, landscape=false)
      prince = Prince.new()
      # Sets style sheets on PDF renderer.
      prince.add_style_sheets(
        "#{RAILS_ROOT}/public/stylesheets/application.css",
        "#{RAILS_ROOT}/public/stylesheets/print.css",
        "#{RAILS_ROOT}/public/stylesheets/prince.css"
      )
      prince.add_style_sheets("#{RAILS_ROOT}/public/stylesheets/prince_landscape.css") if landscape
      # Render the estimate to a big html string.
      # Set RAILS_ASSET_ID to blank string or rails appends some time after
      # to prevent file caching, and messing up local-disk requests.
      ENV["RAILS_ASSET_ID"] = ''
      html_string = render_to_string(:template => template_path, :layout => 'document')
      # Make all paths relative to the file systemm, but only if they don't have http(s):// at the start.
      html_string.gsub!(%r{(src=")([^h][^t][^t][^p][^s]?[^:][^/]*)}, "src=\"#{RAILS_ROOT}/public\\2")
      # Send the generated PDF file from our html string.
      return prince.pdf_from_string(html_string)
    end

    # Makes and sends a pdf to the browser
    #
    def make_and_send_pdf(template_path, pdf_name, landscape=false)
      send_data(
        make_pdf(template_path, pdf_name, landscape),
        :filename => pdf_name,
        :type => 'application/pdf'
      )
    end
end

And just to be precise, here is the diff between the helpers:

--- pdf_helper.rb 2008-06-27 15:05:44.000000000 +1000
+++ new_pdf_helper.rb 2008-06-27 15:05:54.000000000 +1000
@@ -18,11 +18,11 @@
       prince.add_style_sheets("#{RAILS_ROOT}/public/stylesheets/prince_landscape.css") if landscape
       # Render the estimate to a big html string.
       # Set RAILS_ASSET_ID to blank string or rails appends some time after
- # to prevent file caching, fucking up local - disk requests.
+ # to prevent file caching, and messing up local-disk requests.
       ENV["RAILS_ASSET_ID"] = ''
       html_string = render_to_string(:template => template_path, :layout => 'document')
- # Make all paths relative, on disk paths...
- html_string.gsub!("src=\"", "src=\"#{RAILS_ROOT}/public")
+ # Make all paths relative to the file systemm, but only if they don't have http(s):// at the start.
+ html_string.gsub!(%r{(src=")([^h][^t][^t][^p][^s]?[^:][^/]*)}, "src=\"#{RAILS_ROOT}/public\\2")
       # Send the generated PDF file from our html string.
       return prince.pdf_from_string(html_string)
     end

I’d also love it if you could propose a better way to handle the regular expression inside the gsub! call. Leave a comment!

Loading the ActiveRecord SQL Server adapter in a Rails 2.1 app

2008-06-18T03:30:00+00:00

It’s pretty simple. In your config/environment.rb:

config.gem 'activerecord-sqlserver-adapter', :source => 'http://gems.rubyonrails.org', :lib => 'active_record/connection_adapters/sqlserver_adapter'

Then run rake gems:install and away you go.

Hugh has some more tips about the packages and configuration required to connect to an SQL Server from a Linux platform.

Enabling a non-interactive install of Blackdown's j2re1.4 on Ubuntu or Debian

2008-06-12T05:45:00+00:00

When you apt-get install the j2re1.4 Java package in Debian or Ubuntu, it displays a few ncurses-style dialogs to configure the software and to require your acceptance of the license agreement. Working with these dialogs is fine if you are installing the software with apt-get in a typical interactive shell session. If you are installing without this capacity to interact (like in a script), you will run into problems. Here’s how to fix it.

If you install the package with the noninteractive frontend for dpkg, then you’ll get an error like this:

# DEBIAN_FRONTEND=noninteractive /usr/bin/apt-get --yes --force-yes install j2re1.4

Reading package lists... Done
Building dependency tree
Reading state information... Done
Suggested packages:
  mozilla-browser mozilla-firefox galeon ttf-kochi-gothic ttf-kochi-mincho
Recommended packages:
  gsfonts-x11 libx11-6 libxext6 libxi6
The following NEW packages will be installed:
  j2re1.4
0 upgraded, 1 newly installed, 0 to remove and 23 not upgraded.
Need to get 0B/22.5MB of archives.
After unpacking 60.3MB of additional disk space will be used.
Preconfiguring packages ...
j2re1.4 failed to preconfigure, with exit status 10
Selecting previously deselected package j2re1.4.
(Reading database ... 26639 files and directories currently installed.)
Unpacking j2re1.4 (from .../j2re1.4_1.4.2.02-1ubuntu3_i386.deb) ...
dpkg: error processing /var/cache/apt/archives/j2re1.4_1.4.2.02-1ubuntu3_i386.deb (--unpack):
 subprocess pre-installation script returned error exit status 10
Errors were encountered while processing:
 /var/cache/apt/archives/j2re1.4_1.4.2.02-1ubuntu3_i386.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)

The package fails to install because it requires the acceptance of the licence agreement, which it will only allow in an interactive installation.

The typical way to fix this is to pre-seed the debconf database (using @debconf-set-selections@) with the answers to the questions that the j2re1.4 package requires. However, this doesn’t seem to satisfy j2re1.4, and the package still displays the dialog or fails in non-interactive mode. Why is it java that always gives me these hairy problems?

Anyway, here is the way to fix it. If you manually append the values directly to the debconf database file, it will work:

cp /var/cache/debconf/config.dat /var/cache/debconf/config.dat-old

cat << E_O_DEBCONF >> /var/cache/debconf/config.dat

Name: j2re1.4/jcepolicy
Template: j2re1.4/jcepolicy
Value:
Owners: j2re1.4
Flags: seen

Name: j2re1.4/license
Template: j2re1.4/license
Value: true
Owners: j2re1.4
Flags: seen

Name: j2re1.4/stopthread
Template: j2re1.4/stopthread
Value: true
Owners: j2re1.4
Flags: seen

E_O_DEBCONF

(For reference, I found these values by making a copy of the config.dat file, running dpkg-preconfigure on the j2re1.4 package, and then running a diff between the updated config.dat file and my copy.)

Now you’ll be able to successfully install the package in noninteractive mode. This means, for us, one step closer to fully automating our Xen builds!

Automatic Saving Of Invalid Resources in Rails While Maintaining a Clean RESTful Interface

2008-06-09T13:35:00+00:00

or, How To Change Your World In One Line Of Code

One of the cool things that we’re doing at the AMC is building a large collection of loosely coupled Rails applications that communicate using REST. This is slightly unusual, as rails is predominantly used to build single apps that operate in isolation. In our experiences, we’ve picked up a number of tricks that we’d like to share. Here’s the first, on how a single this single line of code has saved us weeks of time and effort. Here’s the line in question:

if @bank_transaction.save && @bank_transaction.valid?

Read on to find out how this helps!

RESTful Resources

This line is part of a new Rails app we’re developing to centrally handle all online payments for our software systems at the AMC. In this payments application, we expose two key resources over REST: (1) line items, which the other apps create with the details of the items to purchase, and (2) bank transactions, which are passed line item IDs and credit card details for the purchase.

Naturally, the models behind these resources have a bunch of validation rules that ensure certain conditions are met before they can be saved successfully. If any of these requirements are not met, then the model fails to save and the error hash is returned to the client app.

For most resources, these error hashes are returned in the usual Rails-like way. Let’s look at how LineItemsController does it:

class LineItemsController < ApplicationController
  # POST /line_items.xml
  def create
    @line_item = LineItem.new params[:line_item]
    respond_to do |format|
      if @line_item.save
        format.xml { render :xml => @line_item, :status => :created, :location => @line_item }
      else
        format.xml { render :xml => @line_item.errors, :status => :unprocessable_entity }
      end
    end
  end
end

To paraphrase: if the items save successfully, return success and the line item in XML, otherwise return the error hash. Nice and predictable, nothing exciting here.

Saving Invalid Resources

One of the resources in the payments app is different. These are the @BankTransactions@, which are about as “mission critical” as we get. Let’s talk about the successful case first: during the creation of the BankTransaction model, if all the validations have passed, a before_create callback is triggered that will talk to the bank (using ActiveMerchant, of course) and ask to make the transaction there. If this succeeds, the model is saved to the database and an XML representation of the saved model is passed back to the client application with a success code.

However, if the transaction with the bank fails, this is still information we care about. A failed transaction could be an indication of a larger problem, and also needs to be recorded for customer service purposes. It makes sense to save every failed transaction as well as every successful one. To this end, the callback that communicates with the bank always returns true, which allows the save continue, and records for both successful and unsuccessful transactions to be kept in the database.

(Another approach to this problem would be to create a separate @TransactionLog@ model to store the transaction data, but this approach requires extra work. Having the BankTransactions save every time is essentially free. Excellent.)

Keeping the REST API Simple

While the payments app is saving unsuccessful transactions, the client apps do not want to keep these records around: all they care about is if a transaction is successful or not. The easiest way to make it simple for the clients is to make the creation of a bank transaction resource behave the same way as creating any other resource over REST in rails. This means that if a transaction with the bank fails, then it should appear to the clients as if the save also failed.

This will require the controller to generate an errors hash if the transaction fails. This means that the model should be invalid at this point. Given that we save to the database even for failed transactions, the model should therefore be invalid after the save:

class BankTransaction < ActiveRecord::Base
  validate :must_be_successful_if_saved
  before_create :transact

  private

  def transact
    # talk to the bank here, and set self.success to true or false pending the results
    # return true to make sure a save always occurs
    true
  end

  def must_be_successful_if_saved
    errors.add_to_base("failed to transact successfully with the bank") if !new_record? && !success?
  end
end

And then, in the BankTransactionsController, that one magic line:

class BankTransactionsController < ApplicationController
  # POST /bank_transactions.xml
  def create
    @bank_transaction = BankTransaction.new params[:bank_transaction]
    respond_to do |format|
      if @bank_transaction.save && @bank_transaction.valid?
        format.xml { render :xml => @bank_transaction, :status => :created, :location => @bank_transaction }
      else
        format.xml { render :xml => @bank_transaction.errors, :status => :unprocessable_entity }
      end
    end
  end
end

Unlike the standard behaviour shown in the LineItemsController above, we only return a successfully created model if the save is successful AND it is still valid afterwards. A saved model for a failed transaction with the bank will be invalid at this point, so it will return the errors hash. To the client app, this looks like the same resource it tried to create initially, and so it can proceed as usual to display the errors, ask for corrections if necessary, and try to save again. In the background, the payments app has saved every failed transaction for safe keeping.

God init script for Debian/Ubuntu systems

2008-05-26T14:45:00+00:00

To compliment the Red Hat init script that is available on the net, here’s the init script we use to launch god on our Ubuntu systems.

To use, save as /etc/init.d/god, make the file executable, then run update-rc.d god defaults. You’re done.

#!/bin/sh

### BEGIN INIT INFO
# Provides: god
# Required-Start: $all
# Required-Stop: $all
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: God
### END INIT INFO

NAME=god
DESC=god

set -e

# Make sure the binary and the config file are present before proceeding
test -x /usr/bin/god || exit 0

# Create this file and put in a variable called GOD_CONFIG, pointing to
# your God configuration file
test -f /etc/default/god && . /etc/default/god
[$GOD_CONFIG] || exit 0

. /lib/lsb/init-functions

RETVAL=0

case "$1" in
  start)
    echo -n "Starting $DESC: "
    /usr/bin/god -c $GOD_CONFIG -P /var/run/god.pid -l /var/log/god.log
    RETVAL=$?
    echo "$NAME."
    ;;
  stop)
    echo -n "Stopping $DESC: "
    kill `cat /var/run/god.pid`
    RETVAL=$?
    echo "$NAME."
    ;;
  restart)
    echo -n "Restarting $DESC: "
    kill `cat /var/run/god.pid`
    /usr/bin/god -c $GOD_CONFIG -P /var/run/god.pid -l /var/log/god.log
    RETVAL=$?
    echo "$NAME."
    ;;
  status)
    /usr/bin/god status
    RETVAL=$?
    ;;
  *)
    echo "Usage: god {start|stop|restart|status}"
    exit 1
    ;;
esac

exit $RETVAL

Testing for elements in ActiveRecord's XML output

2008-04-18T03:25:00+00:00

Following on from my previous post about customising ActiveRecord’s to_xml output, I have had to write specs to make sure a custom attribute I have added to a model’s XML serialisation actually appears as expected.

ActiveSupport’s Hash.from_xml class method makes this a piece of cake. Instead of testing against the XML as a string or parsing it manually, you can turn it into a hash and get directly to the attribute you want. Behold:

describe Product do
  before(:each) do
    @product = Product.new
  end

  # net_price is the custom attribute I have added to the XML serialisation
  it "should include net price in XML serialisation" do
    @product.attributes = valid_product_attributes
    Hash.from_xml(@product.to_xml)['product']['net_price'].should == @product.net_price
  end
end

Activating the screensaver with Quicksilver in OS X

2008-04-11T14:45:00+00:00

There are a few posts around the net that describe some ways to launch the OS X screensaver using a keyboard shortcut with Quicksilver. However, common to all of these approaches is using a code snippet that is launched by a trigger.

Quicksilver triggers can only be launched by global keybindings or mouse gestures. That is, you can’t access them through the main quicksilver interface. A simple way around this is to use Automator to create a custom screensaver launch application:

Save the workflow as an app called, let’s say, “Screensaver”, put it in your Applications folder, and you can then use Quicksilver to activate your screensaver like you would launch any ordinary app.

Setting default arguments for to_xml for your ActiveRecord model

2008-04-11T02:45:00+00:00

Rails provides you with a lot of scope for customising the XML serialisation of models with to_xml. Among other things, you can exclude attributes, include objects that are first-level associations, and include the results of any custom methods for your model.

However, most of the examples for these only show this customisation taking place in the scope of the controller, where the arguments are passed to a single call of to_xml. This is not very DRY if you want to customise the default to_xml output for your model to include or exclude some information.

Customising the default behaviour is pretty easy. Say I have an extra method inside a model that combines price and tax to formulate a net price, and I want this to be included in the XML serialisation every time. Here is what to do:

class Product < ActiveRecord::Base
  def net_price
    self.price + self.tax
  end

  alias_method :ar_to_xml, :to_xml

  def to_xml(options = {}, &block)
    default_options = { :methods => [:net_price]}
    self.ar_to_xml(default_options.merge(options), &block)
  end
end

New Job at the Australian Medical Council

2008-03-27T11:40:00+00:00

Today I began working in a new Rails development role at the Australian Medical Council. This is the second job I have taken since I have moved to Canberra, and I am looking forward to digging in my heels in here.

Shortly after arriving here, I took up a web systems administration position at the Department of Broadband, Communications and the Digital Economy. This was a good opportunity to see the inner working of government, and I enjoyed working with a fun and motivated team.

It’s refreshing to be back into a hands-on programming role at the AMC. Along with my friend Hugh, we’ll be introducing some new, agile development practices while moving most of the organisation’s VB-based internal systems to the web with Rails. It’s going to be a lot of fun and I sense that it will be a good opportunity for growth and learning. I’ll be sure to pass along anything useful that I come across. Watch this space!

Mocking out your Rails helpers in helper specs

2008-03-19T10:35:00+00:00

RSpec provides some pretty good tools for mocking your objects in Rails test specs.

Mocks allow you to set expectations on what kind of messages are passed to your objects, and what values they pass back in return. Among other things, this brings the advantage that your test/example can be truly focused on one unit of code, because the data that the mocks (or stubs) return does not rely on the implementation of the objects or methods that they imitate.

In most cases, mocking with RSpec in your Rails tests is pretty straightforward - you just create the mock object and set your expectations of it:

# mock_model is a convenience provided by rspec_on_rails
my_mock = mock_model(User)
my_mock.should_receive(:authenticate).with('user', 'pass').and_return(true)

Alternatively, you can create partial mocks or stubs on real objects or classes:

# partial stub for a real object
u = User.new
u.stub!(:name).and_return('Phred')

# partial mock for a class method
User.should_receive(:find).and_return(u)

In all of these cases, there is a clear recipient for the mocking or stubbing treatment - a mock object, or real object, or a class. This is what you will mostly use in your model and controller specs.

However, if you are in the specs for your helpers, it’s not self evident what the recipient should be. Here’s an example for a helpers file with two helpers, one calling the other:

module ApplicationHelper
  def user_list(users)
    users.each { |u| user_summary(u) }
  end

  def user_summary(user)
    open :div, {:class => 'user_summary'} do
      open :p, user.name
      open :p, user.birthday
    end
  end
end

These helper methods all belong inside a module, and are not attached to any object or class like the ActiveRecord models in the above examples. So how do we mock a helper? Check it out:

require File.dirname( __FILE__ ) + '/../../spec_helper'

describe ApplicationHelper, "user printing helpers" do
  include ActionView::Helpers
  include Haml::Helpers

  it "should call the user_summary helper for each user object in the array passed to user_list" do
    user = mock_model(User) do |u|
      u.stub!(:name).and_return('Phred')
      u.stub!(:birthday).and_return('April')
    end
    users = []

    5.times do
      users << user
    end

    # This is the clincher!
    self.should_receive(:user_summary).exactly(5).times

    user_list(users)
  end
end

In your helper specs, the self object is what you can use to mock and stub your helpers. It’s that simple. What is self? I’m not really quite sure. It looks like a dynamically created subclass of Spec::Rails::Example::HelperExampleGroup that is particular to the spec example currently being run. Whatever it is, it gets the job done for us.

If you also need to mock or stub your helpers in view specs, you can use the @controller.template object:

@controller.template.stub!(:a_helper_method).and_return(true)

Check out Jack Scrugg’s article for more detail about this.

jQuery datepickers with constrained ranges in Rails forms

2008-03-18T11:20:00+00:00

One of the first things I’ve done since joining the AMC is dive into one of the projects currently in development and start fixing small bugs, to familiarise myself with the codebase and the kind of problems they are solving here. It’s also been a good chance to pick up a few things.

One of these has been jQuery. One of the places we are using it is to provide datepickers in a form to create new Exam records. Here’s the view for the form partial:

%p
  = f.label :name
  %br/
  = f.text_field :name

%p
  = f.label :exam_date
  %br/
  = f.text_field :exam_date, {:value => full_date(@exam.exam_date)}

%h2 Registration Period

%p
  = f.label :open_date, 'From: '
  %br/
  = f.text_field :open_date, {:value => full_date(@exam.open_date)}
  %br/
  = f.label :close_date, 'To: '
  %br/
  = f.text_field :close_date, {:value => full_date(@exam.close_date)}

Mmm, haml. Also note the complete lack of JavaScript inside this view! This is because we are making use of unobtrusive JavaScript, a technique which separates the JavaScript code from the presentation layer and places importance on allowing the views to degrade gracefully (that is, continue to work fully even when JavaScript support is not available).

In this way, our JavaScript is kept inside another file: application.js:

$(document).ready(function() {
  ////
  // ui.datepicker fields
  $('#exam_exam_date').datepicker();
  $('#exam_open_date').datepicker();
  $('#exam_close_date').datepicker();
});

This code initialises the datepickers and attaches them to the text fields with IDs of exam_exam_date, exam_open_date, and exam_close_date. The datepickers will appear when you click or focus any of these fields. They will let you select the a date from an interactive calendar and insert a string representation of that date into the text field.

All very good, but we have another requirement: that the dates available for selection inside the datepickers are constrained. Specifically, the open and close dates should not be after the exam date, and the close date should not be before the open date. These constraints can be set up by passing a function name to the datepickers when they are initialised:

$(document).ready(function() {
  ////
  // ui.datepicker fields
  $('#exam_exam_date').datepicker();
  $('#exam_open_date').datepicker({beforeShow: customRange});
  $('#exam_close_date').datepicker({beforeShow: customRange});

  function customRange(input) {
    return {
      // 8640000 is the number of milliseconds in a day
      // set the maxDate for registrations to be the day _before_ the exam date, or no limit if there is no exam date yet
      maxDate: $('#exam_exam_date').datepicker('getDate') ? new Date($('#exam_exam_date').datepicker('getDate') - 86400000) : null,
      // set the minDate for registration close to be the day _after_ registration open
      minDate: input.id == 'exam_close_date' ? ($('#exam_open_date').datepicker('getDate') ? new Date(new Date($('#exam_open_date').datepicker('getDate')).getTime() + 86400000) : null) : null
    }
  }
});

The customRange function above is called every time before the datepickers appear. It sets the maxDate and minDate properties of the datepicker. This allows the constraints for one field to vary depending on the dates the user has chosen for the others.

When you press save, all of the dates will be sent to the controller and saved to your record. However, if validation fails for some reason and the form is reloaded, and extra step is necessary to make sure the constraints continue to behave properly.

You see, the datepicker widgets are kind of dumb. If they are attached to a field that already contains a textual representation of a date (such as when your form is reloaded after failed validation), they will not set themselves to that date. You must do this manually. Here it is, in the final incarnation of application.js:

////
// Behaviours
$(document).ready(function() {
  ////
  // ui.datepicker fields
  $('#exam_exam_date').datepicker();
  $('#exam_open_date').datepicker({beforeShow: customRange});
  $('#exam_close_date').datepicker({beforeShow: customRange});

  // initialise the date in the datepickers from the text in the input fields
  // this is necessary for the page reload that occurs after a failed validation
  $('#exam_exam_date').datepicker('setDate', new Date($('#exam_exam_date').attr('value')));
  $('#exam_open_date').datepicker('setDate', new Date($('#exam_open_date').attr('value')));
  $('#exam_close_date').datepicker('setDate', new Date($('#exam_close_date').attr('value')));

  function customRange(input) {
    return {
      // 8640000 is the number of milliseconds in a day
      // set the maxDate for registrations to be the day _before_ the exam date, or no limit if there is no exam date yet
      maxDate: $('#exam_exam_date').datepicker('getDate') ? new Date($('#exam_exam_date').datepicker('getDate') - 86400000) : null,
      // set the minDate for registration close to be the day _after_ registration open
      minDate: input.id == 'exam_close_date' ? ($('#exam_open_date').datepicker('getDate') ? new Date(new Date($('#exam_open_date').datepicker('getDate')).getTime() + 86400000) : null) : null
    }
  }
});

What does this all give you?

JavaScript code that is separated from your views, keeping everything nice and clean
Datepickers that will appear when their text fields are focused
Constraints that prevent the user from selecting dates that are out of bounds
Properly initialised datepickers after a form reload, such that they will display the correct date for the fields when clicked, and still adhere to the proper constraints

All up, this amounts to a pretty solid set of date selection widgets. Thanks, jQuery.

Generating semi-private, obfuscated resource sharing URLs in Rails

2008-03-11T12:25:00+00:00

Recently I was working on an app that required semi-private, obfuscated URLs for sharing pages and feeds with non-registered members, much like Backpack does for its page feeds. Specifically, I wanted a URL with a long code in it of some kind that would make it difficult to guess.

I could not find anything on the net addressing this kind of requirement, much less in a RESTful way, so I rolled up my sleeves and built one independently. Here’s how I did it.

In this example, I want to share Activity resources, which I am exposing in the typical way in Rails’ routes.rb:

map.resources :users do |user|
  user.resources :activities, :path_prefix => '/:user_id'
end

This routes provides access to the 7 standard CRUD actions in the ActivitiesController: index, show, new, create, edit, update, destroy. The only one of these that relates to listing multiple activities is index, but that action is already used to display activities to logged in users. So, a new action is needed, which we shall called “shared”. I will create 2 new named routes to provide access to this:

map.shared_activities '/:user_id/activities/shared/:key',
  :controller => 'activities', :action => 'shared', :conditions => { :method => :get }

map.formatted_shared_activities '/:user_id/activities/shared/:key.:format',
  :controller => 'activities', :action => 'shared', :conditions => { :method => :get }

Take note of the :key component of these paths. This is the private code required to ‘unlock’ the shared activity pages. The key belongs to Sharing objects:

class Sharing < ActiveRecord::Base
  belongs_to :user
  validates_uniqueness_of :key
  after_create :create_key

  private

  def create_key
    self.key = Digest::SHA1.hexdigest(Time.now.to_s.split(//).sort_by {rand}.join)
    self.save
  end
end

So, if a user wants to share his activities, he can create a sharing object, which generates the key for the URL that they can share with their non-registered friends. One of these URLs will look like this:

http://myapp.com/tim/activities/shared/a258f423366a2a07ffd3afec8c07f1bed8e07ba9

This will load the shared action in the activities controller, which is also protected by a before_filter that will only allow access if the key in the URL is valid for the user.

class ActivitiesController < ApplicationController
  before_filter :login_required, :except => :shared
  before_filter :get_user
  before_filter :sharing_required, :only => :shared

  def shared
    @activities = @user.activities.paginate(:order => 'created_at DESC', :page => params[:page])
  end

  private

  def get_user
    @user = User.find_by_login(params[:user_id])
  end

  def sharing_required
    # pull the key out of the URL and verify that the user has one to match
    unless Sharing.find(:first, :conditions => ['user_id = ? AND key = ?', @user.id, params[:key] ])
      flash[:error] = 'You do not have permission to view this page'
      redirect_to '/'
    end
  end
end

To flesh out the implementation, all you need to do is the following:

Add views for the shared action in the activities controller
Write a builder view to output the sharing action in RSS (explained here)
Include a sharing resource (nested under the user resource) in routes.rb and add a CRUD interface for creating and manipulating Sharing objects

Canberra Ruby Crew February meeting wrap-up

2008-02-23T08:35:00+00:00

The second Canberra Ruby Crew meeting was a success. Our number grew to seven, and all had a good time chatting, eating, playing guitar hero and sharing ideas and code. Thanks to everyone who came along.

Some things we did:

Hugh introduced some of the crew to the wonders of Haml and Sass and bringing them together with StaticMatic
Shoaib ran us through how to get started with git and gitorious
Matt showed off his use of the Stencil plugin and his rocking Open Flash Charts.

The next meeting will be on Friday the 21st of March. Keep your eyes on the Rails Oceania google group for an announcement closer to the date!

Edit: The next meeting will actualy be on Friday 28th March, beginning a last-Friday-of-the-month pattern.

Naming schemes

2008-02-11T04:30:00+00:00

Garrett Murray writes about his history with naming schemes for hardware and networks.

My favourite scheme so far was in one of my former offices, where we named everything after Street Fighter characters. A few of the notable names:

ehonda was the network file server - big, fat and full of data
guile was the backup server - being a millitary man, he covered our six
blanka - courtesy of his electric shock powers - was the firewall
vega was my workstation, because I could speak Spanish

Another benefit of the scheme was that, given the proliferation of Street Fighter sequels, it was unlikely we were going to run out of names very quickly. The scheme I use at home is characters from Azumanga Daioh. My iMac is Chiyo, and a rarely leave home without Tadakichi-san, my MacBook.

The Canberra Ruby Crew and getting involved

2008-01-21T02:55:00+00:00

Last Friday night we ran the inaugural CRC (Canberra Ruby Crew) meeting. It was a good time; we ate, wrote some code, debugged some code and played some Guitar Hero. There were just three of us in attendance, but there should be at least five for the next month’s meeting.

I’ve been to a couple of Rails nights in Adelaide before I moved here, but this meet left my truly inspired. I put this down firstly to the setting that allowed us to dig our heels into some code, and also the news, energy and wider perspective of the Rails scene that Matt brought from his involvement in the large and active Sydney community. I’ll be definitely visiting their next gathering.

This brings me to my plans for 2008. I’ve been meaning to write for some time about what I plan to achieve this year, and I will start with a themeword, a notion I picked up in a post from Chris Messina.

My themeword for 2008 is involvement. I wrote in my notebook recently that I want to:

Give myself fully to what I am doing at any time, without worrying or getting distracted by the other components of my life.

So this year I plan to get involved in a number of ways: dive fully into my hacking, web projects and Tagalog study, and allow myself to completely unwind when I take myself away from these.

I will also get involved in the community in a way I have never done before. Until now, I haven’t fully engaged in local web and tech communities, forsaking this involvement largely to pursue volleyball ambitions. This year, however, I want to involve myself in and contribute to these communities. Last week’s CRC meeting has already shown me how rewarding this activity is. I’m looking forward to a fruitful and productive year!

Oh, and I must also buy Guitar Hero.

ActionMailer templates in Rails 2.0

2008-01-10T07:15:00+00:00

When upgrading to Rails 2.0, it seems you cannot rename all of our .rhtml templates to .html.erb - it seems ActionMailer will refuse to find templates with this extension. Make sure you rename them to just .erb.

Tim Riley / timriley.info

Tim in open source, September 2024

RedDotRubyConf returns!

Hanami actions, meet validation contracts

Hello again, MySQL

New ways to go multi-database

Did someone say dry-cli?

Just before: a release!

Right now: a break!

Up next: 2.2 by RubyConf!

Thank you (again!) to Hanami’s contributors

A cheeky question

Tim in open source, July 2024

2023 in review

Five conferences

A near-release of Hanami

Family trips!

Work at Buildkite

Reading

Assorted things

Open source status update, September 2023

Assorted small things

Making our better errors better

Kickstarting dry-operation!

Releasing Hanami 2.1.0.beta2!

Open source status update, August 2023

Open source status update, October 2022–July 2023

Released Hanami 2.0

Spoke at RubyConf Thailand

Overhauled hanami-view internals and improved performance

Spoke at RubyConf Australia

Created Hanami::View::ERB, a new ERB engine

Implemented helpers for hanami-view

Added friendly error pages

Worked on integrating hanami-assets

Prepared the Hanami 2.1.0.beta1 release

Spoke at Brighton Ruby!

Took a holiday

What’s next

Open source status update, September 2022

A story about Hanami::Action memory usage

Hanami 2.0, here we come

Open source status update, August 2022

Conditional slice loading in Hanami

Fixed how slice config is applied to component classes

Released Hanami 2.0.0.beta2

Created memory usage benchmarks for dry-configurable

Quietly relaunched Decaf Sucks

Open source status update, May–July 2022

Released Hanami 2.0.0.alpha8

Resurrected work using dry-effects within hanami-view

Unified application and slice

Devised a slimmed down core app structure

Devised a slimmed down release strategy

Released Hanami 2.0.0.beta1!

And a bunch more

Joining Buildkite, and sticking with Ruby

Open source status update, April 2022

Finished centralisation of Hanami action and view integrations

Enabled proper access to full locals in view templates

Merged a fix to stop unwanted view rendering of halted requests

Shipped some long awaited dry-configurable features

Started work on unifying Hanami slices and application

Two years of open source status updates

Salubrious Ruby: Don’t mutate what you don’t own

Open source status update, March 2022

Let the shape of the code reflect its flow

Salubrious Ruby

Open source status update, 🇺🇦 February 2022

Stand with Ukraine 💙💛

Concrete slice classes

Slice registration refactors

The single-file-app dream lives on

Removed some unusued (and unlikely to be used) flexibility

Made Application.shutdown complete

Simplified configuration by permitting env to be provided just once

Worked to define a consistent action and view class structure

Released alpha7

What’s next?

Open source status update, December 2021 and January 2022

Enabled proper access to full `locals` in view templates

Made `Application.shutdown` complete

Simplified configuration by permitting `env` to be provided just once