Walk to Woolacombe and Morthoe, North Devon.

Looe, #Cornwall

Exeter Cathedral, Devon.

As I mentioned in my previous post, this has allowed me to migrate existing content from my Markdown files using a custom CLI tool I've written in Ruby. It also means that - thanks to the obsidian-pagecord plugin - I can use Obsidian to write and publish content.

Email posting is the headline feature that has drawn me to Pagecord for friction-less publishing, but whilst HTML formatting is supported, it doesn't parse Markdown in the emails.

For those longer posts where I would like to be more precise about headings, links, code blocks, etc., the Obsidian Pagecord plugin means I can write in a familiar - and powerful - Markdown editor, whilst easily publishing directly to the site.

☕ Morning #coffeeandcode at Elements Cafe, Plymouth Science Park.

Ever since it was shut down, I've wished to replicate that simplicity in my publishing. I've tried various blogging platforms; started writing my own apps; and put together solutions via tools like Zapier - but nothing felt quite as elegant and efficient as Posterous.

Most recently I've been using 11ty to power this site, but - even though I spend most of my working time in a code editor - I've never found using Zed or VSCode or Vim, or any code editor, a good place for writing.

Coupled with the fact I would procrastinate by running npm updates, wrangling with JavaScript, and trying out the latest version of [insert any framework/tool/package here] to get my site "just right", it has meant that I've always spent more time updating the code rather than the content.

I've always found myself settling back to a static site generator, and the hope that I'll "just write more". That's often not the case though.

So I've been been interested to see the  the development of Pagecord, and light but increasingly-powerful blogging platform written by Olly Headey.

(As an aside, Olly was one of the co-founders of FreeAgent - the bookkeeping software I've used since day one of my business, and wouldn't be without 🙏)

Back to Email

Like Posterous, Pagecord is based on the same idea of posting via email. I love this, and the recently released API - coupled with Olly's fast responses to feedback and adding or tweaking features - has given me the confidence to finally take the plunge and migrate this blog.

I'm also happy knowing that the blog is running on a Rails app again, which I'm far more familiar with!

Migrating Content

To quickly migrate the content over, I put together a simple pagecord-cli tool to import my markdown posts and do some gsub-bing to remap asset URLs.

I've also included a feature in the CLI tool can export my content back to markdown+yaml to ensure data portability.

Future

I really like Pagecord, and Olly has done an amazing job. Pagecord is open source, although contributions are currently closed.

However, one element I'm hoping to add are thumbnail images for my moments posts, rather than the current list of titles. 

But, more than anything, it's great to support a great, independent blogging app. I hope that I'll be able to settle my blog and finally get back into the writing habit unencumbered by package updates, CSS tweaks, breaking changes and specs, and WYSIWYG editors.

PS. Yes - this post was written and sent via email 🎉

Ref: Sky News: Are you using an AI-generated password? It might be time to change it

View through a stone wall looking back to Plymouth near Foggintor Quarry, Dartmoor, Devon.

Happy New Year!

Welcoming 2026, here is what I plan to focus on in the months ahead. The key themes are simplification, lightness or reduction, and relying less on third-party software and services.

• Develop products and productised services to complement my consulting work. Simplify the offering - it feels like I've overcomplicated things. Business goals is a separate post...
• Reduce reliance on hosted services. Default to self-hosted, open-source, and plain-text-first software. See uses for progress, and my current #appdefaults.
• Similarly, simplify the technology I use. Fewer gadgets. Fewer "smart" devices. Fewer apps. Fewer payments.
• Travel more, more lightly.
• Continue to work on strength training, diet, and reducing BF%.
• Read more books. Currently I am tracking books to read, those in progress, and books finished in an Obsidian note.
• Keep walking, and walk more.
• As ever - publish a bit more content here!

See now for an updating list of what I'm focussing on throughout the year.

Reflective Christmas Lights in the trees at Crealy, Devon, UK

Christmas Lights Installation @ Eden Project, Cornwall, UK

Hamilton @ Theatre Royal, Plymouth, UK.

Hockings Ice Cream, Westward Ho!, Devon, UK

Coleton Fishacre, Devon, UK.

Coleton Fishacre, Devon, UK.

Nemesis Reborn, Alton Towers, UK.

Kamal Proxy is a load balancer, designed to be exposed to the Internet. However, in my client's situation, the server itself (on which Kamal Proxy was running) was behind a GCP Cloud Load Balancer.

GCP Load Balancers (and, presumably, similar products such as AWS) don't allow a hostname to be specified when performing a healthcheck. They ping the server on a given port, and expect an HTTP 200/OK response.

If Kamal Proxy is running a single app, or an app that is running on a wildcard hostname, then this isn't a problem. However, as in this case, you may have multiple, isolated apps each with its own specific hostname, e.g:

• httpx://appname-dev-mary.example.com
• httpx://appname-dev-john.example.com
• ...

When the Cloud Load Balancer pinged the server without specifying a host, Kamal Proxy would respond with a 404. The result is that the server was never shown as healthy, so the Cloud Load Balancer would not route any traffic to it.

Workaround

Without a solution directly in Kamal Proxy itself, I worked around the problem by installing a small app directly into Kamal Proxy that simply returned a 200/OK response when hit with /up.

The app uses a customised nginx container, which can be built use the following Dockerfile. For simplicity, you can create this on your target (deployment) server:

# Dockerfile
FROM nginx

RUN <<EOF
echo "server {
    listen       80;
    server_name  localhost;
    location /up {
        default_type text/plain;
    return 200 "OK";
    }
    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   /usr/share/nginx/html;
    }
}" > /etc/nginx/conf.d/default.conf
EOF

Next, build and deploy the image into Kamal Proxy:

$ docker build --tag nginx-healthcheck .
$ docker run -d -it --network kamal --name gcp-healthcheck-1 nginx-healthcheck
$ docker exec -it kamal-proxy kamal-proxy deploy gcp-healthcheck-1 --target gcp-healthcheck-1:80 --health-check-path "/up" --deploy-timeout 5s

You can check the app is working correctly by sending a request to localhost on your server:

$ curl http://localhost/up
# OK

With the health-check app running, your Cloud Load Balancer will now see the server as healthy and start routing traffic to it.

Note: This is OK for internal/development servers, but probably not production.

As the host is not specified (--host *), kamal-proxy will respond to any URL that points to the server and isn't connected to an application. It will also show the server as healthy (to the Cloud Load Balancer), but the individual apps themselves might not be up and running. In that case, Kamal Proxy will return an error.

Porth Beach, Cornwall

Coastal walk near Haugesund, Norway.

Fjords at Olden, Norway.

Sunset, North Sea

St Pancras, London

Lille, France

Happy New Year!

Updating goals for the year ahead...

Christmas Eve at Crownhill Bay, Devon.

Minehead Beach, Somerset.

One of the things I've noticed running Plymouth Software over the years is just how quick and easy it is to over-complicate everyday tasks.

As business owners, there is a vast (vast!) number of apps and services vying for our attention - and curiosity leads me to want to try each of them.

Over months and years, all these things build up until eventually we spend more time navigating between them, learning new interfaces, etc. than actually creating anything…

For this reason, I've spent a little time recently trying to cut complexities out of my business, and settle on using a few simple tools and approaches.

One of the most important services I am using to do this is Zapier, a "no/low-code" application to connect various systems together. With a small amount of custom code to tidy up text and formatting, Zapier can be used to build powerful event-driven processes into your business.

Email as the Interface

As an example, this post was written in my normal email app. I love the simplicity of email-as-the-interface, and wanted to adopt the approach for my mailing lists. I currently use (Convert) Kit, but I find the content editor creates too much friction. I wondered if I could instead write posts in my normal email UI, but still publish them through Kit.

So I created a Zapier automation - or Zap - to do just this. I send the email to a special, unique address created by Zapier. It then parses the content, using a little custom code to do tweak the content so it meets Kit's requirements, and then creates the email draft via their API.

Future improvements

Currently, the automation stops there, leaving the email as a draft in my Kit account. This lets me tweak the audience segments, schedule publication date, and so on.

Once Zapier updates the Kit integration to support the V4 API, this is something that I hope to fully automate.

Christmas Lights Experience at Eden Project, Cornwall.

Cotehill Mill, Cornwall.

Snow skies over Dartmoor, Devon.

Sunset walk at Eden Project, Cornwall.

Morning Coffee (and code) - Nuffield Devonshire.

Sunday afternoon walk along the river at Lydford Gorge, Devon.

{
  "license": "UNLICENSED",
  "private": true,
  "dependencies": {
    // ...
    "package-a": "github:cblunt/package-a#main",
    "package-b": "github:cblunt/package-b#main1",
    // ...
  }

When building the image, the yarn install step attempts to download these packages, but requires the use of my host machine's private SSH key to authenticate and connect to the Github repositories.

Luckily, Docker has introduced the concept of build-secrets to handle just this situation. Build-secrets allow us to mount environment variables temporarily into the build process, discarding them afterwards so that they are not present in any of the image layers.

Furthermore, there is a special SSH build-secret that forwards the host's SSH agent into the image. In the Dockerfile, this can be achieved using the --mount=type=ssh argument:

RUN --mount=type=ssh mkdir -p ~/.ssh && \
    ssh-keyscan github.com > ~/.ssh/known_hosts && \
    yarn install && \
    rm -rf ~/.ssh

The docker image can then be built by passing adding your key to the local SSH agent, and forwarding it into the build (using the identifier default which defaults to the value of the SSH_AUTH_SOCK environment variable):

$ eval $(ssh-agent)
$ ssh-add ~/.ssh/id_ed25519 # path to your private SSH keyfile
# Identity added: ...

$ docker buildx build --ssh default .

Docker will now build the image, temporarily using the forwarded SSH credentials to connect to Github and download packages from private repositories.

1Password Managed Credentials

However, I use 1Password to manage my SSH keys and act as the SSH agent. This means that the private keyfiles (id_ed25519, id_rsa, etc.) are not saved into my home folder.

As 1Password runs as the SSH agent on my machine, instead of mounting the SSHAUTHSOCK, I needed to instead pass the socket file that 1Password creates, the location of which can be found in their documentation.

With this, we can skip adding the key to the default SSH agent, and instead make use of 1Password's agent:

$ docker buildx build --ssh default==$HOME/Library/Group\ Containers/2BUA8C4S2C.com.1password/t/agent.sock .

Docker will now build the image, using 1Password's SSH agent to forward the private key into the build process. This keeps the key securely stored in 1Password.

The same argument can also be passed when using Docker Compose:

$ docker compose build --ssh default=$HOME/Library/Group\ Containers/2BUA8C4S2C.com.1password/t/agent.sock

Passing Github Tokens via 1Password

Similarly, I use 1Password's gh plugin to manage my Github access token.

In the same way as SSH secrets can be mounted, Docker can also forward other secrets into the build process that are stored in environment variables.

For example, to pass your Github token into the build downloading gems from Github:

RUN --mount=type=secret,id=github_token BUNDLE_GITHUB__COM="x-access-token:$(cat /run/secrets/github_token)" \
    bundle install && \
    rm -rf ~/.bundle/ "${BUNDLE_PATH}"/ruby/*/cache "${BUNDLE_PATH}"/ruby/*/bundler/gems/*/.git

Note that the secrets are accessed like a text file, so to get their value in the Dockerfile, we use cat to output the value of the secret: cat /run/secrets/github_token.

To pass this value into Docker:

$ docker buildx build --secret id=github_token,env=GITHUB_TOKEN .

Using the 1Password, op command, these values can retrieved and passed into the build:

$ GITHUB_TOKEN=$(op read "op://Vault/Github Token Item Name/token") \
    docker buildx build --secret id=github_token,env=GITHUB_TOKEN .

Similarly, the token can be configured in Docker Compose:

services:
  # ...

secrets:
  github_token:
    environment: GITHUB_TOKEN

It can then be read when running docker compose build by setting the environment variable:

$ GITHUB_TOKEN=$(op read "op://Vault/Github Token Item Name/token") \
    docker compose build

Using this approach, it is possible to inject both SSH and token secrets managed by 1Password into your Docker build process, while maintaining the benefit of Docker's build secrets.

Sutton Harbour, Plymouth.

Afternoon walk around the lakes at Ferry Meadows.

Coffee meet with Glenn at Rusty Tractor Farm Cafe, Devon.

Weston Super Mare, Somerset.

Rails includes a default script, bin/setup, that runs some basic environment setup such as installing gems, creating and seeding the database, and so on.

Often, though, any further environment setup, such as configuring default user accounts, installing supporting services, and so on, is described in a project's README. With time, this document grows and the installation process becomes more complicated, often including lots of copy-and-paste scripts, manual file edits, and so on.

I'm working on upgrading a legacy application that had used this approach, and so spent some time consolidating the various setup requirements and configurations into an automated bin/setup script.

The result was that the full local development stack, including supporting Docker images, became a simple one-line command:

$ bin/setup

The benefits of such a small investment can be huge, especially when onboarding new members to your dev team, or simply spinning up a new working copy of the application for yourself.

Keeping the setup in a single command also makes it simpler to provision working environments in services such as Github Codespaces.

Westward Ho!, North Devon.

One of the benefits of the Alpine Linux images is that they are very minimal, resulting in much smaller images for the same application.

So, I set about rewriting the default Dockerfile that's generated by Rails to use the ruby:3.3.1-alpine3.18 image, and discovered a few trip-hazards along the way.

Getting Started

In this tutorial, we'll spin up a default Rails app (using SQLite as a database) and update the Dockerfile and requirements to use Alpine. To get started, create a new app and the traditional blog-post scaffold:

# versions: ruby = 3.3.1, rails ~> 7.1.3
$ rails new demo-app
$ cd demo-app
$ bin/rails g scaffold post title:string body:text
$ bin/rails db:migrate
$ bin/rails s

Open localhost:3000/posts to ensure that everything is running smoothly. At this stage, we can also create a new docker-compose.yml file to simplify building images and creating containers:

# docker-compose.yml
services:
  app:
    build: .
    platform: linux/amd64
    environment:
      - RAILS_ENV=development
    ports:
      - 3000:3000
    volumes:
      - .:/rails
    command: ["bin/rails", "server", "-b", "0.0.0.0"]
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:3000/healthz"]
      interval: 30s
      timeout: 10s
      retries: 5

We can now build the default image for the application using docker compose, and see the resulting image is around 730MB in size.

$ docker compose build
$ docker images

# REPOSITORY     TAG     ...     SIZE
# demo-app-app   latest  ...     730MB

Switching to Alpine

Alpine Linux is a lightweight linux distribution, making it well-suited to Docker images.

To switch our application to use Alpine instead of the default Debian-based ruby image, we need to make some changes to Dockerfile base image, and the packages:

# Dockerfile
- FROM registry.docker.com/library/ruby:$RUBY_VERSION-slim as base
+ FROM registry.docker.com/library/ruby:$RUBY_VERSION-alpine3.18 as base

  # Install packages needed to build gems
- RUN apt-get update -qq && \
-    apt-get install --no-install-recommends -y build-essential git libvips pkg-config
+ RUN apk add --update --no-cache build-base git vips sqlite-libs tzdata

  # Install packages needed for deployment
- RUN apt-get update -qq && \
-    apt-get install --no-install-recommends -y curl libsqlite3-0 libvips && \
-    rm -rf /var/lib/apt/lists /var/cache/apt/archives
+ RUN apk add --update --no-cache curl vips sqlite
# ...

# Run and own only the runtime files as a non-root user for security
- RUN useradd rails --create-home --shell /bin/bash && \
+ RUN adduser rails --disabled-password --shell /bin/ash && \

We also need to make a slight change to the default docker-entrypoint file, as bash is not available in Alpine. We'll use the default ash shell instead:

# /rails/bin/docker-entrypoint
- #!/bin/bash -e
+ #!/bin/ash -e

With the files updated, you can now build the new image and see the reduced image size (in this case, a reduction of around 69.5%):

$ docker compose build
$ docker images

# REPOSITORY     TAG     ...     SIZE
# demo-app-app   latest  ...     223MB

Next, restart your container using the new image:

$ docker compose up -d
$ open http://localhost:3000/posts/

You should now be presented with the posts index. Note that because our app is using the default sqlite3 database, the database file (storage/development.sqlite3) has been copied into the container. This is because the local directory is mounted into the container by Docker Compose.

For production, however, these database files won't be present. You can simulate this by removing the bind mount from docker-compose.yml:

# docker-compose.yml
services:
  app:
    # ...
    # volumes:     <- comment out these lines
    #  - .:/rails  <- 

Now recreate your Docker Compose stack:

$ docker compose up -d
$ open http://localhost:3000/

In development mode, you will now see the Pending Migration error, with an option to run the migration. Click the button to run the pending migration, and your app will continue to operate as normal.

Remember that when you remove your container (e.g. with docker compose down), then the database file will also be deleted. For this reason, it is recommended to revert the previous change to docker-compose.yml, which is fine for development and testing:

# docker-compose.yml
services:
  app:
    # ...
    volumes:
     - .:/rails

$ docker compose up -d

All done! Your rails app is now running on the much smaller Alpine linux base image.

Eiffel Tower, Paris.

Louvre, Paris.

Disneyland Paris, France.

I wrote a Pl/pgSQL function to perform the lookup, and - as the value would be reused across a batch of 100-or-so rows per lookup - insert the results of the lookup into a cache table.

The function looked something like this:

-- Example function, not the real code...
CREATE FUNCTION get_document_title(userid int)
  RETURNS TABLE(user_id int, document_label text)
  LANGUAGE plpgsql
AS $$
DECLARE
  cacheresult record;
  result record;
BEGIN

--
-- Check cache
SELECT COUNT(*) INTO cacheresult 
  FROM cache_table WHERE user_id = userid;

IF cacheresult > 0
THEN
  --
  -- Result has been cached
  RETURN QUERY
    SELECT user_id, document_label
      FROM cache_table 
      WHERE user_id = userid
      LIMIT 1;
ELSE
  --
  -- Perform the lookup. 
  -- In the real code, there are actually several branches for looking up from different sources, but for this demo, I've simplified:
  SELECT * INTO result
    FROM documents 
    WHERE user_id = userid 
      AND document_type_id = doctypeid 
      AND project_id IN (
        SELECT id 
          FROM projects 
          WHERE archived IS NOT TRUE
      ) 
    LIMIT 1;

  --
  -- Cache the result
  INSERT INTO cache_table (user_id, document_label) 
    VALUES (userid, result.document_label);

  --
  -- Return the result
  RETURN QUERY
    SELECT document_label
      FROM result;
END IF;
END;
$$

However, with over 100m rows to process, the resulting query was projected to run into days - if not weeks - of processing time. This was running on a fairly large Aurora PG cluster.

Whilst on a walk and not thinking about this problem, the idea suddenly occurred to me of splitting the process across parallel database clusters.

The source table had already been partitioned by year in an attempt to speed things up, but as each row was being updated in sequence, the time required was still too long.

So I set about using RDS to clone the source database - one for each year partition - and then starting the UPDATE individually on each cluster using a small bash command:

$ echo "UPDATE partition_table_2015 SET document_title = get_document_label(user_id) WHERE ..." | psql -h dbcluster_1;

Similarly, on the second DB cluster:

$ echo "UPDATE partition_table_2016 SET document_title = get_document_label(user_id) WHERE ..." | psql -h dbcluster_2;

...and so on.

With each DB cluster having its own resources (and writer instance), the plan was that this would drastically cut down the number of hours required to process the full table.

As each DB cluster finished updating its partition, the partition table was exported via pg_dump and then reimported back into the primary DB cluster:

# Export partition from each individual DB cluster:
$ pg_dump -Fc -h dbcluster_1 --table partition_table_2015 partition_2015.pgdump
$ pg_dump -Fc -h dbcluster_2 --table partition_table_2016 partition_2016.pgdump
$ pg_dump -Fc -h dbcluster_3 --table partition_table_2017 partition_2017.pgdump

# Import all the exports back into the Primary DB cluster:
$ pg_restore -h db_primary /tmp/partition_2015.pgdump
$ pg_restore -h db_primary /tmp/partition_2016.pgdump
$ pg_restore -h db_primary /tmp/partition_2017.pgdump

Whilst this process still took some time, parallelising the work across multiple DB clusters definitely sped things up, and meant that each partition could be independently verified before removing the cluster and importing the pgdump file back into the main DB cluster.

Evening at the beach and waterside in Looe, Cornwall.

Note to Self has a chat-style UI, and while there is no data sync, it has SQLite export for data portability.

I'm hopeful that Markdown rendering is on the roadmap...

City Conversations with Devon and Plymouth Chamber of Commerce at Sutton Harbour, Plymouth.

🌲 Winter Sunday walk across Dartmoor.

Happy New Year!

I can't believe it's been a year since my last post! My hope of regular posting to this blog was dashed by a year of ongoing frustration learning on AWS for a large data migration and transformation project, amongst other client projects.

The silver lining is I've now gained a much better understanding of how the various services work together, the approach that's necessary for large data transform projects, and the (many, many) hoops that must be jumped through to get various AWS services such as DMS, Glue, Athena, Lambda, SQS, and RDS working together.

This has necessitated a lot of working with serverless tools and micro-service architectures, and managing so much service infrastructure to achieve desired outcomes.

In my experience, this new approach might have solved some problems, but has certainly introduced new ones into the mix.

This experience has only re-affirmed my resolve to stick with the simplicity of a "all-in", conventions-based framework such as Rails, coupled with managed infrastructure such as AppRunner or Digital Ocean's App Platform.

At the same time, I've been happy to see an apparent "resurgence" in Rails (and other full-stack) frameworks over 2023, and am excited to see the developments in recent and upcoming releases of Rails.

So with that in mind, I am updating my consulting services and developing a couple of new products for 2024. All being well, I'll post updates here (at a better rate than last year!) as they are released.

Wishing you a happy, healthy, and successful 2024.

current_year += 1

goals = %i[
  :write_more_articles,
  :update_rails_courses,
  :travel,
  :new_products,
]

goals.map do |g|
  # ...
end

I hope that your 2023 is happy, healthy, and successful.

currentYear++

let goals = new Array()
// TODO: ...

Up until last week, that is, when I heard myself uttering the phrase "Oh, it's Christmas" on multiple occassions when a excuse reason was needed for eating something not-traditionally-thought-of as good for you; or was needed to avoid some form of exercise. My Fitbit is begining to wonder if it's been retired.

So in a haze of coffee*-fuelled enthusiasm, I'm starting to plan out what my goals for 2023 will look like. As ever, the world of software is changing very rapidly, so as well as personal goals, it's also a good time to figure out what Plymouth Software will be for the months and years ahead.

* Topped with cream, of course. Because it's Christmas.

More often than not, I'm distracted by the wealth of tools that are available, such that I end up procrastinating over those more than actually writing!

This is something I struggle with in running business as well - there are always so many options and things-to-be-done, that it is far too easy to get stuck very much in the weeds, postponing decisions that would be much better made and forgotten about.

I'm sure technology was supposed to help eliminate such problems, but often the limitless options available can actually hinder progress. Just look at the number of static site generators on which I could run this site.

The developer part of me probably wants to deal with every technicality; whereas what I should be doing is using what the technology affords me to produce the content!

Status

Currently, I'm still bouncing between the plain-text simplicity of Obsidian and the more structured, but seemingly less-portable, Standard Notes.

I've already procrastinated researched several hours trying to get 11ty and new versions of Jekyll working, and whilst I may eventually like to get back to that level of control over this blog, for now I'm sticking with a hosted blog platform - Blot for Obsidian-written posts, and Listed for Standard Notes ones, switching between the two to test and optimise my writing flow.

PS. This post was written on Obsidian :). I used the Sanctum theme, and am publishing via git to Blot.

As I've been promising myself I'd write more on this site for years now - but consistently failing to do so - I thought this would be a good challenge to take up.

There seems to be a bit of a renaissance for traditional blogging, with platforms such as Listed, Hey World, write.as, and so on.

There'll likely be the odd day or two I miss posting, but hopefully I can find enough to fill up this blog with some thoughts, ideas, experiences, and the odd code tutorial (I'm spending a lot of time with cloud and serverless technologies at the moment).

Subscribe to get posts by email or RSS.

Going through this process has highlighted the amount of digital clutter I've archived over the years.

Many things aren't clutter - important documents and records, snippets and audio clips of first words, and various artwork and scribbles from over the years.

But do I also need to muddle them with bills and insurance documents from nearly two decades ago? Ticket confirmations from shows long-since seen?

As Tony asks in his post, I'm questioning the benefits of being encouraged to "save everything" over the years:

Tools like Evernote trained us to capture everything. Scan a document. Send it to the cloud. Keep it forever. In all honesty, you’d have shredded the real thing by now.

I remember that being one of the original features of GMail when I signed up ... No need to worry about storage, just throw everything into the archive...

It may have been a feature, but I'm not sure it's a benefit. It's more to think about; more to manage when you eventually want to move, or even hit the "delete" button.

If nothing else, knowing it is sitting there, wasting taking up storage, and forever building up can feel overwhelming. It's very much the digital equivalent of the messy desk. Multiplied by a seemingly infinite number of bytes.

Will I need that water bill from 2007? Probably not, but I'll leave it archived for now.

Just in case

"Head out for a 20 or 30 minute walk" was my advice to the PSW team whenever they were stuck on a coding problem.

It is one of those lessons in life that I keep re-discovering for myself.

With the simple act of stepping outside, away from the keyboard, thoughts are clarified; solutions to problems suddenly appear; the mind has a chance to reset.

Although it's something that feels like it could be lost in the never-ending streams-of-notifications, time management tools, and productivity-"hacked" tasks that compete to fill our day.

There is something revitalising that comes with being outdoors.

Admittedly, the weather is not always ideal for getting out, but the act of stepping away from the keyboard, and giving your mind some space and time amongst nature, can pay dividends - and often help you fix that bug in your code.

Whilst Evernote allowed exporting of data, its .enex files were a custom format, although there are plenty of importers available. I ended up using Joplin's importer to convert my Evernote exports into folders of Markdown + Front Matter.

Markdown - which is just plain text - is ultimately portable. As long as plain text files are readable, then the notes I've captured over the years should be as well.

This blog has for years been powered by Markdown, starting with Jekyll, and moving to Blot.im and Listed.

Testing for Portability

Standard Notes also uses markdown, so importing my notes was easy. However, as I mentioned yesterday, attachments were not imported, leaving me to manually upload and reconnect attachments with their notes.

Whilst gradually doing this across my notes, I decided to try exporting content from Standard Notes to ensure that my data was portable should I decide to move my notes in the future. (the note-taking apps space seems really busy at the moment!)

Unfortunately, whilst the Markdown data is exported, the attachments are not.

Most of the notes I've captured have been documents, receipt, artwork, or audio, making these exports redundant. I would be constrained to the way Standard Notes organises things (at least, without manual processing should I wish to move to another note platform).

Obsidian

Alongside Standard Notes, I've also been trying out Obsidian, which - using a completely file-based system of organisation - allows attachments to notes to be preserved through simple Markdown [title](./path/to/file) links.

Thanks to the Joplin conversion of my Evernote files to Markdown+Front Matter, along with attachments in their own folder, getting my data into Obsidian was relatively easy.

Whilst this puts the responsibility of data management on the user, a purely file-based structure feels more future proof.

The ability to run scripts and tools across the files (e.g. normalising YAML front-matter, etc.) is an advantage.

Unlike Standard Notes, Obsidian doesn't include a blogging platform. However, being Markdown, the files can easily be published to a static provider, or a service such as Blot, again.

I'll continue working with Obsidian and Standard Notes to determine which of the two best fits my needs. However, the portability of data and ability to migrate to another service without too much manual processing is something that will be a key requirement.

As part of my note-taking journey, I'm currently trying out Standard Notes.

Standard Notes seems to tick most boxes - multi-platform, reliable syncing, privacy-focussed and open-source, offline support, and it's really fast. I particularly like multiple editors for different types of content.

Attachments

The one thing that puts me off Standard Notes is its lack of support for embedded attachments, and the seemingly strange way it handles attachments altogether.

I used Evernote to capture everything - documents, artwork, audio recordings, as well as text-based notes. Unfortunately getting these into Standard Notes is a slow, manual process at the moment - and something I keep starting and giving up on.

Unfortunately, Standard Notes doesn't provide a way to import Evernote or Markdown files with attachments. Again, because of the way attachments are handled, this isn't something that can be done with the existing app.

As it's open source, though, I'm exploring if it's possible to add something to a custom build just to perform a one-time import.

Standard Notes has recently added the Super Note type which does allow for embedded content, and is a step in the right direction. The Super Note uses a custom JSON structure, though, so it strays away from the universal Markdown format.

Whilst it's possible to convert the JSON to Markdown, embedded content is lost. For data portability, I'd like to keep things in Markdown wherever possible.

Listed

One of the things I really like about Standard Notes is the built-in blogging platform, Listed. By way of an experiment to see if it helps me to write more consistently, I tried switching this blog over to Listed for a short time.

I much preferred static-site hosting, knowing that the site would never be locked into a platform as the content was stored simply as a set of Markdown text files. This meant that I could easily change things without needing to perform complicated export/imports across platforms.

As a ruby developer, I naturally settled on Jekyll, a popular static site engine that also powers Github Pages.

However, every time I came to write something for the site, I found myself being drawn into the minutae of managing the code that built the site, tweaking configurations and templates, updating gem dependencies, and so on.

As a result of this procrastination development, I hardly ever got round to publishing anything!

Blot

A couple of weeks ago, I stumbled across Blot. Within a few hours of trying out the service, I'd decided to use it for this site.

Just like Jekyll and other static-site generators, Blot compiles a folder of markdown files into a website. Amongst other things, Blot also handles the hosting, CDN/image hosting, theming, integrations, and most other technicalities for you.

Just like Jekyll, the files can be hosted in a Git repository. However, for even easier management, a Dropbox or Google Drive folder can be used instead. You can learn more about Blot works here.

Balance

For me, Blot seems to strike the right balance between a traditional CMS/hosted website (e.g, Wordpress), and the flexibility of a static-generated site.

This site is now running on Blot, and I'll likely be moving other sites here (such as plymouthsoftware.com and valuepricing.uk). I've also taken the opportunity to archive legacy posts and outdated tutorials to make some room for new content.

With Blot taking care of the technical detail, I hope to have far fewer distractions and so spend more time actually writing content and tutorials!

I really enjoy using Calm, mostly for sleep stories. A few years ago, what started out as a couple of minutes each day eventually built to a streak of a few weeks.

This led to my developer-brain thinking "I wonder how it handles 4-digit numbers?". The sort of thing you find yourself wondering when writing software all day...

So started the work towards a goal of a 1000 daily meditations, "breathe" sessions, and/or sleep stories. Even if only a minute or two, it counted towards the goal.

Unfortunately, I broke the first streak at 535 days, and so had to start from scratch...Roughly two years and 9 months later, I finally got to 1000 days.

The next step, seeing how Calm deals with 5-digit numbers, is due in just over 24 years...

When compiling the assets through bin/webpack-dev-server or bin/rails assets:precompile, we were getting the following error message:

Module not found: Error: Can't resolve '../bundles/HelloWorld/components/HelloWorld' in '/path/to/code/react-on-rails-app/app/javascript/packs'

However, it turns out the message is a little misleading. I discovered this by accident by changing the import line:

# hello-world-bundle.js
- import HelloWorld from '../bundles/HelloWorld/components/HelloWorld'
+ import HelloWorld from '../bundles/HelloWorld/components/HelloWorld.jsx'

Trying to compile the assets again gave a different error:

HelloWorld.jsx: Support for the experimental syntax 'jsx' isn't currently enabled (8:5)

The error seems to be that the JSX (the HTML-in-Javascript template language used by React) could not be compiled.

But JSX should come as part of the React installation, so I tried re-running the webpacker react generator:

$ bin/rails webpacker:install:react

From the logs, I could see this added babel (amongst other things) to the package.json.

After restarting bin/webpack-dev-server and bin/rails s, the tutorial page worked as expected!

One of Pipelines' features is allowing the setup of Docker containers to provide services, such as databases, to a job whilst running the job itself inside a container.

Unfortunately, when I configured a service container to run a MariaDB instance, the pipeline would try to run the jobs before the database had initialised:

Waiting for MySQL database to accept connectionsERROR 2002 (HY000): Can't connect to MySQL server on 'db' (115)
.ERROR 2002 (HY000): Can't connect to MySQL server on 'db' (115)
.ERROR 2002 (HY000): Can't connect to MySQL server on 'db' (115)
...

Service Containers in pipelines do make use of the image's HEALTHCHECKcommand, but at the time of writing, the MariaDB images don't appear to include a healthcheck; and the Pipelines config file doesn't support adding one at runtime, in the same way that you might in a docker-compose.yml file.

As a workaround, I tried sleeping the tests to wait for the database to initialise, but regardless of the sleep time this didn't seem to work. I also wasn't happy including a hard-coded "wait" time which, even if it had worked, might lead to randomly-failing job runs depending on conditions.

Eventually, I found a working solution by using a short step that repeatedly attempts to connect to the the database server.

To set this up for your own project, first, make sure the Docker image you are using for your tests installs the mysql command, e.g.:

# Dockerfile (repository/yourappimage)
FROM ruby 2.6

RUN apt-get update && apt-get install -y mariadb-client

# ...

CMD ["bundle", "exec", "rails", "server", "-b", "0.0.0.0"]

Next, in your azure-pipelines.yml file, configure the service and job containers:

# azure-pipelines.yml
trigger:
- master

resources:
  containers:
  - container: db
    image: mariadb:10.1
    env:
      MYSQL_DATABASE: my_app_test
      MYSQL_ROOT_PASSWORD: superSecret123

jobs:
- job: specs
  displayName: Run the specs

  pool:
    vmImage: 'ubuntu-latest'

  services:
    db: db

  container:
    image: 'repository/yourappimage'
    endpoint: 'your-docker-hub-or-acr-service-connection'
    env:
      RAILS_ENV: test
      RACK_ENV: test
      DATABASE_URL: mysql2://root:superSecret123@db:3306/my_app_test

  steps:
  - script: |
      printf 'Waiting for MySQL database to accept connections'
      until mysql --host db --user=root --password=superSecret123 --execute "SHOW DATABASES"; do
        printf '.'
        sleep 1;
      done;
    displayName: Wait for database to initialise

  - script: bundle exec rails db:create db:migrate
    displayName: Prepare test db

  - script: bundle exec rails test
    displayName: Run specs

The step that checks for connectivity to the database server is at at line 32. It attempts to run a simple SHOW DATABASES command from our job container.

If that command fails, it sleeps for 1 second, and then tries again, repeating this process until the connection is successful (or the job times out).

This prevents the next step - creating the test database schema - from running until our app's container has confirmed that it can connect to the database server.

I hope this is useful for anyone using Azure Devops Pipelines as part of their CI setup. It may also be applicable to other CI services, including the new Github Actions, which allow the use of service containers to provide services to your tests.

This post summarises some lessons I've learned on my journey so far.

Last week, Plymouth Software proudly sponsored the FutureSync conference held at Plymouth University.

It was fantastic to see how much the conference has grown in just a couple of years. The event was a huge success, and I was also lucky enough to have the opportunity to give another talk. The topic I chose was somewhat-loosely titled “The Business of Software: Lessons Learned So Far”.

Taking some time away from the day-to-day running of business is a crucial - but also incredibly difficult to do when there is so much else to be done “in the business”!

In writing and practicing the talk, I had a lot of time to reflect on my experiences so far running Plymouth Software, and take some time to think about the bigger strategy moving the business forward.

Here I've summarised some of the key topics from the talk.

Start-up Days

When I first started the business, I made a point of writing about my experiences. I really enjoyed the process of writing about my business highs and lows “in the open”. I was inspired by people such as Nathan Barry and the team at Basecamp (then 37signals), and countless others.

However, as time when on two things happened that reduced the frequency of (and eventually stopped!) these posts :

1. As things got busier, I didn't devote as much time to blogging (and marketing in general - a big mistake); and
2. The more I was learning about the running business, the more I began to feel the effects of imposter syndrome.

Both of these are still the case - business is running well at the moment, and I have some ambitious growth plans for 2019-20; and to some degree I still spend a lot of time wondering if I'm doing things “the right way” to build my business (whatever that may be!)

Looking Back

In 2020, Plymouth Software will turn 10 years old. That alone is worrying, as it really does feel like I'm just getting started.

But as I came to terms with this whilst writing my talk, it seemed like a good opportunity to reflect on some of the lessons I've learned, and things I wish I'd have known when starting out.

It turns out there are so many things I could have spoken about. There were so many things I feel I've learned (but not necessarily taken action on!) that it was impossible to squeeze them all into a 20-minute session.

So instead, I focussed on four of the “headline” lessons I felt summed up where I've been, and where I want to be.

Build Your Audience

Quite simply, a business should have the biggest audience possible.

Most advice out there is to niche, and that is important, but I think - especially when starting out - you just need to gain exposure for your business.

With the wealth of networks (both on- and off-line) that are available nowadays, audiences will naturally build around the content you are delivering.

Whilst I've spent many years trying to remain “small” and focussed, I'm now trying to grow the businesses audience as much as possible.

"Obscurity is your Enemy" - Grant Cardone, The 10X Rule

As soon as I heard this quote by Grant Cardone, it resonated immediately. I realised that was describing how I'd been running my business - remaining obscure and trying to fly “under the radar”, rather than thinking about growing awareness of the brand and building an audience.

Rather than worrying about niching, just try to attract as many people to your brand and content as possible. You will naturally niche your audience by attracting people with whom your content resonates.

One of the most important things I missed for several years was to build an email list. A well-managed, quality email list gives you an audience of people who are interested in, and have given you their permission, to talk about your business, products, and services.

In the early days, I started with an email newsletter using Mailchimp. But, after sending a handful of newsletters, the “news” dried up!

Nowadays, I build knowledge-based email courses using ConvertKit. Building courses and sequences in this way provides genuine value to new and existing members of the list.

The primary course I offer is the free Keep Your Rails Apps Healthy Course, which gives practical guidance on how to secure, monitor, maintain and optimise Ruby on Rails applications (You can join for free at https://healthyrailsapps.com).

Whichever tool you use (ConvertKit, Drip, Mailchimp, etc.), the important thing is to build in valuable content and set up automated sequences to help automate a big section of your marketing.

Understand Value

I talk a lot about value pricing, but pricing is only a small part of the story. The bigger goal is to shift your mindset away from providing a service towards providing a solution.

Yes, providing the highest quality of service is critical - but only as a means of delivering solutions to your clients' business problems.

I spent years talking about the tools and technologies, processes and methodologies I'd be using to work with clients. It's only natural, as developers, that we focus on the code (and editors, frameworks, servers, and so on…)

But that is generally not relevant to a client. There are exceptions, but a client who comes to you with a business problem (as opposed to a client who is simply looking to outsource an employee's role) is interested in solving that problem using the most effective, efficient methods possible.

It's taken me a long time to come to terms with this, but good clients shouldn't be as interested in how well-formed your code is (or worse - how many hours you'll be at your desk!) as much as they are by how effectively you can solve their business problems[^1].

Ultimately, the client is focussed on their problem, and how you can solve it.

Jonathan Stark talks about trying to talk new clients out of working with you. This is a great way to ensure your sales process is truly about the value you can deliver, and that the client has a real interest in engaging your services.

If your potential client is able to exhaust all your reasons for not working with them, you have a pretty strong case that you can deliver a solution, and that they will value your work.

Productise

Bringing together the previous two lessons - building an audience, focussing on business problems, and ultimately extracting common needs, leads to productisation.

Productising services is really the key to shifting away from a service provider model to a solution provider.

The goal is to identify common needs and requirements amongst your audience and customer base, and begin to extract common activities that deliver results.

These activities can be documented; in fact, the more you can put into step-by-step instructions, the better. Whilst there are apps out there that will help you build these standard operating procedures (SOPs), I find that Google Docs or Evernote works just fine.

As you build more and more systems and processes in your business, you can document, repeat, delegate and ultimately scale.

For in-depth resources on this topic, and great information and courses on how to productise your services, check out Brian Casel's work, and specifically the Productize course.

Business Mindset

This was the big lesson, that feels like it sums up my current thinking.

It's the inevitable result of the other three, especially productisation. The mindset shift from employee/freelancer/contractor/consultant to business owner is huge, and probably one of the most difficult things to do when you're running your own business.

As a solo consultant, you wear all the hats in your business, from big-strategy direction through sales, project management, and operations to payroll, bookkeeping and ordering the coffee (one of the most critical roles).

The biggest lesson, then, is perhaps to let go of certain tasks in your business. Accept that you cannot do everything yourself, and look for ways to delegate the repeatable parts of your business.

As more of your business becomes repeatable - more systematised - you should be able to start focussing more on the more valuable parts of your business. Delivering on the stuff that gives the greatest, most beneficial return to your clients, and thinking more about the bigger strategy rather than worrying about the minutiae.

Today

That pretty much sums up where I feel I am now. It's taken me a lot longer than it should to realise, but I feel the time is right to grow the business, by starting to build a team, focussing on our core services, constantly improving what we offer to clients, and growing the overall number of clients we serve.

Running your own business gives you incredible freedom and flexibility, but also brings with it huge challenges, worries, highs and lows.

It seems strange to say after so many years, but it really does feel like I'm just getting started. Whilst I've learned so much, I know that there is so much more and there are new ideas, opportunities and lessons in the years ahead.

I can't wait to get started.

[^1]: Caveat: Obviously, your code should be well-structured, architected, fully tested and compliant with best practices, but that should be implicit as part of you providing a professional, high-quality service to clients.

The past 12 months have been quite an experience in my business journey. I was lucky enough to work on some great new projects, and continued my drive towards a more focussed, product-orientated business.

But I also found myself having moments of uncertainty with value pricing.

This post is a summary of what I feel went right and wrong, and how I plan to update my value pricing strategy in 2019.

Productising Services

This year I made a big effort to standardise my services, replacing where I could bespoke offers with products and packages. The goal of productising a service is to make the work - or at least the process - as repeatable as possible.

I think productising can work hand-in-hand with value pricing as transition your offer to be more about that value of the service and outcomes you deliver.

A productised service should be more easy to scale and increase its value to both parties. As time goes on, prices can be adjusted to better reflect value, and optimise the service to increase efficiency (and the value to you)

That being said, one conflict between Productised services and Value Pricing is that by publishing a price upfront, you are not charging for the true value. This, then, may not be truly value pricing (or perhaps it is not true productising), but it has certainly felt like a good stepping stone on the journey.

As Jonathan mentions in his article, I've ended up using productised services as a part of a product ladder, with fixed-price smaller engagements, and bespoke-priced larger engagements such as service plans and Rails application upgrades (see below).

Code Reviews

Code Reviews were perhaps my most frequent example of a productised service, as I found they became a pre-requisite to any higher-value service that I offered.

Code Reviews were easy to define and limit in their scope. They had clear outcomes, and an easy to convey value for customers. The report customers received as a result of the review gave them a snapshot of their current situation, and recommendations for the next best steps. Code Reviews acted as a great first engagement, with low risk for all parties.

For me, the Code Reviews were a great way to gain a deeper understanding of an application's code and infrastructure, and better appreciate what might be required to achieve the customer's goals.

I'll continue to offer stand-alone Code Reviews and incorporate them into other services.

Consultations

I recognised that sometimes a full-blown code review (which can take a few days to produce, depending on the complexity of the underlying code) may be too much at the very early stages of an enquiry.

For smaller engagements, such as those initial conversations that establish the value of a project, I offered a consultation service.

I've found the consultations haven't had much uptake though. Most incoming projects have progressed from a quick 15-minute phone call to the Code Review itself.

The Consultation offer is a little too generic at the moment, with little definition about who they are aimed at. I plan to change this in the coming weeks.

CodeShot: Ruby on Rails

I offered CodeShots as a "one-hit" piece of development work. Ideally a small, discrete piece of work that can be turned around quickly.

Whilst I have carried out a handful of these with some success, I've found it to be very difficult to isolate development projects to such a degree.

Based on what I've appreciated about value pricing this year, and recognising that such small jobs have little value beyond "needing to be done", I plan to retire (or at least redesign) the CodeShot offer.

Rails Upgrades

Rails Upgrades have been a staple product this year, and seem popular as Rails and applications built on it mature. That being said, some of those upgrades I've carried out have presented huge challenges.

This has been difficult to recognise (and admit to myself!), but where I thought I was value pricing upgrades as a service, looking back I can see that I was really pricing them as fixed-priced projects. I really had little understanding of the value that such upgrades deliver.

As Rails Upgrades can hide so many unknowns (although I've learned a few new red flags to look out for this year!), I plan to change how they are offered in the new year.

Sidenote: GitHub published a great article on how they upgraded from Rails 3.2 to 5, with some important lessons they learned during the process.

Service Plans

One of my earliest attempts at productisation was to introduce service plans. These are essentially retainer agreements that are limited by scope rather than time.

The scope of each service agreement is determined in conversation with the customer, based on their longer-term goals.

However, the service plans all feature some common benefits, such as ongoing upgrades and bug-fixes, monitoring, etc.

In the early days I made some major pricing mistakes with service plans. I've also learned it's fairly difficult to scope retainer agreements where there is a lot of ambiguity about the end project.

In line with the rest of the business, I've also spent the past year or so constraining service plans to Rails applications (rather than including mobile apps as was the case).

In 2018, a number of service plans came to their natural end. Whilst this may be beneficial from a long term business strategy perspective, there's no denying that losing that regular monthly income was difficult.

I still plan to offer service plans as a core part of the business, although will continue to revise what is offered, and spend more time assessing the value for each plan.

To that end, it may be that Service Plans have an annual review, or are offered as a foundation for other services. I also feel that more vertical specialisation may be required, knowing that this would affect the business as a whole.

Moments of Uncertainty

The previous 12 months have forced me to think about my approach to value pricing, as doubt has started to creep into my mind due to a number of value pricing mistakes I've made.

Rediscovering Value Pricing Benefits

Whenever I've had concerns or doubts about using value pricing, I usually read great articles for inspiration and listen to podcasts and find myself re-discovering its benefits.

I understand that the mistakes are mine and due to my misinterpreting or incorrectly applying value pricing principles. Whilst when I started this journey, I was fairly insistent on applying value pricing to everything.

I've had to accept there are situations where it is not always best-suited.

When In Doubt…

Agency-style work is one such example that I now avoid value pricing. This year I've worked on a couple of agency-style projects.

These projects have been genuinely interesting, and rewarding project. I get to work with great teams on much larger projects than I would normally.

However, it's clear that there is nothing like going back to filling in time sheets and concerning yourself with inputs over outcomes to remind you of the benefits of value pricing!

Onwards

I always intended this experiment to be a journey. Admittedly at the start, I hadn't wanted to accept that there might be areas where value pricing wasn't appropriate - especially as I felt so strongly that time-based billing is bad for everyone.

Three years on, though, I've come to accept that such circumstances can and do occur.

Rather than stubbornly trying to fit everything into a value pricing, I think it may be best to accept these, and instead concentrate on adjusting the business itself such that value pricing is the default.

In 2019, I plan to further tighten up the services that are offered, specialise more, concentrate on products (and productisation).

How was 2018 for your business? Did you try value pricing? Please comment below or get in touch .

expect(find('img.my_asset_image')[:src]).to end_with('/my-image.png')

This expectation would pass given the following ERB output:

# <%= image_url('my-image.png') %>
# => <img src="/assets/my-image.png" />

In Rails 5, though, assets are compiled and fingerprinted during the test runs. This resulted in the generated URL containing a random fingerprint that could not be determined until runtime:

# <%= image_url('my-image.png') %>
# => <img src="/assets/my-image-aa00bb11cc22dd33ee44ff55.png" />

Writing a Simple Custom Matcher

To resolve this issue, I ended up writing a small custom matcher for RSpec that matched the URL against a simple regular expression.

# spec/support/custom_matchers.rb
RSpec::Matchers.define :eql_asset_url do |asset_name|
  match do |actual|
    pattern = asset_matcher_pattern(asset_name)
    expect(actual).to match(pattern)
  end

  failure_message do |actual|
    pattern = asset_matcher_pattern(asset_name)
    "expected #{actual} to contain an image asset url matching #{pattern}"
  end

  def asset_matcher_pattern(asset_name)
    # Split the asset's path, filename and extension for matching
    pathname = Pathname.new(asset_name)
    dirname = pathname.dirname.to_s
    dirname = '' if dirname == '.'
    extname = pathname.extname.to_s
    basename = pathname.basename.to_s.gsub(extname, '')

    %r{/assets#{dirname}/#{basename}-[a-f0-9]*#{extname}}
  end
end

Now in my feature specs, I can use the eql_asset_url matcher to test generated asset URLs:

expect(find('img.my_asset_image')[:src]).to eql_asset_url('my-image.png')

For example, when simulating a micro-services architecture in development, you might want to share a single database container across two or more applications, so that they can both access the same data.

Docker and Docker Compose make this possible through the use of Docker networks, allowing containers from different compose projects to be attached to the same network.

Sharing Containers in Compose Files

Suppose we have two entirely separate Rails applications that need to connect to a single database container. We can achieve this by using two docker-compose files, and specifying a single network to which our containers will be connected.

→ Create a configuration file, docker-compose.1.yml:

# docker-compose.1.yml
version: '3.2'

services:
  db:
    image: postgres
    environment:
      - POSTGRES_DB=my_app_development
      - POSTGRES_PASSWORD=password

  web_one:
    image: cblunt/rails-basic-app
    depends_on:
      - db
    ports:
      - '3000:3000'
    environment:
      - RAILS_ENV=development
      - DB_HOST=db
      - DB_USER=postgres
      - DB_PASSWORD=password
      - DB_NAME=my_app_development
    command: /bin/sh -c 'bin/rails db:migrate && bin/rails s -b 0.0.0.0 -P 3000'

→ Next, create a second configuration file, docker-compose.2.yml, in the same folder.

This configuration file will declare our 'second' web application (in this case, just a second instance of the same application).

Note that we only need to declare the web application itself in this configuration. We'll also expose web_two on port 3001, to prevent port clashes on our network:

# docker-compose.2.yml
version: '3.2'

services:
  web_two:
    image: cblunt/rails-basic-app
    ports:
      - '3001:3001'
    environment:
      - RAILS_ENV=development
      - DB_HOST=db
      - DB_USER=postgres
      - DB_PASSWORD=password
      - DB_NAME=my_app_development
    command: /bin/sh -c 'bin/rails s -b 0.0.0.0 -p 3001'

Now run docker-compose up on the two files. As they are in the same directory, Docker Compose will create a single default network, and attach all the containers to that network:

$ docker-compose -f docker-compose.1.yml -f docker-compose.2.yml up -d

Creating network "project_default" with the default driver
Creating project_db_1 ... done
Creating project_web_two_1 ... done
Creating project_web_one_1 ... done

In your browser, open the first application (running on port 3000). If you've used my example image above, open http://localhost:3000/posts. Use the simple scaffold to create a new blog post entry:

Once you have added a post, visit the second instance of the application (running on port 3001). You will see that the post you added shows up here as well, so both our applications are talking to the same database:

Sharing Containers Across Different Projects

So far, everything has worked pretty much as expected when your compose files are in the same project folder, but what if you have two compose files in entirely separate projects that need to share a common container?

For example, lets assume you have a project with the following folder structure:

+ project_one/
|  - docker-compose.yml
|
+ project_two/
|  - docker-compose.yml

As before, we can use Compose's ability to load multiple configuration files to bundle everything into one network:

$ docker-compose -f project_one/docker-compose.yml -f project_two/docker-compose.yml up -d

Again, Compose will create a single default network, and attach all the containers to that network.

Although this works as before, if dealing with multiple projects, I'd prefer to explicitly declare to which network the containers are attached. This helps to prevent any confusion when debugging our configuration.

You can declare a default network using the networks entry in your docker-compose.yml files:

# project_one/docker-compose.yml
version: '3.2'

networks:
  default:
    external:
      name: my_net

services:
  # ...

# project_two/docker-compose.yml
version: '3.2'

networks:
  default:
    external:
      name: my_net

services:
  # ...

Note that when declaring an external network manually, Compose will not automatically create it. Therefore, we need to create the network manually using docker network first:

$ docker network create my_net

Now we can use the compose configurations independently, knowing that any containers declared within them will be attached to the same network:

$ docker-compose -f project_one/docker-compose.yml up -d
$ docker-compose -f project_two/docker-compose.yml up -d

As before, both web containers will have access the same database.

Next Steps

Docker's networking allows you to build complex micro-service architectures, and share container resources with multiple applications. Docker Compose is great for simulating this type of set up in development.

Once you start running your apps in production, you can take advantage of more powerful orchestration tools, such as Docker Swarm, to configure and manage your applications and their environment.

This file is created when you run rails server to start your app. If your container is not shut down properly, this server.pid file is not removed. This prevents you from starting a new container; as the file is copied into the new container, you will be confronted with the following message when you try to run your app:

web_1    | A server is already running. Check /usr/src/app/tmp/pids/server.pid.
...
web_1    | Exiting

Of course, the simple way to resolve this is to remove the offending file ($ rm tmp/pids/server.pid) and restart your container.

This can quickly become annoying, though, and won't help if (for example) you are trying to scale the number of containers in your development environment. Every new container will inherit the tmp/pids/server.pid file, and so exit immediately.

Using an Entrypoint File

To properly handle this situation is to use a docker entrypoint file. This is a simple shell script that becomes the command that is called when your container starts.

Entrypoint files are commonly used to set up or configure a container at runtime. For example, when you run the postgres or mariadb containers, their entrypoint files create and configure a default database.

An example entrypoint file for our Rails app to resolve the problem above might be:

#!/bin/sh

rm -f tmp/pids/server.pid
bin/rails server -b 0.0.0.0 -p $PORT

Save this file as docker-entrypoint.sh in your app's folder (in the same location as Dockerfile). You'll also need to make sure it is executable by modifying its permissions:

$ chmod u+x ./docker-entrypoint.sh

Next, in your app's Dockerfile, set the CMD line to run your new entrypoint file:

# Dockerfile
FROM ruby:2.5
# ...
EXPOSE $PORT
CMD ./docker-entrypoint.sh

Note that you could use the _ENTRYPOINT_ command in this instance, but as we're supplying default arguments (_server_, _-b_ and _-p_) to the _bin/rails_ command, I prefer to use _CMD_ in line with the Docker documentation.

Alternatively, if you only want to use your entrypoint file for local development, you could simply override the container's default command in your docker-compose.yml file:

# docker-compose.yml
# ...
web:
  build: .
  command: ./docker-entrypoint.sh
  # ...

Now you can spin up your container(s) in the normal way using Docker compose:

$ docker-compose up -d --scale web=2

Next Steps

In this simple example, you've seen how to use an entrypoint file to initialise the state of a container at runtime. You can also use entrypoint files to perform much more complicated setup, such as creating runtime files and databases, configuring local services, and more.

In this first post, I'll be looking at ConvertKit, which I use for email marketing automation.

When I started my business (which is a worrying 8 years ago now!), my knowledge of email marketing went no further than publishing a newsletter.

At the time, I set up a Mailchimp account, and (very) rarely published a newsletter with what I thought would be interesting information for the slowly growing number of subscribers.

As things got busier, though, the newsletter became less and less frequent, and when it was published, the open/read rates were low.

Discovering Email Marketing

Somewhere along the line, I'd heard about email marketing tools, but paid them little attention. I was already in Nathan Barry's audience as a customer of the Designing Web Applications book, and was loosely following his journey building ConvertKit.

About 2 years ago, tired of managing Mailchimp lists and emails, I decided to give other platforms a go. I quickly settled on ConvertKit. The simple web-based UI made building email sequences and forms really easy, and it integrated well with Gumroad, the e-commerce site I use to sell my digital books and courses.

If you're looking to start building an audience for your content, ConvertKit is an excellent choice. It provides enough power "out of the box" to build landing pages, forms, email courses and more.

The most difficult part of using ConvertKit (for me, at least) is knowing what content to write! Let's look at how you can get started building your audience and sharing your content.

Building a Simple Newsletter with ConvertKit

After signing up to ConvertKit (they offer a 14-day free trial), you'll be presented with the option to build Forms, Landing Pages and Sequences.

Forms are used to capture leads and emails on your website (e.g. signing up for a newsletter or special offer), whilst landing pages provide a fully-hosted web page on which you can add text and images to describe your service or offer.

The landing page includes an integrated form to capture leads, and ConvertKit have recently launched a new landing page editor and some great default theme templates.

Building a Newsletter Sign-up Page

Let's start by building a newsletter sign-up page. The latest release of ConvertKit provides a great template for us to do this.

Here's an example of the landing page you'll build (see the original):

→ Click on Forms along the top of the screen, then Create Form

You'll be asked if you want to create a form or landing page. Let's build a complete landing page, which you can share to readers.

→ Click Landing Page, then choose the Profile Template

The Profile template is designed for a personal newsletter (although you tweak the content for anything, really).

The next screen you'll see is the page editor. You can click on the different elements to edit their content and colours. For example, click on Your name and change it to your own name.

→ Click on "Your Name" and replace the text with your own name.

You'll also see a sidebar open up, where you can edit the font style (colour, size, etc.) of the text.

Edit the other elements on the page until you're happy with the design. You can also click on the image selector to choose a profile image for your page.

→ Click on the "Magic Wand" icon to choose your own colour scheme

The styles panel allows you change various colours on your page. You can also change the colour of the Subscribe button by clicking on it.

→ Click the envelope icon to configure your double opt-in / incentive email.

The incentive email panel allows you to customise (and enable/disable) the double opt-in verification email that your subscribers receive when signing up to your newsletter.

For compliance with GDPR for EU citizens, I recommend you always enable a double opt-in email. ConvertKit also has GDPR options which you can manage in your account settings

In this screen, ConvertKit also allows you to upload an incentive attachment that will be downloaded automatically when the subscriber verifies their email address.

Connecting Your Form to a Sequence

Once you've built your landing page, the next step is to connect it to an email sequence. Sequences are chains of emails that are delivered to your subscribers on a timed basis. They are great for building evergreen newsletters, and also simple email courses.

Let's start by building a welcome email for our sequence.

→ Click Sequences, then Create Sequence

ConvertKit will ask you to name your sequence. We'll call it Newsletter for now (you can always change this later).

→ Write an introductory email

The ConvertKit email editor is simple yet powerful. It let's you create great looking emails without having to worry about coding HTML or advanced templates.

ConvertKit's emails are designed to be read, so they look more like personal written emails than traditional, graphics-heavy newsletters you might be familiar with.

→ Set sending options

Finally, we need to mark our email as published by changing it's status in the drop-down menu. As this is a welcome email, we should also make sure it delivers immediately to subscribers once they have confirmed their email address.

To do this, click the after 1 day setting under When to Send, and enter 0 into the field. This will remove any delay before the email is sent.

Putting Everything Together

ConvertKit offers a couple of ways to connect forms and sequences: Rules and Automations. Automations allow us to design an advanced flow-chart style set of rules for our subscribers to follow.

I'll cover automations in a future post. For now, though, we'll use a simple rule to connect our form to the sequence, so that when somebody subscribes to our form, the rule automatically subscribes them to our sequence.

Rules are a great way to build If this happens, then do that conditions.

→ Click Automations, Rules, then Add Rule

ConvertKit presents "Triggers" on the left and "Actions" on the right. When a trigger occurs, the corresponding Action(s) will run.

→ Click the Subscribes to a Form trigger

In the drop-down list that appears, choose the Profile landing page you created before.

→ Click the Subscribe to a Sequence action

In the drop-down list, choose the Newsletter sequence.

→ Click Save Rule to create the rule.

ConvertKit will now show your rule, along with a green Enabled light to show that rule will process when the trigger occurs.

Your Newsletter is Ready!

That's it! Your newsletter is now ready to go! You can try out your new rule by signing up to your landing page - just open the link that ConvertKit generates for your landing page, and enter your email address.

After confirming your subscription, you'll receive your new Welcome email!

Next up...

I hope you've found this guide helpful. To get started with ConvertKit, click here to sign up and start your 14-day free trial.

In upcoming posts, I'll look at some of the more advanced techniques I've learned when using ConvertKit, including how to build more complex automation flows.

As part of this Business Tools series, I'll explore some of the other systems I use in the day-to-day running of my business.

Let me know your thoughts and comments by tweeting me or getting in touch via email.

Along with the changes came the secrets:edit task, allowing you to edit the credentials YAML file whilst automatically decrypting and encrypting the file using a master key.

The key is provided either through config/master.key or supplying an RAILS_MASTER_KEY environment variable. (More on this below…)

Editing Credentials in your Docker Container

When using Docker to develop and run your apps, though, getting secrets:edit to work may requires a few tweaks to your Docker image.

Depending on how the image is configured, you'll need to ensure that an editor is installed in your container.

For one-off editing, we can do this using a simple command line:

$ cd /path/to/your/app
$ docker run --rm -it --mount type=bind,src=${PWD},target=/app my_app /bin/sh -c 'apt update && apt install -y vim && EDITOR=vim bin/rails credentials:edit'

Alternatively, you can ensure that vim (or your editor of choice) is installed into the Docker image:

FROM ruby:2.5

RUN apt update -qq && apt install -y vim # nano ...< your choice of editor

# ... continue your Dockerfile

$ docker build -t my_app .
$ docker run --rm -it --mount type=bind,src=${PWD},target=/app -e EDITOR=vim my_app bin/rails credentials:edit # Assume your app's code is ADDed to /app in the Dockerfile.

Using this technique, you can use Rails' built-in credentials editor without the need to install rails itself (and other dependencies) on your workstation machine.

Using Docker Compose

If you're using Docker Compose to manage your containers, credentials can be edited in the same way using the docker-compose run command.

Assuming a declaration similar to the following in your docker-compose.yml:

# ...
services:
  web:
    build: .
    volumes:
      - .:/app
    # ...

$ docker-compose run --rm -e EDITOR=vim web bin/rails credentials:edit

Supplying Your Master Key as an Environment Variable

By default, Rails creates a config/master.key file which is used to decrypt/encrypt your credentials file. As this file is by default excluded from your code repository, you may need to supply the key as an environment variable to your containers:

$ docker run --rm -it --mount type=bind,src=${PWD},target=/app -e EDITOR=vi -e RAILS_MASTER_KEY=your-master-key my_app bin/rails credentials:edit

Note that this is only necessary if you do not have the config/master.key file in your workspace

I hope that by documenting them here, you’ll be able to learn from my mistakes to make your journey to value pricing that little bit easier.

Above all, though, remember that it’s inevitable that moving entirely away from time billing is challenging — you are going against very established norms.

1. Not Recognising Value

This is perhaps the single mistake I’ve made in value pricing that underpins all others, and (anecdotally, at least) seems to be very common.

Recognising what your customer values, and — just as importantly — what they do not value, is absolutely critical to successful value pricing.

That does feel like stating the obvious, and on the surface, value pricing does sound pretty obvious. But as I’ve mentioned before, value is often very subjective, and based on situation. It is often difficult to put into words, let alone numbers, what the value of something may be.

There have been many circumstances where I’ve struggled to identify value, and I know this is because I’ve failed to fully carry out the value conversation. This has led to lazy fixed-price billing, which is far from ideal.

I’ve always packaged the outcomes into a fixed price, though, and have managed to avoid time-based billing. In those situations where I’ve been tempted to apply a time-based rate as a “quick fix”, I’ve avoided it by reminding myself how time-billing is painful for everyone involved.

To help avoid this lazy pricing, I’ll continue to productise all of my service offerings, and gradually establish systems that encourage a lot more focus on that in-depth value conversation.

However — and as difficult as it has been for me to accept— there are a couple of situations I’ve come across where this more approach to projects isn’t the right fit, and where value pricing may not be the best approach…

2. Value Pricing Everything

My goal was to become 100% value pricing, but there have been a couple of situations where I’ve really struggled to do much more than offer a fixed price:

Minor Maintenance Tasks

These are the small and often rush/panic jobs that come in such as fixing a minor bug or tweaking a layout.

The technical effort is usually lower and there’s no value in discussing any “bigger picture” subjects (at this stage).

It is sometimes difficult to remember when you spend most of your time in code, but as Jonathan Stark reminds us on this episode of his podcast, code has no value to your customer:

My approach to this type of quick-maintenance work has been to wrap it into a fixed-price, limited-scope product. There are some obvious risks in doing this, but that is reflected in the price/value to me.

Productising the process also greatly simplifies things for the customer, and allows them to quickly weigh up their options.

The CodeShot productised service has been particularly well received, and has led on to further projects. In this way, it acts as a great introductory service for the customer, and is now one of the first things I’ll recommend to new customers who are looking at larger or longer-term projects.

Agency-style work

In this style of work, you are being taken on as simply a “pair of hands” to top up an existing workforce. Although I’ve said agency-style work, your client may not be an actual agency.

For example, you might be contracted in to a larger organisation’s team, but the model is similar. Generally, I’m considering work as agency-style if you could easily be mistaken for, or replaced by an employee, and have little or no interaction with the end-customer.

In this situation, your customer is the agency. The value to them is in maximising their profit on your effort, and so value conversations are difficult (simply put, the maximum value to them is minimising your price). This leads to normal market pressures and a focus on input effort rather than outcomes.

I’ve previously tried to apply value pricing to these types of engagement, but it has never been particularly successful. I’ve also used a more productised approach (such as CodeShots) to handle smaller jobs. But as much as it is difficult to admit, I think there is little point in trying to apply value pricing to agency-style work.

As I don’t tend to do a lot of this type of work, my approach moving forward is simply to apply a minimum daily or weekly rate based on my perceived value, and accept the issues that come with that.

Of course, the priority is to continue finding and building projects where I am working directly with the end-customer, as this allows me to deliver far more value, and work on a partnership level with them and their business.

I’d love to hear from you if you have successfully value-priced agency style work.

3. Underestimating Valuable Outcomes

Closely linked with not recognising value above, I still find myself underestimating the true value of outcomes to the customer, even when I know what those outcomes are (i.e. I’ve recognised them, but underestimated their impact).

I’m getting better with this as time goes on — it helps to remember value pricing is a journey, so adjustments can be made with time. Productising my services has certainly been one of the best tools to help do this.

Sometimes, though, it can be very much a case of not seeing the wood for the trees.

Ironically, this is not the case when looking at other businesses from the outside in . Often when working with customers, I’ve found it easy to spot value leaks in their businesses, and can quantify that value without question. Yet doing the same in my own business seems almost impossible!

Perhaps it helps being an outsider to the business, and also being able to appreciate what their valuable outcomes might be worth to me as a customer.

I’m fairly sure I’m not alone in this, and am considering business coaching or mentorship to help put a second pair of eyes on my business and help spot any value leaks.

Having a mastermind group, such as Kirk Bowman’s Art of Value Society is a great way to bounce value pricing ideas and challenges.

Would you like to get another perspective on your software business? As I’m considering doing exactly this for my own business, I’d love to help others who are on the journey to value pricing.

Start a discussion by getting in touch.

A value-based price is one that reflects the outcomes that your customer will receive as a result of your inputs. The effort, time and energy required for those outcomes isn’t relevant to the price.

Your customer places little, if any, value on your time.

As a knowledge-based professional, this can be a tough pill to swallow. After all, you’ve spent years learning your craft; and the amount of time you put into a project is likely to be very valuable to you.

So should you be tracking your time, even if you’re pricing based on value?

Time is an Operational Cost of Business

Regardless of how you price your services, you need to see yourself as an operational cost of your business.

And just like any cost, you need to make sure that you are providing a return for the business; ultimately, you need to be profitable for your business.

Value-pricing purists might say time tracking can be abandoned, but despite every effort, I’ve really struggled with this. I’m still not quite there yet, and throughout my transition to value pricing, I’ve continued to track my time (using Toggl) for pretty much everything I do in my business.

This includes not only “billable” work, but also internal tasks, such as updating accounts, filling in tax returns, writing books and courses, marketing, and so on.

I’m even tracking the time I spend writing and editing this article (1h 25m).

Why would I do this? After all, keeping track of time is an inconvenience at best.

But keeping an track of how I spend my time allows me to know my cost to the business, as well as giving me the opportunity to identify inefficiencies and delegate tasks that get in the way of more valuable work.

Whilst it may be irrelevant to your customer, I think knowing how you spend your time is still important for you and your business.

Time is a floor to (but does not determine) price

Despite tracking my time, it’s crucial to remember that it is not used to determine the value-based price of a service, except perhaps as a baseline, i.e. a break-even point.

After all, you want to ensure that your business creates a profit on your input.

If you work solo, it can be easy to forget this — you almost have to see yourself as two separate entities: the business owner and a separate, salaried team member.

When determining a value-based price, regardless of the value to the customer, you need to ensure you are covering your operating costs. Tracking your time allows you to benchmark different jobs and help provide a guesstimate of how long a piece of work might take.

As you begin to productise your services, tracking your time can also help you monitor efficiency.

Remember, though, that the value of the outcomes you deliver should be several orders of magnitude above your operating costs, and at least several times the price your charge.

This can often be difficult to determine; if you’re struggling to justify (to yourself) a value based fee that meets this criteria, then it’s likely you’ve mis-identified where the value lies for the customer; or — and it has taken me a long while to accept this — it may be that the work is simply too small or not suitable for a value based price.

This is a subject I’ll cover in an upcoming post, but generally, the former can be resolved by continuing the value conversation with your customer. The latter might actually be better resolved with a different pricing model or, preferably, offering a value-priced, productised service that fulfils the needs of the client whilst maintaining a profit margin for your business.

Conclusion

As a business owner, it’s essential to understand your operating costs and where (in)efficiencies lie in your business.

With this in mind, should you track your time when adopting value-based pricing in your business?

My thinking on this may change in the future, but at the moment I would say yes. With the strict caveat that time reports are “for your eyes only”. Your timesheet should never be a concern for your customers, nor used as the basis for pricing your services.

Would you like to get another perspective on your software business? As I’m considering doing exactly this for my own business, I’d love to help others who are on the journey to value pricing.

Start a discussion by getting in touch.

Here are five things I like to check prior to deploying a Ruby on Rails app:

1. Customise the pages your users shouldn't really see

It's inevitable that at some point, your app will throw an error page - be it a server issue (500) or even a missing page. Rails provides some basic templates for these pages, but they aren't very useful to users.

Instead, you should customise the public/404.html and public/500.html pages to provide some useful feedback to site users, and even an escape route (such as a link to your homepage or site search) to help them resolve their issue.

2. Ensure the app enforces an encrypted connection

With the arrival of LetsEncrypt, there really is no excuse not to use an SSL certificate for your apps now. Even the most basic sites benefit from the security that an SSL certificate provides to its users.

You can enforce SSL easily across your app by setting in your environment configuration file:

# config/environments/production.rb
# ...
  config.force_ssl = true

3. Don't leak database details through your URLs

The default behaviour of Rails routing is to use database ID fields for URLs. For example, to edit a user's account, the URL might be yourapp.com/users/5/edit. Using the database fields in this way could potentially leak data from your application, and offer a potential security vulnerability in your application if (for example) an authorization check is not correctly carried out on the action.

An easy workaround is to use a separate, auto-generated UUID field on your model, and override the model's to_param method:

class User < ApplicationRecord
  has_secure_token :uuid # Ideally, add a UNIQUE index to this field on your database

  def to_param
    self.uuid
  end
end

Alternatively, a more complete approach is to use the excellent FriendlyId gem to take care of this easily. FriendlyID also gives you the benefit of being able to generate URL-friendly slugs for your model, and redirections should they change.

4. Check for Availability

Once your app is up and running, you want to make sure it stays that way! This means monitoring the various parts that make up your app. Thankfully, there are a number of services to make this task easy.

For example, StatusCake and Pingdom perfor regular checks on your website, and can even be configured to ensure that the response includes a particular string of text (e.g. "Welcome to my App"). This helps to ensure not only that your app is responding correctly, but that (for example) users are not seeing unexpected content (such as a directory listing!)

5. Automate your Deployment

Automating your configuration and deployment is one of the best steps you can take to keeping your app stable in production. CI/CD automation tools can be configured to perform a number of checks on your code both during and after the deployment to ensure that everything has gone smoothly.

By setting this up once, you can be far more confident about the state of your application and its services every time you make a change. There are a number of CI / CD tools available, and the best one for you will depend on your particular setup.

However, to get started, I highly recommend checking out either Drone, or Gitlab CI (especially if you are already using a Gitlab instance to host your code repository).

Get the 30-Step Pre-Launch Checklist for Healthy Rails Apps

These are just 5 things you should check as part of your app's launch into production, but there are plenty more!

To help you out, I've put together a handy reference worksheet that includes 30-step checklist to step through when preparing your app. You can download the checklist using the form below.

If you're using Docker to develop your app, however, running those tests/specs can be time consuming. Spinning up a container for every run soon becomes time consuming and wastes resources.

A better approach during development is to use a container for your app that is running Guard. This container will monitor changes to your files and run your tests within itself.

Whilst there have previously been issues with Guard picking up file changes across the Docker mount (due to the underlying use of VirtualBox, for example), recent releases of Docker (for Mac) seem to have resolved the issue.

In this tutorial, you'll see how to use a Docker container to run Guard and monitor your app's code for changes.

A Basic Dockerised Rails App

Starting with an app that has some tests (minitest) or specs (rspec) in place, add the guard gem and appropriate plugins for your test suite (e.g. guard-minitest and/or guard-rspec):

# Gemfile
group :development do
  gem 'guard'
  gem 'guard-minitest'
  gem 'guard-rspec' # If you use Rspec instead of Minitest
end

If your Rails app doesn't already have a Dockerfile, the following example is a good starting point. You might need to include more packages depending on your app's specific needs. See Getting Started with Docker and Ruby on Rails for more details about how to set this up.

# Dockerfile
# /path/to/your/app/Dockerfile
FROM ruby:2.4.3

RUN apt-get update -qq && apt-get install -y nodejs

# Cache Gems
WORKDIR /tmp
ADD Gemfile .
ADD Gemfile.lock .

RUN bundle install --jobs 4

# Copy your app's code into the image
WORKDIR /usr/src/app
ADD . /usr/src/app

# Precompile assets
RUN bundle exec rails assets:precompile

# RUN apt-get install -y libreadline6 libreadline6-dev

# Expose port 3000 to other containers
ENV PORT 3000
EXPOSE $PORT

# Run the rails server
CMD rails s -b 0.0.0.0 -p $PORT

When your Dockerfile is ready, you can build (and tag) an image from it. In this tutorial, the image tag will be myapp:latest:

$ docker build -t myapp:latest .

Next, check your suite runs by running your tests/specs inside a Docker container:

$ cd /path/to/your/app
$ docker run --rm -it -v ${PWD}:/usr/src/app myapp:latest bin/rails test # or bin/rails spec

Running via Spring preloader in process 19
Run options: --seed 61489
# Running:
...
Finished in 1.462146s, 2.0518 runs/s, 6.1553 assertions/s.
3 runs, 9 assertions, 0 failures, 0 errors, 0 skips

Installing Guard

With our tests running inside Docker, it's time to set up Guard. Again, this can be done using the existing Docker image for your app:

$ docker run -it -v ${PWD}:/usr/src/app --rm myapp:latest bundle exec guard init minitest rspec

Here, I've included both the minitest and rspec guard plugins, but you might only want to include the plugin that is relevant for your app.

Your app's folder will now include a Guardfile. Open it up, and edit according to your needs. For example, if you're using Rspec, the default configuration will look like:

# Guardfile
guard :rspec, cmd: "bundle exec rspec" do
  require "guard/rspec/dsl"
  dsl = Guard::RSpec::Dsl.new(self)

  # Feel free to open issues for suggestions and improvements
  ...
end

See the README file for your plugin(s) to learn about their various configuration options.

Run Guard

With Guard configured, we can now spin up a new container that is running Guard and monitoring the app's code. Be sure to mount your code folder into the container, so that changes are applied as you make them:

$ docker run -it -v ${PWD}:/usr/src/app --rm myapp:latest bundle exec guard

The container will attach to Guard as if it was running on your host machine. For example, you can press enter to run all your tests:

[1] guard(main)> # Enter
12:09:56 - INFO - Run all
12:09:56 - INFO - Running all specs
....

Finished in 0.39492 seconds (files took 4.05 seconds to load)
4 examples, 0 failures

Now if you edit one of your test/spec files, Guard should pick up the change and automatically run your test/spec, as if it was running locally!

To exit Guard (and remove the container), just type exit or hit Ctrl+D

Adding Guard to Your Compose File

Now that we know Guard is working and monitoring file changes inside our Docker container, you can add a declaration to your Docker Compose file. This will allow you to run Guard automatically as part of your development suite when you call docker-compose up.

One caveat, though, is that we can't use Guard's interactive console when calling docker-compose up. The process does not attach to the console, and so the container immediately fails. A workaround for this is to add the --no-interactions argument to the container's command:

# docker-compose.yml
version: '3.2'

services:
  web:
    build: .
    ports:
      - '3000:3000'
    volumes:
      - .:/usr/src/app
    environment:
      - RAILS_ENV=development
    # ...

  guard:
    build: .
    volumes:
      - .:/usr/src/app
    environment:
      - RAILS_ENV=development
    command: bundle exec guard --no-bundler-warning --no-interactions

Now when you call docker-compose up, you will see the Guard container start up and waiting for notifications from the filesystem:

$ docker-compose up
guard_1  | 12:23:39 - INFO - Guard::RSpec is running
guard_1  | 12:23:39 - INFO - Guard is now watching at '/usr/src/app'
web_1    | => Booting Puma
web_1    | => Rails 5.1.4 application starting in development
web_1    | => Run `rails server -h` for more startup options
web_1    | Puma starting in single mode...
web_1    | * Version 3.10.0 (ruby 2.4.3-p205), codename: Russell's Teapot
web_1    | * Min threads: 5, max threads: 5
web_1    | * Environment: development
web_1    | * Listening on tcp://0.0.0.0:3000
web_1    | Use Ctrl-C to stop

As before, if you now edit one of your test/spec files, you'll see Guard triggers the appropriate tests:

12:24:16 - INFO - Running: spec/mailers/admin_mailer_spec.rb
guard_1  | ....
guard_1  |
guard_1  | Finished in 0.6642 seconds (files took 3.75 seconds to load)
guard_1  | 4 examples, 0 failures

With everything working, you can now continue building your app as before, safe in the knowledge that whilst it is running, Guard is monitoring your tests, and will flag any breaking tests.

If you're using Yarn to manage your app's frontend libraries, you'll need to ensure that Yarn is available to use when adding or precompiling your assets in your Docker container. To do this, you'll need to add a few extra lines to your app's Dockerfile.

Let's start by creating a new Rails app, and creating a basic Dockerfile for our app's image:

$ rails -v # Make sure you are using >= Rails 5.1
$ rails new my-app -d postgresql -B
$ cd my-app

Create your Dockerfile and add the following:

# /path/to/app/Dockerfile
FROM ruby:2.3-alpine

# Set local timezone
RUN apk add --update tzdata && \
    cp /usr/share/zoneinfo/Europe/London /etc/localtime && \
    echo "Europe/London" > /etc/timezone

# Install your app's runtime dependencies in the container
RUN apk add --update --virtual runtime-deps postgresql-client nodejs libffi-dev readline sqlite

# Install Yarn
ENV PATH=/root/.yarn/bin:$PATH
RUN apk add --virtual build-yarn curl && \
    touch ~/.bashrc && \
    curl -o- -L https://yarnpkg.com/install.sh | sh && \
    apk del build-yarn

# Bundle into the temp directory
WORKDIR /tmp
ADD Gemfile* ./

RUN apk add --virtual build-deps build-base openssl-dev postgresql-dev libc-dev linux-headers libxml2-dev libxslt-dev readline-dev && \
    bundle install --jobs=2 && \
    apk del build-deps

# Copy the app's code into the container
ENV APP_HOME /app
COPY . $APP_HOME
WORKDIR $APP_HOME

# Configure production environment variables
ENV RAILS_ENV=production \
    RACK_ENV=production

# Expose port 3000 from the container
EXPOSE 3000

# Run puma server by default
CMD ["bundle", "exec", "puma", "-C", "config/puma.rb"]

Notice the lines to install Yarn. This will install Yarn into your Docker image, and update the $PATH variable so that it is available to use on the command line:

$ docker build . -t my-app

Add Frontend Depedencies

Now your app's Docker image is built with Yarn installed, you can use it to spin up a container and add a frontend dependencies to your app.

To demonstrate this, let's add the Tachyons CSS framework:

docker run --rm -it --env RAILS_ENV=development --volume ${PWD}:/app my-app bin/yarn add tachyons

yarn add v1.3.2
info No lockfile found.
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
success Saved lockfile.
success Saved 2 new dependencies.
├─ tachyons-sass@4.8.1
└─ tachyons@4.9.0
Done in 2.10s.

If you look inside your app's folder, you'll see that yarn has added a yarn.lock file and a node_modules folder.

The lockfile is used by Yarn to determine exactly which versions of dependencies to install (similar to your app's Gemfile.lock for gems), and node_modules is where those dependencies are saved.

Note that the node_modules folder should not be committed to your code repository (it is add to .gitignore by default).

You should also ensure it is not added to you Dockerfile when building by adding it to .dockerignore:

# .dockerignore
node_modules/

Using Frontend Dependencies

Rails 5.1 is configured to search the node_modules folder by default when precompiling assets. This means we can easily use the Tachyon's CSS framework by calling it in application.css:

/* app/assets/stylesheets/application.css */

/**
 * ...
 * = require tachyons/css/tachyons.min.css # <- Path relative to node_modules/
 * = require_tree .
 * = require_self
 */

(Note you can see the node_modules folder added to the asset pipeline in config/initializers/assets.rb)

Next, create a simple home controller that we can use the build a demo page:

$ docker run --rm -it --env RAILS_ENV=development --volume ${PWD}:/app my-app bin/rails g controller home index

And finally, add some Tachyons CSS classes to your app's layout. For now, we'll just create a simple red header across the top of the page:

    <!-- app/views/layouts/application.html.erb -->
    ...
    <body class="ma0">
      <header class="bg-dark-red ma0">
        <div class="mw8 center">
          <h1 class="f1 fw6 white ma0 pv3">Hello Rails</h1>
        </div>
      </header>

      <main class="mw8 center">
        <%= yield %>
      </main>
    </body>
    ...

$ docker run --rm -it --env RAILS_ENV=development --volume ${PWD}:/app --publish 3000:3000 my-app

With your app running, let's check that everything looks as expected. Open http://localhost:3000/home/index in your browser and you should be greeted with the (basic!) styled layout:

Finishing Up

That's all there is to using Yarn! By installing it inside your app's Docker container, you'll also be able to successfully precompile assets when deploying your app to CI or production servers.

Yarn is called as part of the assets:precompile task to ensure that frontend dependencies are available to the asset pipeline. You can see this in action by running the task through your container:

$ rm -rf node_modules # Remove the existing node_modules folder first

$ docker run --rm -it --volume ${PWD}:/app --publish 3000:3000 my-app bin/rails assets:precompile
yarn install v1.3.2
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
Done in 0.72s.
I, [2017-11-24T10:12:57.634438 #1]  INFO -- : Writing /app/public/assets/application-dcfb333542166c7e176ac13945f5c27350627b932d212a870a5a112daf6d4db9.js
I, [2017-11-24T10:12:57.636782 #1]  INFO -- : Writing /app/public/assets/application-dcfb333542166c7e176ac13945f5c27350627b932d212a870a5a112daf6d4db9.js.gz
...

In the previous tutorial, you learned how to get a simple Ruby on Rails app up and running on Docker containers.

As you work on your app, it often becomes difficult (and tiresome) to remember all the different configurations and arguments you need to create and link containers. The complexity increases as your app's dependencies grow (such as when you need to using Sidekiq and to handle background jobs).

Thankfully, Docker comes with a great solution to this problem in the form of Docker Compose.

Compose allows you to declare your app's requirements and configuration a simple yaml file. From this file, Docker will create, configure and start the containers. You can also use Compose to specify data volumes for your containers, and connect containers across an isolated virtual network.

In this tutorial, we'll take the simple Rails app from last time and use Compose to replicate the container setup. This will make creating an "instance" of our Rails app much faster, and allow configurationt to be easily shared between developers.

Need the code? If you don't have the Rails application code from last time, you can create it as new Rails app using the following template:

$ rails new --database=postgresql --skip-bundle --template=https://gist.githubusercontent.com/cblunt/1d3b0c1829875e3889d50c27eb233ebe/raw/8fce95533db3a3be19cb1aa31054589d219433c2/rails-docker-pg-template.rb my-app

Creating a Compose File

The simple rails app we built makes use of two containers: one for the app itself and one for the PostgreSQL database. To run them, we created and linked two containers using the following commands:

$ cd my-app
$ docker build . -t my-app
$ docker run -d -it --env POSTGRES_PASSWORD=superSecret123 --env DB_NAME=my-app_development --name mydbcontainer postgres:9.6
$ docker run --rm -it --env RAILS_ENV=development --env POSTGRES_USER=postgres --env POSTGRES_PASSWORD=superSecret123 --publish 3000:3000 --volume ${PWD}:/app --link mydbcontainer:db my-app

We'll recreate this setup in a new docker-compose.yml file:

# ./my-app/docker-compose.yml
version: '3'

services:
  db:
    image: postgres:9.6
    environment:
      - POSTGRES_PASSWORD=superSecret123
      - DB_NAME=my-app_development
  web:
    build: .
    environment:
      - RAILS_ENV=development
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=superSecret123
    ports:
      - '3000:3000'
    volumes:
      - .:/app
    depends_on:
      - db

Notice that while the db container uses the postgres:9.6 image, the web container (which is our app) will build a new docker image from the existing Dockerfile in your app's folder.

You can see that the YAML file reflects the command-line arguments we previously supplied to Docker. However, it is much easier to read and manage!

ProTip: What are Services?

Rather than specify each individual container, in Compose we specify services. Each service (db, web in the file above) can be thought of as representing one or more containers that running a particular Docker image.

Services allow us to create any number of identical containers running the same application. This is important when we come to scale services (i.e. run 2 or more copies of our Rails application and load-balance between them).

For this tutorial, we'll only be using one container per service so you can think of each service as a single container.

Starting Services

Next we'll use Docker Compose to create and start the containers (services) specified in the docker-compose.yml file. Compose will automatically download or build any images it needs to create each container:

$ docker-compose up -d
Creating network "myapp_default" with the default driver
Creating myapp_db_1 ... done
Creating myapp_web_1 ... done

Once your image is built, Compose will create and start the containers necessary for your app to run. The -d flag detaches your console allowing the containers to run in the background. You can tail the log messages from the containers just like you would with docker, but using the docker-compose command instead.

Compose will tail the logs for all of the containers specified in the docker-compose.yml file:

$ docker-compose logs -f
Attaching to myapp_web_1, myapp_db_1
web_1  | Puma starting in single mode...
...
web_1  | * Listening on tcp://0.0.0.0:3000
web_1  | Use Ctrl-C to stop
...
db_1   | LOG:  database system was shut down at 2017-09-09 11:20:25 UTC
db_1   | LOG:  MultiXact member wraparound protections are now enabled
db_1   | LOG:  database system is ready to accept connections
db_1   | LOG:  autovacuum launcher started

Hit [Ctrl-C] to stop tailing the logs. You can also show or tail logs from just one of the containers by using the name given to it in docker-compose.yml. For example, to tail only our app's logs:

$ docker-compose logs -f web
web_1  | Puma starting in single mode...
...
web_1  | * Listening on tcp://0.0.0.0:3000
web_1  | Use Ctrl-C to stop

Hit [Ctrl-C] to exit the logs (note if you omit the -f flag, the logs command will automatically exit)

You can also see the app is running by opening your browser and visiting localhost:3000.

Running Single-Task Containers with Compose

Just as before, your app will throw a missing database error. This is because the container that was created by Compose is a completely separate instance from the previous container (remember that containers are ephemeral. As soon as they are removed, any data within them is lost).

As in the previous tutorial, we'll need to run a database migration task to create our app's database inside the db container. Just like normal Docker, we can use Compose to run create, use and remove a single-task container. Compose will take care of creating, configuring and running any dependent containers (e.g. the database container) as specified in the docker-compose.yml file before running the task.

To create and migrate our database, use the docker-compose run command (again, this is nearly identical to the standard docker run command:

$ docker-compose run --rm web bin/rails db:create db:migrate
Starting myapp_db_1 ... done
Created database 'my-app_development'
Created database 'my-app_test'
== 20170909110604 CreatePosts: migrating ======================================
-- create_table(:posts)
   -> 0.0221s
== 20170909110604 CreatePosts: migrated (0.0223s) =============================

Specifying the --rm flag tells Compose to remove the container once the task has finished. Notice that although we used the web specification from our docker-compose.yml file, Compose created, used and removed a completely separate container to run this task. The previous instance of our web container is still running:

$ docker-compose ps
   Name                  Command               State           Ports
-----------------------------------------------------------------------------
myapp_db_1    docker-entrypoint.sh postgres    Up      5432/tcp
myapp_web_1   bundle exec puma -C config ...   Up      0.0.0.0:3000->3000/tcp

Now if you revisit localhost:3000, you'll see the familiar Posts screen.

Stopping and Tidying Up Containers

Compose takes care of tidying up containers for us just as easily as creating them. You can stop and start containers in the normal way. This will stop but not remove them, allowing them to be restarted with data intact:

$ docker-compose stop
Stopping myapp_web_1 ... done
Stopping myapp_db_1 ... done

# Visiting localhost:3000 will now show a connection refused error as the containers are not running.

$ docker-compose start
Starting db ... done
Starting web ... done

To stop and completely remove the containers (including all their data), use the docker-compose down command:

$ docker-compose down
Stopping myapp_web_1 ... done
Stopping myapp_db_1 ... done
Removing myapp_web_1 ... done
Removing myapp_db_1 ... done
Removing network myapp_default

(Note that you can also stop and remove individual service containers using, for example, docker-compose stop web and docker-compose rm web).

Using docker-compose down will also remove any other unnecessary resources that were created, such as the virtual network (in this case myapp_default). Data volumes (which we've not used here) are not removed by default, allowing you to persist data between runs.

Use Docker Compose in your Apps

Compose is a great tool, and forms the basis of more advanced Docker tools (such as configuring stacks for Docker Swarm orchestration).

You can see how using Docker Compose greatly simplifies the configuration of your containers. By specifying the various services necessary for your app in a single place, it is easy to create disposable instances of your app for development, and share configuration with other developers.

Adding a docker-compose.yml file to your repository is a great way to make getting started with your app as easy as possible. Often, your README file for people to get started using your app can be as simple as:

$ git clone https://github.com/your-app.git
$ cd your-app
$ docker-compose run --rm web bin/rails db:setup
$ docker-compose up -d
$ open http://localhost:3000/

Next Steps

Thanks for reading this tutorial. I hope you've found it useful and you can begin using Docker and Docker Compose in your own projects. Let me know how you get on by getting in touch.

Next time, we'll explore how you can use additional services for your Rails apps with Docker compose, such as Sidekiq and redis.

The benefits of value pricing sound obvious. No ambiguity; fair compensation; a multiple-of-X return on your customer’s investment; no surprise invoices (or subsequent “difficult conversations”), and so on. It seems like a clear win-win.

So why is it that when you’re ready to use value pricing and talking with a prospective customer, you suddenly get hit with the fear?

It usually happens just after you’ve been given a long list of requirements, and hear the words:

So, how much will this cost…?

Despite all the reading, all the podcasts, all the books and all the time spent convincing yourself of the benefits of value pricing, a dark cloud of doubt is suddenly cast across your mind.

As you try to consider your customer’s needs, and understand their goals, your head races with questions, such as:

• Do I know enough about this project?
• What if they want to do X?
• If I give a price now, what if they decide to use Y?
• What if it takes longer than I think?
• Is my price too high?
• Is my price too low?
• What if doesn’t take as along as I think? Will they want a refund?
• What if they want to add more stuff?
• But what if…?

You start to second-guess yourself. Are you really ready for this value pricing? Imposter syndrome sets in, and suddenly time-based billing doesn’t seem so bad after all (it is). You’ll just do it one last time, but next time you’ll definitely try out value pricing…

The reason behind this fear, I believe, is that we often don’t accept (rather than don’t know) the value we deliver.

It is an indicator that there you are missing a piece of the puzzle. You haven’t yet uncovered the real value — the pain that has caused the customer to come to you in the first place.

It’s a sign that you need to continue the conversation.

What Does Your Customer Value?

Value to your customers is likely to be very different to value to you. Perfect test coverage and a flawless CI setup probably won’t interest them — unless you can demonstrate the benefit to them (e.g. “we won’t unknowingly break something when we deploy”, or perhaps “we can confidently deploy at 5pm on a Friday…”).

To your customer, though, perhaps there is immense value in a simple conversation; learning about how the latest release of iOS or Android will help benefit their customers; a video on the latest trends in their industry; implementing an off-the-shelf piece of software that will make their lives easier, and so on.

Even something as simple as sending an article or blog post to your customer can provide incredible value — especially if they receive it at just the right time to be relevant to their needs.

Take the time to have conversations with your customer, and understand the benefits they receive a result of your work. This is the best way to conquer the fear, and be confident that your proposal represents immense value to both you and your customer.

Have you started using value pricing in your business? Do you get hit with the fear when asked to give a price? I’d love to hear how you’re conquering those fears and using value pricing in your business. Let me know by getting in touch.

Docker is a fantastic tool for isolating your app and its environment, and allows easy distribution and state-replication across multiple environments (dev, test, beta, prod, etc.). Using Docker can get rid of the "it works on my machine" problem, and help you to easily scale your app as it grows.

Docker is particularly great when your app has a lot of dependencies, or requires specific versions of libraries and tools to be configured.

In this tutorial, you'll learn how to take a basic Rails app and prepare it for use in a Docker container ("dockerise" it).

PreRequisites

For this tutorial, I'm using a simple Rails 5 application configured to use a PostgreSQL database. If you use a different database, you'll need to tweak a few of the files below.

You can use the following template to create a basic Rails application that is configured with a Dockerfile and config/database.yml as below:

$ rails new --database=postgresql --skip-bundle --template=https://gist.githubusercontent.com/cblunt/1d3b0c1829875e3889d50c27eb233ebe/raw/8fce95533db3a3be19cb1aa31054589d219433c2/rails-docker-pg-template.rb my-app
$ cd my-app

Database Configuration

We can make use of environment variables to configure the details for our app's database. You'll use this later so that your app's docker container can connect to a PostgreSQL container.

Edit your config/database.yml configuration

(Note: You don't need to do this if you've used the application template above)

Update your app's config/database.yml to use environment variables:

# config/database.yml
default: &default
  adapter: postgresql
  encoding: unicode
  pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
  host: db
  username: <%= ENV.fetch('POSTGRES_USER') %>
  password: <%= ENV.fetch('POSTGRES_PASSWORD') %>

development:
  <<: *default
  database: my-app_development

test:
  <<: *default
  database: my-app_test

production:
  <<: *default
  database: my-app_production

Create a Dockerfile

With your app prepared, it's time to start using Docker. Let's start by creating a Dockerfile. This is a plain text file that instructs Docker how to build an image for your application.

You use it to install dependencies, configure default environment variables, copy code into the container, and so on.

To keep your image size small, I prefer to use the alpine-linux Ruby base image. Alpine linux is a tiny linux distribution that's perfect for containers, and Docker provides a default ruby:alpine base image that we can use.

Write Your Dockerfile

Let's start by creating a basic Dockerfile for your rails app. In your app's folder, create the following Dockerfile.

(Note: You don't need to do this if you've used the application template above)

What if I don't use PostgreSQL?

If you're using a different database server (e.g. MySQL), you'll need to tweak the Dockerfile to install the appropriate packages.

You can search for the correct package(s) using the following Docker command:

$ docker run --rm -it ruby:2.5-alpine /bin/sh -c 'apk update && apk search mariadb | sort'
...
mariadb-client-10.2.15-r0
mariadb-common-10.2.15-r0
mariadb-connector-c-3.0.4-r1
mariadb-connector-c-dev-3.0.4-r1
mariadb-dev-10.2.15-r0
...
mysql-10.2.15-r0
mysql-bench-10.2.15-r0
mysql-client-10.2.15-r0
...

With your Dockerfile written, you can now instruct Docker to build an image for your app:

Build an image for your app

$ docker build . -t my-app

Once the image is built, we're ready to get started! You can spin up a new container based on your app's image using the following command:

$ docker run --rm -it --env RAILS_ENV=development --env POSTGRES_USER=postgres --env POSTGRES_PASSWORD=superSecret123 --publish 3000:3000 --volume ${PWD}:/app my-app

We're passing a few arguments to the Docker run command:

• -it is actually two arguments that allow you to interact with your container via the shell (e.g. to issue Ctrl+C commands).
• --env allows you to pass environment variables into the container. Here, you're using it to set the database connection values.
• --rm will instruct Docker to remove the container once it finishes (i.e. when you hit Ctrl+C).
• --publish forwards port 3000 from the container to port 3000 on your host. This allows you to access the container as if it was running on your host (i.e. http://localhost:3000).
• Finally --volume instructs Docker to "mount" the current folder (from your host machine) into the container. This means as you edit code on your workstation, it will be available to the container. Without this, you'd need to recreate the container with every code change you make.

Once it's up and running, you can access your container by opening localhost:3000 in your browser.

Run a Database Container

Unfortunately, although the container runs successfully, if you try to access the app in your browser, it will crash with a database error.

could not translate host name "db" to address: Name does not resolve

At the moment, there isn't a PostgreSQL server available for the app too connect. We'll fix that now by spinning up a separate Docker container in which PostgreSQL will run:

ProTip Remember that in Docker, a container should be designed to do one thing (and one thing only).

In our case, we'll use two containers: one for our app, and one for our database (PostgreSQL).

Start a new container running PostgreSQL:

Hit Ctrl+C to stop (and remove) your running app container, then spin up a new container for PostgreSQL:

$ docker run -d -it --env POSTGRES_PASSWORD=superSecret123 --env DB_NAME=my-app_development --name mydbcontainer postgres:9.6

The -d flag will detach the container from our terminal, allowing it to run in the background. We also give the container a name (mydbcontainer) which we'll use below.

Using Single–Task Containers

Docker containers are disposable, and their single-purpose nature means that once they have "finished", they are stopped (and, optionally, removed).

This makes them perfect for running one-off tasks, such as rails commands (e.g. bin/rails db:setup).

We'll do that now to setup your app's database on mydbcontainer:

Run the rails db:migrate task using a container:

Use the following command to spin up a copy of your app's container, run the bin/rails db:setup task, and then shut down.

Note that you'll need to configure environment variables for the database connection (these are injected into the config/database.yml file you edited earlier).

You'll also use the --link option which allows the container to connect to PostgreSQL container that is running (mydbcontainer), using the hostname db:

$ docker run --rm --env RAILS_ENV=development --env POSTGRES_USER=postgres --env POSTGRES_PASSWORD=superSecret123 --link mydbcontainer:db --volume ${PWD}:/app my-app bin/rails db:create db:migrate

The --rm flag will remove (delete) the container once it has finished running.

Once that command has run, your app's database will be setup on the mydbcontainer container. Finally, we can run our app!

Run your app!

Let's spin up a new container using our app's image. Notice that there are a couple of additional arguments when running the command:

$ docker run --rm -it --env RAILS_ENV=development --env POSTGRES_USER=postgres --env POSTGRES_PASSWORD=superSecret123 --publish 3000:3000 --volume ${PWD}:/app --link mydbcontainer:db my-app

=> Puma starting in single mode...
=>  * Version 3.8.2 (ruby 2.4.1-p111), codename: Sassy Salamander
=>  * Min threads: 5, max threads: 5
=>  * Environment: development
=>  * Listening on tcp://0.0.0.0:3000
=>  Use Ctrl-C to stop

Open your browser to localhost:3000, and you should see your app running entirely on Docker!

Next Steps

Docker is a great tool for developing your applications. As time goes on, you can begin to move all components of your app (database, redis, sidekiq workers, cron, etc.) to Docker.

The next step is to use Docker Compose to declare all your containers, and how they should work together.

It can be difficult to get started with value pricing, though. It can seem a daunting, and risky, move. I spent months trying to reconcile the “fixed-price, fixed-scope” nature of value pricing with the fluidity of software projects, constantly worrying about how to deal with scope-creep.

This is the second in a series of articles based on my talk at FutureSync held at Plymouth University on 9th June 2017. In Part 1, I discussed why charging for time is against the interests of both you and your customers.

Eventually, I figured the only way was to jump in 100%: force myself to abandon time-billing altogether, and accept that I will make many mistakes along the way…

As with most things, the hardest step was actually getting started. Once I made the leap, things began to change for the better. In fact, the journey has led to major changes in all areas of my business.

In this post, I’ll introduce three steps you can take to start value pricing in your business. These steps help to change your thinking away from the time you put in, towards the value and results your customers get out.

1. Increase Your Time-Billing Unit

Instant, easy and effective. Increasing the time unit for which you charge has an immediate effect on your business’ top line.

If you are currently charging for hours, start charging for days; if you’re charging for days, start charging for weeks; and if you’re already charging by the week, start charging for 2-weekly sprints, or — even better — monthly.

Of course, this isn’t really value pricing — you are still charging based on your time. However, you’ll very quickly find your customer interactions change. As you move up the time-billing units scale, and particularly once you are charging on a weekly basis, the conversations you have with your customers change dramatically.

As small time increments play less of a part in your work (15 minutes here, an hour or two there), and especially once you’re billing by the week, you should find customers become more concerned with the results of your time, rather than the time-sheets you produce, and in turn you can begin to work with them on the bigger-picture goals of their project.

ProTip: Never use fractional units

The key to this step is not to fall back to your previous time unit. If your new billing unit is a day, that becomes your new minimum fee. No half days (or quarter days, or 0.1333 days…).

I made this mistake in the early days, and it meant the conversations I was having with customers never changed — after all, if you’re just changing the label from hours to days, nothing changes in the eyes of the customer.

Some customers will understandably resist when, for example, your price rises from a couple of hours to a “day” for what they consider to be small task.

Remember that your goal is to price your work based on the value your customer receives, rather than your efforts going in. One way to help reduce sticker-shock for your customer is to simply drop the “days” label, and offering the work as fixed price service instead for a set of defined outcomes instead. More on this below.

Needless to say, the quicker you can move away from labelling your work with time (hours, days, weeks, etc.), the better. That’s where the next steps come in.

2. Have Conversations

This is where you begin to discover the value you’re really delivering to our customers. Conversations are critical to the success of your business, and your relationships with your customers.

Now, it’s important to clarify that by conversations, I don’t mean sales meetings, pitches, or job interview-esque discussions (those one-way conversations where you sit with a customer, they dictate a bunch of tasks, and you head off to get started)

Value-driven conversations require really getting to know your customer’s business, understanding their long-term goals, and building huge amounts of trust between you.

Trust is the key that lies behind every successful project, and what you will need in order to make the shift to value pricing.

As you learn more about your customer’s business, you’ll naturally begin to see new opportunities for them.

Remember that you are likely far more aware with what’s going on in your industry than your customer, and so being able to offer them advice and knowledge about what’s going on — and, more importantly, how it can benefit them — is incredibly valuable.

The goal is to become a trusted advisor to your customer and their business; someone they can rely on to act in the best interests of them and their business.

This requires regular engagement, conversations about more than just the project; and also that you become familiar with their business and industry.

ProTip: Say No

A quick way to gain trust is to start with no. Say no to those off-the-cuff ideas that tend to crop up in conversation, and that you know will deliver little or no benefit to your customer.

We all succumb to shiny-new-thing syndrome once in a while, and your customer is no different. Whenever they ask for something out of the blue (e.g. “we just need to implement live chat on our website.”), force them to justify the business case behind it, and how that will help them work towards their goals.

If your customer can’t justify the feature or request yet, but still feels strongly about it, then suggest putting it on an backlog or wish-list for future consideration, and set a date a few weeks out to review it.

What seems critical and urgent today can and will change as things progress and new opportunities arise.

3. Productise

As you move up the scale of time-billing units, the amount of time you put into your work becomes less relevant (it doesn’t necessarily decrease though).

Through the conversations you’re having with your customers, you’ll begin to see common patterns, needs and requirements. As you do, you can begin to extract the work you do to satisfy those requirements and bundle them into products, or productised services.

Doing this allows you to establish standard procedures for certain functions of your business which you can continuously work to make more efficient, review and test with your customers.

One of the greatest benefits to you and your customer of a product(-ised service) is that you can set and advertise it with a price.

Now you can begin to remove reference to your time from the service, and just focus on outcomes.

Of course, you will want to consider your own time as a cost of business — especially if you have a team of employees or contractors.

As far as the customer is concerned, though, they are getting known outcomes for a specified price. They can determine themselves if the value is right for them, greatly speeding up your qualification and sales process.

ProTip: Give Your Customer Options

Options are beneficial to any pricing strategy, but especially helpful in value pricing. Presented with options rather than a binary yes/no proposal, your customer can determine their own level of return on investment based on the options presented.

Three is the popular number for presenting options, but even a simple A/B choice is better than a single offer, so find a way to increase the value between two options, and make sure you highlight them to your customer.

The higher price options would usually build upon earlier options, increasing the value exponentially as you move up the scale, but this can be dependent on the nature of the project.

4. Over-Deliver

(pun intended… :)

Over deliver on absolutely everything you do for your customer.

It’s important to point out that I don’t mean using the tired “under-promise, over-deliver” sales trick (that’s terrible sales advice). Remember, the key to success with value pricing is trust.

You can only really start doing this once you’ve successfully moved your own mind away from time-billing; appreciate the value you are delivering to your customers, and — most importantly — are pricing based on that value.

However, over-delivery comes naturally result of a value-based mindset. You can’t help but want to find new ways to help your customer achieve their goals, because your focus will be on maximising the impact of your work on their business, knowing that you are both getting value out of the transaction.

True value pricing is always a win-win.

I hope you find these techniques helpful, and can use them in your own business. I’d love to know and hear how you’re getting on — get in touch, or reach me on twitter.

This is Part 1 in a series of articles based on a talk I gave at the FutureSync Conference held at Plymouth University on 9th June 2017.

The Problem with Time

As I imagine is the usual story, when I set up Plymouth Software (and indeed, when I was freelancing before), charging by the hour was the norm. After all, it’s just “the way things are done”.

The story goes something like this: Once you’ve decided to make the plunge and set up your own business, you head to one of the many freelance rate calculators. You tap in your living costs; some estimated budgets for nice-to-haves (coffees, holidays, new laptops, etc.); your expected number of holiday days, and a target profit margin. With that all done, you hit submit and out pops your new hourly rate. You round the figure, and set off to work.

Pretty quickly, though, you learn that selling your time is really quite painful. And it’s not just painful for you — it’s just as bad for your customers.

Understandably, they don’t want any ambiguity in what they are going to pay you, and they want to know what they’re going to get for their money. We’ve all been on the other side of those transactions, and nobody likes receiving those surprise end-of-the-month invoices.

It’s a hard lesson, but you quickly come to realise that time-based billing—a relic of the industrial age—simply isn’t fit for purpose in the world of knowledge-based and professional service businesses.

It is fraught with problems and frustrations for everyone involved, but there are three major problems with time-billing that underpin all the others.

1. Time billing puts you in a position of conflict with your customer from day one.

This is perhaps the most fundamental issue with time-based billing — every project becomes a zero-sum game.

As a business, you want to maximise income, so you need to work as many hours onto the timesheet as possible. Naturally, your customer wants to minimise their costs, and so they want to keep those hours down. In this scenario, a win-win scenario is impossible.

Recognising this, we usually try to mitigate the damage by employing fixed-price quotes.

Now, you estimate how long the job will take, and quote a price for the customer. They get some certainty in their spend, but you have taken on virtually all of the risk.

Unfortunately, this type of fixed-price quote doesn’t solve the problem — it simply reverses the dynamic. You want to minimise hours to maximise your effective hourly rate, but your customer wants to maximise their return, squeezing as much as possible from that quote and reducing your effective hourly rate.

This is where (in software development, at least) we fall into the trap of hefty upfront specification documents, and pretending_ to know how things will be three months down the line.

In all but the rarest cases, fixed-price quotes based on time usually result in big squeeze of work near the end, as that estimated number-of-hours draws close, and work is crammed in to maximise the budget.

Unfortunately, the end result is usually poor relationships and awkward conversations about what was included in the original quote, where things went wrong, the finer details of the contract, etc.

This is valuable time and energy that would be much better spent delivering results for your customers.

2. Time billing encourages busy work

Time-based billing naturally encourages doing something rather than the right thing. This often means we jump into a project without really knowing the end goal. We skip learning what success looks like for our client to get hours on the timesheet.

This almost inevitably leads to disappointed customers. Without taking the time to really understand your customer’s needs and goals, we end up delivering nothing of value.

For example, if a customer comes to you saying they need a new website, the last thing you should do is start considering which of the latest cool new frameworks you want to use.

Instead, you should start questioning why they think they need a website — and keep working with them until you reveal the underlying goals behind their request.

Nobody wakes up one morning thinking “I’ll spend a few thousand on a website today.”

There is always an underlying reason.

The problem with time billing is it encourages us to skip the learning phase, and deliver the website.

A few weeks later, our customer has a shiny new website, a reduced bank balance, and the same problems as before.

We’ve delivered what the customer asked, but failed to solve the issue that led to them asking us in the first place.

3. Time Billing puts a ceiling on your income

Admittedly, this is not a problem with which your customer is concerned, but as a business-owner it should worry you. There are only so many hours in a day, and so by billing for your time, you are putting an artificial ceiling on your potential income.

There are 3 main ways we can try to overcome this issue:

i) Increase your workload

This is OK in the short-term, but eventually you’ll burn out or hit the very hard limit of 24 hours a day.

ii) Increase your rate

Certainly this will be the quickest and easiest way to break through the income ceiling. However, your rate is also subject to the market, and the laws of economics mean there will be an increasing downward pressure on your rate. This is especially true as implementation work becomes more commoditised. Ultimately, you’ll hit another ceiling.

iii) Increase your staff

Likely the most effective strategy over the long-term, but taking on staff may require big changes to your business — particularity if you’re currently solo.

Unfortunately, though, increasing staff just scales up the problem of time billing: now you’re selling ten people’s time rather than just one or two—and you’re still subject to the same problems as before.

Finding a Better Way

I knew there had to be a better way than time billing, and set out to find it. Fairly early on, I learned the concept of Value Pricing from the first edition of Brennan Dunn’s Double Your Freelancing Rate (a book I highly recommend)

As I read more into the subject, I became increasingly convinced value pricing was the only way to truly escape time-based billing, and grow my business.

There was one problem though.

I struggled a lot to reconcile how value pricing could work with software development. There are just so many unknowns in software projects…but it turns out this we’re not alone in this!

Changing your own mindset is perhaps the single biggest challenge of switching to value pricing.

After all, I’d spent 30-odd years being taught that time is money and everything in business is reduced down to an hourly rate. The modern world of work — especially in service industries — is still very much designed around the billable hour.

By doing anything different, you’re going up against a lot of expected norms.

In 2016, I decided the only way I could make value pricing succeed was to embrace it 100%, force myself to abandon time billing, and accept that mistakes will be made…

In the next part of this article series, I’ll talk more about what value pricing is, and outline 3 practical steps you can use right away to begin changing your business to value pricing.

After some searching, I realised the found was being configured with hw.sdCard=no during creation, despite specifying an SD card in the UI.

To fix the issue, open up the config.ini for the affected emulator, and change the value to hw.sdCard=yes, e.g:

# ~/.android/avd/Nexus_5X_API_16.avd/config.ini
# ...
hw.sdCard=yes

After restart your emulator, it should now successfully mount the SD Card.

# app/assets/javascript/home.coffee
$ ->
  $('#some_element' ).text('<%= @user.id %>')

If you look at the resulting Javascript, you'll see that the string is interpreted literally, rather than interpolating the user's id:

(function() {
  $(function() {
    $('#some_element' ).text('<%= @user.id %>');
  });
}).call(this);

Quickfix

You can quickly fix this by appending .erb to the Coffeescript's filename. This will instruct the asset compiler to parse the file through ERB before Coffeescript, resulting in the value being correctly interpolated:

# app/assets/javascript/home.coffee.erb
$ ->
  $('#some_element' ).text('<%= @user.id %>')

// (resulting Javascript)
(function() {
  $(function() {
    $('#some_element' ).text('32');
  });
}).call(this);

A More Flexible Approach

However, I feel this too tightly-couples the Javascript and Rails code. I much prefer to follow Rails' own unobtrusive-Javascript approach, and make use of HTML5 data attributes to pass data between the front-end (JS) and back-end (Rails) code.

Using the example above, the id would be passed as an arbitrary data attribute on the HTML element:

# app/views/home/index.html.erb
<div class="user-details" data-user-id="<%= @user.id %>">
</div>

Now in the Javascript, you can easily extract the value of data-user-id. Furthermore, you can use the same code to act on any HTML element containing the data attribute:

# app/assets/javascript/home.coffee
$ ->
  $('div[data-user-id]' ).each ->
    userId = $(this).data('user-id')
    $(this).text(userId)

This post was inspired by a question on StackOverflow. See the original discussion.

As building and supporting Rails apps is a core part of my business, I've adopted a few tools and techniques to ensure that the apps I build and manage are easy to maintain and improve.

Here are 3 things you can do today to keep your Ruby on Rails apps in great condition:

1. Check for Code Smells

A code smell is a section of code in your app that indicates a deeper underlying problem. The problem might be a bug, or a something that has the potential to lead to bugs, such as duplicated blocks of code, or several levels of nested conditions.

Code smells are dangerous for your app because as the amount of code increases, so to does the risk of errors creeping into the code. Catching code smells early and often can help to reduce this risk, and make your code much easier for you (and other developers) to maintain in the future.

RubyCritic is a great tool for identifying code smells and bad practice. You can install it easily by adding the following to your app's Gemfile:

# Gemfile
gem 'rubycritic', require: false, groups: [:development, :test]

Once that's done, just bundle install and run rubycritic against your app's code:

$ cd /path/to/your/app
$ bundle install
$ bundle exec rubycritic --format html

After a while, RubyCritic will generate a report on your app's code quality. If the report doesn't open automatically, you'll find it in your app's tmp/rubycritic folder:

$ open tmp/rubycritic/overview.html

The reports show code smells that Rubycritic has identified. Use the reports as a guide to improve your app's codebase.

2. Scan for security holes

Rails does a lot to protect you from exposing your app to security holes. But like any software, new vulnerabilities are discovered all the time. Keeping your app's version of rails up to date with a supported release is critical to protecting your app and your users.

Similar to Rubycritic for codesmells, we can use tools like brakeman, you can easily scan your app's codebase for security vulnerabilities and known issues. To get started with brakeman, install it as a gem:

$ gem install brakeman

Then jump into your app's code and run the brakeman command:

$ cd /path/to/your/app
$ brakeman

Brakeman will then report on various levels of issues in your code, allowing you to improve the security of your app instantly.

3. Set up Monitoring

Finally, once your app is out in the wild, you'll want to know if anything goes wrong. Errors can occur anywhere - and you definitely don't want them to be silent (to you).

StatusCake's monitoring dashboard.

There are lots of different monitoring services out there - with a range of prices - that will help you monitor your app. Several of them, such as StatusCake, Sentry and Rollbar offer free plans to get you started.

There are also several types of monitoring, from availability checks (as provided by StatusCake and Pingdom) through error reporting (Sentry and Rollbar) and performance monitoring (Skylight and New Relic).

To get started, you can start with simple availability and error checks, as they're going to give you the instant coverage that your app needs. Most of the tools are easy to setup, either through registering a URL or dropping a gem and API key into your app's code.

As your user–base and site traffic increases, you should look at performance monitoring tools to keep an eye on how well your app is responding to customers, and highlight any bottlenecks or opportunities for caching.

Next Steps

Keeping a production app in top condition is an ongoing task. To find out more about keeping your app's healthy, and how to make the process easier with tools and systems, check out my new courses at healthyrailsapps.com.

The good news is that the switch to value pricing has, on the whole, been very successful.

The move to a value pricing model has fundamentally changed my approach to virtually everything I do, and is even driving a change to the business itself.

As I discussed in my previous post, when you start proposing work based on value, it forces you to begin thinking about what it is you’re delivering (lesson learned: it’s not code, designs or apps).

Here are some of the changed I made and benefits I’ve experienced from moving to value pricing.

Breaking the Time Billing Mindset

One of the biggest challenges is overcoming the general expectation for time-based billing, particularly in service-based businesses. This has been as much my own expectation as anyone else’s.

My commitment was to move the business entirely to value pricing, and I’m pleased to say that this has been the case in the vast majority of jobs I’ve taken on. I’ll confess, though, there have been a couple of occasions when I’ve charged for time for very small development work (minor fixes, tweaks and so on) where the “value” was just getting the job done.

I still wrapped these in fixed-fee quotes, but they were very much what Ron Baker describes as Hourly Billing in Drag. I ensured these jobs were small though; nothing more than a day or two.

However, I knew my mindset had changed when I experienced another unexpected outcome of the switch: I now expect other companies to offer me value-based prices and options, rather than giving me a time estimate or rate.

In fact, I’ve found myself specifically opting for services that are priced in this way (even if the price may seem higher than a time-estimate) because I’m aware of the advantages of value-pricing to me as a customer.

Proposals

Without doubt, one of the quickest benefits I saw from the switch to Value Pricing has been in writing proposals. They are now considerably faster to put together, as generally they are more a summary of what has gone before them, rather than the “send-and-hope” pitches they were before.

I changed my proposals from a report style to a personalised letter format. As in any business, my goal is to build lasting business relationships, so this personal touch helps in fostering a more collaborative relationship.

The letter format also allows me to cover various things in a far more conversational manner — which is far easier to write! — and reflect on discussions that have happened in the stages before the proposal.

Customer Satisfaction

As you would expect, customers are far happier when they know what they are getting and how much it will cost, rather than an estimate of hours, days and weeks.

Coupled with additional benefits that can be offered within a price, such as providing a satisfaction guarantee, customers are usually very happy (if not a little surprised) to learn that I offer prices over rates.

Productisation

My move to productisation started with introducing Discovery Workshops for new projects.

Rather than the usual “email, chat, coffee, proposal, cross-fingers” cycle, I recognised that those initial discussions with potential customers can deliver incredible value — even if it ultimately means the project doesn’t go ahead.

There’s a lot of value in saving customers £000’s in development costs with some early validation using the Discovery Workshop approach.

Productising these early stages of a project also helped me to start defining a sales process for incoming leads. I eventually offered 3 levels of discovery product— a short consultation, a full-day workshop, and a workshop plus wireframes/working mockups. This gave new customers 3 levels of investment to choose from, as well as helping me to to qualify leads.

Although I only ran a few of the workshops last year, I learned a lot from each one and got positive feedback from customers. Not all customers opted for Discovery (sometimes they wanted to jump right into building a working prototype), those that did got real value from the sessions, as well as something they could take away and use to further develop, refine or invest in their idea.

Development Sprints

I’ve used the Sprints straight out of SCRUM as part of my agile process for several years, but I’ve always struggled to move the charging structure away from time.

Previously, I would pitch a Sprint as 1 or 2 weeks worth of work. In a (admittedly basic) attempt to price them by value, I removed “weeks” from the equation.

This worked, but it’s not even close to true value pricing — each Sprint could arguably deliver a very different level of value from the next. However, in the early stages of a new software project, it is extremely difficult to determine any value beyond “getting it built”.

Fixed-price, fixed-scope Sprints offered a quick-fix, but they weren’t perfect. However you frame it, development work is difficult to value price because the end-result is bespoke.

So whilst Sprints provided a way to move my charges away from time, they still weren’t a good for value pricing.

As time went on, I came to realise that development work is so difficult to value price because it’s selling the wrong thing.

Real Value

The truism goes that::

People don’t want to buy a quarter-inch drill, they want a quarter-inch hole.

I recently listened to an excellent interview with Eric White where he took this analogy one step further: customer’s don’t want to buy the hole — they want the picture hanging on the wall.

The big realisation after nearly a year of value pricing my work is that the implementation (code in my case, but this could be anything) is of little — if any — real value to customers. Trying to sell development is like trying to sell the drill instead of the hole (or the picture).

It’s this that has got me thinking and driven me to start making some big changes to my business.

As time goes on, I’ll cover those changes in more detail, but the first step is to shift my focus to providing Service Plans rather than new-build software projects.

Service Plans

Service Plans are now my main productised service offering, providing a known scope of work for a known monthly price.

I have purposely left time restrictions out of the Service Plans, and instead base their deliverables on scope of what’s included. As well as standard maintenance and monitoring, this approach has allowed me to include more much more valuable benefits such as consultation and (software) strategy services for customers, which are more suited to value pricing.

Service Plans also represent a shift away from delivering code, and instead concentrate on delivering outcomes. You can find out more about what’s included in the Service Plans at plymouthsoftware.com.

Service Plans represent the first of a number of productised services (and products) that I will offer through Plymouth Software. Subscribe to get email updates as I introduce new products and services, and continue my journey to value pricing.

Have you made the switch to value pricing in your software business? How has it changed your business? Let me know by getting in touch or message me on Twitter.

Without the constraints on time, some projects have dragged. Projects running on and on are nothing new, but it does make things extremely difficult when trying to schedule future work or bring new projects on board.

Perhaps the most most obvious way to resolve this is to time-box work. However, my worry is that by putting an artificial time-limit back on the scope, it will be all too easy to fall back into some form of time-based billing.

I’ve recently had success selling sprints purely based on scope — customers love the safety that the price won’t change, as it is based on the value of the sprint. So whilst weekly billing seems to be the simplest answer, I’m obviously reluctant to go down this route.

However, I’ve begun to realise there’s perhaps a bigger takeaway.

Value pricing is forcing me to completely rethink what my business does.

Those generic development services (“programmer for hire”) are now getting in the way of growing my business.

This realisation probably started when I introduced Service Plans, which have been great for me and my customers. They offer a known scope of work for a known monthly price.

The majority of my Service Plans customers have no time restrictions, but are instead restricted on the scope of work that’s included. Lower-tier plans might include monitoring and maintenance, security patches, etc., whilst higher plans include new feature development, consulting, R&D, and so on.

Service Plans are designed for the modern apps, whose environment changes every day (OS upgrades, security patches, user feedback, bugs, changing APIs, etc.)

However, there’s still much more I can do in this area.

Moving to Productised Services

With this in mind, I’ve begun reducing the number of “traditional” or ad-hoc development projects I’ve taken on, and have concentrated on the Service Plans.

As I’ve learned more about my customers’ needs, I’ve begun to identify common requirements which can be packaged into fixed-price, fixed-scope “productised services”.

Productised services and value pricing seem to go hand-in-hand. In the coming days, I’ll be publishing a new site detailing the first set of services.

Early Productised Services Landing Page — Coming Soon!

With known scope and pre-defined outcomes, I hope to be able to schedule work more confidently, and offer a far more solid value proposition to new and existing customers. The monthly Service Plans will continue to be the main focus, but over time I hope to refine what’s included in the plans (beyond the code) to offer even more value to customers.

Have you started using value pricing in your business? Do you get hit with the fear when asked to give a price? I’d love to hear how you’re conquering those fears and using value pricing in your business. Let me know by getting in touch.

Whilst there are workarounds that can help to speed things up, any change to your Gemfile results in a lengthy (re)download and install of all the gems in your app.

Sometimes I only needed to generate a new Gemfile.lock so that the image could be built, tested and published by our continuous deployment server. Frustratingly, this meant having to wait for the redundant local bundle install to complete before I could commit the new lock file.

Today, I discovered a way to save the hours wasted downloading gems: bundler's lock command.

This gem[^1] of a command resolves your app's dependencies and writes out the appropriate Gemfile.lock – without installing any of the gems themselves.

Using Bundle Lock with Docker

Use the following command in your app's path to quickly create your Gemfile.lock:

# cd /path/to/my-rails-app

$ docker run --rm -v $(pwd):/usr/src/app -w /usr/src/app ruby:2.3.1 bundle lock

If you have an existing lock file, but want to update the gems (equivalent to bundle update), just add the --update flag:

# cd /path/to/my-rails-app

$ docker run --rm -v $(pwd):/usr/src/app -w /usr/src/app ruby:2.3.1 bundle lock --update

Hopefully this will save you some time (and bandwidth) when building your Rails apps on Docker.

[^1]: Pun very much intended

To begin with, I thought this was just another name for the initial call or meet with a potential new customer. The usual meet-and-greet over coffee, getting a broad overview of their idea, listening to their requirements, and trying to formulate in my head (as quickly as possible) some solutions.

After a while, though, I began to see that these initial consultations were, at their worst, actually limiting my work and the value I delivered by solving only surface-level concerns — which often aren’t even a problem!

As I’ve had more and more of these initial conversations over the weeks and months, I’ve come to realise how different they are from a true value conversation.

Digging into the Problem

Let’s start by defining the value conversation as I’ve come to understand it: a true value conversation is far more exploratory, and goes into far more depth than a typical initial consultation — even though it may still be called as such.

By way of an example, I’ll summarise the typical traditional initial discussion I was using:

A potential customer would get in touch via phone or email with a brief outline of task they want to achieve. This may be a cold enquiry, or a referral. The potential customer may have idea for a new app; need some help with a specific technical issue; or even need some new work carrying out on an existing project.

After answering some initial qualifying questions via email, a phone call or meeting is booked to find out more about the project. What generally happened, though, is that “finding out more” is almost entirely focussed on the technical details of the initial enquiry.

The customer says they need a mobile app, so we start discussing what platforms would be best suited, what sort of infrastructure we might need , third-party API integrations, and so on. In some cases, we’d even start napkin-sketching some design and layout ideas (giving away value!)

As I gained more experience, I began to introduce questions to find out a little more about the problem (e.g. what made you think about this idea?, or what is the problem with the current app?), but invariably, any attempts to discover what’s driven the project were either limited to surface level problems — glossed over as a necessary evil, or even entirely brushed aside to get to the excitement of the solution.

At the end of all this, the potential customer asks for some “ballpark” idea of costs and timescales, and numbers are carefully considered plucked from the air and proffered. Even though they requested a ballpark, always know that the customer hears a final price.

After pleasantries are exchanged, some written communication and formalisation of the discussion takes place, and the project is either won or lost. Then we get to dive into the fun stuff (writing code, designing screens, pushing releases). Within a few weeks — once code has begun to generate results, and things are “real” — that initial discussion and the initial “problem” we set out to solve become a long-forgotten memory.

The real problem — lurking in the shadows all the time, but buried under layers of really great features — may never get solved. I’ve seen enough projects that start with such great intention, but go on to solve little or even nothing at all.

Clearly, this approach is damaging for everyone involved.

Focus on the Problem…

So let’s take a step back, and try to explain how a value conversation can help. The difference is in how the relationship starts. A real value conversation establishes a level of partnership between the customer and the consultant. It is in the consultant’s interest to dive as deep into the root cause(s) of the project — the real pain that it is trying to solve. Similarly, it is in the customer’s best interest to realise this themselves, and be able to talk with the consultant about it.

In this way, both parties are working towards a common goal rather than being at odds, and both parties can see the true value in what is being built. In the future, when questions arise about why something is being done, it can always be measured against that initial root problem, and approved or rejected. A bonus of this — for both consultant and customer — is that it can help to contain scope creep / scope seep.

So, in practice, a value conversation is far more exploratory than a traditional initial consult. The goal (as a consultant) is simply to listen, and keep questioning the reasons behind a customer’s problem. Solutions should be avoided at this stage, or kept minimal.

…but allow ideas to flow

However, in my experience, once some root problems have been uncovered, it’s inevitable that my mind starts thinking of solutions. These will usually be small ideas that may solve the customer’s problem.

For example, if the customer is looking to integrate some email marketing solution into their app, then one relatively simple idea is to use an off-the-shelf product or service, rather than writing their own, and having to deal with all the problems and ongoing maintenance that entails.

There are two possible outcomes here. One is that it’s something the customer hadn’t considered, and after some more exploration, it might be that that is enough to solve their need. Except for some short-term integration work, this would be a fairly trivial solution that gets the job done. The customer has received value from the conversation, and that might be as far as the project goes (for now).

Alternatively, they might put up a barrier to the solution — perhaps their IT department refuses to allow third-party systems to be integrated, etc. — and thus reveal a deeper problem to the consultant.

All of this is valuable feedback for the consultant, helping them to build a picture of the true value of the project and what they can deliver.

Actions

For a while now, with all of this in mind, I’ve been trying to move my initial discussions towards a value conversation model. Basically trying to shut my mind up (it tends to start hunting for solutions immediately) and take a lot more time getting an understanding of a customer’s real requirements.

An interesting outcome of this is that I’ve found myself actively trying to put off new customers — finding ways to talk them out of hiring us. If it wasn’t for Jonathan Stark talking about exactly this, then I would be very concerned that something is going wrong…after all, actively discouraging work is generally not good business!

But in my experience, doing exactly that has actually helped to refine the work I’ve taken on. It seems to help customers justify their requirements to themselves, and in doing so better ensures they are getting genuine value from working with me, rather than work for work’s sake.

My work has evolved to spending more time helping customers to understand their true need at these early stages of a project’s inception. The code is just an outcome of that understanding.

By obtaining a much deeper understanding of the problems behind their project, everyone is on the same page. We’re no longer jumping to solutions and chasing false hopes dreams. We’re consistently working towards a common goal and delivering real value, all from simply changing how we frame those initial discussions.

Have you changed your initial discussions to value conversations? How have you found the experience? Do you actively try to discourage new leads from working with you? Let me know by getting in touch.

This has been my experience as I transition my business to a 100% value pricing model. When you can no longer fall back on time-and-materials, you are forced to look at your business in a different way and to figure out the real value you deliver.

This is not to say that it is easy to do. Whilst simple value pricing (if such a thing exists) might be to charge a percentage of monetary return on investment, that is not always something that is available. In fact, it may not even be the primary motivation for a project.

You need to be able to determine what is valued by each individual customer for each project, and that’s something that can take time and require thinking about things a little differently.

Start with Value

When you can no longer fall back on your time, you have to change your approach to business to establish value early on in any engagement. In any other business, this would be the norm, but for some reason, in knowledge businesses we seem to get stuck worrying about our own costs and time rather than what we’re delivering to the customer.

Doing things this way, it’s easy to work backwards: What’s the budget?…What’s our rate?…Here’s what we guess we’ll do in that amount of time. The price becomes the primary and ongoing concern.

Time billing means it is very easy to spend a lot of time (and money) delivering nothing — and expecting to get rewarded for it. In any other context — indeed, in any other type of business — this would be crazy. Imagine paying £30k for a new car, but finding only half of it turns up because “our engineers ran out of time”.

Value pricing doesn’t let you get away with this form of lazy pricing. It forces you to really understand your own business before you even start working with a customer.

In making the switch, I’ve had to completely rethink how Plymouth Software is positioned; the services we offer; and the outcomes of those services to our customers — what we deliver.

In a very short time, I’ve been forced to completely change my business model in three key areas:

Position

If you define your business by the deliverables (a web site designer, for example), your business has a very broad, generic target customer. In fact, if you’re anything like I was, your target customer is anyone who wants an app/website/whatever.

This is a problem as it means you have to redefine your value proposition to every new customer for every new job, and you can easily end up attracting the wrong customers to your business.

When you start thinking about your ideal client — who is it you actually want to work with; what are their hopes, dreams and worries (what my business mentor calls their 2am issues) — you can begin to figure out what they really value.

It took me a little while to realise what I was really doing was positioning my business. This is a whole subject in itself, and something that Philip Morgan covers in his work — I recommend checking out his blog and books if you’re interested.

It took me a little while to realise what I was really doing was positioning my business. This is a whole subject in itself, and something that Philip Morgan covers in his work — I recommend checking out his blog and books if you’re interested.

Product(isation)

Once you have your ideal customer in mind, and have gained an understanding of what they value, you can begin to standardise your offer.

One extreme of this would be an off-the-shelf product of some sort. For software companies, that’s usually a SaaS or downloadable resource such as a paid app, theme, eBook, and so on. However, there’s often a huge amount of upfront investment and risk required to make the leap from client-work to the dream product.

At this stage, I’m productising my consulting efforts — something I first learned about through Brian Casel. Productised consulting sits somewhere in the middle-ground of off-the-shelf and bespoke work. By providing a common structure around your service offering, you can begin to match your services to your customers’ goals. I’ve started this by productising the early stages of any engagement.

As an example, after an initial review, we now follow up with short phone consultation to discuss a business idea, we follow up with a code-review and/or a full-day discovery workshop (depending on the project).

The outcomes from these allow us to dig into a project and put together a realistic and value-based delivery proposal for our customers. Both of these services are offered on a fixed price basis, with a number of scope options to match customers’ needs.

This is by no means a new concept, and all of these services require a lot of input. However, by structuring them in this way, we can tweak and adapt them to better suit the needs of our target customers as we learn more about them.

Price

It’s perhaps surprising that price is the last item to be considered, but it is with good reason.

Only once you’ve figured out your ideal customer, established what are their goals and needs, and honed your business services to meet those goals, can you begin to calculate the value you deliver — and derive a price.

That price cannot be derived by a simple formula. It requires conversations with your customer. Setting the price for your services becomes an art — something that can change with every project; and something that actually feels good to present.

If you have taken the time to understand the value of your work, you know that your price is fair and mutually beneficial to everyone involved.

The Benefits of Value First

Thinking about the value you deliver forces you to reflect on your business in a way that’s easy to brush under the carpet when charging for time. The benefits are immeasurable, though. You become more confident in your business offering; you can standardise (and thus document) your operations and workflows; you can identify an ideal customer and align your business with their goals.

Most importantly, your customers receive real value from working with you, and you build stronger relationships as a result. This can only come from figuring out the value first, and making the price a result of that value rather than the driving force behind it.

It’s a difficult, long and ongoing process, but the switch to value pricing does more than just change the figures in your proposal — it transforms everything about your business.

Have you made the switch to value pricing? How has it affected your own business and the way you operate? Are you worried about making the switch away from time billing? Let me know by getting in touch.

Mini-Project

In February, I proposed a small project to an existing customer. They had requested some additional functionality that was outside the scope of their existing service plan.

In this instance, there were two distinct pieces of work available, so that made for 2 easy options. Thinking of a third option to present was difficult, so unfortunately I fell back to a traditional “buy both and get a lower total price” option.

In the end, the customer chose the middle option, which was the larger of the two pieces of work. Although I wasn’t particularly happy with my third option (which simply offered a reduced total price by committing to two projects instead of one), the act of proposing a fixed-price value agreement was extremely refreshing.

Most importantly, other than target delivery dates, the proposal included no mention of time-cost. Instead, the scope of work was defined as end-user outcomes rather than technical deliverables.

Concentrating on goals rather than implementation has made for a much smoother-running project. As the work has progressed, the implementation has adapted but there has been no need to worry about additional time as everything has been in line with those original user goals.

I can’t say that time was entirely absent from my pricing decision at this stage — I did give consideration to my internal costs when pricing the work, but did not expose this in the proposal. Taking a cue from Jonathan Stark, I priced the project such that minor tweaks and changes would not be an issue. It is part of the value of offering a fixed-fee agreement.

New Service Plan Agreement

In this first quarter, we also won another value-based service plan agreement. This had been in the works a little while beforehand, but during the proposal period I offered the customer a value-based alternative to the more traditional hours-per-month retainer approach.

Again, a few options were presented, based on the levels of scope and access that would be available. We eventually settled on a middle-ground agreement between options 1 and 2.

For both this service agreement, which is an recurring service, and the discrete project above, I applied what I’ve learned in value pricing to offer a fixed-fee agreement to the customer, and weighted the fees according to the additional benefits and reduction in risk they were receiving.

Simplified Proposals

Whilst I know I’ve made some mistakes in these first value-based proposals, there has undoubtedly been a huge advantage to working in this way.

Putting together the proposals took a fraction of the time that is usually necessary, as they were simply a formal, written summary of what had been agreed in earlier conversations.

Without the need to break down time and deliverables, I was free to focus on how each option I presented could offer increasing value the value to the customer.

Each proposal was therefore only a couple of sides of A4 paper, rather than reams of technical breakdown that would be necessary under a time-based agreement.

It certainly helped my own mindset knowing that the proposals presented solutions to the customers’ business goals, rather than simply increasing in the number of days and weeks that I would be available to the customer.

Freedom to Explore

By not worrying about the time and minutiae of (technical) deliverables, and instead concerning myself with achieving the overall goals that were defined in the scope, it has freed me to explore alternative options and put in the extra time necessary to research better ways of doing things.

For example, the service plan includes scope for complete redevelopment of an Android application, which is now underway. Without the clock running, I’ve been able to spend some time carrying out in-depth research into how to effectively tackle some of the business goals the app will meet, without feeling the pressure to keep within a set budget of hours.

The biggest challenge has been identifying all the value that could be delivered, and formulating options that truly reflect that. In the coming weeks and months, I’ll be concentrating on developing my skills in holding the value conversation, and training my mind to stop jumping straight to technical solutions.

Have you made the switch to value pricing? How has it affected your own business and the way you operate? Are you worried about making the switch away from time billing? Let me know by getting in touch.

@b = BlogPost.find(params[:id])
# ==> BlogPost[id: 100, 
polymorphic_url(b)
# ==> blog_post_url(b.id)
# ==> "/blog_posts/hello-world"

However, I recently came across a case where the named route did not match the model name. Using the above as an example:

# config/routes.rb
resources :blog_posts, as: 'posts'

Unfortunately, calling polymorphic_path now throws an error complaining that the generated path method (blog_post_url) does not exist. The path name, which is inferred through the model class BlogPost no longer matches any routes.

Instead, we needed the method to be post_url.

Thankfully, polymorphic_url lets you supply the path as a symbol, rather than inferring it from the model. We can then pass the additional information (in this case, the slug parameter) to the helper as an argument:

polymorphic_url([b.model_name.human.underscore.to_sym], id: b.slug)

==> post_url(b.id)
==> "/posts/hello-world"

As developers, we learn pretty quickly that no two projects are the same, each bringing its own challenges and nuances. (I’m sure this applies to other knowledge/service businesses as well).

It’s perfectly reasonable for a prospective client to ask “how much will this cost?”, but this can be a very difficult question to answer, especially in the early stages of a project where so much is unknown.

Nobody likes surprise invoices, or not knowing how much they will end up spending on a project. Whilst I always preferred to give a fixed fee — and did so in the early days — I soon switched to time-and-materials approach.

At some point, it seemed this was the only way to deal with increasingly complicated projects, unknown requirements, and scope creep (or, far more often the case, scope seep).

Hidden Time and Materials

I thought I had found a good compromise to a simple ‘hourly rate’ by using daily, weekly or “per-sprint” rates as a way to package up the time into small, fixed-fee units of work.

However, it soon became clear that this was just re-framing the problem, and would lead to the same questions from customers. “How many hours?” just became “How many days, weeks, or sprints?”

The underlying problem was that the fixed fees I was giving were still based on an estimate of time.

By pricing in this way, I was still stuck on the cost-plus model. The price a customer paid bore no relation to the value being delivered, and brought with it all the same problems as time-and-materials billing normally would.

It certainly did nothing to mitigate expanding scope and managing the unknown — but that’s the subject for another post.

Moving Back to Fixed Fees

There is no question in my mind that offering a fixed fee for work is better for everyone involved, but a fixed fee based on time is not.

So how can you offer a fixed fee, and still protect yourself from the dangers of run-on projects, scope creep and the unknown? It’s taken me perhaps a year to come to terms with the fact that you simply can’t. There will always be a risk in this type of work — it is the nature of our business.

Offering a fixed price means you are taking on that risk, but by doing so, you are adding value for your customer.

So if you are value pricing, you can and should charge a premium for offering a fixed price and removing that risk for the customer.

A Minimum Price

Even knowing this, though, there’s little doubt that determining a value-derived price is difficult — especially so in these early days.

Despite a conscious effort to avoid it, it’s all too easy to fall back into thinking hours and rates.

To help overcome this, I plan to use time-based estimates to give myself a baseline price. This will simply show the minimum level that my value-derived price must match for the work to be viable.

Aside: I am part-way through Ron Baker’s excellent book, Implementing Value Pricing. Ron introduces the idea of a reservation price, which takes this concept much further.

If the value to the customer cannot exceed the time-based estimate, then the work won’t be taken on.

For the work we do take on, the customer gets a value-based, fixed price (based on output), and I can still monitor the costs (based on input) to the business.

• Pricing based on estimated time is just cost-plus based pricing. A price should instead be based on the agreed value to the customer.
• A fixed fee loads the risk onto the consultant, but this in turn represents added value to the customer.
• Value pricing should ignore time, but in these early days it’s difficult to escape the mindset of hourly rates. For now, I’ll use my time estimates as the minimum value that must be met to take on piece of work.

Have you made the switch to value pricing? How has it affected your own business and the way you operate? Are you worried about making the switch away from time billing? Let me know by getting in touch.

I was recently asked how an app's theme can be toggled by the user in Android. This is a nice feature that is often seen in reading apps.

I've always found Android's approach to theming overwhelmingly complex, but with the Android 5 Lollipop release and Design Support Library, Google have made great progress in simplifying things.

Recent builds of Android Studio have also included an early Theme Editor which can be used to edit and preview your app's themes and how it will affect various widgets.

In this tutorial, we'll build a simple app with a switch that lets the user toggle between dark and light themes.

The full source code for this tutorial is available at https://github.com/cblunt/blog-android-theme-toggler.

$ git clone git@github.com:cblunt/blog-android-theme-toggler.git

Create the project

Note: For this project, I'm using the latest Android Studio 2.0 beta release (preview2).

We'll start by creating a new application:

• Create a new project called ThemeToggler targetting API Level 15 (Ice Cream Sandwich). We'll use the Android Design Support library to support older devices running Android 4.

• Choose Blank Activity to create a basic Activity with a Floating Action button.

• Accept the defaults for the remaining screens to create your new application.

Build the UI

Once generated, we'll add a switch widget for the user to toggle the app's theme:

• Delete the Hello World label from the default layout that was created.
• Add a new Switch widget to the layout with the following attributes:

<!-- res/layout/content_main.xml -->
<Switch
  android:layout_width="wrap_content"
  android:layout_height="wrap_content"
  android:text="Toggle Theme"
  android:id="@+id/switch1"
  android:layout_alignParentTop="true"
  android:layout_alignParentLeft="true"
  android:layout_alignParentStart="true"
  android:layout_alignParentRight="true"
  android:layout_alignParentEnd="true" />

Next, we'll hook up our switch to toggle the application's theme:

• Add an OnCheckedChangeListener to the Switch in your Activity. The state of the Switch will be used to determine which theme is used:

// app/src/main/java/com/example/themetoggler/MainActivity.java
public class MainActivity extends AppCompatActivity {
  // ...
  protected void onCreate(Bundle savedInstanceState) {
    // ...
    Switch toggle = (Switch) findViewById(R.id.switch1);

    toggle.setOnCheckedChangeListener(new CompoundButton.OnCheckedChangeListener() {
      @Override
      public void onCheckedChanged(CompoundButton view, boolean isChecked) {
        toggleTheme(isChecked);
      }
    });
  }

Saving the User's Choice

When the theme is toggled, we'll need to restart the current Activity to use the chosen theme. This is similar to configuration changes in Android (e.g. screen rotation) where the system will destroy and recreate the current Activity.

We'll also need to store the user's chosen theme so the re-created Activity knows which theme to use. It makes sense to store this in the app's SharedPreferences so that the user's choice is persisted across application launches.

After storing the user's preference, we store a reference to the Activity's original Intent, finish the current activity and restart it using that original Intent:

• Add a new method, toggleTheme() to MainActivity to store the user's preference and restart the current Activity:

// app/src/main/java/com/example/themetoggler/MainActivity.java
public class MainActivity extends AppCompatActivity {
  private static final String PREFS_NAME = "prefs";
  private static final String PREF_DARK_THEME = "dark_theme";

  // ...

  private void toggleTheme(boolean darkTheme) {
    SharedPreferences.Editor editor = getSharedPreferences(PREFS_NAME, MODE_PRIVATE).edit();
    editor.putBoolean(PREF_DARK_THEME, darkTheme);
    editor.apply();

    Intent intent = getIntent();
    finish();

    startActivity(intent);
  }
}

Creating the Theme

Finally, we'll need to change the Activity's onCreate method to use the user's preferred theme.

However, we haven't created any alternative themes yet, so let's do that first.

Let's add an alternative AppTheme.Dark which extends the default Theme.AppCompat. We can use the original AppTheme that Android Studio created as the basis for our theme:

• Create AppTheme.Dark in styles.xml:

<!-- app/src/main/res/values/styles.xml -->
<style name="AppTheme.Dark" parent="Theme.AppCompat">
  <!-- Dark theme base colours -->
  <item name="colorPrimary">@color/darkColorPrimary</item>
  <item name="colorPrimaryDark">@color/darkColorPrimaryDark</item>
  <item name="colorAccent">@color/darkColorAccent</item>
</style>

• We'll also need to override some of the system themes, such as the NoActionBar theme. Use the existing AppTheme. styles as a base:

<!-- app/src/main/res/values/styles.xml -->
<style name="AppTheme.Dark.NoActionBar">
  <item name="windowActionBar">false</item>
  <item name="windowNoTitle">true</item>
</style>

<style name="AppTheme.Dark.AppBarOverlay" parent="ThemeOverlay.AppCompat.Dark.ActionBar" />
<style name="AppTheme.Dark.PopupOverlay" parent="ThemeOverlay.AppCompat" />

• I've also used declared colours in the theme. Put these in your colors.xml resource file:

<!-- app/src/main/res/values/colors.xml -->
<color name="darkColorPrimary">#1e2756</color>
<color name="darkColorPrimaryDark">#141831</color>
<color name="darkColorAccent">#2aac4b</color>

We've now declared enough for our themes to be used in the app. You can use Android Studio's experimental Theme Editor to preview and tweak your themes:

• Choose Tools->Android->Theme Editor
• In the Theme dropdown, choose your AppTheme and AppTheme.Dark themes to preview how different widgets will look:

Applying the theme

Finally, back in our code, we can instruct our Activity to use the selected theme in onCreate().

Note that the code must be at the top of onCreate() (before the call to super.onCreate()) so that the app's default theme (specified in AndroidManifest.xml) is overridden

// app/src/main/java/com/example/themetoggler/MainActivity.java
@Override
protected void onCreate(Bundle savedInstanceState) {
  // Use the chosen theme
  SharedPreferences preferences = getSharedPreferences(PREFS_NAME, MODE_PRIVATE);
  boolean useDarkTheme = preferences.getBoolean(PREF_DARK_THEME, false);

  if(useDarkTheme) {
    setTheme(R.style.AppTheme_Dark_NoActionBar);
  }

  super.onCreate(savedInstanceState);
  // ...

• Also in onCreate(), we'll update the switch to reflect the current choice (make sure this line goes before you add the onCheckedChangedListener to prevent an infinite loop:

// app/src/main/java/com/example/themetoggler/MainActivity.java
Switch toggle = (Switch) findViewById(R.id.switch1);
toggle.setChecked(useDarkTheme);
toggle.setOnCheckedChangeListener(new CompoundButton.OnCheckedChangeListener() {
// ...

Finally, run your app and tap the switch to toggle between your dark and light themes. With every toggle, the Activity will be restarted.

This tutorial introduces how you can give the user a choice of theme for your app. The code you added for applying theme (at the top of onCreate()) would be common across all the Activities in your app.

For this reason, it would be a good thing to extract into a BaseActivity from which all your app's Activities inherit.

I hope you've found this tutorial useful. Please let me know your thoughts and feedback in the comments below, or by getting in touch on Twitter.

Full Source Code: https://github.com/cblunt/blog-android-theme-toggler

TinyMCE lets you do this using the content_css option:

# app/assets/javascript/tinymce.coffee
tinymce.init {
  selector: ".tinymce",
  content_css: ["/assets/tinymce-content.css"],
  # ...
}

Unfortunately, whilst this is fine in development, where CSS and other assets are compiled on demand, when deploying to a server and running under production, the URL above becomes invalid, causing 404 errors and broken styles.

Linking to Precompiled Content CSS

To ensure the content CSS assets are compiled, we first need to add them to the precompiled assets list:

# config/initializers/assets.rb
Rails.application.config.assets.precompile += %w( tinymce-content.css )

Next, we need to ensure that the content_css option uses the compiled asset's URL. We can do this by using a snippet of ERB in our coffeescript file:

# app/assets/javascript/tinymce.coffee.erb
tinymce.init {
  selector: ".tinymce",
  content_css: ["<%= asset_url 'tinymce-content.css' %>"],
  # ...
}

Note that you also need to add the .erb extension to the filename.

With that, the correct compiled URL for tinymce-content.css will be inserted into your tinymce.init call, and the styles will be loaded correctly into the editor.

Testing it out in development

You can check the correct styles are loaded in development when precompiled by running:

$ bin/rake assets:precompile

Then restart your server and load the page with your editor.

Don't forget to remove the precompiled assets with:

$ bin/rake assets:clobber

This can be used, for example, in an html img tag to generate charts, icons and other dynamic graphics.

1. Register a new MIME type for the SVG response

# config/initializers/mime_types.rb
Mime::Type.register "image/svg+xml", :svg

2. Register a route to view your generated SVG

# config/routes.rb
get "icons/generate" => "icons#generate", format: :svg

3. Generate your SVG

Here I'm generating the SVG inline, but it could come from anywhere (e.g. model.as_svg).

Be sure to include the xml namespace (xmlns) in the svg tag, otherwise browsers just render the content as XML. This caught me out for a little while.

class IconsController < ApplicationController
  def generate
    svg = '<svg width="300px" height="300px" xmlns="http://www.w3.org/2000/svg">
<circle cx="50" cy="50" r="40" stroke="black" stroke-width="3" fill="red" />
</svg>'
  end
end

4. Add your Respond Block

In your controller, add a respond to block, render the SVG.

def generate
# ...svg = ...

  respond_to do |format|
    format.svg { render inline: svg}
  end
end

To view the generated SVG, start the server and visit http://localhost:3000/icons/generate.svg

In the previous tutorial, you learned how to use Volley's built-in networking tools to quickly download a network resource and display the results.

Volley is a networking library that removes lots of boiler-plate code when dealing with handling data across a network. It provides a number of helpful classes to simplify downloading and parsing data and images, and lets you concentrate on the details of your app rather than dealing with low-level networking technologies.

In this post, we'll start a new project and go further into Volley, using the built-in JSONObjectRequest and ImageRequest classes to make use of the remote data. We'll also use Volley's custom NetworkImageView widget to easily (and safely) load a remote image into our UI.

Create a new Android Studio Project

As in the previous tutorial, we'll use Android Studio to build the project. Once installed, create a new project for our app using the Android New Project setup:

→ Create a new project in Android Studio with the following settings:

• Open Android Studio and choose Start a New Android Studio Project
• Name your new app VolleyApp2 under the example.com company name (this will automatically set your package name to com.example.volleyapps2).
• Choose Phone and Tablet and API 21: Android 5.0 (Lollipop) for the Minimum SDK. This will automatically configure your app to use the Android Support Libraries to support older devices.
• Click Next and then Basic Activity. Later we'll add a ListView to browse the remote data.
• Name your activity ImagesActivity, and layout activity_images.
• Check Use a Fragment.
• Finally, click Finish to create your new project.

Once your project has been created and Gradle has updated, hit Build->Run (Ctrl+R) to test the new app. You should see the standard Hello World! message on your device or emulator.

Add Volley Dependency to your Gradle File

Volley is now officially available via Gradle from the Android Open Source Project (this tutorial previously used this mirror, which is now deprecated).

→ Add the Volley dependency to your module's build.gradle file:

// /app/build.gradle

// ...

dependencies {
  compile fileTree(dir: 'libs', include: ['*.jar'])
  compile 'com.android.support:appcompat-v7:25.3.1'
  // ...
  compile 'com.android.volley:volley:1.0.0'
}

Android Studio will notify you that it needs to Sync the Gradle config. This ensures that your project's configuration matches that of Gradle within Android Studio. Synchronising will also download the Volley as a dependency, making its classes available within our app.

If you don't see this notification, choose Build->Rebuild Project... to refresh and sync the project.

Create a Data Model

We'll set up a simple UI to hold our list of images and their titles. This will use the standard Adapter pattern that is common throughout Android.

Let's start by creating a data model to represent the downloaded data. When consuming a web service, you'll likely want to store a local cache of the downloaded data for using offline. For Android apps, this is likely to be a SQLite database to persist the database.

For this tutorial, though, we'll just cache the data - a list of image URLs and their titles - into an ArrayList of simple ImageRecord objects.

→ Create a new class, ImageRecord with fields for the image URL and title:

// /app/src/main/java/com/example/volleyapp2/ImageRecord.java

public class ImageRecord {
  private String url;
  private String title;

  public ImageRecord(String url, String title) {
    this.url = url;
    this.title = title;
  }

  public String getTitle() {
    return title;
  }

  public String getUrl() {
    return url;
  }
}

→ Create a new class, ImageRecordsAdapter:

This will hold our list of ImageRecords and adapt them to ImagesActivityFragment UI.

// /app/src/main/java/com/example/volleyapp2/ImageRecordsAdapter.java
public class ImageRecordsAdapter extends ArrayAdapter<ImageRecord> {
  public ImageRecordsAdapter(Context context) {
      super(context, R.layout.image_list_item);
  }
}

We haven't yet created the image_list_item layout. Android Studio will notice this and warn us to create the resource. Do that now using the following simple layout.

→ Create a new layout file for image_list_item:

<!-- /app/src/main/res/layout/image_list_item.xml -->
<?xml version="1.0" encoding="utf-8"?>

<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
              xmlns:tools="http://schemas.android.com/tools"
              android:orientation="horizontal"
              android:layout_width="match_parent"
              android:layout_height="wrap_content">

  <com.android.volley.toolbox.NetworkImageView
    android:layout_width="100dp"
    android:layout_height="100dp"
    android:scaleType="centerCrop"
    tools:src="@mipmap/ic_launcher"
    android:id="@+id/image1"/>

  <TextView
    android:padding="8dp"
    android:layout_width="0dp"
    android:layout_height="wrap_content"
    android:layout_weight="1"
    android:textAppearance="?android:textAppearanceLarge"
    tools:text="Image Name"
    android:id="@+id/text1"/>
</LinearLayout>

Notice the use of com.android.volley.toolbox.NetworkImageView instead of the normal ImageView widget. NetworkImageView is a special widget provided by Volley that adds a simple setImageUrl method.

Volley will automatically handle downloading remote images into the NetworkImageView for us, cancelling any requests in progress if the NetworkImageView is scrolled off the screen.

→ Add a new ImageRecordsAdapter field (mAdapter) inside ImagesActivityFragment:

You'll also initialise it in the fragment's onActivityCreated method.

// /app/src/main/java/com/example/volleyapp2/ImagesActivity.java

// ...
  public static class ImagesActivityFragment extends Fragment {
    private ImageRecordsAdapter mAdapter;

    @Override
    public void onActivityCreated(@Nullable Bundle savedInstanceState) {
      super.onActivityCreated(savedInstanceState);

      mAdapter = new ImageRecordsAdapter(getActivity());

      ListView listView = (ListView) getView().findViewById(R.id.list1);
      listView.setAdapter(mAdapter);
    }
  }

→ Open the fragment's layout and remove the placeholder TextView, replacing it with a ListView to display the list of images as they are loaded:

<!-- /app/src/main/res/layout/fragment_image.xml -->
<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"
  xmlns:tools="http://schemas.android.com/tools"
  android:layout_width="match_parent"
  android:layout_height="match_parent"
  tools:context=".ImagesActivity$PlaceholderFragment">

  <ListView
    android:id="@+id/list1"
    android:layout_width="match_parent"
    android:layout_height="match_parent" />
</RelativeLayout>

Running the app, you should now see a blank screen. It's time to fetch some remote data using Volley and populate our list!

Fetch a remote JSON feed using Volley

We'll use Volley in the same way as before to grab a JSON feed. This feed contains a list of images and their titles in the following format:

{
  'images': [
    {
      'url': "http://assets.example.com/image1.jpg",
      'title': "Sunset"
    },
    {
      'url': "http://assets.example.com/image2.jpg",
      'title': "Beach"
    },
    // ...
  ]
}

Start by extending the Android Application class and adding a global RequestQueue field (see the previous tutorial for more details on this).

→ Create a new VolleyApplication class:

// /app/src/main/java/com/example/volleyapp2/VolleyApplication.java

public class VolleyApplication extends Application {
    private static VolleyApplication sInstance;

    private RequestQueue mRequestQueue;

    @Override
    public void onCreate() {
        super.onCreate();

        mRequestQueue = Volley.newRequestQueue(this);

        sInstance = this;
    }

    public synchronized static VolleyApplication getInstance() {
        return sInstance;
    }

    public RequestQueue getRequestQueue() {
        return mRequestQueue;
    }
}

→ Set the new VolleyApplication class in your app's AndroidManifest.xml file:

Now is also a good time to add the android.permission.INTERNET permission, so our app can download the remote JSON file.

<!-- /app/main/AndroidManifest.xml -->
<!-- ... -->
  <uses-permission android:name="android.permission.INTERNET" />

  <application
    android:name=".VolleyApplication"
    ...>
  </application>

With an accessible, global RequestQueue ready to receive Volley requests, we can now add a new private method fetch to ImagesActivityFragment.

→ Add the following fetch() method:

// /app/src/main/java/com/example/volleyapp2/ImagesActivity.java

// ...

  public static class ImagesActivityFragment extends Fragment {
    // ...
    private void fetch() {
      JsonObjectRequest request = new JsonObjectRequest(
        "http://cblunt.github.io/blog-android-volley/response2.json",
        null,
        new Response.Listener<JSONObject>() {
          @Override
          public void onResponse(JSONObject jsonObject) {
            // TODO: Parse the JSON
          }
        },
        new Response.ErrorListener() {
          @Override
          public void onErrorResponse(VolleyError volleyError) {
            Toast.makeText(getActivity(), "Unable to fetch data: " + volleyError.getMessage(), Toast.LENGTH_SHORT).show();
          }
      });

      VolleyApplication.getInstance().getRequestQueue().add(request);
    }
  }

→ Add a call to fetch() at the end of onActivityCreated:

// /app/src/main/java/com/example/volleyapp2/ImagesActivity.java

// ...

  public static class ImagesActivityFragment extends Fragment {
    @Override
    public void onActivityCreated(@Nullable Bundle savedInstanceState) {
      mAdapter = new ImageRecordsAdapter(getActivity());

      ListView listView = (ListView) getView().findViewById(R.id.list1);
      listView.setAdapter(mAdapter);

      fetch();
    }

Parsing the Data

The above code sends a remote request (asychronously) via Volley and calls back with a response, either a successfully parsed JSONObject from the response, or an error (such as a 404).

In our onResponse handler, we'll forward the parsed JSONObject onto a new method, parse which will convert the JSON feed into a list of ImageRecords.

→ Add the parse() method to ImagesActivityFragment:

This method to loop through the array of images in the JSON feed, and create a new ImageRecord for each one.

// /app/src/main/java/com/example/volleyapp2/ImagesActivity.java

// ...

  public static class ImagesActivityFragment extends Fragment {
    // ...
    private List<ImageRecord> parse(JSONObject json) throws JSONException {
      ArrayList<ImageRecord> records = new ArrayList<>();

      JSONArray jsonImages = json.getJSONArray("images");

      for(int i =0; i < jsonImages.length(); i++) {
        JSONObject jsonImage = jsonImages.getJSONObject(i);
        String title = jsonImage.getString("title");
        String url = jsonImage.getString("url");

        ImageRecord record = new ImageRecord(url, title);
        records.add(record);
      }

      return records;
    }
  }

→ Update the onResponse() handler to call our new parse() method and update the ArrayAdapter:

// /app/src/main/java/com/example/volleyapp2/ImagesActivity.java

// ...

  public static class ImagesActivityFragment extends Fragment {
    // ...
    private void fetch() {
      JsonObjectRequest request = new JsonObjectRequest(
        "http://cblunt.github.io/blog-android-volley-2/response.json",
        null,
        new Response.Listener<JSONObject>() {
          @Override
          public void onResponse(JSONObject jsonObject) {
            try {
              List<ImageRecord> imageRecords = parse(jsonObject);

              mAdapter.swapImageRecords(imageRecords);
            }
            catch(JSONException e) {
              Toast.makeText(getActivity(), "Unable to parse data: " + e.getMessage(), Toast.LENGTH_SHORT).show();
            }
          }

          // ...
    }

Next, we need to add the swapImageRecords() method to ImageRecordsAdapter. This simply clears the existing data, adds all the new records, and notifies the adapter that the underlying data has been updated.

→ Add the ImageRecords() method:

// /app/src/main/java/com/example/volleyapp2/ImageRecordsAdapter.java

  public void swapImageRecords(List<ImageRecord> objects) {
    clear();

    for(ImageRecord object : objects) {
        add(object);
    }

    notifyDataSetChanged();
  }

Before the app can be run, we need to override ImageRecordAdapter's getView() method to specify how the data should be adapted to the UI.

→ Override getView and inflate the image_list_item layout:

// /app/src/main/java/com/example/volleyapp2/ImageRecordsAdapter.java

  @Override
  public View getView(int position, View convertView, ViewGroup parent) {
      if(convertView == null) {
          convertView = LayoutInflater.from(getContext()).inflate(R.layout.image_list_item, parent, false);
      }

      // NOTE: You would normally use the ViewHolder pattern here
      NetworkImageView imageView = (NetworkImageView) convertView.findViewById(R.id.image1);
      TextView textView = (TextView) convertView.findViewById(R.id.text1);

      ImageRecord imageRecord = getItem(position);

      imageView.setImageUrl(imageRecord.getUrl(), mImageLoader);
      textView.setText(imageRecord.getTitle());

      return convertView;
  }

Notice the use of setImageUrl to load an image directly into the NetworkImageView. This is provided by Volley.

Just as with the JSONObjectRequest, we need to let Volley know which RequestQueue should be used to manage the downloaded images.

For images, though, Volley provides another layer of abstraction - the ImageLoader class. This takes a RequestQueue and an object implementing the ImageLoader.ImageCache interface.

The ImageCache lets us store downloaded images in memory, only triggering a network download if image isn't present in the cache.

Caching

Unfortunately, Volley doesn't provide a built-in cache, so we'll need to write our own. Fortunately, this is relatively simple using the LruCache (least—recently used) class provided by Android.

→ Create a new class, BitmapLruCache, with the following code:

// /app/src/main/java/com/example/volleyapp2/BitmapLruCache.java

public class BitmapLruCache extends LruCache<String, Bitmap>
    implements ImageLoader.ImageCache {

  static int getMaxCacheSize() {
      int maxMemory = (int) (Runtime.getRuntime().maxMemory() / 1024);
      return maxMemory / 8;
  }

  public BitmapLruCache() {
    super(getMaxCacheSize());
  }

  @Override
  protected int sizeOf(String key, Bitmap value) {
    return value.getRowBytes() * value.getHeight();
  }

  @Override
  public Bitmap getBitmap(String url) {
    return get(url);
  }

  @Override
  public void putBitmap(String url, Bitmap bitmap) {
    put(url, bitmap);
  }
}

Disk-Level Caching

With a simple in-memory cache ready to use, we can now create an new instance of ImageLoader for our adapter to use.

→ Add the following code to create an instance of ImageLoader in the adapter's constructor:

// /app/src/main/java/com/example/volleyapp2/ImageRecordsAdapter.java

public class ImageRecordsAdapter extends ArrayAdapter<ImageRecord> {
  private final ImageLoader mImageLoader;

  public ImageRecordsAdapter(Context context) {
    super(context, R.layout.image_list_item);

    mImageLoader = new ImageLoader(VolleyApplication.getInstance().getRequestQueue(), new BitmapLruCache());
  }

  // ...
}

With that, your app is now ready to download and parse downloaded data, and any external images in the feed. The NetworkImageView (and ImageLoader) will automatically retrieve and display the images.

→ Run your app, and you will be presented with the results of the parsed feed:

Consuming Remote JSON APIs

You have successfully used Volley to consume a simple JSON API. Although there's lots more that can be done (e.g. disk caching, locally caching and synchronising the retrieved data using SQLite, etc.), this tutorial has shown the core of what needs to be done.

Complementing Volley with Picasso

Although Volley provides some great helpers for downloading, caching and displaying images from the network, it can be a little low-level sometimes. I've recently been using Square's Picasso library to handle the image downloading side of things.

Not only does Picasso provide a very nice API, it handles disk and memory based caching without any extra configuration or effort from you - and provides some nice built-in transition effects.

Picasso is a topic for another post, but you can find out more at http://square.github.io/picasso/.

Next Steps

I hope you've found this tutorial helpful for learning about Android's powerful new networking framework. Using Volley to handle your network requests removes vast swathes of boiler-plate code and edge-case tests which means you can concentrate on developing your app's valuable features!

Clone the code on github

In this post, you'll learn how to use Volley to perform a basic download of remote data. This will form the basis of consuming a simple RESTful API.

Clone the code on github

A Note on Android Studio

Whilst it's still very much in Beta, I'm a fan of Android Studio and the new Gradle build system for Android. I use it for all day-to-day Android work without too many issues. In this tutorial, I'll be using Android Studio & Gradle.

If you haven't tried it yet, you can download Android Studio from android.com. At the time of writing, Android Studio is at version 0.5.7.

Creating the project

With Android Studio running, create a new Blank Activity project. After naming your project, using the default settings will be fine:

Add the Volley Library

When your app project is created, we can add Volley to the list of dependencies via Gradle. Somewhat strangely, Google don't provide a build of Volley for Gradle, but thankfully there are mirrored copies of Volley published for use in Gradle.

We can add one of these mirrors from https://github.com/mcxiaoke/android-volley:

// /app/build.gradle
// ...

dependencies {
  // ...
  compile 'com.mcxiaoke.volley:library:1.0.+'
}

Select Tools > Android > Sync Project with Gradle Files (or hit the toolbar icon) to update your project and expose the Volley classes to your code.

Create a RequestQueue

With Volley in our project, we can start using it to download remote content. Volley uses the concept of RequestQueue to manage requests for content and download them. We can create a global request queue for our application in a customer Application subclass.

Create a new VolleyApplication class with a RequestQueue field. We'll initialise the RequestQueue in the application's onCreate method, and keep a static instance of VolleyApplication to let us access the queue from anywhere in our app:

// src/main/java/com/example/volleyapp/app/VolleyApplication.java
// ...

class VolleyApplication extends Application {

  private static VolleyApplication sInstance;

  private RequestQueue mRequestQueue;

  @Override
  public void onCreate() {
    super.onCreate();

    mRequestQueue = Volley.newRequestQueue(this);

    sInstance = this;
  }

  public synchronized static VolleyApplication getInstance() {
    return sInstance;
  }

  public RequestQueue getRequestQueue() {
    return mRequestQueue;
  }
}

Be sure to update your AndroidManifest.xml to specify VolleyApplication as the application's class. We'll also need to request the INTERNET permission:

<?xml version="1.0" encoding="utf-8"?>

<manifest xmlns:android="http://schemas.android.com/apk/res/android"
    package="com.example.volleyapp.app">
  <uses-permission android:name="android.permission.INTERNET" />

  <application
    android:name=".VolleyApplication"
    ...>
    ...
  </application>
</manifest>

Download a JSON Feed

Now we can create a request and pass it to Volley. As well as plain text responses, Volley provides some helper classes to download and parse JSON, which is great when consuming APIs:

// src/main/java/com/example/volleyapp/app/MainActivity.java
public class MainActivity extends ActionBarActivity {
  private TextView mTextView;

  @Override
  protected void onCreate(Bundle savedInstanceState) {
      super.onCreate(savedInstanceState);
      setContentView(R.layout.activity_main);

      mTextView = (TextView) findViewById(R.id.text1);

      JsonObjectRequest request = new JsonObjectRequest("http://cblunt.github.io/blog-android-volley/response.json", null,
                                                        new Response.Listener<JSONObject>() {

                                                            @Override
                                                            public void onResponse(JSONObject response) {

                                                                mTextView.setText(response.toString());
                                                            }
                                                        },

                                                        new Response.ErrorListener() {

                                                            @Override
                                                            public void onErrorResponse(VolleyError error) {
                                                                mTextView.setText(error.toString());
                                                            }
                                                        }
      );
      VolleyApplication.getInstance().getRequestQueue().add(request);
  }

In this example, we're fetching the remote and simply displaying it in a TextView (remember to set the TextView id in your activity_main.xml layout file). If something goes wrong, the ErrorListener will display the error message in the TextView.

The second parameter of the JsonObjectRequest is an optional JSONObject to be sent with the request. If set, the request will be submitted as an HTTP POST request, otherwise it will be a GET request. Again, this is helpful if we're dealing with a RESTful API.

Finally, with the request created, we can add it to the app's RequestQueue for Volley to handle. Run your app to see the returned JSON response:

Fast, Easy Networking for Android

With Volley, performing networking takes no time at all. In upcoming posts we'll look at downloading images using Volley's built-in ImageRequest, and finally put everything together to consume a simple JSON API.

Clone the code on github

My early experience of Gradle seemed to cause more problems than it solved. However, this was probably due to lack of understanding as much as the early status of the tools. As well as taking the time to learn more about Gradle, recent releases of the toolkit also seem far more stable and integrated with the Android Studio/IDEA IDE.

This post is a summary of how I've solved various tasks with Gradle and what I've learned so far. Hopefully it will help guide you if you're getting started with Gradle, and serve as a handy reminder. These instructions are based on using Android Studio 0.4:

Including an external Jar

1. Create a libs folder under your project's top-level folder.
2. Copy the .jar file into the new libs folder, or a sub-folder.
3. In your app's build.gradle, add the jar as a file dependency:

// /MyProject/my-app/build.gradle

buildscript {
    repositories {
        mavenCentral()
    }

    dependencies {
        classpath 'com.android.tools.build:gradle:0.8.+'
    }
}

apply plugin: 'android'
repositories {

    mavenCentral()

    flatDir {
        dirs 'libs'
    }
}

android {
  // ...
}

dependencies {
  compile files('libs/somelibrary.jar')
}

Including an external library module

• Add a new module at the top level of your project (e.g. MyProject/my-library)
• Inside your project's settings.gradle, create a reference to the module. The reference is a path to your module relative to the Project root. Note that you use : as a path seperator rather than /:

// MyProject/settings.gradle

include ':my-library'

• In your app's build.gradle, add the external library as a project dependency:

// /MyProject/MyApp/build.gradle

android {
// ...
}

dependencies {
compile project(':my-library')
}

Including Google Play Services

Google Play Services is increasingly becoming the only way to interact with Google's APIs (this is a good thing, as it will hopefully help to reduce platform fragmentation). The Google Play Services are now provided through a local repository.

• Ensure that the Google Play Services repository is added in the Android SDK Manager:2. Add the dependencies to your build.gradle file corresponding to the services you want to use:

// /MyProject/MyApp/build.gradle

android {
  // ...
}
dependencies {
  compile 'com.google.android.gms:play-services:3.2.+'
}

The same is also true of the Android Support Libraries which are provided through a separate local repository:

// /MyProject/MyApp/build.gradle
dependencies {
  compile 'com.android.support:support-v4:19.0.0'
  compile 'com.android.support:gridlayout-v7:19.0.0'
  compile 'com.android.support:appcompat-v7:19.0.0'
  // ...
}

Product Flavours and Build Types

In all the documentation, I couldn't find a explanation of the difference between a product flavour and a build type.

Put simply, a product flavour is a different edition your app that you might release on Google Play, the most common example being a free and paid edition of your app. I've not really used these much, but one thing you can do is configure a different package name for your app depending on which flavour you use, e.g:

When the app is compiled, it will use the appropriate package name according to the product flavour your have built.

A build type is an environment setting for your app. By default, a debug (development) and release (production) build type are set up. I usually add a third staging build type.

Build types let you set environment. Specific values using the build config object. This is great for setting things like ContentProvider authorities and API endpoints:

// /MyProject/MyApp/build.gradle
android {

  buildTypes {
    debug {
      buildConfigField "String", "PROVIDER_AUTHORITY", "\"com.example.app.debug.provider\""
      buildConfigField "String", "API_ENDPOINT", "\"http://development.localhost/\""
    }

    staging {
      buildConfigField "String", "PROVIDER_AUTHORITY", "\"com.example.app.beta.provider\""
      buildConfigField "String", "API_ENDPOINT", "\"http://beta.example.com/api/\""
    }

    release {
      buildConfigField "String", "PROVIDER_AUTHORITY", "\"com.example.app.provider\""
      buildConfigField "String", "API_ENDPOINT", "\"http://api.example.com/api/\""
    }
  }
}

With product flavours and build types, you could generate a debugFree apk of your app for testing, and a releasePaid apk for distribution on the Play Store.

Keeping release signing configuration out of source control

The keystore and passwords for signing your release jars is now also configured through the Gradle build script. Like any sensitive data, though, you should ensure this is never published to your code repository.

Instead, using the code from this gist lets you keep your signing data in an external file which Gradle loads at compile time. I add a signing.properties.example file to my repository to server as a reminder to set the correct values:

// /MyProject/MyApp/build.gradle

android {
  // ...
}
dependencies {
  // ...
}

def Properties props = new Properties()
def propFile = new File('signing.properties')

if (propFile.canRead()) {
    props.load(new FileInputStream(propFile))

    if (props != null && props.containsKey('STORE_FILE') && props.containsKey('STORE_PASSWORD') &&
            props.containsKey('KEY_ALIAS') && props.containsKey('KEY_PASSWORD')) {
        android.signingConfigs.release.storeFile = file(props['STORE_FILE'])
        android.signingConfigs.release.storePassword = props['STORE_PASSWORD']
        android.signingConfigs.release.keyAlias = props['KEY_ALIAS']
        android.signingConfigs.release.keyPassword = props['KEY_PASSWORD']
    } else {
        println 'signing.properties found but some entries are missing'
        android.buildTypes.release.signingConfig = null
    }
} else {
    println 'signing.properties not found'
    android.buildTypes.release.signingConfig = null
}

# signing.properties.example

STORE_FILE=/path/to/your.keystore
STORE_PASSWORD=yourkeystorepass
KEY_ALIAS=projectkeyalias
KEY_PASSWORD=keyaliaspassword

Android provides powerful database functionality built on SQLite. If you're familiar with SQL from web development, you'll be very comfortable with SQLite. However, what you might find is that it's very easy to deal directly with the database (especially if you're coming from a framework like Rails or Symfony).

Whatever language or platform you're using, it's always good practice to build an abstraction layer between your app's code and its database. In this tutorial miniseries, we'll walk through building a simple app that uses a database abstraction class, MyDatabaseHelper to manage querying, inserting, updating and deleting records from a simple database. In future posts, we'll look at dealing with multiple tables and joins, and finally explore Android's own ContentProvider API to abstract our database in a standard way.

Getting started

Start with a new Android application project (I recommend trying out the new Android Studio preview, especially if you're just starting out with Android), and add the following files:

// src/com/example/MyActivity.java

public class MyActivity extends Activity {
  private MyDatabaseHelper mDatabaseHelper;

  public void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    mDatabaseHelper = new MyDatabaseHelper(this);
  }
}

// src/com/example/MyDatabaseHelper.java

public class MyDatabaseHelper extends SQLiteOpenHelper {
  public static final String TABLE_USERS = "users";
  public static final String COL_ID = BaseColumns._ID;
  public static final String COL_NAME = "name";
  public static final String COL_EMAIL = "email";
  public static final String COL_DOB = "date_of_birth";
  private static final String DATABASE_NAME = "my_app.db";
  private static final int DATABASE_VERSION = 1;

  public MyDatabaseHelper(Context context) {
    super(context, DATABASE_NAME, null, DATABASE_VERSION);
  }

  @Override
  public void onCreate(SQLiteDatabase db) {
    db.execSQL("CREATE TABLE " + TABLE_USERS + " ("
        + COL_ID + " INTEGER PRIMARY KEY AUTOINCREMENT,"
        + COL_NAME + " TEXT NOT NULL,"
        + COL_EMAIL + " TEXT,"
        + COL_DOB + " INTEGER"
        + ");");
  }

  @Override
  public void onUpgrade(SQLiteDatabase db, int oldVersion, int newVersion) {
    db.execSQL("DROP TABLE IF EXISTS " + TABLE_USERS + ";");
    onCreate(db);
  }

  public long insert(String tableName, ContentValues values) {
    return getWritableDatabase().insert(tableName, null, values);
  }

  public int update(String tableName, long id, ContentValues values) {
    String selection = COL_ID + " = ?";
    String[] selectionArgs = {String.valueOf(id)};

    return getWritableDatabase().update(tableName, values, selection, selectionArgs);
  }

  public int delete(String tableName, long id) {
    String selection = COL_ID + " = ?";
    String[] selectionArgs = {String.valueOf(id)};

    return getWritableDatabase().delete(tableName, selection, selectionArgs);
  }
}

The advantage of this approach is that MyDatabaseHelper becomes a single point of access to the raw database (following good DRY principles). It also provides the opportunity to validate data before it is written to the database. For example, we have added a NOT NULL constraint on the name column. If we tried to save a record to the database without setting a value for name, our app would crash with a SQLiteConstraintException. Let's demonstrate this by adding a few users in our Activity, but leaving one of the names blank:

// src/com/example/MyActivity.java

  public void onCreate(Bundle savedInstanceState)
    // ...
    addUser(null, null, 0);
    addUser("Joe User", "joe@example.com", 0);
    addUser("Mary Jones", "mary@example.com", 0);
    addUser("Sue Bloggs", "sue@example.com", 0);
  }

 private void addUser(String name, String email, long dateOfBirthMillis) {
    ContentValues values = new ContentValues();
    values.put(MyDatabaseHelper.COL_NAME, name);

    if (email != null) {
      values.put(MyDatabaseHelper.COL_EMAIL, email);
    }

    if (dateOfBirthMillis != 0) {
      values.put(MyDatabaseHelper.COL_DOB, dateOfBirthMillis);
    }

    mDatabaseHelper.insert(MyDatabaseHelper.TABLE_USERS, values);
  }

Running this code, your app will crash with the SQLiteConstraintException we expected:

  ERROR/SQLiteDatabase(19185): Error inserting name=null

      android.database.sqlite.SQLiteConstraintException: users.name may not be NULL (code 19)
      at android.database.sqlite.SQLiteConnection.nativeExecuteForLastInsertedRowId(Native Method)
      at android.database.sqlite.SQLiteConnection.executeForLastInsertedRowId(SQLiteConnection.java:775)
      at android.database.sqlite.SQLiteSession.executeForLastInsertedRowId(SQLiteSession.java:788)
      ....

Adding Validations

Using our abstract Database helper class though, we can protect against this and handle the error much more elegantly. To do this, we'll add some simple validations at the appropriate points in MyDatabaseHelper:

// src/com/example/MyDatabaseHelper.java
  public long insert(String tableName, ContentValues values) throws NotValidException {
    validate(values);
    return getWritableDatabase().insert(tableName, null, values);
  }

  public int update(String tableName, long id, ContentValues values) throws NotValidException {
    validate(values);
    String selection = COL_ID + " = ?";
    String[] selectionArgs = {String.valueOf(id)};

    return getWritableDatabase().update(tableName, values, selection, selectionArgs);
  }

  // ...

  protected void validate(ContentValues values) throws NotValidException {
    if (!values.containsKey(COL_NAME) || values.getAsString(COL_NAME) == null || values.getAsString(COL_NAME).isEmpty()) {
      throw new NotValidException("User name must be set");
    }
  }

  public static class NotValidException extends Throwable {
    public NotValidException(String msg) {
      super(msg);
    }
  }

Back in MyActivity, wrap the call to insert() in a try...catch block to capture the NotValidException and show a message to the user:

// src/com/example/MyActivity.java
  private void addUser(String name, String email, long dateOfBirthMillis) {
    // ...
    try {
      mDatabaseHelper.insert(MyDatabaseHelper.TABLE_USERS, values);
    } catch (MyDatabaseHelper.NotValidException e) {
      Toast.makeText(this, e.getMessage(), Toast.LENGTH_SHORT).show();
    }
  }

Now if you run your app, it will gracefully handle the error, and continue adding the other user records to the local database.

Querying and Displaying Records

At the moment, we can't see our data though. We'll add a quick ListView to MyActivity to show the list of user's. If one doesn't already exist, create a new layout resource file in your project: res/layouts/my_activity.xml and add a ListView to fill the screen:

// res/layout/my_activity.xml

<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
              android:orientation="vertical"
              android:layout_width="match_parent"
              android:layout_height="match_parent">
    <ListView
        android:layout_width="fill_parent"
        android:layout_height="match_parent"
        android:id="@+id/listView"/>
</LinearLayout>

In MyActivity, make sure that you set the content view to be the new my_activity.xml layout:

// src/com/example/MyActivity.java

  public void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);

    setContentView(R.layout.my_activity);
    // ...
  }

Next, create a SimpleCursorAdapter to retrieve data from the database and display it in our new ListView:

// src/com/example/MyActivity.java

  public void onCreate(Bundle savedInstanceState) {
    // ...
    Cursor c = mDatabaseHelper.query(MyDatabaseHelper.TABLE_USERS, MyDatabaseHelper.COL_NAME);

    String[] from = new String[]{MyDatabaseHelper.COL_NAME, MyDatabaseHelper.COL_EMAIL};
    int[] to = { android.R.id.text1, android.R.id.text2 };

    SimpleCursorAdapter adapter = new SimpleCursorAdapter(this, android.R.layout.simple_list_item_2, c, from, to, 0);

    ListView listView = (ListView) findViewById(R.id.listView);
    listView.setAdapter(adapter);
    // ...
  }

Finally, you'll see that we've called a new method, query(), on our database helper. The parameters specify the table to query, and how to sort results. This will provide a simple way to retrieve the records from our database, wrapping the SQLiteOpenHelper's own query() method. Add the new method to MyDatabaseHelper:

// src/com/example/MyDatabaseHelper.java

  public Cursor query(String tableName, String orderedBy) {
    String[] projection = {COL_ID, COL_NAME, COL_EMAIL, COL_DOB};

    return getReadableDatabase().query(tableName, projection, null, null, null, null, orderedBy);
  }

Of course, we could add other parameters such as search terms to our query(), making it very powerful - and again providing a single point of contact for querying our database.

Running the code, you'll now see the list of names and email addresses as people are added to the database. Note that every time you re-run the app, another set of records will be added as the calls to addUser() are in the activity's onCreate() method.

As you can see, it's relatively easy to build a powerful abstraction layer between your database and application code, and there are huge advantages in doing so. As your app grows, you'll start to build a library of common methods that will save you time and make your code more robust.

Coming Next

In the next tutorial, we'll add another table to our database and adapt our database helper class to handle multiple tables and joins. In the final part, we'll look at Android's ContentProvider API, which provide a standard way to abstracting and manage access to your your app's raw database.

By declaring stored attributes on your model, ActiveRecord will automatically generate the appropriate setter/accessor methods, and validations work just as you'd expect.

# db/migrate/create_cars.rb

# ...

create_table do |t|

  t.references :model

  t.references :manufacturer

  t.text :metadata # Note metadata is just a text column

  t.timestamps

end

# app/models/car.rb

class Car < ActiveRecord::Base

  belongs_to :model

  belongs_to :manufacturer

  # Manufacturer and Model are 'real', database-backed attributes

  attr_accessor :model_id, :manufacturer_id, :colour, :size, :notes, :product_url

  store :metadata, :accessors => [:colour, :size, :notes, :product_url]

  # Database-backed attributes

  validates :model, :presence, :presence => true

  validates :manufacturer, :presence => true

  # Metadata stored attributes

  validates :colour, :presence => true

  validates :size, :presence => true, :inclusion => { :in => %w(small medium large) }

  validates :product_url, :format => { :with => /^(https?:\/\/)?([\da-z\.-]+)\.([a-z\.]{2,6})([\/\w \.-]*)*\/?$/ }

end

Note that the metadata is just a text column. Rails will automatically serialise the hash into this column.

With our Car model defined, and its metadata attributes declared with store, we can use them in just the same way as normal, database-backed attributes:

car = Car.new

car.model = CarModel.first

car.manufacturer = Manufacturer.first

car.colour = "Red"

car.size = "medium"

car.product_url = "http://www.example.com/"

car.save

# => true

car.colour

# => "red"

car.size

# => "medium"

Pros and Cons

The biggest advantage to this type is store is it provides a flexible data schema. This is perfect for storing non-indexed metadata about a model, and a use-case that often crops up when building apps. It provides a compromise between common relational databases (such as MySQL and PostgreSQL) and the flexibility of NoSQL databases such as MongoDB and CouchDB. In practical use, it means attributes can quickly be added to a model without the need to perform any migrations on the database schema.

A possible disadvantage is that stored attributes can not be indexed or used in queries (as they have no corresponding database column). For example, you could not call Car.where(:colour => "red"). However, as Garry Tan points out in his post, if this becomes a requirement in the future, you could always add a database-column to the schema at a later date, and just move the data at that time.

References

1. http://axonflux.com/one-of-my-favorite-additions-to-rails-3-activ

In this post, I'll document a brief summary of what's happened, and offer my thoughts and goals for the new year.

The Past Year

In January, I moved Plymouth Software from the incubating LaunchPad to its own offices on Tamar Science Park Scott Building. The business has grown its client base and income over the year, and whilst by no means enough to let me take a mini-retirement yet, feels a lot more solid than 12 months ago.

As well as winning several new clients (commercial, academic and charitable), I also expanded the company's service offer to include development of native iOS applications, and I am currently engaged developing an iPhone app due for release in February 2013.

In June, I ran the Plymouth Half Marathon, shaving a couple of minutes off my 2011 time. Thanks to the generosity of friends, family, colleagues, and everyone else who donated, I managed to raise over £600 for Macmillan Cancer Support.

We travelled to Spain, Jersey, Scotland, several cities in Northern England, lots of spots around Devon and Cornwall; and of course, drank plenty of tea.

Of course, the highlight of the year occurred in August, when Claire and I celebrated the arrival of our baby daughter. Becoming a Dad really puts a new perspective on things, which has inevitably changed my thinking for what I want to achieve in the next 12 months.

2013: The Goals

Produce More, Consume Less.

With our amazing new addition, running and building Plymouth Software, and generally trying to balance life and business, my time seems to have become incredibly limited. This is evident by the sheer lack of posts here. I realise though that this is just an excuse - it's all too easy to be suckered into the endless river of social media streams, news feeds, blog , etc. and procrastinatory opportunities that present themselves when you run your own business. My first goal then is to focus on producing rather than consuming. The Internet is endless in its ability to offer more, so it's better to try to output (quality) content rather than read it.

Goal: Publish at least one relatively in-depth blog post to this site each month. Write content for products (see goals below).

Focus

Lack of focus really hit me this year when I realised just how many task and project management apps I'd signed up for in an effort to become more organised. The time spent trying out what are essentially list apps could have been put to far better use. Again, the endless distraction of the Internet was the cause, offering an escape from dealing with code problems and producing.

Back in 2011, I found the Pomodoro technique incredibly useful (using the excellent Pomodroido app), but for some reason I stopped. The resulting lack of focus has led to frustration.

Goal: Start using the Pomorodo technique again to focus on work that_ generates value_. I've also signed up to Freckle for the quickest way to see where my time is being spent.

Sidenote: For task management, I settled on Wunderlist. With the release of version 2, they've added the absolutely vital Reminders and Repitition to tasks, which were essential. Wunderlist is also multi-platform, and now has a nice native Android app.

Balance

As part of developing Plymouth Software's business model, one of my aims is to ensure a good work-life balance. As more client work has come in and deadlines approach, I've felt more of a drag to do just a few minutes in evenings and weekends.

Goal: Hit the reset button to keep work at the office. Establish business hours with myself and clients, and stick to them.

Sidenote: According to others' experiences, reducing time available to work talks should actually help to improve my focus and efficiency.

Products

When it's your own, I don't believe selling time is a viable way to build a sustainable business - irrespective of your rate. In order to grow Plymouth Software financially, it needs to a fundamental change away from the freelance model (selling my time) to a business model, sustainable through the sale products, or services which can be packaged as products, that begin to remove my time from the equation.

This has been the aim since day one. I've tried to figure this out before, but could never make the break away from selling time. The freelance model also provided the quickest way to generate some income and build capital, which is essential for a bootstrapped business. Now, though, in order to grow the company (both financially and taking on staff), this approach needs to change.

Reading about very talented people in similar situations have made the switch inspired me to get thinking, and taking Brennan Dunn's excellent Consultancy Masterclass opened my eyes to a new way of productising my services, as well as offering products that are not a traditional part of the business, such as eBooks, code plugins, and workshops.

Goal: Schedule time and funds to develop revenue-generating assets (more details on the first of those in the next post).

Finance

To my mind, self-employment - having the ability to quickly adapt to situations and generate new streams of income - has always been far more stable than being an employee, where your income is firmly in the hands of somebody else. The real advantage of working for yourself is the freedom to be in control of my own future, as well as the sheer enjoyment that comes with building up a business.

Plymouth Software has so far granted an income and turned a profit in Year 1. However, in 2013, with lots of new responsibilities, I would like to at least double the turnover that the company has made in its first year (as a limited company).

This should be easily achievable by fulfilling some of the goals above, as well as following in the footsteps of bigger consultancy firms from whom I take inspiration. For example, I hadn't come across weekly billing until taking Brennan's Consultancy Masterclass course, but it seems to be quite common in software development shops, particularly those practicing agile techniques.

Goal: Double the company's turnover for Year 2. Acquire at least 4 retainer-style contracts for clients. Set value-based standard rates, and move to a weekly billing model for project/consultancy jobs.

Brand / Marketing

I'll be the first to admit the business must improve its marketing output. The company website is functional, but the portfolio is already outdated, and doesn't much promote the core service offering of the company. Along with changes the underlying business model above, I'd like to take the opportunity to refine the company brand itself.

One thing I've recognised is that the business actually offers clients much more than simple code-and-deliver development services. As I've taken on more involved projects, I've found myself consulting on a range of subjects, from business development and marketing, through to experience design and technical environment considerations.

As I move the company away from a freelance model, I need to ensure that the brand reflects the full scope of what the company offers.

Goal: Reduce friction of updating company website by moving to more flexible content management system. Rewrite marketing copy to focus on full range of services offered. Send at least one email newsletter per month. Write one white-paper on mobile apps, to be distributed though the company web site.

Fitness

In both 2011 and 2012 I ran the Plymouth Half Marathon, and the fitness that came with running was great. However, with no running goal for 2013 - and a lot of sleepless nights since August - my running has taken a back seat. In 2013, I'd like to rectify this by committing to get back into running, as well as taking advantage of January membership discounts for the local gym and swimming pool.

Goal: Get back into running at least once a week, minimum of 3 miles per run (road or gym). Gym or Swim session twice a week.

Final Thoughts - The Overall Goal

Hopefully, as we move to 2014, I'll be able to look back over this post and check off all those goals.

I'm sure though, like any plan, they will change a little throughout the course of 2013, but it feels good to have committed to writing where I want to be. Ultimately, the most important goal (and that which has influenced all others) is to provide and spend as much quality time with our daughter as she grows up.

2013 is going to be a great - Happy New Year!

In memory of a close friend, Pat Mourton, who sadly passed away from cancer in August 2011, I am running the Plymouth Half Marathon again on the 3 June 2012. My goal is to raise at least £1k for Macmillan Cancer Support, who gave incredible and ongoing support to both Pat and her family throughout her battles with cancer.

You can be part of helping me to reach (or beat!) my target, and raise as much as possible for Macmillan by donating on my Just Giving page at http://www.justgiving.com/chrisblunt-plymouth2012.

If you can't donate today, then please help spread the word by posting the link on your favourite social networks.

Thank you!

In 1996, Pat Mourton was diagnosed with breast cancer. I was lucky enough to meet Pat through her son Richard and husband Alan, during my first days at secondary school. They quickly became very close friends.

Pat was well known for her work in the local community, and everyone who was fortunate enough to meet her couldn’t help but be inspired by her spirit, determination, and (perhaps most of all) her unfaltering sense of humour.

In the nearly 2 decades she lived with the disease, Pat fought and beat the breast cancer, and bravely endured bone cancer and later metastatic liver cancer. She ran countless fundraising events for the charities that would become close to her heart.

Sadly, Pat lost her battle with cancer on 15 August 2011.

Macmillan nurses provide exceptional support and care not only to cancer sufferers, but their families who face this horrible disease. Macmillan nurses truly went above-and-beyond the call of duty for Pat, Alan and Richard.

In memory of Pat, and to help Macmillan carry on this tireless work, I’m running the Plymouth Half Marathon for a second time on 3 June 2012 and need to raise £1000 for Macmillan.

Please donate whatever you can to help me reach - and smash - my target of raising £1000 for Macmillan Cancer Support so that other people can continue to receive the support that was so important to Pat, Alan and Richard.

Please take a moment to send this page to everyone you know to help support those who are suffering with cancer.

Thank you,

Chris