CoderJourney

Kubernetes Frontend Service with Nginx

keith@coderjourney.com (Keith Thompson) — Wed, 12 Apr 2017 02:30:15 +0000

With both Rails and Postgres running smoothly in your Kubernetes cluster it’s time for you to create a frontend service to handle the web requests. In this tutorial, we’ll walk through what it looks like for us to set up Nginx as the frontend service for our Rails application.

Goals:

Create Nginx Deployment/Service in front of Rails
Understand why a separate frontend vs backend is good.

If you went through the Ruby on Rails series then you’ve already worked a little with having Nginx in front of your Rails application, but today we’re going to translate that over to working with Kubernetes.

Before we get started, if you’ve stoped and restarted minikube and you attempt to connect to the mealplan service in your browser you’ll be met with this error:

An unhandled low-level error occurred. The application logs may have details.

Not the way we wanted to start this tutorial.

Fixing the Postgres Persistence Layer for Minikube

This is caused by an oversite when we created the PersistentVolumeClaim in the previous tutorial. Our database was removed sadly. The reason that we lost our database after spinning down the minikube was because we mapped our PersistentVolumeClaim to the wrong directory in our postgres container. We pointed to /var/lib/postgresql/db-data, but the PGDATA directory is actually at /var/lib/postgresql/data. Let’s delete what currently exists so that we can patch it up:

$ kubectl delete -f deployments/postgres.yml
service "postgres" deleted
deployment "postgres" deleted
persistentvolumeclaim "postgres-pv-claim" deleted

We removed them entirely since we working locally, obviously, if we made this sort of mistake in a non-development set up we would probably do something more graceful. In addition to mapping our directory incorrectly there is also an issue with where we’re storing the data on the minikube VM. By default, new PersistentVolumes with minikube store their data in /tmp/hostpath-provider and it isn’t supposed to be removed, but at the time of writing this it was being removed when we run minikube stop. Because of this issue we’re going to manually create a PersistentVolume that stores the data under /data/ which does indeed persist between minikube starts.

deployments/postgres.yml

# Skipping the Service portion of the file
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: postgres
spec:
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
        - image: "postgres:9.6.2"
          name: postgres
          env:
            - name: POSTGRES_USER
              valueFrom:
                configMapKeyRef:
                  name: mealplan-config
                  key: postgres_user
            - name: POSTGRES_PASSWORD
              valueFrom:
                configMapKeyRef:
                  name: mealplan-config
                  key: postgres_password
          ports:
            - containerPort: 5432
              name: postgres
          volumeMounts:
            - name: postgres-storage
              mountPath: /var/lib/postgresql/data
      volumes:
        - name: postgres-storage
          persistentVolumeClaim:
            claimName: postgres-pv-claim
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: postgres-pv-claim
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi
  volumeName: postgres-pv
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: postgres-pv
spec:
  accessModes:
    - ReadWriteOnce
  capacity:
    storage: 5Gi
  hostPath:
    path: /data/mealplan-postgres

Now we can bring up all of these objects and recreate our database (your pod ID will be different):

$ kubectl create -f deployments/postgres.yml
service "postgres" created
deployment "postgres" created
persistentvolumeclaim "postgres-pv-claim" created
persistentvolume "postgres-pv" created
$ kubectl exec mealplan-4029866644-1srx8 --stdin --tty -- bundle exec rake db:setup db:migrate

Now the database is back and we don’t have to worry about it getting deleted again.

Building the Nginx Image

If we think about the services that we have already, Rails & Postgres, they are both what we would call “backend” services. They aren’t really meant to be directly exposed to the web because that’s not where they shine. The Rails application’s main purpose is to handle the creation of recipes and meal plans, not serving static assets (stylesheets, javascript, etc.). For this reason, we want to put a more specialized tool in front, our “frontend” server, which in this case is going to be Nginx.

Thankfully, our project already has a Nginx configuration in the form of the site.conf file. Let’s take a look at that and see what we will need to change to make it work with Kubernetes.

site.conf

server {
  listen 80;

  # Properly server assets
  location ~ ^/(assets)/ {
    root /usr/share/nginx/html;
    gzip_static on;
    expires max;
    add_header Cache-Control public;
    add_header ETag "";
  }

  # Proxy request to rails app
  location / {
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_pass_header Set-Cookie;
    proxy_pass http://prod_app:3000;
  }
}

This looks pretty good, though we’re going to need to make will be changing prod_app to be the name of the service for our Rails application. We’ll also add an upstream block that we can use later. Here’s the modified file:

upstream mealplan {
  server mealplan;
}

server {
  listen 80;

  # Properly server assets
  location ~ ^/(assets)/ {
    root /usr/share/nginx/html;
    gzip_static on;
    expires max;
    add_header Cache-Control public;
    add_header ETag "";
  }

  # Proxy request to rails app
  location / {
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_pass_header Set-Cookie;
    proxy_pass http://mealplan:3000;
  }
}

Unlike how you may have deployed this if you followed along in the Ruby on Rails series, we’ll need to build the image before we can deploy it. If you’re following allowing using minikube then make sure you are connected to the minikube host as your Docker host (eval $(minikube docker-env)):

$ mkdir nginx
$ mv site.conf nginx/
$ touch nginx/Dockerfile

Let’s open up the nginx/Dockerfile and set it up.

nginx/Dockerfile

FROM nginx:1.11.13-alpine

COPY site.conf /etc/nginx/conf.d/default.conf

With the Dockerfile in place, we’re able to create the image we need before we’ll be able to create our Kubernetes Deployment & Service.

$ docker build -t coderjourney/mealplan-frontend:1.0.0 nginx/

Creating the Front-end Deployment & Service

Now that we have an image in place we need to specify what Kubernetes needs to run and maintain through a Deployment and Service. Similar to the postgres tutorial we’ll define our Service and Deployment in the same file.

deployments/frontend.yml

apiVersion: v1
kind: Service
metadata:
  name: frontend
spec:
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
  selector:
    app: mealplan
    tier: frontend
  type: LoadBalancer
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: frontend
spec:
  template:
    metadata:
      labels:
        app: mealplan
        tier: frontend
    spec:
      containers:
        - image: "coderjourney/mealplan-frontend:1.0.0"
          name: nginx
          lifecycle:
            preStop:
              exec:
                command: ["/usr/sbin/nginx","-s","quit"]

We’ve seen most everything in this file except the extra lifecycle bit in the container declaration. We’re specifying what command should be running before the container is stopped so that we don’t drop connections when stopping this particular container. With our Service and Deployment defined it’s time to create them.

$ kubectl create -f deployments/frontend.yml
service "frontend" created
deployment "frontend" created

With our service up and running we should be able to connect to our Nginx service through the web browser after we get the URL for it from minikube:

$ minikube service frontend --url
http://192.168.99.100:31844

Visiting that URL you should now show our rails application through the eyes of Nginx.

Recap

In this tutorial, we continue to familiarize ourselves with Kubernetes by creating another service that will act as the frontend for our Rails backend service. We also fixed an issue that we created when originally setting up the persistent storage for our PostgreSQL database. There are still a few more things to tackle before we’ll have a production grade deployment setup for our Rails application, but we’re getting close.

Using Kubernetes Persistent Volumes

keith@coderjourney.com (Keith Thompson) — Tue, 04 Apr 2017 02:00:00 +0000

In the previous tutorial we looked at running a Rails application utilizing a Kubernetes Deployment, Service, and ConfigMap. This week, we’ll take a look at running a stateful database in Kubernetes.

Goals:

Create a service for our Postgres database.
Connect our mealplan service to our database.

Stateful vs Stateless

Our Rails application is an example of a “stateless” application because there is no state maintained between requests. It might seem like there is, but all of the actual state is managed in the database. It’s very easy to tear down and spin up stateless applications because you just need to make sure that you’re not terminating a request while it’s being processed. Stateful applications aren’t quite so easy to work with in this way because the data could/would be lost.

For us to run our Postgres database in Kubernetes we’re going to need to use the following Kubernetes object types:

Service
Deployment
ConfigMap
PersistentVolumeClaim

We’ve seen Service, Deployment, and ConfigMap, but PersistentVolumeClaim is new. A PersistentVolumeClaim object indicates that we need access to some storage that isn’t going to go away. It’s not quite like a Docker volume because we will need to specify more information like how much storage and how many containers can access it.

Creating a Database Deployment

For this tutorial, we’re going to group all of the definitions necessary to run our database in Kubernetes into the same YAML file. Before we get too far let’s create the Service and Deployment that we need without any persistence (running it this way would cause data loss when the container stopped):

deployments/postgres.yml

apiVersion: v1
kind: Service
metadata:
  name: postgres
spec:
  ports:
    - port:5432
  selector:
    app: postgres
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: postgres
spec:
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
        - image: "postgres:9.6.2"
          name: postgres
          env:
            - name: POSTGRES_USER
              valueFrom:
                configMapKeyRef:
                  name: mealplan-config
                  key: postgres_user
            - name: POSTGRES_PASSWORD
              valueFrom:
                configMapKeyRef:
                  name: mealplan-config
                  key: postgres_password
          ports:
            - containerPort: 5432
              name: postgres

This isn’t super far off of how we defined our mealplan Deployment or Service. We’re reusing the configMapKeyRef lines from earlier since we want to use the same user and password. The biggest differences are that we’re defining both objects in the same file and we’re separating them with three dashes ---.

If we run this YAML file using the kubectl create -f command we should see the following:

$ kubectl create -f deployments/postgres.yml
service "postgres" created
deployment "postgres" created

Adding Data Persistence

Now that we have our Service and Deployment created successfully we’re going to set up our PersistentVolumeClaim so that we can persist the data. The postgres image that we’re using sets up the username/password combination when the container is first created with no database, because of this we’re going to remove our deployment and service for the time being:

$ kubectl delete -f deployments/postgres.yml
service "postgres" deleted
deployment "postgres" deleted

Now we will define the PersistentVolumeClaim and also mount the volume into our deployment’s containers. The Service portion of the file will be unchanged so we’ll leave that out.

# Servie declaration left out for brevity
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: postgres
spec:
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
        - image: "postgres:9.6.2"
          name: postgres
          env:
            - name: POSTGRES_USER
              valueFrom:
                configMapKeyRef:
                  name: mealplan-config
                  key: postgres_user
            - name: POSTGRES_PASSWORD
              valueFrom:
                configMapKeyRef:
                  name: mealplan-config
                  key: postgres_password
          ports:
            - containerPort: 5432
              name: postgres
          volumeMounts:
            - name: postgres-storage
              mountPath: /var/lib/postgresql/db-data
      volumes:
        - name: postgres-storage
          persistentVolumeClaim:
            claimName: postgres-pv-claim
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: postgres-pv-claim
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi

We needed to add a volumeMounts section to our container in the Deployment and then also tell the deployment what volumes it has access to. Lastly, we created a PersistenVolumeClaim. The claim itself exists to say that whatever is using this claim is allowed to access the PersistentVolume behind it. Let’s create the resources and do some poking around.

$ kubectl create -f deployments/postgres.yml
service "postgres" created
deployment "postgres" created
persistentvolumeclaim "postgres-pv-claim" created

First taking a look at our PersistentVolumeClaim using the describe command:

$ kubectl describe pvc postgres-pv-claim
Name:           postgres-pv-claim
Namespace:      default
StorageClass:   standard
Status:         Bound
Volume:         pvc-9af882c8-1348-11e7-9408-080027b11ce5
Labels:         <none>
Capacity:       5Gi
Access Modes:   RWO
No events.

We can see from the describe output that a PersistentVolume was created for our claim. Getting the description of that volume should give us even more insight how this is set up:

$ kubectl describe pv pvc-9af882c8-1348-11e7-9408-080027b11ce5
Name:           pvc-9af882c8-1348-11e7-9408-080027b11ce5
Labels:         <none>
StorageClass:   standard
Status:         Bound
Claim:          default/postgres-pv-claim
Reclaim Policy: Delete
Access Modes:   RWO
Capacity:       5Gi
Message:
Source:
    Type:       HostPath (bare host directory volume)
    Path:       /tmp/hostpath-provisioner/pvc-9af882c8-1348-11e7-9408-080027b11ce5
No events.

From the looks of it, the volume was created as a directory that we have read/write access to. That works perfectly fine for our needs.

Connecting Rails to our Database

Now that we have a volume and Postgres running we need to reconfigure our Rails application to connect to it. To complete our set up we need to do a few more things:

Change our ConfigMap’s POSTGRES_HOST value so Rails knows what to talk to.
Deploy the ConfigMap changes.
Set up our new database (we’re not going to migrate any of the data).

Updating the mealplan-config ConfigMap

Thankfully, Kubernetes makes it easy for us to edit the data in a config map using the kubectl edit command. Let’s change the POSTGRES_HOST value now:

$ kubectl edit configmap mealplan-config

This will open up your $EDITOR (in my case that’s Vim) and you should edit the POSTGRES_HOST from being an IP address to having the value of postgres. It should look like this when we’re done:

# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: v1
data:
  postgres_host: postres
  postgres_password: dF1nu8xT6jBz01iXAfYDCmGdQO1IOc4EOgqVB703
  postgres_user: meal_planner
  rails_env: production
  rails_log_to_stdout: "true"
  secret_key_base: 7a475ef05d7f1100ae91c5e7ad6ab4706ce5d303e6bbb8da153d2accb7cb53fa5faeff3161b29232b3c08d6417bd05686094d04e22950a4767bc9236991570ad
kind: ConfigMap
metadata:
  creationTimestamp: 2017-03-21T22:46:45Z
  name: mealplan-config
  namespace: default
  resourceVersion: "82058"
  selfLink: /api/v1/namespaces/default/configmaps/mealplan-config
  uid: 427c493c-0e88-11e7-99a4-080027b11ce5

When you save in your editor it will update the ConfigMap object that Kubernetes maintains.

Deploying ConfigMap Changes

Unfortunately, it was very easy to update the ConfigMap, but slightly less easy or understandable to get those changes reflected in our deployment. Similar to the way that environment variable changes are deployed to Docker containers using docker-compose, we need to stop our container and deploy a new one for the value to be updated. Since this is done on a deployment level we will be using the kubectl scale command to stop our container(s) and spin some new ones back up:

$ kubectl scale deployment/mealplan --replicas=0; kubectl scale deployment/mealplan --replicas=1
deployment "mealplan" scaled
deployment "mealplan" scaled

Now our new container will be connected to the Kubernetes maintained postgres service.

Setting up the Database in Kubernetes

Now that our rails application is connected to the postgres service we need to do the work to set up the database initially. Since this is something that we’ll only be doing once we will run this manually by using the kubectl exec command. First, we need to find the container that we’ll use to exec:

$ kubectl get pods
NAME                        READY     STATUS    RESTARTS   AGE
mealplan-4029866644-mhhtf   1/1       Running   0          5m
postgres-4220950182-qphj2   1/1       Running   0          15m

The container name that I’ll be using will be mealplan-4029866644-mhhtf but yours will be different. Now let’s use the kubectl exec command to make sure we’re talking to the proper host and get up our database:

$ kubectl exec mealplan-4029866644-mhhtf -it -- env | grep POSTGRES_HOST
POSTGRES_HOST=postgres
$ kubectl exec mealplan-4029866644-mhhtf -it -- rake db:setup

Tearing Down the Old Postgres Container

Let’s make sure that we don’t leave the old Postgres container running outside of Kubernetes. Set the minikube host as your DOCKER_HOST if it’s not already set:

$ eval $(minikube docker-env)

Now we can stop & remove the database container that we were using:

$ docker stop mealplan_prod_db_1
mealplan_prod_db_1
$ docker rm mealplan_prod_db_1
mealplan_prod_db_1

Recap

In this tutorial, you went through and learned how to set up a Kubernetes PersistentVolume and run a stateful application within Kubernetes. You also learned how to edit a ConfigMap and deploy the changes to an existing deployment. We had some unfortunate downtime that we had to deal with, but in later tutorials, we will address how to overcome that hurdle.

Define Kubernetes Objects using Yaml

keith@coderjourney.com (Keith Thompson) — Tue, 21 Mar 2017 01:00:00 +0000

You’ve taken Kubernetes for a spin using the command line utility, but eventually you’re going to need set up more customized pods & deployments. In this tutorial, you’ll learn how to set up YAML files to create custom Kubernetes objects.

Goals:

Create a deployable image for a Rails application
Generate a ConfigMap to define environment variables in our pods
Configure deployment for our Rails application

Our goal for today is to get a Rails application running with Kubernetes. To get the application up and running we need to figure out how to use environment variables with Kubernetes, how to package the application to run with Kubernetes, and lastly how to define our configuration.

Using the Minikube Docker Server

While we’re working locally using minikube we’re going to need to be building our applications into images that we can run in pods. We’ll do this using the docker build command, but by default, our docker client won’t be talking to the minikube Docker.

We’ll get around this by using a line similar to how we set which docker-machine to use:

$ eval $(minikube docker-env)

This will set our Docker client to use the proper server.

Building our Rails Image

We’ll be using the meal planning application that can be found here. Download this application to follow along.

From within the application’s directory, let’s build the image and give it a meaningful name:

$ docker build -t meal_plan:1 .

We’re giving this a version number of 1 since it’s our first time packaging it up. Having a version number like this allows us to ensure that we’re deploying the proper version.

Remember: latest is not a version number.

Starting the Database on a Separate Docker Host

We’re not going to be running the database within Kubernetes just yet, but we still need one to be running so our application can connect to it. We’re going to run the database from the VM that Kubernetes is running in so we can easily connect our application to it. You would never actually do this, but we don’t have time in this tutorial to coverage running Postgres in Kubernetes.

Before we start it, we’ll need to expose the database port to the Docker host so that our Kubernetes cluster can access it:

docker-compose.prod.yml

# Only showing the `prod_db` portion of the file
  prod_db:
    image: postgres
    env_file: .env.prod
    ports:
      - "5432:5432"
    volumes:
      - db-data:/var/lib/postgresql/db-data

Now with that changed we can start the database using:

$ script/prod up -d prod_db

To make it easier on ourselves we should also make sure that the database exists before continuing.

$ script/prod run --rm prod_app rake db:create db:migrate

Creating a Rails Deployment

Now that we have an image we could create a deployment using the kubectl run command, but that can’t be peer reviewed and might be hard to replicate so we’re going to create this deployment as a YAML file.

deployments/mealplan.yml

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: mealplan
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: mealplan
        tier: backend
        track: stable
    spec:
      containers:
        - name: mealplan
          image: "meal_plan:1"
          ports:
            - name: rails
              containerPort: 3000

Besides the image declaration, you’ll notice that we use mealplan without and underscore, this is because the underscore is not allowed. This will create an application container for us, but it’s not going to work without the additional configuration values that we had been storing in the .env file.

Going through this file. We define the type of Kubernetes object using the kind key, set the name as a metadata subkey, and then we set the spec for the deployment. The spec is a lot like what we would define as part of a docker-compose.yml file. The notable differences here are the replicas setting, the template key, and the fact that we’re able to name the port. The labels allow us to specify ways to classify our objects.

For us to utilize the environment variables that we have been using we’re going to set up a ConfigMap, which is another object in Kubernetes that you can use to hold configuration data.

Creating the Meal Plan ConfigMap

ConfigMaps exist within Kubernetes and allow us to set specific keys (literals) or entire files. Unfortunately, our application isn’t set up to read an entire file in as environment variables (you could use dotenv for that). We’re going to need to set up the values from our .env.prod as literal values within a ConfigMap.

$ kubectl create configmap mealplan-config \
--from-literal=postgres_user=meal_planner \
--from-literal=postgres_password=dF1nu8xT6jBz01iXAfYDCmGdQO1IOc4EOgqVB703 \
--from-literal=postgres_host=$(minikub ip) \
--from-literal=rails_env=production \
--from-literal=rails_log_to_stdout=true \
--from-literal=secret_key_base=7a475ef05d7f1100ae91c5e7ad6ab4706ce5d303e6bbb8da153d2accb7cb53fa5faeff3161b29232b3c08d6417bd05686094d04e22950a4767bc9236991570ad

We can make sure that we set everything properly by looking at the data section of the ConfigMap resource:

$ kubectl get configmap mealplan-config -o yaml
apiVersion: v1
data:
  postgres_host: 192.168.99.101
  postgres_password: dF1nu8xT6jBz01iXAfYDCmGdQO1IOc4EOgqVB703
  postgres_user: meal_planner
  rails_env: production
  rails_log_to_stdout: "true"
  secret_key_base: 7a475ef05d7f1100ae91c5e7ad6ab4706ce5d303e6bbb8da153d2accb7cb53fa5faeff3161b29232b3c08d6417bd05686094d04e22950a4767bc9236991570ad
kind: ConfigMap
metadata:
  creationTimestamp: 2017-03-19T21:08:25Z
  name: mealplan-config
  namespace: default
  resourceVersion: "12711"
  selfLink: /api/v1/namespaces/default/configmaps/mealplan-config
  uid: 86160c7a-0e7a-11e7-99a4-080027b11ce5

Adding Configuration to the Meal Plan Deployment

With the config map created we can now use these in our deployment file.

deployments/mealplan.yml

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: mealplan
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: mealplan
        tier: backend
        track: stable
    spec:
      containers:
        - name: mealplan
          image: "meal_plan:1"
          ports:
            - name: rails
              containerPort: 3000
          env:
            - name: POSTGRES_USER
              valueFrom:
                configMapKeyRef:
                  name: mealplan-config
                  key: postgres_user
            - name: POSTGRES_PASSWORD
              valueFrom:
                configMapKeyRef:
                  name: mealplan-config
                  key: postgres_password
            - name: POSTGRES_HOST
              valueFrom:
                configMapKeyRef:
                  name: mealplan-config
                  key: postgres_host
            - name: RAILS_ENV
              valueFrom:
                configMapKeyRef:
                  name: mealplan-config
                  key: rails_env
            - name: RAILS_LOG_TO_STDOUT
              valueFrom:
                configMapKeyRef:
                  name: mealplan-config
                  key: rails_log_to_stdout
            - name: SECRET_KEY_BASE
              valueFrom:
                configMapKeyRef:
                  name: mealplan-config
                  key: secret_key_base

For each environment variable we need to set the name as it would be exposed in the environment, all uppercase, and then we tell the deployment where it should pull this value. In this case, we want to use values from a ConfigMap so we’re using the configMapKeyRef option, providing the name of the ConfigMap that we created, and then specifying the key from the ConfigMap.

The configuration story here is a little tedious, but it does require us to be explicit. Now we can create our deployment using the kubectl create command and our deployment file:

$ kubectl create -f deployments/mealplan.yml

Exposing our Rails Application with a Server

To ensure that our application is running we should expose it using a Kubernetes service. Services can also be configured using configuration files so we’ll create that now.

$ mkdir services

services/mealplan.yml

kind: Service
apiVersion: v1
metadata:
  name: mealplan
spec:
  selector:
    app: mealplan
    tier: backend
  ports:
    - protocol: TCP
      port: 80
      targetPort: rails
  type: LoadBalancer

Setting up the Service works a lot like setting up the deployment earlier. One thing to notice is that we’re setting the targetPort using the port name that we created in our deployment. The selector section is important because that tells the Service how to find the pods to direct traffic to.

Let’s create the service using the same kubectl create command as before with our new file:

$ kubectl create -f services/mealplan.yml

Get the URL for this service from minikube:

$ minikube service mealplan --url
http://192.168.99.100:32261

If all went well, visiting that URL in your web browser should show you the unstyled home page of our meal planning application.

Recap

In this tutorial, you created Kubernetes objects to run a Ruby on Rails application in a repeatable way. This didn’t cover even close to everything that we’ll be able to do with Kubernetes, but it is a pretty good start. If you would like to improve this, you should move some of the configuration values into Kubernetes Secrets. In the next tutorial, we’ll cover running stateful applications (like databases) in Kubernetes.

The code from this tutorial can be found here.

Run Kubernetes Locally Using Minikube

keith@coderjourney.com (Keith Thompson) — Tue, 14 Mar 2017 22:00:00 +0000

If you’re just getting started with Kubernetes then it may be a little intimidating to start running of servers in a cloud provider like Google Cloud or AWS. Thankfully, you don’t need to use a cloud provider if you’re just dabbling with Kubernetes thanks to minikube.

Minikube will allow you to run a Kubernetes cluster on your development environment. This won’t exactly match the experience that you’ll have in a deployed environment, but it will be close enough to let you get your bearings.

Installing Kubectl

Minikube will help when learning Kubernetes, but it won’t actually be of much use unless you have the kubectl command installed.

You’ll want to follow these instructions to install kubectl for your operating system. I’m working on macOS so I’ll be using homebrew to install.

$ brew update && brew install kubectl

Make sure that you’re set up properly before continuing.

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"5", GitVersion:"v1.5.4", GitCommit:"7243c69eb523aa4377bce883e7c0dd76b84709a1", GitTreeState:"clean", BuildDate:"2017-03-08T02:50:34Z", GoVersion:"go1.8", Compiler:"gc", Platform:"darwin/amd64"}

Installing Minikube

Installing minikube works in a similar way to most of the installation directions for kubectl. Follow the instructions for the latest version found here. Some of the commands utilize sudo, but depending on your development environment set up you may not need them to.

Since I’m on macOS, I’ll use the corresponding instructions:

$ curl -Lo minikube https://storage.googleapis.com/minikube/releases/v0.17.1/minikube-darwin-amd64 && chmod +x minikube && mv minikube /usr/local/bin/

With minikube installed you’re ready to start your first cluster (this might take a little while):

$ minikube start

The minikube start process configures your kubectl to connect to it.

Your First Deployment

The atomic unit of Kubernetes is the “pod” and a pod represents everything that is necessary for an application to run. Pods include a lot of the same information that you may have seen in a docker-compose.yml file (images, volumes, networks, etc). We won’t be creating a pod directly right now because we’re going to instead create a “deployment” that will create a pod behind the scenes. For this example, we’ll use the nginx:alpine image since it is small and exposes a port for us to view the default page.

To create a new deployment you can use the kubectl run command in this way:

$ kubectl run [name-of-deployment] --image=[image-name]

In this case, we’ll call the deployment webserver and use the nginx:alpine image.

$ kubectl run webserver --image=nginx:alpine

Now you can take a look at the deployment & pod using the kubectl get command:

$ kubectl get deployments
NAME        DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
webserver   1         1         1            1           7s

$ kubectl get pods
NAME                         READY     STATUS    RESTARTS   AGE
webserver-2615124133-bgjjv   1/1       Running   0          11s

Notice that the pod has an extra identifier as part of the name. We don’t need to manage extra info because our deployment will do it for us.

Once of the nice features of deployments is that they can keep your pods running even when something goes wrong. Let’s delete the pod that was created with the deployment and see what happens. Whenever you delete a resource like a pod, deployment, or service you’ll use the kubectl delete command.

$ kubectl delete pod webserver-2615124133-bgjjv
pod "webserver-2615124133-bgjjv" deleted

Now if we look at the pods again we would expect to not see anything, but our deployment says otherwise.

$ kubectl get pods
NAME                         READY     STATUS    RESTARTS   AGE
webserver-2615124133-d03zm   1/1       Running   0          10s

The deployment created a replacement pod since the previous one was destroyed.

Exposing the Deployment

This webserver isn’t all that useful to us right now because it can’t be accessed, but that can be changed by creating a “service”. We’ll create a service for our “webserver” deployment by using the kubectl expose command:

$ kubectl expose deployment webserver --type=LoadBalancer --port=80

We’re using the LoadBalancer type here, but you could have also used the NodePort, and we need to specify the port that will be exposed. Take a look at this service using kubectl get once again:

$ kubectl get services
NAME         CLUSTER-IP   EXTERNAL-IP   PORT(S)        AGE
kubernetes   10.0.0.1     <none>        443/TCP        3d
webserver    10.0.0.247   <pending>     80:30349/TCP   8s

Since we’re using minikube accessing this service requires another step since we can’t get an external IP. You’ll take the exposed port (in this case 30349) and access that on the minikube IP address. You can get all of this information in a single minikube command thankfully:

$ minikube service webserver --url
http://192.168.99.101:30349

Viewing the URL that the command gave you from your browser will show you the default nginx landing page.

Visualizing Kubernetes

Working with a cluster of nodes, numerous pods, and services can get a little confusing, but kubernetes can be visualized through dashboards and minikube comes with one baked right in. You can launch the dashboard using this command:

$ minikube dashboard

Recap

You went through quite a bit in this tutorial. You created your first Kubernetes cluster using minikube & you made a deployment that you exposed using a service. As this series continues you’ll deploy increasingly more advance setups, but this is a really good start.

Learning Rails – Improving Deployment (Part 9)

keith@coderjourney.com (Keith Thompson) — Tue, 24 Jan 2017 13:00:15 +0000

Today we’re going to be improving deployment of the meal plan application by setting up a web server (Nginx) in front of our application server.

Goal for this Tutorial:

Pre-compile assets as part of our image building process
Add Nginx to our web application stack
Handle Deploying New Application Changes

We current have our application running in the wild, but pointing a user at an IP address with a port number isn’t a great approach. It’s also not great that there is a really long first load time when the application has to compile the status assets (javascript, styles, etc). We can fix both of these issues by using a web server and pre-compiling our assets.

Before we get going too far though we’re going to improve our ability to run commands using our docker-compose.prod.yml file. Let’s create script/prod to make this a little easier to type:

script/prod

#!/bin/sh

docker-compose -f docker-compose.prod.yml "$@"

After you save the file don’t forget to make it executable:

$ chmod +x script/prod

Note: If you’re on windows you can created a console alias by following the instructions here

Pre-compiling Application Assets

We’re going to pre-compile assets as part of our Docker image build so that we can do it before we ever mess with our containers that are already running. This will make our application much faster on the first load and force our ruby process to do less once we have Nginx in place. We’ll start by creating a separate Dockerfile for production.

$ cp Dockerfile Dockerfile.prod

We’re calling this Dockefile.prod, but it would really work for any configuration environment that we want to have our assets pre-compiled into. Next, we’ll make a few changes to the file to get our assets to be pre-compiled:

Dockerfile.prod

FROM ruby:2.3.1

RUN apt-get update -yqq \
  && apt-get install -yqq --no-install-recommends \
    postgresql-client \
    nodejs \
  && apt-get -q clean \
  && rm -rf /var/lib/apt/lists

# Pre-install gems with native extensions
RUN gem install nokogiri -v "1.6.8.1"

WORKDIR /usr/src/app
COPY Gemfile* ./
RUN bundle install
COPY . .

# Pre-compile assets
ENV RAILS_ENV production
RUN rails assets:precompile

CMD script/start

Next, we will need to set up our docker-compose.prod.yml file to use the proper file to build our image.

docker-compose.prod.yml

version: "2"

volumes:
  db-data:
    external: false

services:
  prod_db:
    image: postgres
    env_file: .env.prod
    volumes:
      - db-data:/var/lib/postgresql/db-data

  prod_app:
    build:
      context: .
      dockerfile: Dockerfile.prod
    env_file: .env.prod
    ports:
      - "3000:3000"
    depends_on:
      - prod_db

Let’s build it and watch the assets be precompiled using our new script/prod

$ script/prod build prod_app

You can see that writes the values out to /usr/src/app/public/assets within the image and even creates gzipped versions for us. This is important to notice because we’ll need to share these files from our prod_app container over to our web server that we’re about to get up and running.

Setting up Nginx as our Web Server

There are a few things that we want Nginx to handle for us:

Proxying for our Rails application. This will allow us to set up multiple backends when we need them.
Serving static assets that have been precompiled.

For us to be able to do this without creating a custom Nginx image we will need to set up a volume to share our assets from our prod_app container over to our webserver container, and a volume to share our Nginx site configuration into the container also. Let’s lay out our docker-compose.prod.yml file to let us accomplish this:

docker-compose.prod.yml

version: "2"

volumes:
  assets:
    external: false
  configs:
    external: false
  db-data:
    external: false

services:
  webserver:
    image: "nginx:1.11.8"
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - assets:/usr/share/nginx/html
      - configs:/etc/nginx/conf.d

  prod_db:
    image: postgres
    env_file: .env.prod
    volumes:
      - db-data:/var/lib/postgresql/db-data

  prod_app:
    build:
      context: .
      dockerfile: Dockerfile.prod
    env_file: .env.prod
    ports:
      - "3000:3000"
    volumes:
      - assets:/usr/share/nginx/html
      - configs:/etc/nginx/conf.d
    depends_on:
      - prod_db
      - webserver

We’re also going to need to create a new file that will represent our site configuration in Nginx so that it will proxy to our Rails app.

site.conf

server {
  listen 80;

  # Properly serve assets
  location ~ ^/(assets)/ {
    root /usr/share/nginx/html;
    gzip_static on;
    expires max;
    add_header Cache-Control public;
    add_header ETag "";
  }

  # Proxy requests to rails app
  location / {
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_pass_header Set-Cookie;
    proxy_pass http://prod_app:3000;
  }
}

This brings us to a “gotcha” with volumes that we hadn’t previously looked at. Once a volume has been created and is connected to a container, it won’t be removed or overwritten by simply connecting another container. This will have an effect because our webserver container will create our assets and config containers with its own default values and then when we create our prod_app container we won’t be able to simply share our assets over because they volume already exists.

Serving Proper Assets on Update

Now that we have our site file up we’re going to need to rebuild our image so that it is packaged in, but before we do that we’re going to make sure that when we start our container in the production environment that we are ensuring the newest assets are in the right spot. We’ll do this by modifying our script/start:

script/start

#!/bin/bash -e

if [[ -a /tmp/puma.pid ]]; then
  rm /tmp/puma.pid
fi

if [[ $RAILS_ENV == "production" ]]; then
  rake assets:precompile
  mkdir -p /usr/share/nginx/html
  cp -R public/* /usr/share/nginx/html/
  mkdir -p /etc/nginx/conf.d/
  cp site.conf /etc/nginx/conf.d/default.conf
fi

rails server -b 0.0.0.0 -P /tmp/puma.pid

Since we precompile assets when we build the image doing so again here doesn’t really cost us anything. We ensure that the html directory exists so that we can copy the assets from the public directory over. Finally, we move our site.conf file into the configuration directory for Nginx. It’s safe for us to now flip the flag we set in our .env.prod file that was making rails serve and compile assets on the fly.

POSTGRES_USER=meal_planner
POSTGRES_PASSWORD=dF1nu8xT6jBz01iXAfYDCmGdQO1IOc4EOgqVB703
POSTGRES_HOST=prod_db

RAILS_ENV=production
# RAILS_SERVE_STATIC_FILES=false
RAILS_LOG_TO_STDOUT=true
SECRET_KEY_BASE=7a475ef05d7f1100ae91c5e7ad6ab4706ce5d303e6bbb8da153d2accb7cb53fa5faeff3161b29232b3c08d6417bd05686094d04e22950a4767bc9236991570ad

Now we can rebuild our image and spin up our containers. We’ll start the webserver container first so that we do get the Nginx static files as the first files in the assets and config volumes.

$ script/prod up -d webserver prod_db
$ script/prod up -d --build prod_app

If you head to localhost you should see the Nginx splash page, but that’s not the goal. The reason that you’re seeing this instead of our application is that we didn’t restart Nginx after we dropped off a new config for it to read. This is one occasion that I will actually say that using docker-compose restart will work just fine for us:

$ script/prod restart webserver

Now if you head back to localhost you should see the meal plan application being served up (with assets) although we didn’t specify a port.

Recap

Today we vastly improved the deployment of our rails application by having a web server serve our static assets and proxy requests to our application so that we don’t need to connect to port 3000 directly. There’s still more improvement that can be done by digging deeper into Nginx, but for now, this works.

Learning Rails – Deploying Rails (Part 8)

keith@coderjourney.com (Keith Thompson) — Thu, 05 Jan 2017 17:30:40 +0000

Today we’re going to take a big step with meal planning by deploying the application. Since our application is already built on top of Docker we will be using that as our means of deployment.

Goal for this Tutorial:

Create production docker-compose file
Provision Docker host in the cloud
Deploy our meal planner

Our meal planning application isn’t exactly feature complete, but it’s in a good enough spot that we might want to get it out there for some people to beta test for us. For that to happen we need to put it onto a server that is publicly accessible. Heroku is a popular deployment target for Ruby on Rails applications because it is very easy to work with and requires very little knowledge from an operations standpoint, but since we’re already familiar with Docker we’re going to use that and deploy to our own server.

Note: It is possible to use Docker to deploy to Heroku also, but I will leave that as an exercise for the reader if interested.

Prepping the Application for Production

Before we can actually deploy we need to make sure that our application can run in the production environment. Thankfully, we can run test deploys on our local docker machine before we ever provision a server and be relatively certain that it will work in a production environment. We’ll start by creating a production specific docker-compose configuration that we’ll use for deployment only. We’ll start by copying our development configuration and then make some modifications:

$ cp docker-compose.yml docker-compose.prod.yml

We only need to change a few lines. We can’t mount a local directory as a volume on a remote server so we need to remove that section. Additionally, we will want to use a separate list of environment variables in production than we use in development so we’ll specify a different env_file. Lastly, we’ll ensure that the RAILS_ENV is set to production:

version: "2"

volumes:
  db-data:
    external: false

services:
  prod_db:
    image: postgres
    env_file: .env.prod
    volumes:
      - db-data:/var/lib/postgresql/db-data

  prod_app:
    build: .
    env_file: .env.prod
    ports:
      - "3000:3000"
    depends_on:
      - prod_db

We can create the .env.prod file by copying our original .env file and changing the username/password:

$ cp .env .env.prod

We’ll also want to generate a new password, and we can use ruby to do that:

$ docker-compose run --rm app ruby -r securerandom -e "puts SecureRandom.base64(30)"

Copy the output of that command and use it in the new environment file, and also assign the POSTGRES_HOST value to prod_db for the time being.

.env.prod

POSTGRES_USER=meal_planner
POSTGRES_PASSWORD=exh2uRzCrbjfRNj4GpiaZuToJa6QrFHCdmDbKLNe
POSTGRES_HOST=prod_db

We’re not quite finished though because we will need to set our SECRET_KEY_BASE value and set the RAILS_ENV to production. This is automatically set in development and test, but should be passed in as an environment variable in other application environments. We can generate that using rake secret

$ docker-compose run --rm app rake secret

.env.prod

POSTGRES_USER=meal_planner
POSTGRES_PASSWORD=exh2uRzCrbjfRNj4GpiaZuToJa6QrFHCdmDbKLNe
POSTGRES_HOST=prod_db
RAILS_ENV=production
SECRET_KEY_BASE=afaad8af5acc8841359153f01f970827e815def7e71d50fd5213bb8e2fcee4cc0c16d61bf30ec3a435d4849e1c4e666c653c5ddd6e407f95aa2fab938b8965b8

Testing Production Locally

Now that the application has been configured with the environment variables that we need for production we can test this locally by creating the containers and running this the same way we normally would. A big difference here is that if we make any file changes we will need to rebuild the prod_app image. First, we need to create our new database.

$ docker-compose -f docker-compose.prod.yml build prod_app
... After the image is built
$ docker-compose -f docker-compose.prod.yml run --rm prod_app rake db:create
$ docker-compose -f docker-compose.prod.yml run --rm prod_app rake db:migrate

That should complete without any errors and we’ll now have a database. Next, we’ll start the application and make sure that we can visit the homepage. You’ll need to make sure that your containers for development are no longer running before doing this because we will be exposing the same port.

$ docker-compose -f docker-compose.prod.yml up -d

Visiting http://localhost:3000/ should show you our welcome page, but you’ll notice that it doesn’t have any of our styles. This is happening because our production rails configuration is set to not serve static assets. Serving these assets is the job of a web server like nginx or apache and if we don’t have to have our ruby process do that then we won’t. For the time being though we’re going to set this up to be configurable with an environment variable so we can toggle this on an off as we need it.

Configuring Asset Handling

As of Rails 5 there is already an environment variable that can be used to determine if the rails application should serve up assets in the /public directory. We will modify the config/environments/production.rb file so that also use that variable to optionally handle asset compilation if it’s necessary.

config/environments/production.rb

Rails.application.configure do
  # ... additional configuration left out for brevity

  # Compress JavaScripts and CSS.
  config.assets.js_compressor = :uglifier
  config.assets.css_compressor = :sass

  # Do not fallback to assets pipeline if a precompiled asset is missed.
  config.assets.compile = ENV['RAILS_SERVE_STATIC_FILES'].present?

  # ... additional configuration left out for brevity
end

When we set the RAILS_SERVE_STATIC_FILES value to true in our environment file it will set our application to serve up and compile assets on the fly.

.env.prod

POSTGRES_USER=meal_planner
POSTGRES_PASSWORD=exh2uRzCrbjfRNj4GpiaZuToJa6QrFHCdmDbKLNe
POSTGRES_HOST=prod_db
RAILS_ENV=production
SECRET_KEY_BASE=afaad8af5acc8841359153f01f970827e815def7e71d50fd5213bb8e2fcee4cc0c16d61bf30ec3a435d4849e1c4e666c653c5ddd6e407f95aa2fab938b8965b8

Since we’ve made a change to one of the ruby files we will need to rebuild the application before we can re-run it to make sure that our efforts were successful.

$ docker-compose -f docker-compose.prod.yml stop prod_app
$ docker-compose -f docker-compose.prod.yml build prod_app
$ docker-compose -f docker-compose.prod.yml up -d prod_app

When you try to reload the application in browser you might not see anything, and if you check the logs using docker logs mealplan_prod_app_1 you might see something like the following:

rm: cannot remove '/tmp/puma.pid': No such file or directory

This is caused by an issue in script/start and we can fix that pretty easily:

#!/bin/sh

if [[ -a /tmp/puma.pid ]]; then
  rm /tmp/puma.pid
fi

rails server -b 0.0.0.0 -P /tmp/puma.pid

Now we will only delete the pid file if it already exists. If you rerun the docker stop, build, and up commands again and reload the web page it will take a little longer than expected, but then render with the styles. That delay is from the assets being compiled on the first load, and subsequent page loads should be fast.

Creating our Docker Host in Digital Ocean

Docker provides us with a really nice tool for building docker hosts easily in quite a few hosting providers. For this tutorial, we’re going to host with Digital Ocean (that link will get you a $10 credit for signing up) and creating our host using docker-machine. You’ll need to sign up with digital ocean before we begin because you’ll need to grab your API token. Once you have your account you can go here to generate a new API token.

I have my token store in the environment variable DO_TOKEN (you can set that for yourself using export DO_TOKEN="YOUR_TOKEN". Now that we have our API token, we can use docker-machine to actually create a droplet for us and set it up to be a docker host for us.

$ docker-machine create --driver=digitalocean --digitalocean-access-token=$DO_TOKEN --digitalocean-size=1gb meal-planner

Once that command has finished, it’ll make our life a little easier if we designate one terminal window/tab specifically to interacting with that docker host. From that window/tab run the following command to set docker to interact with that server instead of your local docker:

eval $(docker-machine env meal-planner)

Deploying the Application

Our terminal window is now set to connect to the proper server and we can go through the same steps that we went through on our local machine to get the application running in the cloud. This new docker host doesn’t have any images downloaded so it will take a little while the first time you run these commands.

$ docker-compose -f docker-compose.prod.yml build prod_app
... After the image is built
$ docker-compose -f docker-compose.prod.yml run --rm prod_app rake db:create db:migrate

Once our database is set up all that is left to do is run the application:

$ docker-compose -f docker-compose.prod.yml up -d

Now we can find the docker host’s IP address and view our application running in the cloud. To get the IP address you can use the following command:

$ docker-machine ip meal-planner
104.236.250.214

In this case, I can go to http://104.236.250.214:3000/ to view the application.

Recap

We’ve take our meal planning application from being nothing more than an ideal to have a lot of the functionality flushed out and being deployed publicly. This is the first iteration of an application like this, but you can take it quite a bit further and improve it in any number of ways. For our purposes though, this application is finished. Next week will improve the deployment of the application by layering Nginx into the mix as the web server, and that will conclude the introduction to Rails series.

Learning Rails – Application Styling (Part 7)

keith@coderjourney.com (Keith Thompson) — Wed, 21 Dec 2016 23:21:45 +0000

Today we’re going to move our application from being completely unstyled to looking like something that people wouldn’t mind using. Since this tutorial series is about Rails we’ll learn how to interact with some additional front-end tools, but won’t necessarily cover all of the style details.

Goal for this Tutorial:

Add Website Navigation
Pull in Bootstrap Gem
Style application

Styling our application won’t be the most difficult thing that we’ve done because we’re not here to learn more about CSS/SCSS or HTML. As of right now, the application is a little clunky to navigate around so we’ll start off by adding some navigation to allow a user to better move around the application. From there, we will be styling the main pages of our application. Although it’s a little cliche we will be using Twitter Bootstrap for our styles so that we don’t have to lay out a ton of CSS and we preemptively added some classes to our markup when we were writing it originally so that today’s work would be easier.

Installing Bootstrap

There are multiple ways to work with front-end dependencies in Rails, but the built in one is by using sprockets and front-end packages wrapped in gems. If you’re coming from a mostly front-end development background then this will feel really wrong, and I completely understand where you’re coming from, but we won’t be covering the use of bower, yeoman, yarn, webpack, etc.

The gem we will be using can be found here. Let’s add this to our Gemfile now.

Gemfile

gem 'bootstrap-sass', '~> 3.3.6'

We already have sass-rails installed so we don’t need to worry about that portion of the README. After we rebuild our image and restart our application we’ll be set up to actually use Bootstrap in our application.

With the gem installed we can now load this into our main sass file. First, we’re going to switch from having an application.css to having an application.scss file so that we can use native sass features.

$ mv app/assets/stylesheets/application.css app/assets/stylesheets/application.scss

Next we can clean out this file and import Bootstrap:

@import "bootstrap-sprockets";
@import "bootstrap";

Lastly, we’ll set up our application.js to load the additional Bootstrap JavaScripts:

//= require jquery
//= require bootstrap-sprockets
//= require jquery_ujs
//= require_tree .

If you refresh any page in the application now it will look slightly different based on the base styles provided by Bootstrap.

Before we get too involved with styling our individual pages we’re going to work on our application layout so that there will be some shared navigation and each page will have a similar frame. We want the user to be able to access the following if signed in:

Recipes
Meal Plans
Log Out

If he/she isn’t logged in then the navigation should only show the “Sign In” option. Let’s tweak our app/views/layouts/application.html.erb to make this happen.

app/views/layouts/application.html.erb

First we will modify some of the informational HTML tags, setting a language and a viewport:

<html lang="en">
  <head>
    <title>MealPlan</title>
    <%= csrf_meta_tags %>

    <%= stylesheet_link_tag    'application', media: 'all' %>
    <%= javascript_include_tag 'application' %>
    <meta name="viewport" content="width=device-width, initial-scale=1">
  </head>

Next we’ll change out our header to be a bootstrap navigation item:

<header class="container">
  <nav class="navbar navbar-default">
    <div class="container-fluid">
      <div class="navbar-header">
        <%= link_to "Meal Planning", root_path, class: "navbar-brand" %>
      </div>

      <ul class="nav navbar-nav navbar-right">
        <% if signed_in? %>
          <li><%= link_to "Meal Plans", meal_plans_path %></li>
          <li><%= link_to "Recipes", recipes_path %></li>
          <li><%= link_to "Sign out", sign_out_path, method: :delete %></li>
        <% else %>
          <li><%= link_to "Sign In", sign_in_path %></li>
        <% end %>
      </ul>
    </div>
  </nav>
</header>

<section class="container">
  <div id="flash">
    <% flash.each do |key, value| %>
      <div class="flash alert <%= alert_class(key) %>"><%= value %></div>
    <% end %>
  </div>

  <%= yield %>
</section>

This will create a consistent layout and automatically style and of the flash messages that we create in our application, but we need to define the alert_class method first.

_app/helpers/applicationhelper.rb

module ApplicationHelper
  def alert_class(flash_type)
    case flash_type.to_sym
    when :notice
      "alert-success"
    when :alert
      "alert-warning"
    when :error
      "alert-danger"
    end
  end
end

That added a lot more structure to our application and now we’re ready to tackle individual pages.

Styling Recipes

When I think of recipes I actually think on Pinterest. That’s where a lot of people find recipes and I think mimicking that representation for our recipe index would be great. Thankfully, Bootstrap provides a thumbnail component that will get us most of the way there (I’ll leave the variable height blocks up to you). Let’s change our recipes to lay them out in rows of 3/2/1 in small panels depending on screen size (using the thumbnail component).

app/views/recipes/index.html.erb

<div class="title-container">
  <h1>Recipes</h1>
  <%= link_to "New Recipe", new_recipe_path, class: "btn btn-default" %>
</div>

<div class="recipes">
  <% @recipes.each do |recipe| %>
    <div class="recipe">
      <div class="thumbnail">
        <div class="caption">
          <h3><%= link_to truncate(recipe.name), recipe_path(recipe) %></h3>
          <p><%= truncate(recipe.description, length: 150) %></p>
          <p>
            <%= link_to "Edit", edit_recipe_path(recipe), class: "btn btn-default" %>
            <%= link_to "Delete", recipe_path(recipe), data: {
              confirm: "Are you sure you want to delete: #{recipe.name}?",
            }, method: :delete %>
          </p>
        </div>
      </div>
    </div>
  <% end %>
</div>

We did change up the markup a fair bit here, and we used the truncate helper so that we can make our recipe boxes all of a specific size. For that we do need some additional CSS though that we’re going to add to our application.scss. We’ll be using flexbox to help keep our layout sane and get our content to flow the way we expect it to.

app/assets/stylesheets/application.scss

.title-container {
  display: flex;
  justify-content: space-between;
  align-items: center;
  margin-bottom: 20px;

  h1 {
    margin: 0;
  }
}

.recipes {
  display: flex;
  flex-wrap: wrap;
  justify-content: space-around;
}

.recipe {
  width: 30%;
  min-width: 300px;
  }

The “new” and “edit” views don’t have much in them, but we’ll want to wrap our titles in the title-container class that we created so that our application looks consistent.

app/views/recipes/new.html.erb

<div class="title-container">
  <h1>New Recipe</h1>
</div>

<%= render partial: "form" %>

The first thing we’ll want to do is get it styled with all of the proper bootstrap form classes and then see what we can do from there. Mostly just adding form-control classes and setting the submit button to look like a bootstrap button.

_app/views/recipes/form.html.erb

<%= render partial: "shared/errors" %>

<%= form_for @recipe do |form| %>
  <div class="form-group">
    <%= form.label :name %>
    <%= form.text_field :name, class: "form-control" %>
  </div>

  <div class="form-group">
    <%= form.label :description %>
    <%= form.text_area :description, class: "form-control" %>
  </div>

  <div class="actions">
    <%= form.submit class: "btn btn-primary" %>
  </div>
<% end %>

If you submit the form now though you can see that our errors container isn’t being rendered with any flare and bootstrap gives us some styles to make that more apparent.

_app/views/shared/errors.html.erb

<% if @errors.present? %>
  <div class="alert alert-danger errors">
    <ul>
      <% @errors.each do |error| %>
        <li><%= error %></li>
      <% end %>
    </ul>
  </div>
<% end %>

The display of a recipe isn’t actually too bad in the recipe show page so I’ll leave adjusting this styles up to the reader. You’ll probably just want to change how the controls are displayed.

Styling Meal Plans

We’ll tackle the “new” view for meal plans to start. This view is a bit more complicated than the edit is because we have a second form, but it won’t take us that long to tweak.

_app/views/mealplans/new.html.erb

<div class="title-container">
  <h1>New Meal Plan</h1>
</div>

<div class="panel panel-default">
  <div class="panel-body">
    <%= form_for @meal_plan, url: new_meal_plan_path, method: "GET", html: { class: "form-inline" } do |form| %>
      <div class="form-group">
        <%= form.label :start_date %>
        <%= form.text_field :start_date, type: "date", class: "form-control" %>
      </div>

      <div class="form-group">
        <%= form.label :end_date %>
        <%= form.text_field :end_date, type: "date", class: "form-control" %>
      </div>

        <%= form.submit "Generate Plan", class: "btn btn-default" %>
    <% end %>
  </div>
</div>

<%= render(partial: "form") %>

This changes our top form to be in a panel to make it visually separate and we also make it an inline form since it’s so small.

Next, we’ll tackle the actual form, which currently looks horrible because we used the wrong class originally on our form groupings.

app/views/meal_plans/_form.html.erb

<%= render partial: "shared/errors" %>

<%= form_for @meal_plan, html: { class: "form-horizontal" } do |form| %>
  <%= form.hidden_field :start_date %>
  <%= form.hidden_field :end_date %>

  <div class="meals">
    <%= form.fields_for :meals do |meal_fields| %>
      <div class="meal">
        <%= meal_fields.hidden_field :id %>
        <%= meal_fields.hidden_field :date %>

        <div class="form-group margin-vertical-none">
          <div class="col-sm-11 col-sm-offset-1">
            <p><%= localize(meal_fields.object.date) %></p>
          </div>
        </div>

        <div class="form-group">
          <%= meal_fields.label :recipe_id, class: "col-sm-1 control-label" %>
          <div class="col-sm-11">
            <%= meal_fields.select :recipe_id, current_user.recipe_options, {}, class: "form-control" %>
          </div>
        </div>
      </div>
    <% end %>
  </div>

  <div class="form-group">
    <div class="col-sm-11 col-sm-offset-1">
      <%= form.submit class: "btn btn-primary" %>
    </div>
  </div>
<% end %>

This form probably has the most changes of any of our pages to improve the layout of the items. We’re not quite finished. We need to create this new class to remove vertical margin.

app/assets/stylesheets/application.scss

.margin-vertical-none {
  margin-top: 0 !important;
  margin-bottom: 0 !important;
}

Additionally, now is the time for us to improve how we display times. We called localize on the date for each Meal, but what does that do? It uses the date format that is localized for the user’s locale. We set this up by modifying the confing/locales/en.yml file and setting the date format.

config/locales/en.yml

en:
  date:
    formats:
      default: "%m/%d/%Y"

This doesn’t quite deal with all of it though. We also want to change how dates are rendered into strings to handle the to_s on MealPlan. So we need to utilize the localize method on any date we render in the views, and use I18n.localize when we interpolate a Date into a String in our models. We’ll start with MealPlan.

_app/models/mealplan.rb

def to_s
  "#{I18n.localize(start_date)} - #{I18n.localize(end_date)}"
end

Now in our views when we use <%= @meal_plan %> it will always render the date’s in the proper format. I’ll leave it up to you to click around the site and see what other areas you would need to change to display all of the dates properly.

Recap

We went through most of the pages in our application to apply some styles and make the application look better. These changes included utilizing some flex-box and Bootstrap styles. Additionally, we took a glimpse at some of the internationalization (I18n) features that we have at our disposal. There are still some more styles that need to be written and I’ll leave that up to you (or you can check the associations pull-request). Next week we will deploy this application to a server on Digital Ocean.

Learning Rails – Meal Plan UI (Part 6)

keith@coderjourney.com (Keith Thompson) — Thu, 15 Dec 2016 12:31:29 +0000

Today we’re going to build on the meal plan functionality from the previous tutorial and create the UI for a user to view and create new meal plans.

Goal for this Tutorial:

Display a Meal Plan.
Allow a User to create a meal plan.

The ability to have a MealPlan pre-fill its own meals took out the most complicated portion of a meal plan, and now we get to put a face on that functionality. Each MealPlan is not set to be specific length so we need to have a fairly dynamic user interface that can handle multiple lengths.

Here’s how I imagine we can work through creating a meal plan:

1) Specify the meal plan start and end dates (default to today, and six days from now). Submit the form. 2) Return a form that has a pre-filled Meal for each date in the date range. 3) Allow user to customize the recipe in each meal before creating the MealPlan.

After the MealPlan has been created we will allow people to delete the plan and edit the recipes. We’ll use a _form partial to handle the new/edit views since they’ll be of dynamic length based on the meal plan duration. We’ll be creating all of the CRUD actions for MealPlan in the same way that we did for Recipe a few tutorials back.

Creating a New Meal Plan

The first thing we’re going to work on will be the initial creation of a MealPlan. The previous tutorial took us through flushing out a simple plan before we ever display it to the user, and now we get to utilize that.

To start we need to create our routes and controller.

config/routes

resources :meal_plans

app/controllers/meal_plans_controller.rb

class MealPlansController < ApplicationController
  before_action :require_login

  def new
    @meal_plan = current_user.meal_plans.build(
      start_date: params[:start_date] || Date.today,
      end_date: params[:end_date] || 6.days.from_now.to_date
    )

    @meal_plan.build_meals
  end
end

We learned last time that we need to remember to require a user to be logged in for them to be able to work with things such as recipes or meal plans.

By default we’re going to create meal plans that are for a whole week, and before we render the view we want to call build_meals so that we have some pre-populated Meal objects.

The view for creating a new MealPlan will actually be a little different than what we did for Recipe, it won’t be the same view that we render for edit later on. We will actually have two forms on the page:

A form to select the date range. Submitting this will re-render the new meal plan page with the proper number of meals.
The Meal Plan form that allows you to customize the meals that were pre-populated.

New Meal Plan Forms

Let’s build write out the date range form first.

app/views/meal_plans/new.html.erb

<h1>New Meal Plan</h1>

<%= form_for @meal_plan, url: new_meal_plan_path, method: "GET" do |form| %>
  <div class="form-control">
    <%= form.label :start_date %>
    <%= form.text_field :start_date, type: "date" %>
  </div>

  <div class="form-control">
    <%= form.label :end_date %>
    <%= form.text_field :end_date, type: "date" %>
  </div>

  <div class="action">
    <%= form.submit "Generate Plan" %>
  </div>
<% end %>

<%= render partial: "form" %>

Notice that this is a pretty standard form other than the fact that we’re defining a url and method in our form_for call. We’re doing this because by default it would create a form that pointed to the “create” action in our controller and use the “POST” method. We want this to send our new dates to the “new” action and re render the page so we had to customize this a bit.

Next, we’ll create the form for customizing the meals. Since we know we’ll be working with this portion of the page in the “edit” flow we’ll go ahead and put it in the _form.html.erb partial now.

app/views/meal_plans/_form.html.erb

<% if @errors.present? %>
  <div class="errors">
    <ul>
      <% @errors.each do |error| %>
        <li><%= error %></li>
      <% end %>
    </ul>
  </div>
<% end %>

<%= form_for @meal_plan do |form| %>
  <%= form.hidden_field :start_date %>
  <%= form.hidden_field :end_date %>

  <div class="meals">
    <%= form.fields_for :meals do |meal_fields| %>
      <div class="meal">
        <%= meal_fields.hidden_field :id %>
        <%= meal_fields.hidden_field :date %>
        <p><%= meal_fields.object.date.to_s %></p>
        <div class="form-control">
          <%= meal_fields.label :recipe_id %>
          <%= meal_fields.select :recipe_id, current_user.recipe_options %>
        </div>
      </div>
    <% end %>
  </div>

  <div class="actions">
    <%= form.submit %>
  </div>
<% end %>

This form is quite a bit more complicated than our previous forms. We need to store off a good deal of information in hidden_fields, but the really interesting part is the form.fields_for :meals block. fields_for let’s us iterate through each of the Meal objects that we’ve created. meal_fields is an object that works just like our original form builder (form) except that it sets up the name for each field a little differently. We have hidden fields for the id of both the MealPlan and each Meal because we will need those when we get around to allowing a meal plan to be edited.

We are calling a method on current_user that doesn’t actually exist yet that will render out the options for the select box. Select fields in rails expect to receive and array of arrays in the following shape:

[
  ["Displayed Name 1", "hidden_value1"],
  ["Displayed Name 2", "hidden_value2"],
]

To do this with the user’s Recipe objects we’ll need to return the recipe name and the recipe id. Let’s add that method to User now.

app/models/user.rb

def recipe_options
  recipes.map do |recipe|
    [recipe.name, recipe.id]
  end
end

Meal Plan Create Action

Now that we have our forms we should be able to actually generate the MealPlan. Let’s create our create and meal_plan_params methods:

app/controllers/meal_plans_controller.rb

def create
  @meal_plan = current_user.meal_plans.build(meal_plan_params)

  if @meal_plan.save
    redirect_to meal_plan_path(@meal_plan), notice: "Meal plan created!"
  else
    @errors = @meal_plan.errors.full_messages
    render :new
  end
end

private

def meal_plan_params
  params.require(:meal_plan).permit(
    :start_date,
    :end_date,
    meals_attributes: [
      :id,
      :date,
      :recipe_id
    ]
  )
end

There really isn’t a whole lot going on in the create action that we haven’t seen before, but meal_plan_params does look a different. Since we used fields_for in our form we’re going to be receiving an array of Meal object fields when the form is submitted and we need to allow those to come through the strong parameters. We’re not quite done though, because the MealPlan model doesn’t know that it should be on the look out for meals_attributes. We need to add a line to MealPlan before we continue.

app/models/meal_plan.rb

accepts_nested_attributes_for :meals

If you submit the form now, you won’t see an error, but the form will re-render. This happens because our Meal objects can’t save. This happens because of the we’re not associating the uncreated MealPlan to the uncreated Meal. We can hook this up by adding another key to the has_many and belongs_to lines of our models:

app/models/meal_plan.rb

has_many :meals, -> { order(:date) }, inverse_of: :meal_plan

And in the Meal model:

app/models/meal.rb

belongs_to :meal, inverse_of: :meals

Now if we submit our form we should successfully create our meal plan and receive the error that we don’t have a show action. Before we build show though let’s clean up our view a little by removing the duplication in our application when it comes to rendering errors.

DRYing Up Form Errors

Our form partial and the recipe form share the same common error displaying markup and logic. We can extract that out so that when we change it both of the forms will be updated. Let’s create a subdirectory in views to hold our shared views (called shared)

$ mkdir -p app/views/shared

Now we’ll create a new partial called _errors.html.erb.

app/views/shared/_errors.html.erb

<% if @errors.present? %>
  <div class="errors">
    <ul>
      <% @errors.each do |error| %>
        <li><%= error %></li>
      <% end %>
    </ul>
  </div>
<% end %>

Now in the meal plan and recipe _form partials replace the error rendering block with a new render partial call.

<%= render partial: "shared/errors" %>

Showing & Listing Meal Plans

Displaying our meal plan won’t be too different from what we’ve done in the past. Here’s the show action:

app/controllers/meal_plans_controller.rb

def show
  @meal_plan = current_user.meal_plans.find(params[:id])
end

Here’s the corresponding view:

app/views/meal_plans/show.html.erb

<h1>Meal Plan: <%= @meal_plan %></h1>

<h2>Meals</h2>

<ul class="meals">
  <% @meal_plan.meals.each do |meal| %>
    <li class="meal">
      <%= meal.date %> - <%= link_to meal.recipe, recipe_path(meal.recipe) %>
    </li>
  <% end %>
</ul>

<div class="actions">
  <ul class="recipe-actions">
    <li>
      <%= link_to "Edit Meal Plan", edit_meal_plan_path(@meal_plan) %>
    </li>
    <li><%= link_to "Delete", meal_plan_path(@meal_plan), data: {
      confirm: "Are you sure you want to delete meal plan: #{@meal_plan}?",
    }, method: :delete %></li>
  </ul>
<div>

We’re trying to create a link to a meal plan by passing it in as the first argument to link_to. From a programmer point of view this is great, but unfortunately that won’t quite work as expected out of the gate. link_to is going to look for the String representation of the first argument and render that. With that knowledge in mind we can customize MealPlan and the way that it displays itself as a String.

app/models/meal_plan.rb

def to_s
  "#{start_date} - #{end_date}"
end

Now when we pass a MealPlan object into a helper that renders a string or simple render it to the page using a <%= meal_plan %> we will see a human readable start date and end date.

We will also define to_s on Recipe so that it prints out the name.

app/models/recipe.rb

def to_s
  name
end

Listing Meal Plans

Before we go about listing out meal plans we should think about what makes that index view for meal plans useful. When a user goes to this page what they probably want to see is his/her previous weeks, but more importantly the currently active plan. Because of this we’re not simply going to return a list to the view to have it rendered.

Here’s the index action:

app/controllers/meal_plans_controller.rb

def index
  @current_meal_plan = current_user.meal_plans.where("start_date <= ? AND end_date >= ?", Date.today, Date.today).first
  @meal_plans = current_user.meal_plans.order("start_date desc")
end

We need to use a custom SQL query in our where statement to ensure that we’re finding the currently active meal plan (or one of them). As for the rest of the @meal_plans we just want to make sure that we’re showing the most recent plans first.

Let’s take a look at the view that will display the plans.

app/views/meal_plans/index.html.erb

<h1>Meal Plans</h1>

<% if @current_meal_plan.present? %>
  <h2>Current Meal Plan</h2>

  <div>
    <ul class="meals">
      <% @current_meal_plan.meals.each do |meal| %>
        <li class="meal">
          <%= meal.date %> - <%= link_to meal.recipe.to_s, recipe_path(meal.recipe) %>
        </li>
      <% end %>
    </ul>
  </div>
<% end %>

<h3>All Meal Plans</h3>

<% @meal_plans.each do |meal_plan| %>
  <div class="meal-plan">
    <%= link_to meal_plan, meal_plan_path(meal_plan) %>
  </div>
<% end %>

<div class="actions">
  <%= link_to "New Meal Plan", new_meal_plan_path %>
</div>

Deleting a Meal Plan

Similarly to how we allow a user to delete a recipe, we will allow them to delete meal plans that they no longer want. The view logic already exists to make this work so we need to implement the controller action.

app/controllers/meal_plans_controller.rb

def destroy
  @meal_plan = current_user.meal_plans.find(params[:id])
  @meal_plan.destroy
  redirect_to meal_plans_path, notice: "Meal plan deleted!"
end

This basically matches what we’ve done before. Remember how we create a Meal for each date in the MealPlan? We should probably delete those whenever we delete the parent MealPlan. Thankfully, Rails/ActiveRecord makes this easy for use to set up on the association.

app/models/meal_plan.rb

has_many :meals, -> { order(:date) }, inverse_of: :meal_plan, dependent: :destroy

Adding the dependent: :destroy portion ensures that when this object is destroyed that it also destroys the meals.

Editing a Meal Plan

Lastly, we need to allow for a user to edit a meal plan. Here’s the edit view:

app/views/meal_plans/edit.html.erb

<h1>Edit Meal Plan: <%= @meal_plan %></h1>

<%= render(partial: "form") %>

Now onto the controller actions (edit & update). These should be looking familiar by now:

app/controllers/meal_plans_controller.rb

def edit
  @meal_plan = current_user.meal_plans.find(params[:id])
end

def update
  @meal_plan = current_user.meal_plans.find(params[:id])

  if @meal_plan.update_attributes(meal_plan_params)
    redirect_to meal_plan_path(@meal_plan), notice: "Meal plan updated!"
  else
    @errors = @meal_plan.errors.full_messages
    render :edit
  end
end

Recap

That’s the whole thing, we’ve successfully implemented the CRUD interactions for meal plans. Since this isn’t the first time we’ve implemented CRUD we went through this pretty quickly, but there were a few new things shown today:

Customizing how an object is rendered to the page via to_s.
Accepting nested attributes as part of a form submission.
How to potentially clean up views by extracting shared partials. We’ve seen partials before but they were always used by views in the same controller so shared/errors is a little different.

I hope you’ve enjoyed this tutorial, next week we’ll take a detour from adding new functionality to the application to improve the aesthetics and usability of our pages.

Learning Rails – Modeling Meal Plans (Part 5)

keith@coderjourney.com (Keith Thompson) — Thu, 08 Dec 2016 12:32:53 +0000

Today we’re going to start developing our main feature: meal planning. We’ll use test driven development to build out how a meal plan should work.

Goal for this Tutorial:

Test Drive the behavior of a meal plan.
Database modeling a meal plan.

Meal planning is the bread and butter of our application so we’re going to spend quite a bit of time thinking through how we’re going to model this so that it makes sense.

We need to find a way to structure a meal plan in our database and create a good interface for us to use a meal plan to build out a sane list of recipes to cook.

Renaming our Application

Before we get too far we’re going to need to make a change to our application config. When we generated the rails application we used the name of meal_plan which created a module for us to house our application configuration. This module is named MealPlan and because of that we can’t create a model named MealPlan. We’re going to change this module name to be MealPlanner. Here’s the new application.rb:

config/application.rb

require_relative 'boot'

require "rails"
# Pick the frameworks you want:
require "active_model/railtie"
require "active_job/railtie"
require "active_record/railtie"
require "action_controller/railtie"
require "action_mailer/railtie"
require "action_view/railtie"
require "action_cable/engine"
require "sprockets/railtie"

# Require the gems listed in Gemfile, including any gems
# you've limited to :test, :development, or :production.
Bundler.require(*Rails.groups)

module MealPlanner
  class Application < Rails::Application
    # Settings in config/environments/* take precedence over those specified here.
    # Application configuration should go into files in config/initializers
    # -- all .rb files in that directory are automatically loaded.

    config.generators do |g|
      g.test_framework :minitest, spec: true
    end
  end
end

Modeling a Meal Plan

Up until now we haven’t really thought about what a meal plan consisted of, but in order for us to model it in our database we need to lay out what it means to be a meal plan. Here are the requirements that I can think of off hand:

Start Date
End Date
List of Recipes
User

That seems pretty simple, but it’s going to require us to create a few different database tables. The meal_plans table isn’t actually going to have much on it besides the start_date and end_date. The more involve model will actually be the Meal model which will act to join together a Recipe and a MealPlan. Our Meal model will need to have the following:

date
recipe_id
meal_plan_id

We need the date so that we know when we’re planning to cook the recipe. The Meal will hold onto both the meal_plan_id and recipe_id because recipes will be on numerous meal plans. We could get around this by having the MealPlan model have seven different recipe columns (monday_recipe, tuesday_recipe, etc), but this is less than ideal because what do we do when we also want to plan out lunches instead of just dinner? By having a separate model to associate recipes and meal plans we can be more explicit about what a meal is and that will allow us to be flexible moving forward.

Let’s create these models now:

$ rails g migration create_meal_plans
      invoke  active_record
      create    db/migrate/20161126154052_create_meal_plans.rb

Modifying the migration that was created we’ll add our date fields.

db/migrate/20161126154052_create_meal_plans.rb

class CreateMealPlans < ActiveRecord::Migration[5.0]
  def change
    create_table :meal_plans do |t|
      t.date :start_date, null: false
      t.date :end_date, null: false
      t.references :user, null: false, foreign_key: true
      t.timestamps null: false
    end
  end
end

Now we can generate our meals table:

$ rails g migration create_meals
      invoke  active_record
      create    db/migrate/20161126154355_create_meals.rb

db/migrate/20161126154355_create_meals.rb

class CreateMeals < ActiveRecord::Migration[5.0]
  def change
    create_table :meals do |t|
      t.date :date, null: false
      t.references :meal_plan, null: false, foreign_key: true
      t.references :recipe, null: false, foreign_key: true
      t.timestamps null: false
    end
  end
end

Since we have two new models we will go ahead and also create our factories now, leaving the implementations empty for a second:

$ rails g factory_girl:model MealPlan
      create  test/factories/meal_plans.rb
$ rails g factory_girl:model Meal
      create  test/factories/meals.rb

Finally, we’ll run our migrations and get ready to test drive these new models:

rake db:migrate

Validating the MealPlan and Meal Models

The initial set up of our MealPlan model will be pretty simple so we’re going to quickly write the tests for that.

test/model/meal_plan_test.rb

require "test_helper"

describe MealPlan do
  describe "validity" do
    let(:meal_plan) { MealPlan.new }

    before do
      meal_plan.valid?
    end

    it "requires a start_date" do
      meal_plan.errors[:start_date].must_include "can't be blank"
    end

    it "requires an end_date" do
      meal_plan.errors[:end_date].must_include "can't be blank"
    end

    it "requires a user" do
      meal_plan.errors[:user].must_include "can't be blank"
    end
  end
end

We can implement this right away based on what we’ve learned in a previous tutorial.

app/model/meal_plan.rb

class MealPlan < ApplicationRecord
  belongs_to :user

  validates :start_date, presence: true
  validates :end_date, presence: true
  validates :user, presence: true
end

Moving onto Meal we’ll have similarly simple tests to write:

test/models/meal_test.rb

require "test_helper"

describe Meal do
  describe "validity" do
    let(:meal) { Meal.new }

    before do
      meal.valid?
    end

    it "requires a date" do
      meal.errors[:date].must_include "can't be blank"
    end

    it "requires a meal_plan" do
      meal.errors[:meal_plan].must_include "can't be blank"
    end

    it "requires a recipe" do
      meal.errors[:recipe].must_include "can't be blank"
    end
  end
end

The initial implementation for Meal is also simple:

app/model/meal.rb

class Meal < ApplicationRecord
  belongs_to :meal_plan
  belongs_to :recipe

  validates :date, presence: true
  validates :meal_plan, presence: true
  validates :recipe, presence: true
end

Generating a Meal Plan

While we will definitely allow users to create meal plans customizing all of the recipes that are used, but we’re really wanting meal planning to take little thought so we’ll generate the initial list of recipes ourselves. Thankfully we can write tests to ensure that this works. Let’s open the test file for MealPlan back up and go through what we would like it to look like to generate a meal plan.

test/models/meal_plan_test.rb

require "test_helper"

describe MealPlan do
  # Validation tests ...

  describe "generating a weekly plan" do
    let(:meal_plan) { build(:meal_plan) }

    before do
      7.times do
        create(:recipe, user: meal_plan.user)
      end
    end

    it "populates a meal for each date between start and end" do
      meal_plan.meals.size.must_equal 0

      meal_plan.build_meals

      meal_plan.meals.size.must_equal 7
    end
  end
end

Running the test right now will show you that we haven’t implemented our factory yet, so let’s do that now:

test/factories/meal_plans.rb

FactoryGirl.define do
  factory :meal_plan do
    start_date { Date.today }
    end_date { 6.days.from_now }

    association(:user)
  end
end

Running the tests now (rake tests), we get the error that build_meals doesn’t exist, which is just what we needed. Let’s follow strict TDD and add an empty method.

app/models/meal_plan.rb

  def build_meals
  end

With that we get an expected error saying that we expected 7 but got 0. Now we get to actually implement this method. We need to create a meal for each date so we can use a ruby range for that.

app/models/meal_plan.rb

  def build_meals
    user_recipe_ids = user.recipes.pluck(:id)

    (start_date..end_date).each do |date|
      available_recipe_ids = user_recipe_ids - meals.map(&:recipe_id)

      meals.build(date: date, recipe_id: available_recipe_ids.sample)
    end
  end

This will get our test passing, but just looking at this it won’t handle the case where a user doesn’t have enough recipes to fill out a full week. Let’s write some tests to handle that and tweak this to work.

test/models/meal_plan_test.rb

  describe "generating a weekly plan" do
    let(:meal_plan) { build(:meal_plan) }

    before do
      7.times do
        create(:recipe, user: meal_plan.user)
      end
    end

    it "populates a meal for each date between start and end" do
      meal_plan.meals.size.must_equal 0

      meal_plan.build_meals

      meal_plan.meals.size.must_equal 7
    end

    it "builds valid meals" do
      meal_plan.build_meals

      meal_plan.meals.all?(&:valid?).must_equal true
    end

    it "uses unique recipes" do
      meal_plan.build_meals

      uniq_ids = meal_plan.meals.map(&:recipe_id).uniq
      uniq_ids.size.must_equal 7
    end

    describe "with more days than recipes" do
      let(:meal_plan) { build(:meal_plan, end_date: 8.days.from_now) }

      it "build valid meals" do
        meal_plan.build_meals

        meal_plan.meals.all?(&:valid?).must_equal true
      end

      it "reuses recipes where necessary" do
        meal_plan.build_meals

        uniq_ids = meal_plan.meals.map(&:recipe_id).uniq
        uniq_ids.size.must_equal 7
      end
    end
  end

Our extra test cases make this work by extending the length of the meal plan to have more days than recipes we created. These two tests will fail because we will attempt to build meals with nil for the recipe_id. Here’s how we can fix this:

app/models/meal_plan.rb

  def build_meals
    user_recipe_ids = user.recipes.pluck(:id)

    (start_date..end_date).each do |date|
      unused_recipes = user_recipe_ids - meals.map(&:recipe_id)

      available_recipes = unused_recipes.empty? ? user_recipe_ids : unused_recipes
      meals.build(date: date, recipe_id: available_recipes.sample)
    end
  end

This change will change the recipe ids we pull from after we’ve used all of the unique ones, and will get our tests passing.

Recap

We’ve created a few more models that we’ll use to create a useful meal planning application and we wrote some logic to actually randomize a list of meals. In the next tutorial we’ll use this logic to build out a meal plan and pre-populate the user interface while still allowing the user to customize the recipes that are being used.

Learning Rails – Recipes Controller and Views (Part 4)

keith@coderjourney.com (Keith Thompson) — Tue, 29 Nov 2016 15:05:49 +0000

Today we’re going to use the Recipe that we created in the previous controller, setting up controller actions and views so that a user can create and manage his or her recipes.

Goal for this Tutorial:

Create CRUD actions for the Recipe model.
Restrict Routes to only logged in Users.

In the previous tutorial we did most of the heavy lifting around Recipes. Knowing that our data makes sense, and preventing obvious data corruption problems was a good start. Today we’re going to create the common workflow views that a User will take to manipulate recipes.

For this to work we’re going to need to set up the following routes:

/recipes - Listing out all of the recipes that a User has. Will also show a link to create a new recipe.
/recipes/new - Shows a form for creating a new Recipe. Should be able to cancel and return to the listing.
/recipes/:id - Shows the details for an individual recipe. Should also show buttons to edit and delete the recipe.

All of the interactions we’ve just described are often referred to as “CRUD” which stands for Create Read Update Delete.

Recipes Controller, Views, and Routes

Before we get too far into the code that will actually perform the CRUD we’ll set up the portions that view content to the user. We’re not going to spend any time making this look pretty, but we will make sure that the html structure is semantic so that when we come back later to style this we don’t have a bad time.

To start we know we’re going to need a new view directory for recipes so we’ll create that now:

$ mkdir -p app/views/recipes

We’re also going to need the controller so we’ll add an empty on at app/controllers/recipes_controller.rb

app/controllers/recipes_controller.rb

class RecipesController < ApplicationController
end

Lastly, we’re going to need to set up our routes.

config/routes.rb

Rails.application.routes.draw do
  # For details on the DSL available within this file, see http://guides.rubyonrails.org/routing.html
  root to: "welcome#show"

  resources :recipes
end

Now if we check our rake routes it should now include the following:

           recipes GET    /recipes(.:format)                      recipes#index
                   POST   /recipes(.:format)                      recipes#create
        new_recipe GET    /recipes/new(.:format)                  recipes#new
       edit_recipe GET    /recipes/:id/edit(.:format)             recipes#edit
            recipe GET    /recipes/:id(.:format)                  recipes#show
                   PATCH  /recipes/:id(.:format)                  recipes#update
                   PUT    /recipes/:id(.:format)                  recipes#update
                   DELETE /recipes/:id(.:format)                  recipes#destroy

Rendering Recipes

Now that we have the backbone of this portion of the app there we can start building on it. The first thing we’re going to do is display the existing recipes. We’ll do this in the index.html.erb view.

app/views/recipes/index.html.erb

<h1>Recipes</h1>

<div class="recipes">
  <% @recipes.each do |recipe| %>
    <div class="recipe">
      <h3><%= link_to recipe.name, recipe_path(recipe) %></h3>
      <p><%= recipe.description %></p>
      <ul class="recipe-actions">
        <li><%= link_to "Edit", edit_recipe_path(recipe) %></li>
        <li><%= link_to "Delete", recipe_path(recipe), data: {
          confirm: "Are you sure you want to delete: #{recipe.name}?"
        }, method: :delete %></li>
      </ul>
    </div>
  <% end %>
</div>

We need to create the @recipes instance variable in the index action of the RecipesController.

app/controllers/recipes_controller.rb

class RecipesController < ApplicationController
  def index
    @recipes = Recipe.all
  end
end

Now if you go to localhost:3000/recipes you should see the following:

Creating Fake Recipes

When you bring a new person onto an application it’s usually good to set them up with some example data. Thankfully, Rails provides us with a means to do this easily through the db/seeds.rb file and the rake db:seed rake task. If you run rake db:seeds in a development environment we’ll delete the data that’s currently in the system and set up a new User who has 20 Recipes. To make the data slightly more useful we’re going to pull in the “faker” gem to make our recipes more unique. Let’s first grab the gem:

Gemfile

# other gems
group :development, :test do
  gem 'byebug', platform: :mri
  gem 'minitest-rails'
  gem 'factory_girl_rails'
  gem 'faker'
end
# other gems

We put it into the development and test groups so that we can use Faker in our recipe factory. Let’s change the factory to use that now:

test/factories/recipes.rb

FactoryGirl.define do
  factory :recipe do
    name { Faker::Hipster.sentence }
    description  { Faker::Hipster.paragraph }
    association(:user)
  end
end

Putting the Faker portions in a block (the { ... }) it means that every time FactoryGirl generates a recipe it will call those methods again. If we didn’t do that then every factory would come out have the same name and description. Now we can set up our seeds file

db/seeds.rb

exit if !Rails.env.development?

puts "Deleting Records"
Recipe.delete_all
User.delete_all

puts "Creating User"
user = FactoryGirl.create(:user, email: "test@example.com")

puts "Creating Recipes"
20.times do
  FactoryGirl.create(:recipe, user: user)
end

Since we’re taking the very destructive action of deleting all Recipes and Users we want to make sure that this file will never accidentally be run in an environment that isn’t used for development purposes. It would currently fail to finish in say production because the “faker” gem would be missing, but it wouldn’t hit that failure until after the damage was already done.

Run rake db:seed now and you’ll populate your database with some recipes. If you then refresh localhost:3000/recipes you should see something like this:

While it’s not pretty, the data is structured in the way that we expect.

Viewing an Individual Recipe

While it’s great to see the entire list of recipes we also need to be able to see and link to a single recipe (by clicking on its name). For this to work we need to build the “show” action and view. Let’s build the action first.

app/controllers/recipes_controller.rb

def show
  @recipe = Recipe.find(params[:id])
end

Pretty simple, we just need to grab the recipe from the database. Onto the view:

app/views/recipes/show.html.erb

<h1><%= @recipe.name %></h1>

<h2>Description:</h2>
<p><%= @recipe.description %></p>

<div>
  <ul class="recipe-actions">
    <li><%= link_to "Back", recipes_path %></li>
    <li><%= link_to "Edit", edit_recipe_path(@recipe) %></li>
    <li><%= link_to "Delete", recipe_path(@recipe), data: {
      confirm: "Are you sure you want to delete: #{@recipe.name}?"
    }, method: :delete %></li>
  </ul>
</div>

We duplicated the recipe actions from our index page, but we added an additional item for “Back” which will take the user back to the list of recipes. We also had to make sure that we were using @recipe in our embedded ruby instead of recipe like we did on the index page.

Now let’s wire up the “Delete” link to remove the recipe.

Deleting a Recipe

Our “Delete” link is already set up to go to the proper route, but the thing that makes it different is that it’s set up to use the method of delete. This is referring to the HTTP method, and if you look at the routes again, you’ll see that the line including the “DELETE” shows that it should go to “recipes#destroy”.

DELETE /recipes/:id(.:format) recipes#destroy

This is the action that we’ll need to set up in the controller to make this link work properly. Let’s create that now.

app/controllers/recipes_controller.rb

def destroy
  recipe = Recipe.find(params[:id])
  recipe.destroy
  redirect_to recipes_path, notice: "Deleted Recipe: #{recipe.name}"
end

First we find the recipe. We then call the destroy method which will trigger any callbacks that may exist and remove the database row. We then redirect back to the index page and include a flash message to say that the recipe was deleted. Notice that we were still able to access the name of the recipe after we “destroyed” it.

Creating a New Recipe

The create and update portions of CRUD are remarkably similar. This will allow us to build out the “New Recipe” workflow and basically get the “Edit Recipe” workflow for free.

The first thing that we need to do is set up the “new” action and view.

app/controllers/recipes_controller.rb

def new
  @recipe = Recipe.new
end

app/views/recipes/new.html.erb

<h1>New Recipe</h1>

<%= render partial: :form %>

We’re going to separate out the “form” portion of this view into a “partial” view so that we can use it in multiple views. Partial views have special names that begin with an underscore, so our “form” partial will be “_form.html.erb” Here’s what that’s going to look like:

app/views/recipes/_form.html.erb

<% if @errors.present? %>
  <div class="errors">
    <ul>
      <% @errors.each do |error| %>
        <li><%= error %></li>
      <% end %>
    </ul>
  </div>
<% end %>

<%= form_for @recipe do |form| %>
  <div class="form-group">
    <%= form.label :name %>
    <%= form.text_field :name %>
  </div>

  <div class="form-group">
    <%= form.label :description %>
    <%= form.text_area :description %>
  </div>

  <div class="actions">
    <%= form.submit %>
  </div>
<% end %>

There are quite a few things going on here, but those will make more sense after we finish the “create” action. If you head to “localhost:3000/recipes/new” you should see the form now.

Before we try to use the form let’s build out the “create” action (otherwise the form will fail on submit).

app/controllers/recipes_controller.rb

def create
  @recipe = Recipe.new(recipe_params)

  if @recipe.save
    redirect_to recipe_path(@recipe), notice: "Recipe Created!"
  else
    @errors = @recipe.errors.full_messages
    render :new
  end
end

def recipe_params
  params.require(:recipe).permit(:name, :description)
end

The create method is where the magic happens, but we’re also using a feature of Rails called “strong parameters” in the recipe_params method. Strong parameters allow us to specify which parameters are allowed to be submitted and ensures that we’re not passing malicious fields from the user to our models (and thus our database connection). This is a great security feature.

Looking at the action again, if the new recipe with the data from the form successfully saves then we will redirect to the show page for the recipe. If the call to save fails, then we’ll store the errors that occurred in a variable so that we can display these on the form that we render. To see this in action submit the form without filling in any of the values. You should see this:

This is great besides the missing user bit. Up to this point we’ve just been working with Recipes without worrying about the user, but that also meant that we were showing and allowing any visitor to edit/delete recipes. Time to add some security.

Authenticating the Recipes Controller

Thankfully, we won’t have to make a ton of changes to get the recipes controller locked down and working with the logged in user’s content. The first thing that we need to do is set up a before_action

app/controllers/recipes_controller.rb

class RecipesController < ApplicationController
  before_action :require_login

  # ...
end

With just that change, going to the recipes index page will redirect us to the sign in page. That did the entire job of restricting access to the pages, but it didn’t ensure that the user is only working with his or her recipes. To do that we’re going to remove all of the uses of Recipe in this controller, instead accessing the recipes through the current_user. Here’s our current controller, modified to work this way:

app/controllers/recipes_controller.rb

class RecipesController < ApplicationController
  before_action :require_login

  def index
    @recipes = current_user.recipes.order(:id)
  end

  def show
    @recipe = current_user.recipes.find(params[:id])
  end

  def new
    @recipe = current_user.recipes.build
  end

  def create
    @recipe = current_user.recipes.build(recipe_params)

    if @recipe.save
      redirect_to recipe_path(@recipe), notice: "Recipe Created!"
    else
      @errors = @recipe.errors.full_messages
      render :new
    end
  end

  def destroy
    recipe = current_user.recipes.find(params[:id])
    recipe.destroy
    redirect_to recipes_path, notice: "Deleted Recipe: #{recipe.name}"
  end

  private

  def recipe_params
    params.require(:recipe).permit(:name, :description)
  end
end

In a nut-shell, we’ve made the following changes:

"Recipe.all" became "current_user.recipes"
"Recipe.new" became "current_user.recipes.build"
"Recipe.find" became "current_user.recipes.find"

That won’t quite work yet though because the User class doesn’t know that it has recipes yet.

app/models/user.rb

class User < ApplicationRecord
  include Clearance::User

  has_many :recipes
end

Now if we go to “localhost:3000/recipes/new” and submit the form without any data again we should see a slightly different set of errors:

If we fill out the form properly and submit again we will have successfully created a recipe!

Editing a Recipe

I mentioned before that creating a new recipe and editing a recipe were very close to being the same thing. We’re going to reuse the “form” partial so our edit view will be the following:

app/views/recipes/edit.html.erb

<h1>Edit Recipe</h1>

<%= render partial: "form" %>

Looks pretty familiar. From here we need to set up the “edit” and “update” actions in our controller. Those will also look pretty familiar:

app/controllers/recipes_controller.rb

def edit
  @recipe = current_user.recipes.find(params[:id])
end

def update
  @recipe = current_user.recipes.find(params[:id])

  if @recipe.update_attributes(recipe_params)
    redirect_to recipe_path(@recipe), notice: "Recipe Updated!"
  else
    @errors = @recipe.errors.full_messages
    render :edit
  end
end

The differences here are that instead of using build we’re using find because the recipes already exist, and instead of save we use update_attributes. Besides those small changes these actions have the exact same shape as the “new” and “create” actions.

Recap

We covered a lot in this tutorial. Our application can actually do things! Since we have the UI and associated actions wrapped around recipes it could actually work as a functional recipe book for a User. The application is better rigged up to onboard new team members with our seed and fake data generation. You’ve now seen a lot of the features of Rails that thousands of developers use every day to be productive in their jobs.

Learning Rails – Modeling a Recipe and Testing (Part 3)

keith@coderjourney.com (Keith Thompson) — Tue, 22 Nov 2016 17:30:49 +0000

Today we’re going to build out the data model for managing recipes. In Rails, models are very important so we’re going to do a deep dive on just that portion of the feature. We’ll also write automated tests to help us better understand the code that we’re writing.

Goal for this Tutorial:

Create a new model for Recipe.
Write specs to ensure we’re modeling our data correctly.
Understand model validations.

One of the main features of our application is that you can create recipes and eventually be able to generate meal plans from it. Before we can have meal plans we’re going to need Recipes. We’ll tackle most of how a Recipe will work today.

Most of what we do today will be in code, but we’re going to need to run some things from the command line also, so we’re going to keep a bash prompt opened up to our rails container:

docker-compose exec app bash

Modeling a Recipe

Our Recipe model is going to start out very simple, but we will likely make it more complicated in the future. We’re going to have our recipes’ have a unique name and a description. The first thing that we need to do is create our database table using the migration generator:

$ rails generate migration create_recipes

We’re only creating the file with the timestamp using that generator, we’ll write the contents ourselves. Your file name will be slightly different.

db/migrate/20161119131322_create_recipes.rb

class CreateRecipes < ActiveRecord::Migration[5.0]
  def change
    create_table :recipes do |t|
      t.string :name, null: false
      t.text :description, null: false
      t.belongs_to :user, null: false, foreign_key: true

      t.timestamps null: false
    end

    add_index :recipes, [:user_id, :name]
  end
end

That will set up the fields that we want and ensure that a single user doesn’t create the same recipe over and over again. Don’t forget to run this migration:

$ rake db:migrate

Introducing Test Driven Development (TDD)

Since Recipe is going to be one of the main models that we use in our application we want to make sure that we’re setting it up properly. In Ruby, writing automated tests is a common practice, as it should be in most settings. Testing is even more important in Ruby since we don’t have a compiler to check our work as we go. We’re going to test drive our Recipe model so that we know it works the way we expect it to.

When we generated this application we skipped test, but we’re going to set it up now to use minitest.

Setting up Minitest & FactoryGirl

We’ll use the minutest-rails gem and the generators and rake tasks that it provides to get our test suite up and running. This gem needs to be added to the development and test groups, because it’s not necessary when we put our application into production. We’ll also pull in factory_girl_rails to help us generate data in our tests.

Gemfile

# Other gems
group :development, :test do
  gem 'byebug', platform: :mri
  gem 'minitest-rails'
  gem 'factory_girl_rails'
end
# Other gems

From there we’ll, need to rebuild our image and restart our container:

$ docker-compose stop app && docker-compose build && docker-compose up -d app

From within the container (docker-compose exec back in), we’ll run the generate to create our test helper:

$ rails g minitest:install

We’ll modify the test helper to not load fixtures since we won’t be using them:

test/test_helper.rb

ENV["RAILS_ENV"] = "test"
require File.expand_path("../../config/environment", __FILE__)
require "rails/test_help"
require "minitest/rails"

# To add Capybara feature tests add `gem "minitest-rails-capybara"`
# to the test group in the Gemfile and uncomment the following:
# require "minitest/rails/capybara"

# Uncomment for awesome colorful output
# require "minitest/pride"

class ActiveSupport::TestCase
  # Add more helper methods to be used by all tests here...
  include FactoryGirl::Syntax::Methods
end

We’ll make some modifications to how generators work so that rails knows to create a minutest-spec file if we generate something that would create a test in the future. Here’s the complete config that our application has thus far:

config/application.rb

require_relative 'boot'

require "rails"
# Pick the frameworks you want:
require "active_model/railtie"
require "active_job/railtie"
require "active_record/railtie"
require "action_controller/railtie"
require "action_mailer/railtie"
require "action_view/railtie"
require "action_cable/engine"
require "sprockets/railtie"

# Require the gems listed in Gemfile, including any gems
# you've limited to :test, :development, or :production.
Bundler.require(*Rails.groups)

module MealPlan
  class Application < Rails::Application
    # Settings in config/environments/* take precedence over those specified here.
    # Application configuration should go into files in config/initializers
    # -- all .rb files in that directory are automatically loaded.

    # Configure Generators
    config.generators do |g|
      g.test_framework :minitest, spec: true
    end
  end
end

Now we’ll set up our test directory to have some subdirectories.

$ mkdir -p test/{factories,models}
$ touch test/factories/.gitkeep

Finally, we will create our first spec to make sure that we set everything up properly. It’s important that we name our file properly, with a “_test” at the end. We’re looking to make a test that fails with the error we expect.

test/models/recipe_test.rb

require "test_helper"

describe Recipe do
  it "will fail" do
    assert true == false
  end
end

Now we can run this test:

$ rails test

And we should see the following failure:

Test Driving our Recipe Model

Now that we have our testing environment it’s time to actually do the testing. The idea with Test Driven Development is that we’ll write our tests first, showing how we want our finished code to behave. Running the test it will fail, and we will incrementally make the test pass by reading the errors that are given and making those errors go away. When you’re first starting out with a language I think TDD is great for helping you get to know the error messages.

Our tests will revolve around making sure that our recipe validations work properly. We’ll write all of the specs up front, but when we’ll only worry about fixing them one at a time.

test/models/recipe_test.rb

require "test_helper"

describe Recipe do
  describe "validity" do
    let(:recipe) { Recipe.new }

    before do
      recipe.valid?
    end

    it "requires a user" do
      recipe.errors[:user].must_include "can't be blank"
    end

    it "requires a description" do
      recipe.errors[:description].must_include "can't be blank"
    end

    it "requires a name" do
      recipe.errors[:name].must_include "can't be blank"
    end

    it "requires the name be unique for the same user" do
      existing_recipe = FactoryGirl.create(:recipe)
      recipe.name = existing_recipe.name
      recipe.user = existing_recipe.user
      recipe.valid?

      recipe.errors[:name].must_include "has already been taken"
    end

    it "does not require the name be unique with different users" do
      existing_recipe = FactoryGirl.create(:recipe)
      recipe.name = existing_recipe.name
      recipe.valid?

      recipe.errors[:name].wont_include "has already been taken"
    end
  end
end

Before we can run these tests we’ll need to create our factories. We’ll use the factory_girl:model generator to create the files and then fill out the values that we need.

$ rails g factory_girl:model User
$ rails g factory_girl:model Recipe

Now let’s fill in the information that would work to create a valid User:

test/factories/users.rb

FactoryGirl.define do
  factory :user do
    sequence(:email) { |n| "example_user_#{n}@example.com" }
    password "password"
  end
end

We didn’t create any validations on our User, but the module that we included from Clearance did. One of those validations is that the email must be unique. Because of this we use a sequence to make sure that every time we use FactoryGirl to create a User it will have a different email address.

Next we will need to create our recipe factory.

test/factories/recipes.rb

FactoryGirl.define do
  factory :recipe do
    name "Mom's Spaghetti"
    description "The best pasta in the world."
    association(:user)
  end
end

The only thing that’s really notable here is that we use the association(:user) method to make sure that every recipe we create has an associated User object. We can specify a different user when we use the factory, but by default it will create a new one.

Now we should be able to run our tests:

Getting the Tests to Pass

A few of our tests fail because of an undefined method user=, since this is failing before we even hit the assertion we should fix that first. The reason for this failure is because the Recipe model doesn’t actually know that is associated to a User yet. It’ll only take one line to fix this error.

app/models/recipe.rb

class Recipe < ApplicationRecord
  belongs_to :user
end

If we rerun the tests now we won’t see the undefined method error any more. Next, we’re going to fix the errors from the missing “can’t be blank” errors. These message come from the “presence” validation that we haven’t set up yet.

app/models/recipe.rb

class Recipe < ApplicationRecord
  belongs_to :user

  validates :name, presence: true
  validates :description, presence: true
  validates :user, presence: true
end

Rerunning the tests now will show that we only have one test failing. We’re missing a “uniqueness” validation on the name attribute. Let’s add that now and see what happens:

app/models/recipe.rb

class Recipe < ApplicationRecord
  belongs_to :user

  validates :name, presence: true, uniqueness: true
  validates :description, presence: true
  validates :user, presence: true
end

Now we’re getting a different failure. The validation that we currently have for uniqueness ensures that there will only ever be one recipe in the database with a specific name, but that’s not exactly what we want. We want to make sure that each User can only have one recipe of a specific name, but multiple users can have recipes that share a common name. We can accomplish this by tweaking our uniqueness validation a little to use a scope.

app/models/recipe.rb

class Recipe < ApplicationRecord
  belongs_to :user

  validates :name, presence: true, uniqueness: { scope: :user_id }
  validates :description, presence: true
  validates :user, presence: true
end

Rerunning the tests show that they’re all passing! Using a hash instead of a true value allowed us to be a little more specific and say that the uniqueness only matters if the recipe rows have the same “user_id” value.

Recap

This was a feature packed portion even though we didn’t completely finish a feature. Models and validations are really important when working with Rails so it’s really good to build a good understanding of them. We also went through a whirlwind tour of testing. Automated testing is one of my favorite tools to use in programming, both for learning and also for ensuring that I don’t write bugs as I continue development on a project. Since the project is set up for testing now it will be a lot easier for us to use them moving forward.

Learning Ruby on Rails – User Sign Up and Authentication (Part 2)

keith@coderjourney.com (Keith Thompson) — Thu, 17 Nov 2016 23:30:16 +0000

Today we continue our series learning Ruby on Rails. Having set up the project in the first episode, we can now start developing our first feature: User sign up, log in, and log out.

Goal for this Tutorial:

** Set up Authentication in a Rails App. ** Weigh Gem Pros/Cons to pick dependencies. ** Create simple Controller, View, and Route for the home page.

User sign up and authentication is pretty much a solved problem. So many applications have user sign in. Because of this, we’re not going to build this feature from scratch. We’re going to pull in a library to handle this part of the application for us.

Before we can get started we’re going to need to pick a library, and we’ll be picking one from these three:

The features that we really want are: user authentication over email/password, and password reset.

Deciding on a Gem

All three of the gems we reviewed are great. Devise is without a doubt the most popular gem for authentication and is well battle tested. Sorcery is the least intrusive solution, providing us with methods that we manually set up and not having an opinion on routes/views. Clearance is simple and maintained by a very reputable company in the Ruby/Rails community.

Clearance is what we’re going to go with for this project. It meets our requirements, and doesn’t try to do much more.

Installing the Gem

Clearance has good documentation, and we need to follow the installation portion right now. The firs thing that we need to do is add the gem to our Gemfile:

# ... Other gems
gem 'clearance', '~> 1.15.1'
# ... Other gems

Take note of the ~> sign in the version number. That does “pessimistic” version pinning, which means that it will allow installation of iterations in the right most version position. In our case this means we could install 1.15.1 now, but if 1.15.2 came out tomorrow we could bundle update to that without changing our Gemfile. We would not be able to update to 1.16.0 without changing this line though. We would write the current line as ~> 1.15 if we wanted to be able to install any minor version above 15.

Since we’ve added a new gem we need to bundle install that. We’re going to stop our containers, rebuild, and start them back up. Building does the bundle install step that we need, but it also bakes the gems into the image.

$ docker-compose build && docker-compose run --rm app bash

Once the application container is rebuilt we’re going to do a few more things before restarting it.

Setting Up Clearance

Clearance comes packaged with some commands to help us get up and running quickly. The first one we’re going to use is clearance:install

$ rails generate clearance:install

That command will create or change the following:

Create the User model or change it if you already had one
Generate a migration to set up the fields on User that clearance needs
Change the ApplicationController to include the clearance methods (through a module)
Create an initializer to allow us to configure clearance

The output from the installation also gives us a few more changes we can make and tells us to migrate our database. We’ll do those things also, doing the migration first.

$ rake db:migrate

Here is the change we’ll make to our app/views/layouts/application.html.erb:

<!DOCTYPE html>
<html>
  <head>
    <title>MealPlan</title>
    <%= csrf_meta_tags %>

    <%= stylesheet_link_tag    'application', media: 'all' %>
    <%= javascript_include_tag 'application' %>
  </head>

  <body>

    <header>
      <% if signed_in? %>
        Signed in as: <%= current_user.email %>
        <%= button_to 'Sign out', sign_out_path, method: :delete %>
      <% else %>
        <%= link_to 'Sign in', sign_in_path %>
      <% end %>
    </header>

    <section>

      <div id="flash">
        <% flash.each do |key, value| %>
          <div class="flash <%= key %>"><%= value %></div>
        <% end %>
      </div>

      <%= yield %>
    </section>
  </body>
</html>

Now that we have the clearance initializer we can start the application.

docker-compose up -d

Signing Up (and Creating a Home Page)

Clearance by default provides us with routes and views through a rails engine. This means that we don’t need to set up much of any code in our application to get up and running. It’s suggested that you not use this feature so that you have more control, so we will actually create routes/views, but for the time being we’re going to take this for a spin.

You can head to localhost:3000/sign_up to actually create a User for yourself. The form isn’t pretty, but that’s ok since it’s our responsibility to style our site.

After you sign up you’ll be redirect back to the “Yay! You’re on Rails” page, which doesn’t actually show us the change we made to our application layout. To get around this we’re going to create our home page.

First we need to create our controller, we’ll call it the WelcomeController and put it at app/controllers/welcome_controller.rb. In Rails the naming of classes and files is very important so make sure that these match up.

class WelcomeController < ApplicationController
  def show
  end
end

That’s all we need in the controller because there won’t be any dynamic data passed to the view we’re going to display (our show view). Next, we’ll create the directory in views, and the view itself.

$ mkdir -p app/views/welcome
$ echo "<h1>Welcome</h1>" > app/views/welcome/show.html.erb

Lastly, we need to set up our root route in config/routes.rb.

Rails.application.routes.draw do
  # For details on the DSL available within this file, see http://guides.rubyonrails.org/routing.html
  root to: "welcome#show"
end

Now if we look at localhost:3000 again we can see that we’re logged in:

Recap

Later on, we will customize the views and routes that clearance has given us, but for the time being we’ve successfully implemented the following features:

User Sign Up
User Sign In/Out

Moving forward we will implement features that are specific to our domain so we won’t be able to use a gem to do all of the work.

Learning Ruby on Rails – Project Setup & Planning (Part 1)

keith@coderjourney.com (Keith Thompson) — Tue, 08 Nov 2016 23:27:24 +0000

Today we kick off our series to learn Ruby on Rails. In this tutorial, we’re going to be laying out our project structure, preview the features that we’re going to build, and do some project planning. We’re not going to write a ton of code today, but we will look into some of the very important non-coding work that goes into building a web application.

Goal for this Tutorial:

Explore Project Goals
Create basic Rails app
Use Git and Github
Plan Project features

The application that we’re going to be building over the course of this tutorial series is an application for planning meals based on recipes that a user stores. I originally built some of this application for a conference talk I gave in April 2016, but I built it using Elixir and Elm. We’ll be rebuilding this application from scratch using Ruby and Rails 5, but we can take a look at the original application for a general idea of some early features to build.

This entire tutorial series is going to be developed using Docker to run our development environment. I released a two part tutorial about using Docker while developing a Rails app, so check those out before we get started:

Docker in Development

Exploring the Existing Project

Before we start anything related to code, let’s go over the problem that we’re trying to solve.

My wife and I use a weekly meal plan to help us determine what we should buy when we go grocery shopping, but it’s annoying to come up with the actual meal plan each week. We have a list of recipes that we rotate through, but it’s a little too big to think about each week so we often repeat the same meal multiple times before remembering one of the other ones that we like. It would be great if there was an easy way for us to generate our meal plan so that it wasn’t so taxing on us and we could more evenly rotate through our meals.

The application we’ll build to solve this problem could be incredibly complex, but we’re going to try to build this piece by piece so that it’s useful all while we’re building it. Additionally, we’re going to build this to work for more than just my wife and me, so we’ll need to have the concept of a User, authentication, and authorization.

In the Elixir version that I built for my talk, I only built the user sign up, authentication flow, and basic recipe creation, editing, and deletion.

If you use your imagination you can probably think about how we could extend this system to eventually generate a grocery list based on ingredients or balance a weekly menu based on meat selection.

Generating our App Skeleton

The first thing that we’re going to do is generate our rails application, using a container to do so like we have in the past:

$ docker run --rm -v $PWD:/usr/src -w /usr/src ruby:2.3 bash

From inside the container, we’ll install rails and generate our application with some predefined configuration:

$ gem install rails
$ rails new meal_plan --skip-turbolinks --skip-spring -d postgresql --skip-bundle --skip-test

Now we have our rails application so we can exit the container. Before we call our application finished though we’ll create our Dockerfile, docker-compose.yml, .dockerignore and .env files:

Dockerfile:

FROM ruby:2.3.1

RUN apt-get update -yqq \
  && apt-get install -yqq --no-install-recommends \
    postgresql-client \
    nodejs \
  && apt-get -q clean \
  && rm -rf /var/lib/apt/lists

# Pre-install gems with native extensions
RUN gem install nokogiri -v "1.6.8.1"

WORKDIR /usr/src/app
COPY Gemfile* ./
RUN bundle install
COPY . .

CMD rails server -b 0.0.0.0

docker-compose.yml

version: "2"

volumes:
  db-data:
    external: false

services:
  db:
    image: postgres
    env_file: .env
    volumes:
      - db-data:/var/lib/postgresql/db-data

  app:
    build: .
    env_file: .env
    volumes:
      - .:/usr/src/app
    ports:
      - "3000:3000"
    depends_on:
      - db

.dockerignore:

.git
.gitignore
README.md

Dockerfile
docker-compose.yml
.dockerignore

log/*
tmp/*
.rake_tasks*

.env:

POSTGRES_USER=coderjourney
POSTGRES_PASSWORD=abcd1234

We’re going to strip down the Gemfile a bit and add nokogiri so we can pin it to 1.6.8.1. That will speed up our image building time any time after the initial build.

source 'https://rubygems.org'

gem 'rails', '~> 5.0.0', '>= 5.0.0.1'
gem 'pg', '~> 0.18'
gem 'puma', '~> 3.0'
gem 'sass-rails', '~> 5.0'
gem 'uglifier', '>= 1.3.0'
gem 'jquery-rails'

gem 'nokogiri', '1.6.8.1'

group :development, :test do
  gem 'byebug', platform: :mri
end

group :development do
  gem 'web-console'
  gem 'listen'
end

# Windows does not include zoneinfo files, so bundle the tzinfo-data gem
gem 'tzinfo-data', platforms: [:mingw, :mswin, :x64_mingw, :jruby]

Lastly, we need to adjust our config/database.yml file to work with containers:

database.yml:

default: &default
  adapter: postgresql
  encoding: unicode
  host: db
  username: <%= ENV["POSTGRES_USER"] %>
  password: <%= ENV["POSTGRES_PASSWORD"] %>
  pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>

development:
  <<: *default
  database: meal_plan_development

test:
  <<: *default
  database: meal_plan_test

production:
  <<: *default
  host: <%= ENV["POSTGRES_HOST"] %>
  database: meal_plan_production

Now that we have all of the pre-requisite files that we want, we can build our image and generate our database:

$ docker-compose run --rm app rake db:create db:migrate

Finally, let’s make sure our rails app is running. Start the containers, daemonized, and navigate to http://localhost:3000:

$ docker-compose up -d

Saving our Work

We’re going to treat this project like we would a client project. Looking at it through that lens, one of the first things I would do would be to get this into source control and set up the repository on a git server somewhere. We’ll initialize the repo and create our first commit:

$ git init
$ git add . && git commit -m 'Initial creation of dockerized rails application'

Github is going to act as our git server, so we need to create a repo there. If you’re unfamiliar with git and github then I suggest following Github’s awesome tutorials. I’m going to house my version of this application here.

Now we can push up our existing code:

$ git remote add origin git@github.com:coderjourney/meal_plan.git
$ git push -u origin master

Planning Out Features

For managing our project we’re going to use the “Projects” tab in Github. Our project is going to have 4 columns to track the status of our features: “Backlog”, “In Progress”, “Completed”, “Deployed”.

When we start implementing a feature or fixing a bug we will move a card from “Backlog” into “In Progress”. When the feature is completed we will open a pull request in Github to merge the feature brach into master. After our branch is merged we will move our card from “In Progress” to “Completed”. Each time we deploy our “master” branch to a server we will move the card collected in “Completed” into the “Deployed” column.

We’ll start off by creating cards for the features that we want to build and try to order them in a way that makes sense. Here are the features that I’m imagining we start with:

User Sign Up
User Login/Logout
Allow User to View/Create/Edit/Delete Recipes
Allow User to generate Meal Plan from Recipes

Recap

We’ve laid out the plan for building our rails application, and generated our initial project structure. In addition to the things we’ve done before, we used git for source code version control and github for project planning. In the next tutorial we will take our first card from the backlog and start building the user registration process.

Command Line Intro – Part 2

keith@coderjourney.com (Keith Thompson) — Tue, 01 Nov 2016 17:02:51 +0000

Today we’re going to dig a little deeper into using the command line, focusing on some of the most useful commands that you’ll likely use in day-to-day development. We’ll specifically look at working with files and searching.

Goal for this Tutorial:

Create, move, and remove files and directories.
Understand file permissions on a Unix system.
Learn to install new tools on Ubuntu/Debian Linux.
Learn how to search within file and by file name.

We learned in how to move around with in a terminal in the previous tutorial, and today we’re going to focus on manipulating files.

Files are the core of the computer from the user’s view. We put our source code in files, our executables are files, and images and other data formats are all files. Because of this, being confident when working with files is very important to how we operate as programmers.

There are some differences between how some commands work between macOS and linux, so for this tutorial I’m going to work from within a Linux container. If you have Docker installed and have a docker server that you can connect to, run this command to follow along:

$ docker run --rm -it  -w /usr/src ubuntu:16.04 bash

Creating Files and Understanding Permissions

To learn how file’s work we’re going to cover how to create, copy, move, and delete them. Let’s start with creating files.

Creating files on a unix system can happen in a few different ways. You can create a file by writing it from your text editor, if the file doesn’t exist it will be created when you save your changes. The other way that we create files is by using the touch command.

$ touch example.txt

Now if we use ls -al we should see the new file:

$ ls -al
total 8
drwxr-xr-x  2 root root 4096 Oct 29 16:36 .
drwxr-xr-x 12 root root 4096 Oct 29 16:36 ..
-rw-r--r--  1 root root    0 Oct 29 16:36 example.txt

Let’s break down that first section of our line for example.txt:

-rw-r--r--

The first dash represents whether or not the item is a file or directory. If example.txt were a directory we would see a d there like we do for the .. line.

Positions 2-4 represent the permissions for the owner of the file. Positions 5-7 represent the permission for the file’s group members. Lastly, positions 7-9 represent the permissions for anyone on the system. All of these sections work the same way, but they represent access for a different level of user.

The three positions in each of the groupings represent:

read
write
execute

Knowing that we can see that the root user (that’s us in this container) can read and write the file, anyone in the root group can read the file, and anyone on the system can read the file.

There are tools specifically for changing file permissions and ownership. I’m going to list them out with a brief description, but I encourage you to look them up using man:

chmod - Change the file’s permissions.
chown - Change the file’s owner.
chgrp - Change the file’s group.

Moving, Copying, and Removing Files

Using our example.txt file, we can go over the rest of the common actions that you’ll perform on files, but before we do that we’re going to create a new directory to move it to.

mkdir is the tool that you’ll use to make directories.

$ mkdir -p test/examples

That command actually creates two directories for us. If the test directory doesn’t exist it will create it, and then it will also create an examples directory underneath of test.

Now we can move our file to one of these different directories. When moving a file sometimes what we actually want to do is create a copy of the file in a different spot. Let’s do that now using cp.

$ cp example.txt test/examples
$ ls test/examples
example.txt
$ ls
example.txt test

You can see that we now have an example.txt in the test/examples directory, but what would it look like if we wanted to move a file completely? For that we’ll use the mv utility.

$ mv test/examples/example.txt test/sample.txt
$ ls test
examples sample.txt
$ ls test/examples
$

That completely moved the file from test/examples/example.txt to be at test/sample.txt. In the process we also renamed the file. The mv tool is also the tool that you would use if you want to rename a file in the same directory.

We no longer need the test directory or files in it so let’s look at how we would get rid of those using rm.

$ rm test/sample.txt
$ ls test
examples
$ rm test
rm: cannot remove 'test': Is a directory

rm is used for removing files by default. It is also the tool that we’ll use to remove directories, but we’ll have to provide it with a specific flag, -r (for recursive).

$ rm -r test
$ ls
example.txt

The recursive flag means that it will also remove the contents of the directory. We no longer have either test or test/examples.

Searching and Finding Files

Knowing how to manipulate files will allow you to perform the file maintenance that you’ll need to do from day-to-day, but one of the more useful things that you’ll need to be able to do is search for files by name and by content. We’ll use the tools find and grep respectively to accomplish these goals.

Before we can search through files we’re going to need to have something of substance to search through. For that we’re going to pull down the rails application that we worked with before Hours, but we can’t easily get that into our container without installing some additional tools.

Installing Packages on Ubuntu/Debian

We’re going to use the built-in package manager for Ubuntu and Debian to install wget so that we can download the zip archive of the Hours project.

$ apt-get update -yqq && apt-get install -yqq wget

This line actually does quite a bit. We’re first updating the list of known packages on our system so that it knows what we can download and install. From there we use && to chain on another command, this time we’re installing the wget command. In both of these commands we’re passing the -yqq flag to suppress some of the output that these commands give (making them more “quiet”) and we’re also saying “yes we want to perform this action”. Without the -y we would need to type y again to allow the command to complete.

Downloading and Unzipping Hours

With wget installed we can grab the zip link from github to download the repo.

$ wget https://github.com/DefactoSoftware/Hours/archive/development.zip

Now we have a development.zip directory that we can unzip using the pre-installed unzip command.

$ unzip development.zip -d Hours

That will give us a sufficiently big project to search through.

Finding Files by Name

If you are working with a codebase that you’re not very familiar with it can be helpful to search for files that contain specific words as a way of exploring. We’ll do this now by looking for file names that contain the word “secret”

$ find . -name "*secret*"
./Hours-development/config/initializers/secret_token.rb

The . after find indicates that we want to look at all files under our current directory. By passing the -name flag we’re stating that we want to search by name. The token "*secret*" means we want to look for any file name including secret and we don’t care what characters come before or after it. The * wildcard character is very helpful when using find, otherwise we would be looking for a file that’s entire name was “secret”.

Searching File Contents

Admittedly, you’ll search for files by name every now and then, but you’ll search the contents of files all the time if you’re anything like me.

It’s amazing how often I need to see where a specific class or method is used in a code base. For that we use tools like grep. Let’s see where in the codebase the Tag model is used.

$ grep -Rn "Tag" .

There are quite a few things that match Tag there so I won’t put all of the contents here, but some of the matches are for things unrelated to our model. For this we’ll make our search a little more specific.

$ grep -Rn "Tag\." .

This search looks for anytime we use Tag explicitly followed by a .. If we didn’t escape the period with a \ then wouldn’t work the way we expect because . is a wildcard character when searching with grep.

Breaking this command down all of the way we’re using the -R flag to search in all directories recursively. The -n flag tells grep to return the line numbers where the matches are found in the files. Finally, the . at the end is the path for grep to search. We could be more specific with the directory to further fine tune our output. For instance we could exclude the test results by using Hours-development/app as the directory to search.

Recap

Now that you know the basics of working with and searching files you should feel much more comfortable in the command line. Between working with files and understanding how the shell reads command and where commands come from you’re in a good spot to use the terminal and command line as part of your normal development tool chain.

Let me know if there are any other command line specific topics you’d like me to cover, if not I’ll likely cover new things in passing as we use them in tutorials for other topics.

Command Line Intro – Part 1

keith@coderjourney.com (Keith Thompson) — Wed, 26 Oct 2016 11:26:34 +0000

Today we’re going to take a few steps back to cover a tool that we’ve used in every tutorial, but never actually talked about it: the command line.

Goal for this Tutorial:

Explore the command line shell and prompt.
Understand the $PATH.

When you first see a command line prompt (probably in a TV show) it can be a little intimidating. Most people imagine a mysterious, black screen with bright green letters on it. After you learn how to use the command line or shell it’s amazingly powerful, and sadly I skipped right past it when I was first thinking about tutorial topics.

Nearly every tutorial I’ve created has used the shell at some point, so it’s best that we’re all familiar with what is going on.

My Favorite Tool: The Shell

The shell is probably my favorite tool. I use the shell so much while I’m programming that it’s actually the main reason that I switched from Windows to Mac: Bash (the default shell) was too powerful for me not to use. Thankfully, today Bash is actually useable on Windows (with the anniversary update to Windows 10). Follow these instructions to install bash on Windows.

The shell is the starting point for a lot of programming tasks because you can do things in a way that is reproducible and easy to share with others. It’s hard to share what you’re doing with your mouse with someone else. It would take a long time to teach someone how to use Docker if you were only able to create containers by clicking, dragging, and hovering over items on the screen. It’s quite easy for me to tell you to type the following with confidence about what will happen:

$ docker run -it ubuntu echo "Hello, World!"

Exploring the Shell

Debatably, the most commonly used tools that your command line gives you are cd and ls. We’ve used these quite a few times up to this point. These are important because you’ll need to move around from directory to directory and see what’s around you.

We’ll use pwd to demonstrate a little of what these other two tools do. pwd *p*rints your *w*orking *d*irectory, it’s useful for when you need to see where you’re currently at.

~ $ pwd
/Users/explorer

The user that I use to make tutorials is named explorer and this output will definitely be different for you.

cd allows you to *C*hange your current working *D*irectory. Here are a few examples:

~ $ cd code
code $ pwd
/Users/explorer/code

We can see there that our working directory changed. We’ll switch back to our home folder to explore ls. To do that we can use either a full path or a relative path, either of these two lines would work:

code $ cd ..
~ $ pwd
/Users/explorer

or with the full path:

code $ cd /Users/explorer
~ $ pwd
/Users/explorer

ls is used to list out the files and directories inside of the directory that you pass it (or the current directory).

~ $ ls
Applications	 Documents Library Pictures	code
Desktop	Downloads	Movies Public	src
Dockerfile	 Dropbox Music Videos

That’s not super useful at first glance. They’re in alphabetical order by column (why not row?), but not giving us a lot of information beyond that. Thankfully, ls can take quite a few flags so that we can change the output that it gives us. The -l flag will list them out vertically and give us the permissions and last changed times.

~ $ ls -l
drwx------   5 explorer  staff   170 Sep 11 07:43 Applications
drwx------@  4 explorer  staff   136 Oct 11 17:06 Desktop
-rw-r--r--   1 explorer  staff    76 Jul 30 15:09 Dockerfile
drwx------@  3 explorer  staff   102 Jul 26 19:15 Documents
drwx------+  5 explorer  staff   170 Oct  4 08:31 Downloads
drwx------@ 23 explorer  staff   782 Oct 25 18:43 Dropbox
drwx------@ 63 explorer  staff  2142 Oct 18 20:05 Library
drwx------+ 13 explorer  staff   442 Oct  3 19:26 Movies
drwx------+  4 explorer  staff   136 Jul 30 10:08 Music
drwx------+  3 explorer  staff   102 Jul 26 19:15 Pictures
drwxr-xr-x+  5 explorer  staff   170 Jul 26 19:15 Public
drwxr-xr-x  14 explorer  staff   476 Oct 18 18:55 Videos
drwxr-xr-x   2 explorer  staff    68 Oct 25 19:12 code
drwxr-xr-x   4 explorer  staff   136 Jul 28 07:35 src

Much more useful. The version that I usually use also includes the -a flag so that I can see hidden files. This can sometimes return you a lot of files though, but the extra ones are normally hidden, you likely won’t see them in Finder.app on Mac or Windows Explorer.

Where Do These Commands Come From?

One question you might be asking yourself is “Where does ls come from?“, and that’s a great question. One of the biggest reasons that I think the command line seems strange at first is because there is a lot that isn’t shown to you unless you ask.

We can see where a specific command comes from using the which command. Let’s see where ls lives:

~ $ which ls
/bin/ls

That means that there is an actual executable file at in the /bin directory named ls that provides the functionality that we’ve seen for ls. That doesn’t entirely answer our question though because we didn’t type /bin/ls, we simply typed ls.

This leads us to the “PATH”. The PATH is an environment variable that tells your shell which directories to look for to find commands. Let’s see what my PATH looks like using echo.

~ $ echo $PATH
/Users/explorer/.gem/ruby/2.3.1/bin:/Users/explorer/.rubies/ruby-2.3.1/lib/ruby/gems/2.3.0/bin:/Users/explorer/.rubies/ruby-2.3.1/bin:/Users/explorer/bin:/usr/local/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/usr/local/share/dotnet

Notice: that we had to use a $ when we referenced the variable name. That’s how you access the value of a variable in bash, otherwise it’s just a string “PATH”.

My path is pretty long because it includes a lot of ruby related directories, but we can still break it down. If we take all of the ruby related paths out this is what we’re left with:

/Users/explorer/bin:/usr/local/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/usr/local/share/dotnet

When we type ls the shell will attempt to find an executable in /Users/explorer/bin. If no executable is found there it will move onto /usr/local/bin, so on and so forth until it either finds an executable or it will exit with an error like this (trying use the obj tool):

~ $ pbj
-bash: pbj: command not found

If you ever see that error you know that what you typed isn’t in your $PATH. This can either be because the file doesn’t exist or isn’t installed, or because you had a typo (pretty common).

Learning More

Almost every time we’re typing into our command line we’re running some program on our system. Most of these tools give us a way to learn more about them through either a --help flag or by giving us a “man page”. If you want to learn more about a command I would suggest first using the man tool like this:

~ $ man pwd

This will open up a scrollable “page” that you can move down by hitting or holding your j key, move up with k, or quit by hitting the q key.

That doesn’t also work, and when it doesn’t then it’s a safe bet to try using --help. Not all tools respond to this, but if they don’t they’ll probably tell you what you could have typed to make the tool work. cd is a good example of this:

~ $ cd --help
cd: usage: cd [-L|-P] [dir]

Other tools will give you “man page” style information without opening in a page so you can scroll through your terminal how you normally would (docker is a good example of this).

Recap

I might be a little too enthusiastic about the shell, but I use it every day and the more that I learn to do from my prompt the better I seem to be at my job. This tutorial was meant to demystify the command line a little and give you a means to learn more on your own. I’ll dig deeper in a later tutorial to show how we can customize our shell experience and some of my favorite command line utilities to use.

Getting Started with Python Functions

keith@coderjourney.com (Keith Thompson) — Wed, 19 Oct 2016 00:02:48 +0000

Today we’re going to learn how to package up small chunks of code for reuse using functions.

Goal for this Tutorial:

Create our first function to execute code multiple times.
Learn about range.
Learn about function arguments.

In most of our programming, we won’t be working with a script that we’re going to run from top to bottom. Most of our time is going to be spent grouping code together in a way that makes sense to use and then combining the piece that we build up in order to create our final product. The simplest way we can package up our code is by using a function.

Python Functions are Pretty Awesome

Functions are the building block that virtually every programming language has in common, and are even the most important building block in some languages. Python functions aren’t the most important, but they are very powerful.

I like to think of functions in programming in the way that you might learn about functions in mathematics. Let’s break down a math function and turn it into a Python function:

f(x) = x + 1

Our function has the name of f and it requires a single variable x. We will define this function in Python using the def keyword:

def f(x):

We follow our function name and arguments with a : and then we will indent the actual logic of the function by 4 spaces. After the body of the function we’ll insert a blank line which will end the function.

def f(x):
    x + 1

We’ve defined our function, now let’s take it for a spin:

>>> f(1)
>>>

That’s not quite right. In our math function if we use 1 for x then the answer should be 2, but we didn’t get anything. Our function works this way because unlike mathematical functions which always produce an output, Python functions can take an action and then return nothing to us. To make this work like the math function we need to use the keyword return.

def f(x):
    return x + 1

If we try to run the function again, we should get 2 back:

>>> f(1)
2
>>>

Beyond Math… Onto Fizz Buzz

Figuring out how to represent a math function in Python is one thing, but that really doesn’t event scratch the surface of Python functions.

Functions are not restricted to being a single line, so we can package up a series of expressions to execute. We’ll demonstrate this using a problem from programming interviews called “fizz buzz”.

For “fizz buzz” we’re going to take a number and print from 1 to that number and for each number we will print one of the following:

“Fizz” if the number is divisible by 3
“Buzz” if the number if divisible by 5
“Fizz Buzz” if the number is divisible by 3 and 5
The number itself if not divisible by 3 or 5

Here’s an example of one way we could write this function:

def fizz_buzz(limit):
    for num in range(1, limit):
        output = ""
        if num % 3 == 0: output += "Fizz "
        if num % 5 == 0: output += "Buzz"
        print(output or num)

This function combines loops, conditionals, strings, conditionals, printing, and even throws in something new: range.

A range is a sequence of numbers, in our case it’s from 1 to whatever the passed in limit is, excluding the limit itself. Ranges are great for when you want to iterate through something a specific number of times or in our case where we want to do something with each number, but we don’t want to create a list ourselves.

This is a fairly complex function, and now we can run it over whatever limit we want, as many times as we want.

>>> fizz_buzz(17)
1
2
Fizz
4
Buzz
Fizz
7
8
Fizz
Buzz
11
Fizz
13
14
Fizz Buzz
16
>>>

Fizz Buzz, but Better

Our original example was a good example of creating a more complex function body, but there is more that we can do in the function declaration that can make it even better. For instance we can give limit a default (we’re going to call this fizz_buzz2 to keep them separate):

def fizz_buzz2(limit=20):
    for num in range(1, limit):
        output = ""
        if num % 3 == 0: output += "Fizz "
        if num % 5 == 0: output += "Buzz"
        print(output or num)

Now if we call fizz_buzz2 without any arguments it will automatically use 20 as the value for limit. Pretty neat!

Let’s take this even one step forward and make the “fizz” and “buzz” messages configurable (this is fizz_buzz3):

def fizz_buzz3(limit, fizz="Fizz", buzz="Buzz"):
    for num in range(1, limit):
        output = ""
        if num % 3 == 0: output += fizz + " "
        if num % 5 == 0: output += buzz
        print(output or num)

This version of fizz buzz is the best we’re going to create. We’re now using multiple arguments, some with defaults and some without. Let’s look at some of the ways we can call this function:

>>> fizz_buzz3(7, "Red", "Blue")
1
2
Red
4
Blue
Red
>>> fizz_buzz3(7, buzz="Boom!")
1
2
Fizz
4
Boom!
Fizz

In our first example we used all positional arguments, meaning the order we passed them in would be interpreted as the order we defined them. The second time we ran it though, we used what’s known as a keyword argument. We let fizz take its default value, but we explicitly set buzz to be Boom!

Recap

We’re finally getting into the more exciting parts of programming. Before today, most of what we’ve learned about were the small building blocks that we’ll use to build up an individual line of code, but with functions we’re learning the tools we need to write more complex programs. From here, there are only a few more broad topics that we’ll need to cover before we can start digging into the problem solving of programming.

Learn How to Use Loops in Python

keith@coderjourney.com (Keith Thompson) — Wed, 12 Oct 2016 00:12:54 +0000

Today we’re going to learn how to execute the same block of code multiple times using loops, and how to take actions on each element in a list.

Goal for this Tutorial:

Create our first loop to execute code multiple times.
Learn how to iterate through a list.

Last time we worked with Python, we learned about lists and tuples, but we didn’t really do anything with them. For lists to really be useful we will need to be able to do things with the data. That means we either need to be able to find and work on a specific piece of data (which we know how to do already), or we need to take each element of the list or tuple and do something with it. We probably won’t do this with tuples, so from this point on I’ll just be talking about lists.

To work on each element we another language feature: loops.

The Original Loop: The While Loop

Like a lot of things in Python, loops are incredible readable, and because of that make a lot of sense when you see them. Let’s start by taking a look at the while loop:

>>> a = 0
>>> while a < 5:
...     print("This is a: %s" % a)
...     a += 1
This is a: 0
This is a: 1
This is a: 2
This is a: 3
This is a: 4

The while loop takes an expression and executes the block underneath if that expression is true, much the same way that an if statement would. The difference here is that it will look at that expression over and over again until it is false.

This type of loop allows us to do something over and over again, but if you forget to modify a variable used in the expression it is possible to run into an infinite loop, so look out for that.

There are a few other keywords associated with loops: break, continue, and else.

Looking at else first, it’s used to run something when the looping finishes. Building on our first loop this is how we could utilize else:

>>> a = 0
>>> while a < 5:
...     print("This is a: %s" % a)
...     a += 1
... else:
...     print("We've reached 5")
...
This is a: 0
This is a: 1
This is a: 2
This is a: 3
This is a: 4
We've reached 5

To learn about break and continue we’ll take a look at the other type of list that you’ll use in Python: the for loop.

Looping Over Lists: The For Loop

The while loop is powerful and can be used to accomplish any type of looping that we need, but most of the time what we want to do is iterate over a list, and for that we have the for loop.

This next example will show both a for loop and the continue keyword in action:

>>> my_list = [1,2,3,4,5]
>>> for num in my_list:
...     if num % 2 == 0:
...         print(num, 'is even')
...         continue
...     print(num, 'is odd')
...
1 is odd
2 is even
3 is odd
4 is even
5 is odd

Looking at the creation of the for loop you can see the readability that I mentioned earlier. We create a temporary variable for each element in our list called num and then we can use that in our loop block. Once we’ve gone through every item of the list then our loop is finished.

We wrote this loop specifically to showcase the continue keyword and we would normally write this differently, but it gets the point across. continue allows us to stop executing our current loop iteration and move onto the next one. If we hadn’t used continue here we would have printed that some numbers were both even and odd.

The last keyword that loops have is break, so let’s see how that works.

>>> my_list = [1,2,3,6,4,5]
>>> for num in my_list:
...     if num <= 5:
...         print(num, 'is less than 6')
...     else:
...         break
...
1 is less than 6
2 is less than 6
3 is less than 6

Looking at the list we should have been able to see all but one of the items print out, except we hit the break statement. Where continue halts a single iteration of a loop, break halts the entire loop. The 4 and 5 that were in the loop after the 6 were never iterated over because the break was triggered when we iterated over the 6.

Recap

Now you know how to create loops in Python. Loops are one of the most common tools that we will use in programming (for more than while), so commit these to memory sooner rather than later.

Convert an Existing Rails Application to Work with Docker

keith@coderjourney.com (Keith Thompson) — Tue, 04 Oct 2016 23:40:02 +0000

Today we’re going to take an existing Ruby on Rails application and convert it so that it be developed/deployed using Docker.

Goal for this Tutorial:

Dockerizing an existing rails application.

Up to this point we’ve been working with “hello world” applications when working with Docker, but today we’re going to take an existing application (that I didn’t write) and get it to work with Docker.

Dockerizing a Rails Application

The application that we’re going to convert today is an open source Rails application for time tracking called “Hours”. This particular application requires a few different services:

Ruby/Rails
PostgreSQL
Memcached

Since this project isn’t currently set up to be used with Docker, the first thing we’ll do is clone the repo and create a docker-compose.yml file to lay out all of our services.

Cloning:

$ git clone git@github.com:DefactoSoftware/Hours.git

Now let’s crack open our docker-compose.yml:

docker-compose.yml

version: "2"

volumes:
  db-data:
    external: false

services:
  cache:
    image: memcached:1.4-alpine

  db:
	  environment:
		  POSTGRES_USER:
		  POSTGRES_PASSWORD:
    image: postgres:9.5
    volumes:
      - db-data:/usr/local/pgsql/data

	jobs:
	  env_file: .env
    build: .
    volumes:
	    - .:/usr/src/app
		command: bundle exec rake jobs:work
    depends_on:
      - db

  app:
	  env_file: .env
    build: .
    volumes:
	    - .:/usr/src/app
    ports:
      - "8080:8080"
    depends_on:
      - db
      - cache

Now we need to set up our Dockerfile so that we can build an image to run our application on. This application uses Capybara-webkit for testing so we’ll need to install some extra dependencies for that to work:

FROM ruby:2.3.1

RUN apt-get update -yqq \
  && apt-get install -yqq --no-install-recommends \
    postgresql-client \
    nodejs \
    qt5-default \
    libqt5webkit5-dev \
  && apt-get -q clean \
  && rm -rf /var/lib/apt/lists

WORKDIR /usr/src/app
COPY Gemfile* ./
RUN bundle install
COPY . .

CMD bundle exec unicorn -c ./config/unicorn.rb

We’ll need to create config/database.yml and config/secrets.yml before our application will work. Hours is build around deploying to Heroku, and Heroku manages some of this configuration for you.

Database:

config/database.yml

# config/database.yml
default: &default
  adapter: postgresql
  host: <%= ENV["POSTGRES_HOST"] %>
  username: <%= ENV["POSTGRES_USER"] %>
  password: <%= ENV["POSTGRES_PASSWORD"] %>
  encoding: utf8
  min_messages: warning
  pool: 2
  timeout: 5000

development:
  <<: *default
  database: hours_development

test:
  <<: *default
  database: hours_test

production:
  <<: *default
  database: hours_production

config/secrets.yml

development:
  secret_key_base: <%= ENV["SECRET_KEY_BASE"] %>

development:
  secret_key_base: <%= ENV["SECRET_KEY_BASE"] %>

production:
  secret_key_base: <%= ENV["SECRET_KEY_BASE"] %>

You’ll notice that we’ve used quite a few environment variables thus far. Environment variables are potentially the easiest way to configure our application, but they can be tedious. Thankfully, docker-compose allows us to use a .env file to store the values that we would like to use in our containers and we don’t need to set them in our own environment.

Let’s create our own .env file now by copying the .sample.env file and making some adjustments.

POSTGRES_HOST=db
POSTGRES_USER=hours
POSTGRES_PASSWORD=super_secure

SECRET_KEY_BASE=development_secret
GMAIL_USERNAME="example@mail.com"
GMAIL_PASSWORD="password"
HELPFUL_URL=https://example.io/incoming_message
HELPFUL_ACCOUNT=helpful_account
SINGLE_TENANT_MODE=true
S3_BUCKET_NAME="s3_bucket_name"
AWS_ACCESS_KEY_ID="aws_access_key_id"
AWS_SECRET_ACCESS_KEY="aws_secret_access_key"

Now we’re ready to take our group of containers for a spin.

Spinning up our Containers

Like we had to when learning to develop a Rails application using Docker, we’re going to need to do a few things before our application will actually run. First, we’re going to need to create our database. We can do that by starting our services that are going to run from images, those shouldn’t have any issues.

$ docker-compose up -d db cache

This will create the network that all of our containers connect to to communicate with one another and also start our memcached and postgres containers.

Next up, we’ll need to build our app image before we can run commands. This is going to need to install all of the gems and dependencies so it will likely take awhile. A nearly identical image will be used for our jobs container so we’ll build that now also.

$ docker-compose build app jobs

After we’ve build our image we can run some commands with it. First, we need to create and migrate our database:

$ docker-compose run --rm app rake db:create db:migrate

Second, we want to create a user for ourselves within the application.

$ docker-compose run --rm app rake create_user
First name: Coder
Last name: Journey
email: keith@coderjourney.com
Password:
The user is created

Now we’re good to spin up our app and jobs containers.

$ docker-compose up -d app jobs

Before moving any further we should make sure that all of our containers are running using docker-compose ps:

$ docker-compose ps
    Name                   Command               State           Ports
-------------------------------------------------------------------------------
hours_app_1     /bin/sh -c bundle exec uni ...   Up      0.0.0.0:8080->8080/tcp
hours_cache_1   docker-entrypoint.sh memcached   Up      11211/tcp
hours_db_1      /docker-entrypoint.sh postgres   Up      5432/tcp
hours_jobs_1    bundle exec rake jobs:work       Up

Now we can head over to localhost:8080 and check out our new time tracking application.

Fixing the Test Suite

Right now, if we try to run the test suite we’ll get some errors because it can’t find certain pieces of the page to click on. This is caused by this application already using a .env file to load environment variables into the application and us having explicitly defined environment variables in our containers. We’re going to get around this by overloading the environment in the test so that it’s always the same.

spec/spec_helper.rb

# Environment set to "test" above

require "dotenv"
Dotenv.overload(".sample.env")

# environment file loaded below

It’s important that we put these two lines in the right spot so that we set the environment variables before loading the rails app for our tests, just in case those ENV values are stored in constants somewhere.

Now we can see if it worked by running:

$ rake

The tests should all be green! Congrats on converting your first application to using Docker.

Recap

All of the code for this tutorial can be found on Github.

I’m looking for feedback on upcoming topics. Head over and answer my single question survey go let me know what you think I should cover.

Deploy with Docker and Digital Ocean

keith@coderjourney.com (Keith Thompson) — Sat, 24 Sep 2016 15:56:51 +0000

Building on top of our docker-machine tutorial, we’re going to use the docker toolbox to create provision a server and deploy a Ruby on Rails application to the cloud.

Goal for this Tutorial:

Create a Docker host with docker-machine.
Deploy a Ruby on Rails application to the server using docker-compose.

Up to this point we’ve looked at how to utilize the Docker in our development environment, how to package up our own images, and how to use docker-compose to orchestrate different services in our development environment. These things are all great for having a sharable development, but there’s nothing stopping us from using this for deploying applications.

Our Rails Application

We’re actually going to build on top of the previous Docker in Development tutorial for our application. It shouldn’t be too difficult to take the basic application that we created in that tutorial and run it in the cloud.

The first thing that we need to do though is grab the code. We’ll use git to pull the sample code from the previous tutorial from github. We’re placing a . at the end of this command so that the contents of the repository are placed in our current directory.

$ git clone git@github.com:coderjourney/03-learn-to-use-docker-in-development.git .

We’ll need to change into the blog directory, but besides that we’re good to go with a Rails application.

Looking at the Docker Setup

Before we get too far we should refresh our memories as to how we’ve set up Docker and docker-compose in this application.

Dockerfile:

FROM ruby:2.3

RUN apt-get update -yqq \
  && apt-get install -yqq --no-install-recommends \
    postgresql-client \
    && rm -rf /var/lib/apt/lists

WORKDIR /usr/src/app
COPY Gemfile* ./
RUN bundle install
COPY . .

EXPOSE 3000
CMD rails server -b 0.0.0.0

docker-compose.yml

version: "2"

volumes:
  db-data:
    external: false

services:
  db:
    image: postgres
    env_file: .env
    volumes:
      - db-data:/var/lib/postgresql/db-data

  app:
    build: .
    env_file: .env
    volumes:
      - .:/usr/src/app
    ports:
      - "3000:3000"
    depends_on:
      - db

We’ve also packaged up the .env file that holds the environment variables that we’re passing into the containers. Normally I wouldn’t store this file in version control because it contains secrets and passwords. Let’s look at that file again.

.env:

POSTGRES_USER=coderjourney
POSTGRES_PASSWORD=abcd1234

Everything here looks pretty good so let’s see if we can deploy this as is.

Docker-machine & Digital Ocean

There are essentially two things that we need to deploy a container for a web application into the wild:

1) A docker host server to run the container 2) An image to download or a Dockerfile to build the image from

Thankfully, Docker provides us with a really nice tool for building docker hosts easily in quite a few hosting providers. For this tutorial, we’re going to host with Digital Ocean (that link will get you a $10 credit for signing up) and creating our host using docker-machine. You’ll need to sign up with digital ocean before we begin because you’ll need to grab your API token. Once you have your account you can go here to generate a new API token.

$ docker-machine create --driver=digitalocean --digitalocean-access-token=$DO_TOKEN --digitalocean-size=1gb blog

Now we have our first Docker host running “in the cloud” 😀. There are a lot more configuration values that you can pass to the digital ocean driver, so check those out in the Docker docs.

After our machine is up and running we’ll want to set that as our active machine using the env command docker-machine gives us:

$ eval $(docker-machine env blog)

Deploying with Docker

$ docker-compose up -d db

This will create the network that all of our containers connect to to communicate with one another and also start our postgres container.

Next up, we’ll need to build our app image before we can run commands. This is going to need to install all of the gems and dependencies so it will likely take awhile.

$ docker-compose build app

After we’ve build our image we can run some commands with it. First, we need to create and migrate our database:

$ docker-compose run --rm app rake db:create db:migrate
/usr/src/app/Gemfile not found

Well that’s not good. This is an issue caused by our docker-compose.yml file sharing through our local volume so that we could develop using our normal tools and run the application in a container. That doesn’t work with a remote docker host like we’re working with now.

To fix this problem we need to change the docker-compose.yml file, but that will mean that we won’t be able to work with our application in development 😱. Thankfully, we can get past that by create a different compose file for our “production” deploys.

Let’s make a copy of our current file and call it docker-compose.prod.yml

$ cp docker-compose.yml docker-compose.prod.yml

We can make a few tweaks to this file and we’ll be well on our way to having this app deployed.

docker-compose.prod.yml

version: "2"

volumes:
  db-data:
    external: false

services:
  db:
    image: postgres
    env_file: .env
    volumes:
      - db-data:/var/lib/postgresql/db-data

  app:
    build: .
    env_file: .env
    environment:
      RAILS_ENV: production
    ports:
      - "3000:3000"
    depends_on:
      - db

We’ll also adjust the “production” block of our config/database.yml and add “POSTGRES_HOST” to our .env file to make sure we can connect to the database.

config/database.yml

# Extra values excluded
production:
  <<: *default
  host: <%= ENV["POSTGRES_HOST"] %>
  database: blog_production

.env:

POSTGRES_HOST=db
POSTGRES_USER=coderjourney
POSTGRES_PASSWORD=abcd1234

Since we’ve made a modification to the files that are in the image we’ll need to rebuild our app image before we continue, thankfully the caching will allow this to be quick. Notice that we’re using the -f flag to specify a different docker-compose file.

$ docker-compose -f docker-compose.prod.yml build app

After that’s finished we should be able to create our database:

$ docker-compose -f docker-compose.prod.yml run --rm app rake db:create db:migrate

Yay, Our database was successfully created! Let’s start our application and see if it works. We’ll run in the foreground so that we can see the logs.

$ docker-compose -f docker-compose.prod.yml up app

Connecting to the App

We should be running now. Our next step is to try to access the application. For that we’ll need the IP address of our machine (your IP will be different).

$ docker-machine ip blog
45.55.89.18

Now if we go to http://45.55.89.18:3000 we should see our application… Except we get this message:

An unhandled lowlevel error occurred. The application logs may have details.

Fixing the App

Thankfully, if we look at our logs we see this error:

#<RuntimeError: Missing `secret_key_base` for 'production' environment, set this value in `config/secrets.yml`>

We need to define a SECRET_KEY_BASE in our .env file since that’s the environment variable used in config/secrets.yml.

POSTGRES_HOST=db
POSTGRES_USER=coderjourney
POSTGRES_PASSWORD=abcd1234
SECRET_KEY_BASE=cb6b786b1ec490b51694594a1243cffef162655f93f5f0927ef7de7553039d7440560cba832a5ec53c0e7293f3382ce707628367bde3b8290255ada8a3f64737

After that’s finished we’ll run our application again, this time in the background (confidence!).

$ docker-compose -f docker-compose.prod.yml up -d app

If we try to connect to http://45.55.89.18:3000 now we should see our fancy “Hello from Docker” message.

Recap

We’ve successfully deployed a “hello world” rails application to a docker host running in the cloud. There were a few snags that we had to deal with that don’t happen when running in development, from a local VM, but we made it through it. This deploy is not “good”. There’s quite a bit more that we should have done to get this production ready, but we did get the application to run.

Use Docker-machine to Create Docker Servers

keith@coderjourney.com (Keith Thompson) — Tue, 20 Sep 2016 10:09:42 +0000

Continuing our series looking into Docker, we’re going to bust open the docker toolbox and look at docker-machine.

Goal for this Tutorial:

Create a local Docker host using docker-machine.
Look at the difference between docker-machine and Docker for Mac/Windows.

Up to this point we’ve used Docker for Mac (or Windows) exclusively as our Docker server, but you’re more than likely going to run Docker on a Linux machine when not in development. The docker-toolbox has a tool to make it easy to spin up Docker hosts in a variety of linux environments, docker-machine. We’re going to use docker-machine in combination with VirtualBox to manage a Docker host VM locally. Docker for Mac and Windows are graphical applications for working your local Docker host and depending on the platform utilizes different technologies under the hood. Docker for Mac utilizes HyperKit to manage a Linux virtual machine for you, and Docker for Windows utilizes Microsoft Hyper-V. VirtualBox is a cross-platform solution for running virtual machines, and was the primary way of running docker on Mac or Windows before the GUIs were released. There are also performance differences that you might run into using the various technologies.

Creating a Docker Host with Docker-machine

Once you’ve installed VirtualBox to your system you can head to your command line to create your first VM manually.

$ docker-machine create --driver=virtualbox test

This will create a light-weight, Linux VM running a distro called Boot2Docker. You’ll notice that the output says to run docker-machine env test to learn how to connect to the machine, so let’s do that now.

$ docker-machine env test
export DOCKER_TLS_VERIFY="1"
export DOCKER_HOST="tcp://192.168.99.100:2376"
export DOCKER_CERT_PATH="/Users/explorer/.docker/machine/machines/test"
export DOCKER_MACHINE_NAME="test"
# Run this command to configure your shell:
# eval $(docker-machine env test)

Your values will be different, but it should look similar.

All that we need to do to connect to our new Docker host is set some environment variables. This works because Docker uses a clint-server architecture and our docker command line tool is the client. We configure the client so that it knows which server to talk to using these four environment variables. This makes it easy for us to access multiple Docker hosts from the same machine by changing some of these values. We’ll run the command that it suggests to configure our docker client.

$ eval $(docker-machine env test)

If we run Docker commands now we should get results because it actually has a docker host to connect to. Let’s try this out by listing our our images.

$ docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE

We didn’t receive any images because we’ve never run a docker pull when connected to our new VM. Since images are stored on the docker host, we’ll have our images again when we connect back up to the Docker for Mac or Windows VM.

Starting, Stopping, and Removing a Docker Host

Now that we have our VM and we’re connected to it, we’re back to where we were before in terms of using Docker. Rather than go through more docker commands, we’re going to look into what else we can do with docker-machine.

The most basic thing that we’ll do with docker-machine is look at the status of the machines that we’re managing.

$ docker-machine ls
NAME   ACTIVE   DRIVER       STATE     URL                         SWARM   DOCKER    ERRORS
one    -        virtualbox   Stopped                                       Unknown
test   *        virtualbox   Running   tcp://192.168.99.100:2376           v1.12.1

We now see the list of machines and some basic information about them.

The state is helpful for seeing if the machine is currently running, and active will let us know which machine our docker command line tool is configured to connect to.

We can start and stop the machines easily to avoid using too many resources on our own machine using similar commands and the name of the machine that we want to work with.

$ docker-machine stop test
Stopping "test"...
Machine "test" was stopped.

Now if we look at the listing they should both have a state of “stopped”. Starting test back up takes a similar command.

$ docker-machine start test
Starting "test"...
(test) Check network to re-create if needed...
(test) Waiting for an IP...
Machine "test" was started.
Waiting for SSH to be available...
Detecting the provisioner...
Started machines may have new IP addresses. You may need to re-run the `docker-machine env` command.

And we’re back to where we were originally.

The last thing we’re going to look at is how to remove a machine if we no longer need it. Removing them will be particularly useful in this case so that we’re not wasting hard drive space on our computers, but will be essential later on so we don’t pay for server that we no longer need.

$ docker-machine rm test
About to remove test
Are you sure? (y/n): y
Successfully removed test

Recap

Today we learned how to create Docker hosts, specifically local VMs using docker-machine. After we created our VM we then went through some of the life-cycle operations that we can perform on the machine. These operations will be important as we start moving into using Docker on machines other than our development machines. I wanted to make sure that we have a good grasp on how Docker hosts work before we get into deployments with Docker.

In the next tutorial we’re going to use docker-machine to create a server on Digital Ocean, and deploy a Ruby on Rails application to it for the public to access.

Learn Python – Lists and Tuples

keith@coderjourney.com (Keith Thompson) — Sun, 11 Sep 2016 17:26:58 +0000

Continuing our series on Python, we’re going to look at how we can work with grouped data using two of the data structures in Python: Lists and Tuples.

Goal for this Tutorial:

Learn how to store data in Lists.
Learn what a Tuple is and how to use it.

Up to this point, we’ve only worked with individual values. Whether it’s been numbers or strings we’ve only worked with 1 or 2 of them at a time. For us to write programs that solve real problems we’ll probably need to work with groups of data.

The Primary Collection: List

The main collection type that we’ll use while programming Python is the list. Let’s open up the REPL and see what they can do.

$ docker run --rm -it python:3.5

>>> my_list = [1, 2, 3, 4]
>>> my_list
[1, 2, 3, 4]

Lists are not limited to holding the same type, but it’s in your best interest to keep them the same. That being said, the following is a completely valid list:

>>> mixed_list = [1, 'a', 2.5, True]
>>> mixed_list
[1, 'a', 2.5, True]

Reading, Adding, and Editing a List

Whenever we create a list, it’s likely that we’ll want to access the information within the list at some point. Let’s first look at how we can access 1 item at a time:

>>> my_list[0] # Index a single element, indexes start at 0 and iterate up.
1
>>> my_list[3]
4
>>> my_list[10] # Using an index that's higher than the length of the list.
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IndexError: list index out of range

Some languages don’t raise errors when you try to access an index that is out of range for an Array/List, but Python does, so be careful.

Besides just accessing one item in a list, we can slice a list to give us a subset of the items. We saw the slice syntax briefly when learning how to reverse a string, but most of the time you’ll use this feature when working with lists.

>>> my_list[1:3] # Includes index 1, up to, but not including index 3.
[2, 3]
>>> my_list[:3] # From the beginning up to index 3.
[1, 2, 3]
>>> my_list[2:] # From index 2 to the end (including the end).
[3, 4, 5]
>>> my_list[::2] # From beginning to end, every other item
[1, 3, 5]

Now we have a better grasp on how to read from a list, but what if we want to add to it or change what’s in it? Adding to a list is easy enough, there’s a method for it:

>>> my_list.append(6)
>>> my_list
[1, 2, 3, 4, 5, 6]
>>> my_list.append(7)
>>> my_list
[1, 2, 3, 4, 5, 6, 7]
>>> len(my_list) # Get the length of the list
7

Our list will grow as we append new data to it, but this will get a little tedious if we need to add many items to it. We were also able to see how long our list had become using the len function. Thankfully, lists support the + operation so we can insert multiple items at once. We’ll reset my_list to equal [1, 2, 3] just so it’s a little short for our examples.

>>> my_list + ['a', 'b', 'c']
[1, 2, 3, 'a', 'b', 'c']
>>> my_list
[1, 2, 3]

Notice that the plus operation returns a different list that contains all of the items, but it does not change the contents of our variable my_list. If we want to store off this new combined list we would have to set assign the result to a variable.

Sometimes you want to replace content within a list. The syntax for assigning values looks like a mix between reading from the list and assigning a variable.

>>> my_list[0] = 'a' # Replacing a single value, at index 0.
>>> my_list
['a', 2, 3]
>>> my_list[1:] = ['b', 'c'] # Replacing a slice
>>> my_list
['a', 'b', 'c']
>>> my_list[:] = [1]
>>> my_list
[1]

Notice that we were able to replace the entire list’s contents by creating a slice from beginning to end and assigning [1] to it. When you are replacing a slice the lengths don’t need to be the same.

Removing an Item from a List

We now know how to create, read from, and update lists, but we need to also know how to remove items from the list. We’ll use a few different ways to remove items, depending on what we’re trying to do. The first one is the method pop:

>>> my_list.pop() # Removes the last item in the list
3
>>> my_list
[1, 2]
>>> my_list.pop(0) # Given an argument removes item(s) at the given indexes
1
>>> my_list
[2]

You can use pop to remove elements at a specific position in the list, or by default it will remove the last item.

Tuples

Lists are flexible and can be changed, they’re what is called “mutable”. Tuples on the other hand will have the same contents and length from the time they are created; we can’t change them. We’ve seen tuples in action a few times before when we’ve used print, here’s an example:

>>> print("There are %s planets and %s." % (8, "Pluto"))
There are 8 planets and Pluto.

The (8, "Pluto") is the tuple. Tuples are similar to lists in that they are what’s known as a “sequence” in Python, but you’ll usually use them the data of different types unlike you normally will with a list. The data within a tuple will normally be accessed by indexing, whereas you’re more likely to iterate over a list.

Recap

Today, we took a look at two of the collection data types that we’ll use in Python. Lists are one of the most used things in all of programming so we’ll need those before we begin writing more complicated programs in Python.

Learn about Python Variables and Control Flow

keith@coderjourney.com (Keith Thompson) — Sat, 03 Sep 2016 18:13:26 +0000

Today we’re going to continue to our series on Python. We’re going to learn how to name data using variables, and how to make our programs behave differently based on the values in that are passed in.

Goal for this Tutorial:

Learn how to use variables to name data.
Learn how to use if/else for control flow.
Learn how to take user input from the command line in our scripts.

Today’s tutorial is going to take place almost exclusively in a script file, so have your favorite text editor handy and lets make sure we have a container created that we can run our script in.

$ docker run --rm -it -v $PWD:/usr/src/python -w /usr/src/python python:3.5 bash

We’re going to create a script that will allow the user to input a string and it will output whether or not the string is a palindrome (a word or phrase that reads the same in both directions excluding spaces and punctuation, examples: “radar” or “nurses run”). Here are the steps that we’re going to take to accomplish this:

Store user’s input into a variable, I’m going to call it “phrase”.
Create another string by removing spaces from “phrase” so that we give the proper response for phrases like “nurses run”. We’ll call this one “sanitized_phrase”.
Compare “sanitized_phrase” with “sanitized_phrase” backwards. If they are the same then output that the phrase was a palindrome, otherwise output that the phrase isn’t that exciting.

Naming Data with Variables

Up to this point in our exploration of Python, we’ve run each line as though it wasn’t connected to any of the code around it, but today we’re going to learn to how to keep a piece of data around for awhile so that we can do change it and use it in comparisons before completing our program. Let’s look at how to create a variable in the Python REPL.

>>> my_var = "This is my var"
>>>

You’ll notice that it didn’t output anything because nothing was returned or written to the screen. If we enter my_var by itself we can see the string we entered.

>>> my_var
'This is my var'

Anytime we use my_var it will give us this value, until we set a different value for that variable.

Accepting User Input

Now to start our palindrome checker script, open up palindrome.py. The first step in our process is to take in user input so that we can see if it’s a palindrome. Let’s learn how to do do that now.

palindrome.py

phrase = input("Enter a phrase: ")
print("Your phrase was '%s'" % phrase)

Before we talk about what we just wrote let’s run it:

$ python palindrome.py
Enter a phrase: This is my phrase
Your phrase was 'This is my phrase'
$

Our first line creates a prompt and waits for the user to hit the return key. Additionally, input also returns the text that was entered before the return key was hit. We then echo that back out to the screen just to make sure that it worked.

Removing Spaces

Up to this point we’ve only used functions that we pass data into, types have functions that are tied to them known as methods and we get to use some of those now. To remove the spaces from the string we’re going to use the method replace the type string has. We will be replacing spaces with empty strings, which causes them to be removed.

palindrome.py

phrase = input("Enter a phrase: ")
sanitized_phrase = phrase.replace(" ", "")
print("The phrase without spaces '%s'" % sanitized_phrase)

Reversing a String

Reversing a string in Python is pretty interesting, but it’s not the most intuitive. Strings are essentially groups of characters and we can break them into smaller groups by slicing them. I’ll explain slices a little more when we talk about Arrays, but for now we’re going to go straight to reversing a string.

palindrome.py

phrase = input("Enter a phrase: ")
sanitized_phrase = phrase.replace(" ", "")

print(sanitized_phrase[::-1])

The [::-1] means we want the string starting from the beginning, going all the way to the end, but we want to step through them backwards (that’s the -1 in action).

If palindrome then …

We’re to the final step of our first script. We have our sanitized_phrase and we know how to reverse it, and now we just need to compare it to it’s reversed self. We’ll make our comparison with the == operator, but that doesn’t get us 100% of the way to printing out different messages. Of that we need if and else.

palindrome.py

phrase = input("Enter a phrase: ")
sanitized_phrase = phrase.replace(" ", "")

if sanitized_phrase == sanitized_phrase[::-1]:
    print("'%s' is a palindrome!" % phrase)
else:
    print("'%s' isn't that exciting." % phrase)

A few things to note here: 1. if and else both need to be followed by a : 2. The code inside of each conditional branch needs to be indented 4 more spaces than the if/else.

This is the first time that we’ve seen the whitespace dependency of Python. Indenting is how we separate code groups into different levels.

If we run our script now and use a phrase like nurses run we should see “‘nurses run’ is a palindrome!”. If we use a phrase like “bananas” then we’ll see “‘bananas’ isn’t that exciting.”

Recap

Today we wrote our first more involved program. This script required us to use variables, explore the methods in Python, and use if/else to do different things based on the string that was input from outside of the program. This is just the beginning of our programming adventure, and I hope this has sparked your curiosity to write other little tools. Let me know what you’re building with what you’re learning.

Learn Python – Booleans, Comparisons, and Logic

keith@coderjourney.com (Keith Thompson) — Tue, 30 Aug 2016 11:14:07 +0000

Today we’re going to continue to our series on Python. We’ll learn about Booleans and how to compare values.

Goal for this Tutorial:

Learn about Booleans and comparison operators.
Learn how to compare 2 different values.
Learn how to use logic operators

We’re going to start today’s tutorial where we left off with last weeks. Go to the folder you are going to be using to follow along with the tutorial and run:

$ docker run --rm -it python:3.5

True, False, and Comparisons

Often times when writing programs we need to see if 2 items or equal or we need to take a different approach if something is larger than a specific value. To do this we need to be able to compare things. Let’s take a look at the most common tools that we’ll use when making comparisons in our code.

comparing number

>>> 1 == 1 # Equal to
True
>>> 1 == 2
False
>>> 1 != 1 # Not Equal to
False
>>> 1 != 2
True
>>> 1 < 2 # Less than
True
>>> 2 < 1
False
>>> 1 > 2 # Greater than
False
>>> 2 > 1
True
>>> 1 >= 1 # Greater than or equal to
True
>>> 0 >= 1
False
>>> 1 <= 1 # Less than or equal to
True
>>> 2 <= 1
False

You’ll notice that all of these comparisons give us either True or False, these are the boolean values in Python. “Boolean” values represent true and false from logic. Some languages use true and false all lowercase, but python uses the capitalized variation so you need to make sure that you always capitalize those words.

We can compare more than just numbers, for instance "a" == "a" would work just fine, but the two types that we’re comparing need to be able to compare with one another or be coerced from one type to another. For instance this is what we see if we try to compare a string to a number using greater than or less than:

can’t compare

>>> "a" > 1
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unorderable types: str() > int()

Logical Operators

Now that we have True and False we can also look at what are called the logical operators. These are keywords in python of and, or, and not. Let’s look at them one at a time.

>>> True and False
False
>>> False and True
False
>>> 0 and 1
0
>>> 1 and 0
0

You’ll notice that 1 and 0 returns 0 and that’s because 0 is a falsy value. and will return the first falsy object that is comes across because it’s impossible for them both to be true if one is false. If both are truthy then it returns the item on the right.

If we never need to know whether a value is true or false we can pass it into the bool function like so:

>>> bool(0)
False

Let’s continue looking at the logical operators using the or keyword. I’m going to go back to using True and False to make these operations easier to read.

>>> False or True
True
>>> True or False
True
>>> 0 or False
False
>>> False or 0
0
>>> True or 1
True
>>> 1 or True
1

The or operator return the first truthy value that it comes across going from left to right. If neither of the items is truthy then it returns the item on the right side.

Only one more logical operator to go in the form of not.

>>> not 1
False
>>> not 0
True
>>> not True
False
>>> not False
True

In the same way the bool function returns the truthiness of a value, not returns the opposite.

Recap

Today we took a look what it looks like to compare values in Python using comparison and logical operators, learning along the way how the truth works in python. Comparisons will become very important as we start structuring our code to do things based on inputs that will vary. The logical operators might not seem immediately useful right now, but trust me, we’ll use them a lot.

Learn Python – Strings, Numbers, and the REPL

keith@coderjourney.com (Keith Thompson) — Sun, 28 Aug 2016 22:22:21 +0000

Today we’re going to start our first series on a programming language, and we’ll be digging into Python. Python is a wonderful language to learn and to start with because it is used in so many different fields, from web programming, to data-science, all the way to scripting animations. We’re going to start from the beginning, assuming no prior knowledge of programming and see where this journey takes us. In today’s episode specifically we’re going to become familiar with the a few basic types in python and how we can run our code as we’re developing it.

Goal for this Tutorial:

Become comfortable in the REPL for trying out code in Python.
Create our first python script and learn to run it from the command line.
Investigate a few of the basic types in Python (strings, comments, and numbers).

For this tutorial we’re going to use the official Python image from Docker Hub so that everyone is at the same starting point. If you’ve never used Docker please watch my series on using Docker for Development. To get the image that we’ll be using you can run:

$ docker pull python:3.5

After you’ve downloaded the image start a container running bash so that we can run commands as though we were running python on our machine:

$ docker run --rm -it python:3.5 bash

Getting Started with Python

The first thing we’re going to do is run the python command from the command line:

$ python

You should see something like this:

Python 3.5.2 (default, Aug  9 2016, 20:58:38)
[GCC 4.9.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

This puts us at a prompt known as a REPL, which stands for Read, Evaluate, Print, Loop. We can type lines of Python code here and they will be run write away. REPLs are a great place to play with a language and learn what works and what doesn’t without worrying about setting up a script and running the script.

The first thing every programmer does in any programming language is write his/her “Hello World” program. This program prints the words “Hello World!” to the screen and depending on the language can take quite a few lines of code, but that’s not the case in Python. Let’s write our Hello World now:

>>> print("Hello World!")

You’ll notices that right underneath the line with three greater than symbols it now says “Hello World!” because we printed it out. Breaking this down we needed to use the built-in function print to get our words the print to the screen and we passed in the String “Hello World!”. A string is anything within either single or double quotes in Python (and more programming languages). We’ll be using these a lot, and a good way to think of strings is that they are used if we want to represent words, phrases, or numbers with specific formatting, for instance currency “$1.00”.

Introducing Numbers and Math

Beyond printing our strings, one of the things that we’ll use a lot in Python are numbers. It is pretty common to want to do Math and numbers for the most part work as expected. There are 2 main types of numbers that we’ll work with in Python: Integers and Floats.

Examples of Integers would be the numbers 1 or -12. Examples of Floats would be the numbers 2.5 or -0.5. Floats are the type that we use to represent decimal numbers.

Let’s do some math using these new number types:

>>> 1 + 1 # Addition
2
>>> 10 - 4 # Subtraction
6
>>> 5 / 3 # Division, notice that it returns a float instead of an int
1.6666666666666667
>>> 5 // 3 # Floor division, always returns an int
1
>>> 3 * 9 # Multiplication
27
>>> 8 % 3 # Modulo division, returns an int representing the remainder
2

Creating a python script

We now know how to interact with the Python REPL and work with strings and number types. The last thing we’re going to look at is how to put our code in a script so that we can work with it over type and run more than one line at a time.

For this we’re going to need to stop our container so that we can share through our local directory as a volume to allow us to work with our python script in our text editor. Exiting the REPL requires either hitting Ctrl+d or using the function exit()

$ docker run --rm -it -v $PWD:/usr/src/ -w /usr/src python bash

The -w flag is being used here so that as soon as we’re in the container we’re already in the proper directory to see our source code.

Now let’s open up a new file in our text editor called planets.py.

hello.py

1 + 1
5 / 4

print("There are %s planets" % 9)

Save the file and then from within your running container run

$ python planets.py
There are 9 planets

Our first 2 lines did the math behind the seen and they returned 1 and 1.2 respectively, but we didn’t tell python to print them so we never saw them. on our 4th line we joined our strings and numbers together by using what’s called a format string. The %s is replaced with what follows the % after the string. We could put multiple values in the string also, but we would need to group the values that we want to put in the string together using parenthesis and commas.

Let’s adjust this since I really don’t know what Pluto is at the moment.

planets.py

print("There are %s planets, plus %s" % (8, "pluto"))

Recap

Today we learned how to use the Python REPL, wrote our first Python script, and learned about strings and numbers. This is a good first step in our Coder Journey. In the next episode we’ll look at how we can store information in variables, group pieces of data together in lists, and package up our code to run it again and again using functions.

Learn to Use Docker for Development

keith@coderjourney.com (Keith Thompson) — Fri, 12 Aug 2016 11:36:40 +0000

In this tutorial, we’re going to dig into how we can use Docker for development. Our example will show the development of a Rails application without ever installing Rails on our machine. The process we’re going through today will make you more familiar with how you can use containers in creative ways. In the second part of the tutorial, we will take a look at layering another tool, docker-compose, into the mix to make our development environment even more enjoyable to work with.

Creating our setup using Docker wasn’t the simplest thing possible, but it is repeatable. In the next tutorial, we’ll go through how to use docker-compose to take the hassle out of creating, starting, and connecting our containers for development.

Resources for Using Docker for Development

You should continue looking into the topics mentioned in this tutorial by reading further. Here are the links that I suggest you check out to dig digger into what we did today.

What is Docker?

keith@coderjourney.com (Keith Thompson) — Mon, 08 Aug 2016 17:12:50 +0000

Docker is a tool for working with Linux containers, but knowing that doesn’t make it obvious why that makes it useful. Let’s look at what it is, what it isn’t, and why it might be useful for you.

We’ll be digging into using Docker for development, deployment, testing, and orchestration in the future. The first video utilizing Docker has already been released, learn how to get started.

Begin Learning Docker in 15 Minutes

keith@coderjourney.com (Keith Thompson) — Sat, 30 Jul 2016 20:00:37 +0000

Learning Docker is going to be important in the next few years, so that’s what we’re going to do. In this first episode, we’re going to get up and running with Docker so that we have a little exposure to images & containers. This is a pretty weird place to start, but it will save a lot of time down the road by not needing to cover initial set up for different languages/environments over and over again.

Resources for Learning Docker

Here’s a list of the tools that we interacted with during this tutorial. Docker is very well documented, so feel free to poke around in some of the associated articles when you go to download Docker.

With this short introduction into using Docker, we now have a stepping off point so that we can start using Docker for development and also for packaging and deploying code.

CoderJourney

Kubernetes Frontend Service with Nginx

Goals:

Fixing the Postgres Persistence Layer for Minikube

Building the Nginx Image

Creating the Front-end Deployment & Service

Recap

Using Kubernetes Persistent Volumes

Goals:

Stateful vs Stateless

Creating a Database Deployment

Adding Data Persistence

Connecting Rails to our Database

Updating the mealplan-config ConfigMap

Deploying ConfigMap Changes

Setting up the Database in Kubernetes

Tearing Down the Old Postgres Container

Recap

Define Kubernetes Objects using Yaml

Goals:

Using the Minikube Docker Server

Building our Rails Image

Starting the Database on a Separate Docker Host

Creating a Rails Deployment

Creating the Meal Plan ConfigMap

Adding Configuration to the Meal Plan Deployment

Exposing our Rails Application with a Server

Recap

Run Kubernetes Locally Using Minikube

Installing Kubectl

Installing Minikube

Your First Deployment

Exposing the Deployment

Visualizing Kubernetes

Recap

Learning Rails – Improving Deployment (Part 9)

Goal for this Tutorial:

Pre-compiling Application Assets

Setting up Nginx as our Web Server

Serving Proper Assets on Update

Recap

Learning Rails – Deploying Rails (Part 8)

Goal for this Tutorial:

Prepping the Application for Production

Testing Production Locally

Configuring Asset Handling

Creating our Docker Host in Digital Ocean

Deploying the Application

Recap

Learning Rails – Application Styling (Part 7)

Goal for this Tutorial:

Installing Bootstrap

Adding Navigation

Styling Recipes

Styling Meal Plans

Recap

Learning Rails – Meal Plan UI (Part 6)

Goal for this Tutorial:

Creating a New Meal Plan

New Meal Plan Forms

Meal Plan Create Action

DRYing Up Form Errors

Showing & Listing Meal Plans

Listing Meal Plans

Deleting a Meal Plan

Editing a Meal Plan

Recap

Learning Rails – Modeling Meal Plans (Part 5)

Goal for this Tutorial:

Renaming our Application

Modeling a Meal Plan

Validating the MealPlan and Meal Models

Generating a Meal Plan

Recap

Learning Rails – Recipes Controller and Views (Part 4)

Goal for this Tutorial:

Recipes Controller, Views, and Routes

Rendering Recipes

Creating Fake Recipes

Viewing an Individual Recipe