A couple of months back I was asked if could do a introductory workshop about F# for Active Solution during their internal conference in the Pyrenees, Spain. Of course I said yes and that was the start of this brand new workshop material for F#.

TL;DR: https://mastoj.github.io/FSharpIntro/#/

When I have previously done introductory workshops in F# I have used the awesome material from http://www.fsharpworkshop.com/ by Jorge Fioranelli, but this time decided to create my own material. The reason for doing so is that I wanted the material to feel more like my own while presenting it, and I also wanted to contribute back to the community. Both Jorge's workshop and my have a lot of similarities and cover much of the same material, as they should since they are introductory workshops, but there are small differences in the way it is structured. I really do recommend the workshop by Jorge's as well, and I even think it make sense to go through both for repeated learning.

I felt that the workshop was well received, but I leave that to the kind people at Active Solution to confirm :). The initial plan was that I would get four hours, but due to some delays I got roughly 2-2.5 hours, so you will need at least four hours to cover everything in a workshop setting. It is likely that it will take even longer if you have an curious crowd, as I had, since that will most likely result in a lot of questions, which I think is more important to answer than actual progress in the workshop. So when doing the workshop I recommend time boxing the workshop and do as much as you can.

You can find the start of the workshop at this link https://mastoj.github.io/FSharpIntro/#/, which contains all the presentation slides and links to github repo and exercises.

Let me know if you have any questions. If you find anything that is wrong in the material or have any question regarding the workshop, please comment here or open an issue on github.

To prepare myself for my new job, which will involve some kubernetes stuff, I've played around with it somewhat lately, as you could see in this post. This post is taking things one step further without making it that much more advanced. The goal for this post is to set up a Event Store cluster on google container engine with a simple script. A prerequisite to get any of this working is that you have installed gcloud and kubectl.

If you don't want to read the whole post and just go to the code you can look it up on github. The naïve and master branches will be described in this post.

Disclaimer

What I will describe here will expose the Event Store cluster to the public and is something you should not do, I do it to make it easier to test that it works. I haven't done any performance tests or reliability tests on my setup either, which you should probably do before using it in production.

The end goal

I did two different approaches, which both will be covered in this post, that had the same end goal. I wanted to have a cluster of Event Store nodes running behind a headless kubernetes service and nginx on top of that to add access to the public. Using a headless kubernetes service will result in a service registered with a dns registration that resolves to all the IPs for the associated containers, and this is exactly what EventStore needs to to discovery through dns.

Configuring nginx

I put this section first since it is the same for both approaches.

Nginx configuraion

The configuration of nginx will be stored in a configmap and looks like this, https://github.com/mastoj/eventstore-kubernetes/blob/master/nginx/frontend.conf:

upstream es {
    server es.default.svc.cluster.local:2113;
}
server {
    listen 2113;
    location / {
        proxy_set_header    X-Real-IP $remote_addr;
        proxy_set_header    Host      $http_host;
        proxy_pass          http://es;
    }
}

If you know nginx, this is just basic configuration. First we create an upstream that can be references later on by our proxy_pass when someone is visiting the path /. The url es.default.svc.cluster.local is the dns registration that our Event Store node will get when we get to that point. The server section just defines that we should listen on port 80 and proxy the traffic to the upstream defined.

To create the configmap we can execute this command

kubectl create configmap nginx-es-frontend-conf --from-file=nginx/frontend.conf

The nginx deployment

This is basically the same as I used in the previous post, if you read that one. The specification looks like this:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: frontend-es
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: frontend-es
        track: stable
    spec:
      containers:
        - name: nginx
          image: "nginx:1.9.14"
          lifecycle:
            preStop:
              exec:
                command: ["/usr/sbin/nginx","-s","quit"]
          volumeMounts:
            - name: "nginx-es-frontend-conf"
              mountPath: "/etc/nginx/conf.d"
      volumes:
        - name: "nginx-es-frontend-conf"
          configMap:
            name: "nginx-es-frontend-conf"
            items:
              - key: "frontend.conf"
                path: "frontend.conf"

It only contains one container, the nginx one, which uses the configmap created above for configuration. We only need one replica at the moment.

To create it we run the following command

kubectl create -f deployments/frontend-es.yaml

The nginx service

To create a public IP we need to create a service on top of this deployment. The specification for the service looks like this:

kind: Service
apiVersion: v1
metadata:
  name: "frontend-es"
spec:
  selector:
    app: "frontend-es"
  ports:
    - protocol: "TCP"
      port: 2113
      targetPort: 2113
  type: LoadBalancer

To create the service and finish up the nginx part of the post we run the following command

kubectl create -f services/frontend-es.yaml

First approach - the naïve one

The first thing I wanted to do was to get a cluster up and running without persisting the data to disk, that is, only keep the data in the container. A cluster like that might work for development, but not in production. My grand masterplan was to just add persistent to that cluster after the cluster was up and running, which did not work. How to get persistent will be covered under Second approach. Before we get into the persistent part, let's get this cluster up and running.

Creating the deployment

The deployment file is really simple:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: es
spec:
  replicas: 3
  template:
    metadata:
      labels:
        app: es
    spec:
      containers:
        - name: es
          image: "eventstore/eventstore"
          env: 
            - name: EVENTSTORE_INT_IP
              valueFrom:
                fieldRef:
                  fieldPath: status.podIP
            - name: EVENTSTORE_EXT_IP
              valueFrom:
                fieldRef:
                  fieldPath: status.podIP
            - name: EVENTSTORE_INT_TCP_PORT
              value: "1111"
            - name: EVENTSTORE_EXT_TCP_PORT
              value: "1112"
            - name: EVENTSTORE_INT_HTTP_PORT
              value: "2114"
            - name: EVENTSTORE_EXT_HTTP_PORT
              value: "2113"
            - name: EVENTSTORE_CLUSTER_SIZE
              value: "3"
            - name: EVENTSTORE_CLUSTER_DNS
              value: "es.default.svc.cluster.local"
            - name: EVENTSTORE_CLUSTER_GOSSIP_PORT
              value: "2114"
            - name: EVENTSTORE_GOSSIP_ALLOWED_DIFFERENCE_MS
              value: "600000"
            - name: EVENTSTORE_INT_HTTP_PREFIXES
              value: "http://*:2114/"
            - name: EVENTSTORE_EXT_HTTP_PREFIXES
              value: "http://*:2113/"
          ports:
            - containerPort: 2113
            - containerPort: 2114
            - containerPort: 1111
            - containerPort: 1112

We use the Event Store docker image from docker hub. This image doesn't allow command line arguments, so we need to use environment variables to configure it. You can read about Event Store configuration here. Every container (we are running three here) need to use their own IP during configuration, and with kubernetes we can access that data through status.podIP when we configure the environment variables.

Creating is as simple as before:

kubectl create -f deployments/eventstore.yaml

That should create a three nodes, but as of this moment they will fail to find each other, and that is why we need the service.

Creating the service

Creating the service is even easier than the deployment. The specification file looks like this:

kind: Service
apiVersion: v1
metadata:
  name: "es"
spec:
  selector:
    app: "es"
  ports:
    - protocol: "TCP"
      port: 2113
      targetPort: 2113
  clusterIP: None

We only expose the 2113 port, which means that we will only be able to talk over http to the event store cluster. To the service we name all the containers that has the label app: "es". The last thing to know is that we set the clusterIP to None, this will not create one single IP in the DNS for this service, instead it will resolve to all the IPs of the containers and that is exactly what Event Store needs to be able to configure itself.

Again we are using kubectl to create the service:

kubectl create -f services/eventstore.yaml

When this is created we should now be ready to test it.

Test

To test that it works run the following command:

kubectl get services

In the result from that command you will find an External IP. To access Event Store, open the browser and go to <external ip>:2113. If everything works as expected you should now have access to Event Store.

Challenges with approach one

The major challenge is persistent data, how do you map one persistent volume to each node in the cluster? Leave a comment on the post if you have any idea.

Let us say that you solve the first problem, how do you make sure the nodes get the same persistent volume the next time you restart the cluster?

Both those two problems is what got me started working on the second approach.

There is a third problem, which we won't fix, and that is increasing the number of replicas won't work. The reason be that Event Store doesn't support elastic scaling, so increasing the number of replicas will only add "clones" to the cluster, not increase its size.

Second approach - adding persistent data

We are still going to use deployments, but instead of letting the number of replicas define the number of nodes we will create one deployment with one replica per node. This way we can force each node to get access to the same persistent data volume when it is restarted, and using deployments will also handle restart for us.

Generating deployment

The difference from the first approach here is that we will generate the deployment from a template instead of creating one. For this to work we need both a template and a simple script that generates the deployments. The template file looks like this:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: es-${nodenumber}
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: es-${nodenumber}
        escluster: es
    spec:
      containers:
        - name: es-${nodenumber}
          image: "eventstore/eventstore"
          env: 
            - name: EVENTSTORE_INT_IP
              valueFrom:
                fieldRef:
                  fieldPath: status.podIP
            - name: EVENTSTORE_EXT_IP
              valueFrom:
                fieldRef:
                  fieldPath: status.podIP
            - name: EVENTSTORE_INT_TCP_PORT
              value: "1111"
            - name: EVENTSTORE_EXT_TCP_PORT
              value: "1112"
            - name: EVENTSTORE_INT_HTTP_PORT
              value: "2114"
            - name: EVENTSTORE_EXT_HTTP_PORT
              value: "2113"
            - name: EVENTSTORE_CLUSTER_SIZE
              value: "${nodecount}"
            - name: EVENTSTORE_CLUSTER_DNS
              value: "es.default.svc.cluster.local"
            - name: EVENTSTORE_CLUSTER_GOSSIP_PORT
              value: "2114"
            - name: EVENTSTORE_GOSSIP_ALLOWED_DIFFERENCE_MS
              value: "600000"
            - name: EVENTSTORE_INT_HTTP_PREFIXES
              value: "http://*:2114/"
            - name: EVENTSTORE_EXT_HTTP_PREFIXES
              value: "http://*:2113/"
            - name: EVENTSTORE_DB
              value: "/usr/data/eventstore/data"
            - name: EVENTSTORE_LOG
              value: "/usr/data/eventstore/log"
          ports:
            - containerPort: 2113
            - containerPort: 2114
            - containerPort: 1111
            - containerPort: 1112
          volumeMounts:
            - mountPath: "/usr/data/eventstore"
              name: espd
      volumes:
        - name: espd
          gcePersistentDisk:
            pdName: esdisk-${nodenumber}
            fsType: ext4

The template looks almost the same as in the first case, but we have now added two; nodenumber and nodecount. We have also added a gcePersistentDisk which we mount to usr/data/eventstore, and that is the parent folder we use for logs and data in the configuration of Event Store, EVENTSTORE_DB and EVENTSTORE_LOGS. We have also added a new label escluster which will be used for the service to identify which nodes that should be included in the service.

To generate the actual deploy files we run a bash script that has the following content:

    for ((c=1; c<=$count; c++ ))
    do
        cat ../templates/es_deployment_template.yaml | sed -e "s/\${nodenumber}/$c/" | sed -e "s/\${nodecount}/$count/" > .tmp/es_deployment_$c.yaml
    done

Running that will generate one file for each node, and each file has its own volume mounted. The script as a whole can be found here.

When the files has been generated we can then run:

    for ((c=1; c<=$count; c++ ))
    do
        kubectl apply -f .tmp/es_deployment_$c.yaml
    done

This code will create one deployment per file. Since we are using deployments our pods will be restarted if they crashes.

Creating the service

This is exactly the same as in the first approach, but with a minor change. We will use the escluster label to identify the pods to add to the service. The file is here

The create cluster script

It is not reasonable to execute any of this by hand, so I created this script. I will dissect it here:

#!/bin/bash

function init {
    rm -rf .tmp
    mkdir -p .tmp
}

function validateInput {
    count=$1
    re='^[0-9]+$'
    if ! [[ $count =~ $re ]] ; then
        echo "error: Not a number" >&2; exit 1
    fi
}

The first part is just some house keeping and validating input arguments. The plan is that you should be able to create a cluster of any size by running: ./create_cluster.sh <size>.

function createSpecs {
    local count=$1
    for ((c=1; c<=$count; c++ ))
    do
        cat ../templates/es_deployment_template.yaml | sed -e "s/\${nodenumber}/$c/" | sed -e "s/\${nodecount}/$count/" > .tmp/es_deployment_$c.yaml
    done
}

function createDeployments {
    local count=$1
    for ((c=1; c<=$count; c++ ))
    do
        kubectl apply -f .tmp/es_deployment_$c.yaml
    done
}

The next part defines functions to generate the deployment files and how they should be executed.

function createEsService {
    kubectl create -f ../services/eventstore.yaml
}

This finish of the Event Store part of the script by creating a service on top of the nodes created by deployment.

function addNginxConfig {
    kubectl create configmap nginx-es-frontend-conf --from-file=../nginx/frontend.conf
}

function createFrontendDeployment {
    kubectl create -f ../deployments/frontend-es.yaml
}

function createFrontendService {
    kubectl create -f ../services/frontend-es.yaml
}

The next part is basically what we described in the section about nginx setup.

function createDisks {
    local count=$1
    for ((c=1; c<=$count; c++ ))
    do
        if gcloud compute disks list esdisk-$c | grep esdisk-$c; then
            echo "creating disk: esdisk-$c" 
            gcloud compute disks create --size=10GB esdisk-$c
        else
            echo "disk already exists: esdisk-$c"
        fi
    done
}

A simple helper function to create disks on google cloud.

function createEsCluster {
    local count=$1
    createSpecs $count
    createDeployments $count
    createEsService
}

function createFrontEnd {
    addNginxConfig
    createFrontendDeployment
    createFrontendService
}

The last two functions is just to make it easier to read what is going on.

init
validateInput $1 #sets the variable $count
createDisks $count
createEsCluster $count
createFrontEnd

With all the functions defined it is quite clear what is going on in this script.

Test

The easier way to test it is to create the cluster:

./create_cluster.sh 5

Add some data to the cluster. You do that by finding the external IP of the nginx service and then follow the getting started instructions for Event Store to add some data.

With that in place you can kill pods as you like to simulate failures with:

kubectl delete pod --now <pod id>

When you kill a pod a new should be created, but this time it should use the same persistent disk as the old one.

You can even delete the whole cluster and rebuild it again. I have added delete cluster script that will delete everything but the disks. If you then create the cluster again the data you added should still be there, since we are using the same disks for the cluster.

If you want to change the size of the cluster you can actually do that as well, just delete the cluster and create it again with a larger cluster size and that should work.

Note that if you delete and create the cluster you might end up with a new IP.

Summary

I wouldn't say that this is perfect, but it definitely a start. One drawback is that it doesn't allow for zero downtime increase of cluster size. It could probably be added, but it is out of scope for this post. I haven't tested the performance either, and that is probably something you should do before actually using it. As I mentioned earlier, in a production environment you shouldn't expose Event Store to the public.

There is probably a lot more comments one can have about this setup, but I leave that up to you. Feel free to come with both positive and negative comments :).

I haven't been doing that much F# dotnet core development, but I think it was time for me to try it out. One of the scenarios that I think it will be used a lot is on kubernetes. I choose to run it on google cloud so I didn't have to set up the infrastructure myself. I could probably have used Azure as well since they now have support for it in preview, but I think that google's implementation looks more mature and easier to use from the command line. So let get this little tutorial started.

Creating the application

Before we need start we need to install the latest dotnet core bits, which we find here: https://www.microsoft.com/net/download/core. Clicking on Current tab we will find the latest bits (1.1.0 as of this moment). With dotnet installed we can get going.

Create project

The first is to create the project. Just navigate to a folder where you want your project and run this command to create a new F# project:

dotnet new -l F#

Note that the folder name will be the name of the project, in my case it is suavecore and that will also be the name of the dll file created.

Update references

I made some minor changes to the project.json file:

{
    "version": "1.0.0-*",
    "buildOptions": {
        "debugType": "portable",
        "emitEntryPoint": true,
        "compilerName": "fsc",
        "compile": {
            "includeFiles": [
                "Program.fs"
            ]
        }
    },
    "dependencies": {
        "Microsoft.FSharp.Core.netcore": "1.0.0-alpha-*",
        "Suave": "2.0.0-rc2"
    },
    "tools": {
        "dotnet-compile-fsc": "1.0.0-preview2.1-*"
    },
    "frameworks": {
        "netcoreapp1.0": {
            "dependencies": {
                "Microsoft.NETCore.App": {
                    "type": "platform",
                    "version": "1.1.0"
                },
                "Microsoft.FSharp.Core.netcore": "1.0.0-alpha-161111"
            }
        }
    }
}

I basically changed to the latest version of all the packages and added a reference to Suave.

After the update we need to run

dotnet restore

to install the dependencies.

Implementing the application

The application implemented is really simple, it is a basic Hello World application that also prints the host name. It is a single file application and it all fits in Program.fs:

open Suave
open System.Net

type CmdArgs = { IP: System.Net.IPAddress; Port: Sockets.Port }

[<EntryPoint>]
let main argv = 

    // parse arguments
    let args =
        let parse f str = match f str with (true, i) -> Some i | _ -> None

        let (|Port|_|) = parse System.UInt16.TryParse
        let (|IPAddress|_|) = parse System.Net.IPAddress.TryParse

        //default bind to 127.0.0.1:8083
        let defaultArgs = { IP = System.Net.IPAddress.Loopback; Port = 8083us }

        let rec parseArgs b args =
            match args with
            | [] -> b
            | "--ip" :: IPAddress ip :: xs -> parseArgs { b with IP = ip } xs
            | "--port" :: Port p :: xs -> parseArgs { b with Port = p } xs
            | invalidArgs ->
                printfn "error: invalid arguments %A" invalidArgs
                printfn "Usage:"
                printfn "    --ip ADDRESS   ip address (Default: %O)" defaultArgs.IP
                printfn "    --port PORT    port (Default: %i)" defaultArgs.Port
                exit 1

        argv |> List.ofArray |> parseArgs defaultArgs

    let log x = printfn "%A" x; x

    let getHostName() = 
        Dns.GetHostName()

    // start suave
    startWebServer
        { defaultConfig with
            bindings = [ HttpBinding.create HTTP args.IP args.Port ] }
        (Successful.OK (sprintf "Hello world: %s" (getHostName())))

    0

That is all we need to try the application. If you run

dotnet run

you will start the application and you can now pay a visit to http://localhost:8083.

Publishing the application

The last thing we need to do is to publish the application, this will create the bits that we will add to our docker container later on. Run

dotnet publish -C Release

to publish the application to bin/Release/netcoreapp1.0/publish. If you navigate to that folder it is now possible to run the publish commands by executing

dotnet suavecore.dll

This will start the web server and you can now navigate to http://localhost:8083 again. Note that suavecore is the name of my project, if you have a different name of the project folder your name might differ.

Building the container

To be able to run this on kubernetes later on we will create a docker container. I have docker beta for OSX installed to build and try out the container. If you are following along I assume you to have that installed.

Creating the Dockerfile

The Dockerfile is based on the official dotnet core image from microsoft and looks like this:

FROM microsoft/dotnet:core
COPY ./bin/Release/netcoreapp1.0/publish /app
WORKDIR /app
EXPOSE 8083
ENTRYPOINT ["dotnet", "suavecore.dll"]

It is quite straightforward what is going on. We base our image on the one from Microsoft as mentioned, then we copy our published app to the app folder in the container. We expose port 8083 to be able to access it from the outside and lastly we set the entry point to the command to start the application.

Building the container

Building a container is as simple as

docker build . -t mastoj/suavecore:v1.5

The container is tagged with the name of my repo for this image on docker hub and a version number so we can access the correct version when publishing to kubernetes later on.

Testing the container

Before we publish the container it might be smart to try it out locally first. So to create a container of the newly created image we run

docker run -p=8083:8083 --name suave mastoj/suavecore:v1.5 --ip 0.0.0.0

The command above will start a running container of our image and name it suave. It will also map port 8083 on our local machine to port 8083 on the container. Lastly we will pass the arguments --ip 0.0.0.0 to the application to tell it to listen all request no matter what the IP is.

Again you can try http://localhost:8083, but this time you should get a little bit different response since the host name of the container is probably not the same as your machine.

Publish the container

We are now ready to publish the container. For this tutorial we are using a public repo to keep things simple.

docker push mastoj/suavecore:v1.5

This will upload the image to docker hub and making it accessible to the public.

Setting up google cloud

We are now ready to proceed to the google cloud and kubernetes part. The goal is to host the application with three replicas running behind nginx using https. To be able to try it out on google cloud you need to sign up and create a project. You also need to install the sdk.

Creating the kubernetes cluster

If you have the sdk installed it is really simple to set up a new basic cluster. To create a cluster named k1 run the following

gcloud container clusters create k1

When you have the cluster up and running you need to install kubectl, which is the CLI tool to work with kubernetes.

gcloud components install kubectl

Now you should be all set to operate your google cloud container cluster, hopefully.

Secrets and configmaps

For nginx to run correctly we need to configure it to use https and where our application is in the cluster.

The first part is generating cert.pem and key.pem files for tls to work. If you have openssl installed you can run:

openssl req -newkey rsa:4096 -nodes -sha512 -x509 -days 3650 -nodes -out cert.pem -keyout key.pem

The result from this have I stored in a folder in the repo: https://github.com/mastoj/suavecore/tree/master/kubernetes/tls. You should probably never publish these files, I'm just doing it for demo purpose.

When we have the files we can create a secret that we will be able to mount in our containers when they run in the cluster. To create a secret you use kubectl

kubectl create secret generic tls-certs --from-file=tls/

We will see later how we access the secrets.

Next step is to add the nginx configuration. The configuration is a file, that we will mount when the container starts. The file is also in repo with the name frontend.conf:

upstream hello {
    server hello.default.svc.cluster.local;
}
server {
    listen 443;
    ssl    on;
    ssl_certificate     /etc/tls/cert.pem;
    ssl_certificate_key /etc/tls/key.pem;
    location / {
        proxy_pass http://hello;
    }
}

This configuration file will configure nginx to listen to 443 with ssl enabled and route the traffic to hello.default.svc.cluster.local, which is where our hello nodes will be. You can also see where nginx now expect the secrets to be located.

With this done we can now continue on to creating the deployments and services.

Creating the deployments

If you want to know exactly what a deployment is you should read this: http://kubernetes.io/docs/user-guide/deployments/#what-is-a-deployment. I want go into details about all these concepts, just show you how to configure it.

The frontend deployment

The deployments/frontend.yaml defines the deployment for the frontend, which is the nginx part of our application.

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: frontend
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: frontend
        track: stable
    spec:
      containers:
        - name: nginx
          image: "nginx:1.9.14"
          lifecycle:
            preStop:
              exec:
                command: ["/usr/sbin/nginx","-s","quit"]
          volumeMounts:
            - name: "nginx-frontend-conf"
              mountPath: "/etc/nginx/conf.d"
            - name: "tls-certs"
              mountPath: "/etc/tls"
      volumes:
        - name: "tls-certs"
          secret:
            secretName: "tls-certs"
        - name: "nginx-frontend-conf"
          configMap:
            name: "nginx-frontend-conf"
            items:
              - key: "frontend.conf"
                path: "frontend.conf"

In the file we first define that it is a deployment and some metadata. The interesting part is the containers part where we define that we will use nginx and also add a correct shutdown command when the container is stopped. In the volumes section we define that we want access to our secret named tls-certs, and the configMap named nginx-frontend-conf. For the configMap we are only interested in the key frontend.conf and we are going to name that file the same as the key. When we have defined the volumens we can reference them in the volumeMounts section of the file and define where they should go. That is it for the frontend deployment.

To create our deployment in the cluster run

kubectl create -f deployments/frontend.yaml

Application deployment

The application deployment is defined in deployments/hello.yaml

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: hello
spec:
  replicas: 3
  template:
    metadata:
      labels:
        app: hello
        track: stable
    spec:
      containers:
        - name: hello
          image: "mastoj/suavecore:v1.5"
          args: ["--ip", "0.0.0.0"]
          ports:
            - name: http
              containerPort: 8083

This is a little bit simple deployment. What interesting here is the number of replicas, 3, and that we are referencing the docker image we have created earlier. To create the deployment of the application we need to run

kubectl create -f deployments/hello.yaml

Creating services

Next up is creating services, which allow us to balance the load requests between our nodes and expose the application to the public.

The frontend service

The services/frontend.yaml is what defines the service for the frontend.

kind: Service
apiVersion: v1
metadata:
  name: "frontend"
spec:
  selector:
    app: "frontend"
  ports:
    - protocol: "TCP"
      port: 443
      targetPort: 443
  type: LoadBalancer

We create the service with

kubectl create -f services/frontend.yaml

When the service is created you can run the command

kubectl get services

to check the status. There you will see the public ip when it is available.

The hello application service

The definition in services/hello.yaml is similar to the frontend definition

kind: Service
apiVersion: v1
metadata:
  name: "hello"
spec:
  selector:
    app: "hello"
  ports:
    - protocol: "TCP"
      port: 80
      targetPort: 8083

The difference here is that we don't have the LoadBalancer part, which means our app will NOT be accessible from the outside, you have to go through our frontend. We also routes the traffic from the service port 80 to the container port 8083 which our containers use. Creating the service is as easy as for the frontend

kubectl create -f services/hello.yaml

Win

Everything should now be configured and up and running. You can find the public IP that you should be able to navigate to be executing

kubectl get services

Remember that it is https://<your public ip>, and the cert we are using is self signed so you will probably get a warning about that as well.

The source code for everything is available here: https://github.com/mastoj/suavecore/

If you have any comments or questions, feel free to post them at the comment section.

The last couple of days I had two experiences that triggered this post. The first one was a question at work regarding how to model a finite state machine (FSM) in java or a language similar to java. The second one was a tweet by Isaac Abraham:

It's really disappointing in 2016 to read this "F# tends to be used in domains that have a lot of scientific or financial computations."

— Isaac Abraham (@isaac_abraham) February 12, 2016

The quote was from an interview with Eric Lippert, a former member of the C# team.

So how are these two events related? They are related because the greater community still has a misconception of functional languages, and to find better solutions to problems we need to look further than C# and java. I agree with Isaac 100 % that it is disappointing, and sad, to read and hear this misconception of F# and functional programming. At the same time I do see a change happening for everyone, with a more functional approach in the front-end with libraries like react and more functional aspects in languages like java and C#. What I don't understand is why doesn't people try harder to stay ahead of the game and learn functional programming, even though they do like it when it gets accepted by the greater community. Almost every C# developer I know is now comfortable, and like, lambda expressions. More and more developers discover that immutability isn't that bad after all, but still don't use it as much since it is so easy to use mutability in the languages we use the most. Everyone is now comfortable with the var keyword in C#, but still want to write the whole type on the other side of the equal sign.

F#, and probably scala and other functional languages that are strongly type handle all this in better ways than we are used to in java and C#. So my answer to the question was how I should define an FSM in F#. This is a problem that is not scientific and it has nothing to do with finance. This is a real problem that anyone of us could have in almost any type of application where you need to model a FSM, and model a FSM is probably something we should do more often. Before we get to my answer to the question, let's look at one of the references in one of the other answers. One colleague linked to Martin Fowler's book Domain-Specific Languages which you can read an example dealing with FSM here: http://www.informit.com/articles/article.aspx?p=1592379. Reading through the example gave me a headache and almost made my eyes bleed. The java code in the example is probably good java code, and I would probably think that all those fancy patterns where nice a couple of years back when the book was written. What is the problem with the code in the book? As I said, it is probably nice java code, but that doesn't mean it is readable. I took me a while to understand how the State Machine Model worked as described here: http://www.informit.com/articles/article.aspx?p=1592379&seqNum=2. Why was it so? The main reason it was hard for me to understand the sample code there was probably all the noise and extra syntax. The noise and extra syntax distracted me from the actual model which can never be a good thing. Let's go to the actual FSM defined on page 3: http://www.informit.com/articles/article.aspx?p=1592379&seqNum=3. The code here is definitely easier to understand since the level of noise has decreased due to the previous model, but it is still much noise. In fact, Fowler agree since he also provides multiple way to specify the model in other format than java code. There is one xml version, and two other versions. The only reason you need this is due to the fact that java is to verbose and the level of noise is too high.

F# to the rescue

The whole purpose of this post is to show you that it is possible to solve real world problem with F# in more elegant ways than you would in languages like C# and java. I'll show you my code and then explain why I think it is better. I don't say that Fowler's java code is bad, more that the language might not be right tool.

Let me walk you through this piece of code. First I define a FSM module where all the general logic for implementing the FSM is defined. I define one type that represents the FSM. In the module I have also defined a set of helper functions that helps me create a FSM and also one function that handles an event given an event and a FSM as input. All the helper functions takes a FSM as the last argument, since that allows me to pipe that argument in making it possible to have a really nice DSL in the language. The helper functions takes the provided FSM and returns a new FSM based on the provided FSM and the extra input. registerTransition takes mapping from one state to another given an event.

I defined all the valid events, commands and states on these lines: https://gist.github.com/mastoj/9dfc21848c449fadcc93#file-fowler_fsm-fsx-L54-L72. I really don't think they need any explanation.

The creation of the FSM is done here: https://gist.github.com/mastoj/9dfc21848c449fadcc93#file-fowler_fsm-fsx-L75-L88. As you can see I'm using the helper functions which defines a really nice DSL for me.

When I have the FSM defined I can actually try that it works and that is done here: https://gist.github.com/mastoj/9dfc21848c449fadcc93#file-fowler_fsm-fsx-L90-L105. First I define a helper infix operator that print the current state before calling the next function in each step.

As I hope you can see in this example there are many advantages compared to the java version:

• No nulls
• Impossible to represent bad states
• The ration of noise vs. relevant code is significantly lower
• Shorter code, so it is easier to get the full picture
• No need for an external dsl

Wrap up

F# (and most likely Scala) is a language you can, and should use, to solve your everyday problem. It will give you more concise code, easier to read code, and easier to maintain code. This will also have the positive side effect of less bugs. I understand that it might be a little bit weird at start since it is a whole new paradigm, but the reward is high on the other side. Learning to program functionally will help you write better programs in any language and you will also be ahead of the game when the functional features comes to java and C#. Note that even though those languages get some functional features they will never be as functional as F# and Scala since the base design of the languages are different. Do your self a favor and learn you some FP :)

Last year I wrote a post about Pure Functional Application for the F# advent calendar, I really think it is a great initiative so I signed up again. This is my contribution to this year's F# advent calendar, you can find all the other excellent posts on @sergey_tihon's blog: https://sergeytihon.wordpress.com/tag/fsadvent/

One would expect that I would write a totally different post this year, but instead I decided to make my last year's post more concrete. With that I mean I would like to introduce a tutorial for you to follow. I won't cover the whole tutorial in this post since it is described in the tutorial which you can find on github. The tutorial covers how it might be to work in a project where you have put in the time to set up the boilerplate for a project using CQRS and event sourcing in F#. It might not be production ready either, but it might give you some inspiration of how you can approach application development.

Covering the exercises in the tutorial here would be a little bit boring since they are covered in the tutorial so instead I thought I would explain how the in-memory event store is implemented using an F# agent. You can find the code I will cover in this folder on github: https://github.com/mastoj/LibAAS/tree/master/ex4/done/LibAAS.Infrastructure.

Agents

I hope there are some people out there not that familiar with F# that follow along in this calendar since it is a great opportunity to learn some F#. I'll try to make this post understandable for most developers out there and that is why I'll write a short section about agents. Agents in F# is usually used as alias for MailboxProcessor, so you often see something like this in code where agents are used:

type Agent<'TMessage> = MailboxProcessor<'TMessage>

The way I see agents is like in-process workers that can keep some kind of state. You can compare it to actors in the actor model but much simpler. They are really great if all you need is a async worker inside your process or a nice way of storing state in your application.

Agents work with an inbox to which you can send messages, when a message arrives in the inbox the agent will read it an act upon it. The type of the message must be defined before hand and it can be of any type, a discriminated union is often used as message type. You can use simple types like string in this example

type Agent<'T> = MailboxProcessor<'T>

let agent = Agent.Start(fun (inbox:Agent<string>) ->
    let rec loop() =
        async {
            let! msg = inbox.Receive()
            match msg with
            | "" -> 
                printfn "Stopping agent"
            | _ -> 
                printfn "Message recieved: %s" msg
                return! loop() 
        }
    loop() 
)

let post (agent:Agent<'T>) message = agent.Post message

"hello" |> post agent
"" |> post agent
"hello" |> post agent

We first make an alias for the MailboxProcessor type. When the alias is created it can be used to start the agent with Agent.Start. The Start function takes a function as argument and this is the body of the agent. The structure you see in this simple example is probably the most common one as far as I know. The body is usually a recursive function in which you listen to new messages with the inbox.Receive, if you want to continue process after a message you just make a recursive call. You can define the recursive function to take a state parameter to keep track inside the agent between messages. I also defined a simple helper so it is easier to post messages to agents with the pipe operator. If you run the code above you should see two messages printed, the last one will not be printed since we are not doing a recursive call on empty messages and that stops the agent.

Event store

What is an event store? Short answer: a data store that store events. It is almost that simple. The simplest possible event store need two functions:

• Get events given a stream id
• Save events given a stream id, expected version and events

The first function should return the list of events for the given stream id. A stream is just a way to group events that belong together.

The second function should append the events to a given stream with the stream id given that the version of the stream is the same as the expected version. A version of a stream is basically the number of events in the stream, this prevents concurrency issues and is also the transaction boundary when working against the event store.

That was a short introduction to what an event store is, there is plenty of more information online, but feel free to ask here if you have questions. Next up is the implementation of the event store in F#.

Event store implementation

The implementation I will describe here is using agents, mainly because it is a nice way to abstract away the basics of an event store. With that in place you can easily create different types of event stores by changing two functions as you'll see.

The messages

First let's define some simple helpers for our agent:

module AgentHelper

type Agent<'T> = MailboxProcessor<'T>
let post (agent:Agent<'T>) message = agent.Post message
let postAsyncReply (agent:Agent<'T>) messageConstr = agent.PostAndAsyncReply(messageConstr)

Now when we got that out of our way we can start with the acutal implementation. We will start with the messages and some types that help us stay out of trouble.

type StreamId = StreamId of int
type StreamVersion = StreamVersion of int

type SaveResult = 
    | Ok
    | VersionConflict

type Messages<'T> = 
    | GetEvents of StreamId * AsyncReplyChannel<'T list option>
    | SaveEvents of StreamId * StreamVersion * 'T list * AsyncReplyChannel<SaveResult>
    | AddSubscriber of string * (StreamId * 'T list -> unit)
    | RemoveSubscriber of string

Just by reading this type definitions you can almost understand how the event store is supposed to work. We have a generic Messages type, where the generic parameter defines the type of event that we want to store in the event store. We have four actions we will be able to do against the event store:

1. Get the events for a stream.
2. Save events for a stream.
   a. When saving you can have version conflict and to indicate that we use the SaveResult type.
3. You can add multiple subscribers, where the first string is an id of the subscriber (should probably be wrapped in a type). A subscriber will be called every time some events have been saved.
4. You can remove an existing subscriber based on the string id.

State format

To make the agent flexible we need to keep an internal state that can be provided when creating the agent. The definition of the state type looks like this:

type internal EventStoreState<'TEvent,'THandler> = 
    {
        EventHandler: 'THandler
        GetEvents: 'THandler -> StreamId -> ('TEvent list option * 'THandler) 
        SaveEvents: 'THandler -> StreamId -> StreamVersion -> 'TEvent list -> (SaveResult * 'THandler)
        Subscribers: Map<string, (StreamId * 'TEvent list -> unit)>
    }

What I call EventHandler here is the the "thing" that stores the actual events, it can be an internal map or a connection to an external db. The methods GetEvents and SaveEvents uses the EventHandler to get or save events. The last thing in the state is the subscribers which we also need to keep track of.

Agent body

Next up is the actual implementation of the agent. I give you the code right away and then walk you through it:

let eventSourcingAgent<'T, 'TEventHandler> (eventHandler:'TEventHandler) getEvents saveEvents (inbox:Agent<Messages<'T>>) = 
    let initState = 
        {
            EventHandler = eventHandler
            Subscribers = Map.empty
            GetEvents = getEvents
            SaveEvents = saveEvents
        }
    let rec loop state = 
        async {
            let! msg = inbox.Receive()
            match msg with
            | GetEvents (id, replyChannel) ->
                let (events, newHandler) = state.GetEvents state.EventHandler id
                replyChannel.Reply(events)
                return! loop {state with EventHandler = newHandler}
            | SaveEvents (id, expectedVersion, events, replyChannel) ->
                let (result, newHandler) = state.SaveEvents state.EventHandler id expectedVersion events
                if result = Ok then state.Subscribers |> Map.iter (fun _ sub -> sub(id, events)) else ()
                replyChannel.Reply(result)
                return! loop {state with EventHandler = newHandler}
            | AddSubscriber (subId, subFunction) ->
                let newState = {state with Subscribers = (state.Subscribers |> Map.add subId subFunction)}
                return! loop newState
            | RemoveSubscriber subId ->
                let newState = {state with Subscribers = (state.Subscribers |> Map.remove subId )}
                return! loop newState
        }
    loop initState

To create the agent we need the eventHandler a function to getEvents and saveEvents, nothing to fancy about that. The first thing we do in the function is to create the initState with the input and an empty Map for our subscribers. Next up is the recursive loop (remember it from the section above?). We first wait until there is a new message in the inbox, when we get one we match on the message type.

For GetEvents we use the GetEvents method on the state passing in the EventHandler and the id of the string. When we have the events we reply back to the callee and finish it of with a recursive call with the new state (if it has changed).

SaveEvents works almost the same way, with the addition of notifying the subscribers if we manage to save the events. We also reply back to the callee with the result of the save operation before making the recursive call to wait for the next message.

The implementation of AddSubscriber and RemoveSubscriber do what you would expect them to, it adds or removes a subscriber for the Subscribers map we have in the state and make a recursive call to wait for the next message.

In-memory implementation

To make it a little bit easier for a user to work with an agent it make sense to hide it behind some kind of type, which also make it easier to swap for another implementation later, and that type looks like this:

type EventStore<'TEvent, 'TError> = 
    {
        GetEvents: StreamId -> Result<StreamVersion*'TEvent list, 'TError>
        SaveEvents: StreamId -> StreamVersion -> 'TEvent list -> Result<'TEvent list, 'TError>
        AddSubscriber: string -> (StreamId * 'TEvent list -> unit) -> unit
        RemoveSubscriber: string -> unit
    }

The SaveEvents and GetEvents method returns something of type Result, and that is taken from Railway Oriented Programming which is a really nice way to handle errors in an application without introducing side effects like exceptions. The Result type is defined as:

[<AutoOpen>]
module ErrorHandling
 
type Result<'TResult, 'TError> = 
    | Success of 'TResult
    | Failure of 'TError

let ok x = Success x
let fail x = Failure x

Together with the type we have defined two helpers ok and fail to make it easier to create a Result through piping.

We also need a function to create a wrapper around an agent that create an instance of an EventStore.

let createEventStore<'TEvent, 'TError> (versionError:'TError) agent =
    let getEvents streamId : Result<StreamVersion*'TEvent list, 'TError> = 
        let result = (fun r -> GetEvents (streamId, r)) |> postAsyncReply agent |> Async.RunSynchronously
        match result with
        | Some events -> (StreamVersion (events |> List.length), events) |> ok
        | None -> (StreamVersion 0, []) |> ok

    let saveEvents streamId expectedVersion events : Result<'TEvent list, 'TError> = 
        let result = (fun r -> SaveEvents(streamId, expectedVersion, events, r)) |> postAsyncReply agent |> Async.RunSynchronously
        match result with
        | Ok -> events |> ok
        | VersionConflict -> versionError |> fail

    let addSubscriber subId subscriber = 
        (subId,subscriber) |> AddSubscriber |> post agent

    let removeSubscriber subId = 
        subId |> RemoveSubscriber |> post agent

    { GetEvents = getEvents; SaveEvents = saveEvents; AddSubscriber = addSubscriber; RemoveSubscriber = removeSubscriber}

It is nothing to complicated going on, the getEvents function takes a stream id and wrap it in a GetEvents message which is sent to the agent. After sending the message we wait for the reply and return the events together with the current version of the stream wrapped in a Result type. The saveEvents method works almost the same way, that is, we wrap the input in a SaveEvents message and pass it to the agent and wait for the reply. If we get a VersionConflict back we translate it to the provided error to keep this code isolated from other code.

Now we have all the pieces to put together our in-memory event store. The in-memory event store will use a simple map as a storage for the events for easy lookup.

let createInMemoryEventStore<'TEvent, 'TError> (versionError:'TError) =
    let initState : Map<StreamId, 'TEvent list> = Map.empty

    let saveEventsInMap map id expectedVersion events = 
        match map |> Map.tryFind id with
        | None -> 
            (Ok, map |> Map.add id events)
        | Some existingEvents ->
            let currentVersion = existingEvents |> List.length |> StreamVersion
            match currentVersion = expectedVersion with
            | true -> 
                (Ok, map |> Map.add id (existingEvents@events))
            | false -> 
                (VersionConflict, map)

    let getEventsInMap map id = Map.tryFind id map, map

    let agent = createEventStoreAgent initState getEventsInMap saveEventsInMap
    createEventStore<'TEvent, 'TError> versionError agent

• The initState is of course an empty map since we don't have any events when we start.
• The saveEventsInMap uses the id argument to lookup in the map argument (current state), if the result is None the entry is added to the map with the events. If the entry already exist we check the version before appending the events to the stream.
• The getEventsInMap will just do a lookup in the map and returning an Option type together with the new map which is the same as the input.

With these three functions we can now call the createEventStore function to create our in-memory event store and we are done.

Taking it out for a spin

The simplest way to actually try the event store out is to use it in a fsharp script. So in the same folder as the I have the files for the implementation I also have a simple script with the following content:

#load "AgentHelper.fs"
#load "ErrorHandling.fs"
#load "EventStore.fs"

open EventStore

let inMemoryEventStore = createInMemoryEventStore<string,string> "This is a version error"
inMemoryEventStore.AddSubscriber "FirstSubscriber" (printfn "%A")
let res0 = inMemoryEventStore.SaveEvents (StreamId 1) (StreamVersion 0) ["Hello";"World"]
let res1 = inMemoryEventStore.SaveEvents (StreamId 1) (StreamVersion 1) ["Hello2";"World2"]
let res2 = inMemoryEventStore.SaveEvents (StreamId 1) (StreamVersion 2) ["Hello2";"World2"]

[res0;res1;res2] |> List.mapi (fun i v -> printfn "%i: %A" i v)

We keep it really simple and only storing strings, as well as using a string as our error indicator. Executing this code with mono fsharpi --exec Script.fsx or on Windows fsi --exec Script.fsx should give the following output:

(StreamId 1, ["Hello"; "World"])
(StreamId 1, ["Hello2"; "World2"])
0: Success ["Hello"; "World"]
1: Failure "This is a version error"
2: Success ["Hello2"; "World2"]

The first two lines are from the subscriber and last in the script I print all the results.

Now it is your turn

There is room for a lot of improvement here I guess, but it is a good starting point. Feel free to try the tutorial and also come with suggestion to what can simplify the infrastructure part. The goal of this implementation was to make it easy to use in a tutorial, and I think I manage that since the user only need to use code like the one in the last script.

With all this in place it shouldn't be that hard to implement an agent that is using eventstore or a event simple one backed by a SQL database. All you need to do is send in the connection as the EventHandler and then implement the GetEvents and SaveEvents method accepting the connection (EventHandler) as an argument and returning the result for these two methods together with the new EventHandler state, the state could be the same as the input to the function.

And this finishes of my contribution to this year's F# calendar. I hope you enjoyed the read and learned something. Let me know if you have any questions!

Merry Christmas!

Most web project to day has some javascript in them, and you should really build the javascript to minify them and also to find stupid errors. It would be stupid to implement the build part all over again in F#, instead you should use the tooling that already exists, like node and npm. Even though npm is used to build the javascript application I still want to control the overall build flow with FAKE, and for that reason I created the FAKE NpmHelper.

Configure FAKE

The easiest way to get started is to install node and npm with nuget as part of the build.cmd before calling build.fsx. This will add npm to the default paths that is used by the helper. Don't worry, it is possible to override which npm file that should be used. A sample build.cmd can be found in my FAKE sample and looks like this:

echo off
cls
NuGet.exe "Install" "FAKE" "-OutputDirectory" "packages" "-ExcludeVersion"
NuGet.exe "Install" "OctopusTools" "-OutputDirectory" "packages" "-ExcludeVersion"
NuGet.exe "Install" "Node.js" "-OutputDirectory" "packages" "-ExcludeVersion"
NuGet.exe "Install" "Npm.js" "-OutputDirectory" "packages" "-ExcludeVersion"
"packages\FAKE\tools\Fake.exe" build.fsx %*

Supported commands

There are only two supported commands where you get some type check, Install and Run. Below is the simplest possible sample to use those two.

Npm (fun p ->
  { p with
      Command = Install Standard
      WorkingDirectory = "./src/FAKESimple.Web/"
  })

Npm (fun p ->
  { p with
      Command = (Run "build")
      WorkingDirectory = "./src/FAKESimple.Web/"
  })

I figured those two commands are the one you usually would like to run in this kind of scenario, but if you do want to run any of the other npm commands you can do so by using the Custom command parameter and just pass in the string you like. Or if it is something you think is commonly used send a PR or ping me about it :).

That's all, let me know if you have any questions.

One or two months ago I saw a presentation by Dan Abramov about hot loading and react. I have't done much react and I haven't done much webpack which is used to do this, but I know I wanted a sample for ASP.NET 5 and now I finally had the time to try it out. Even though the titles says ASP.NET 5 the solution can be used for basically any web project running on Windows.

TL;DR

I've put together a small sample with react hot load and ASP.NET 5. You can find the code on github.

What is it?

React hot load let you change the style files (css/less/scss) and javascript files on the server and see the result directly in the browser while keeping the browser state. This make it really easy to try things out and see the result directly instead of reloading the browser after every minor change. An example is when working with styling, instead of tweaking around in the dev tools you can just change the style file directly. Another example is if you're working with a javascript wizard, then it might be annoying to go through every step when you find an error, hot loading the javascript lets you continue where you are after you change the javascript.

Below is a demo showing the result. I use atom for changing less and jsx files since the support for jsx+ES6 in Visual Studio isn't the best yet.

How it sort of works (my version)

I won't cover the details of how hot loading works but the short version of how it works is something like this. Instead of reading the javascript files you read them from a node server, the node server "injects" a "wrapper" between the physical file on disc and the one you pointed too from your web page that routes your javascript command to the file that is currently up to date. If you make a change to a file the wrapper will see this and make a rebuild swapping out what has changed while keeping the state. Keeping the state is possible because of the architecture of react.

Gotchas

My goal was to get a sample up and running with both javascript and less since that is what I most likely will use in a real project, I also wanted everything to run with the latest version of everything. This caused some problems during the setup and here are a list of things that you need to do to your dev environment to get things working:

• I wanted to run regular npm tasks and to do so in Visual Studio you need the extension NPM Scripts Task Runner
• For some reason node 5 was needed, so install that.
• To get node 5 to play nice in Visual Studio you need to add the path for External Web Tools. You find that setting under Tools->Options->Projects and Solutions->External Web Tools. Just add the path to node 5 above the line that points to external tools. It looks like node 5 installs node modules directly under modules folder and not in a hierarchy as before. This makes Visual Studio to think that a lot of the packages are extraneous, but you can just ignore that.
• One of the packages I used got some weird error if python wasn't installed, so python needs to be installed. I couldn't use version 3 of python so I installed version 2 from chocolatey.
• This last part is a little bit tricky and only works because I have both VS2013 and VS2015 I think. I got some error which I found the answer on at stackoverflow. Npm needed to run cl.exe for some task, but it didn't find it for VS2015 so changing Npm to use VS2013 helped as the SO answer says. The command to do so is npm config set msvs_version 2013 --global.

The setup

I won't cover the whole application and what it does, I'll just cover the pieces to get this up and running.

The package.json file

The file I ended up with looks like this

{
  "version": "0.0.0",
  "name": "",
  "scripts": {
    "start": "node server.js",
    "build": "set NODE_ENV=production && webpack -p --progress --colors"
  },
  "devDependencies": {
    "babel-core": "^6.1.2",
    "babel-loader": "^6.0.1",
    "babel-preset-es2015": "^6.1.2",
    "babel-preset-react": "^6.1.2",
    "css-loader": "^0.22.0",
    "less": "^2.5.3",
    "less-loader": "^2.2.1",
    "react-hot-loader": "^1.3.0",
    "style-loader": "^0.13.0",
    "webpack": "^1.12.2",
    "webpack-dev-server": "^1.12.1"
  },
  "dependencies": {
    "jquery": "2.1.4",
    "marked": "^0.3.5",
    "react": "^0.14.0",
    "react-dom": "^0.14.0"
  }
}

I've added to npm scripts to the file, build that should be used when packaging for production on a build server and start that is used to start the server for development. This only works with webpack so I installed that and all the loaders I needed to build jsx and less files.

The server.js

The server.js is a small node server that will host the javascript for us during development. It is started with the start script task defined in package.json.

var webpack = require('webpack');
var WebpackDevServer = require('webpack-dev-server');
var config = require('./webpack.config');

new WebpackDevServer(webpack(config), {
    publicPath: config.output.publicPath,
    hot: true,
    historyApiFallback: true,
    headers: { 'Access-Control-Allow-Origin': '*' }
}).listen(3000, 'localhost', function (err, result) {
    if (err) {
        console.log(err);
    }

    console.log('Listening at localhost:3000');
});

The server is based on this boilerplate code but I added headers: { 'Access-Control-Allow-Origin': '*' }. When I upgraded everything to the latest bits CORS was required.

Webpack configuration

You can run webpack directly from the command line, but you usually use a configuration file to do so. This is my first time using webpack so it is most likely a guide to how you should do it, more a sample of how I got it to do what I wanted it to do. The webpack.config.js I ended up with looks like this:

var webpack = require('webpack');
var path = require('path');
var outFolder = path.resolve(__dirname, "./wwwroot/app");
var isProduction = process.env.NODE_ENV === 'production ';
var jsxLoaders = isProduction ?
    ['babel?presets[]=es2015,presets[]=react'] :
    ['react-hot', 'babel?presets[]=es2015,presets[]=react']; // only react hot load in debug build
var entryPoint = './content/app.jsx';
var app = isProduction ? [entryPoint] : [
    'webpack-dev-server/client?http://0.0.0.0:3000', // WebpackDevServer host and port
    'webpack/hot/only-dev-server', // "only" prevents reload on syntax errors
    entryPoint
];

module.exports = {
    entry: {
        app: app
    },
    output: {
        path: outFolder,
        filename: "[name].js",
        publicPath: 'http://localhost:3000/static/'
    },
    devtool: "source-map",
    minimize: true,
    module: {
        loaders: [{
            test: /\.(js|jsx)$/,
            loaders: jsxLoaders,
            exclude: /node_modules/
        },
        {
            test: /\.(css|less)$/,
            loaders: ['style','css','less']
        }]
    },
    plugins: [
      new webpack.HotModuleReplacementPlugin()
    ],
    resolve: {
        extensions: ["", ".webpack.js", ".web.js", ".js", ".jsx"]
    },
    devServer: {
        headers: { "Access-Control-Allow-Origin": "*" }
    }
};

First I define some settings that differs depending on environment. The environment is set as an environment variable, see the build script task in the package.json file. One important part is this one:

var jsxLoaders = isProduction ?
    ['babel?presets[]=es2015,presets[]=react'] :
    ['react-hot', 'babel?presets[]=es2015,presets[]=react']; // only react hot load in debug build
var entryPoint = './content/app.jsx';
var app = isProduction ? [entryPoint] : [
    'webpack-dev-server/client?http://0.0.0.0:3000', // WebpackDevServer host and port
    'webpack/hot/only-dev-server', // "only" prevents reload on syntax errors
    entryPoint
];

This is what make the actual server running, I used to port 3000 to host the files. I also needed to specify the publicPath under output, that's because the files are not served from the same application as the consumer of the files. As you can see, if we are doing a production build, by running npm rum build, we will only use the actual app.jsx as entry point. Also, we won't add react-hot (alias for react-hot-loader) to the list of jsxLoaders since I don't want hot loading enabled in production.

I won't try to cover webpack in more depth since all this is sort of new to me.

The ASP.NET part

If you haven't figured it out by now, the ASP.NET solution stays mainly the same to get this working. The trick is actually just to fire up a node server to host your static content and then point the script tags in your solution to that server. So my simple index page looks like this:

@{
    // ViewBag.Title = "Home Page";
}
<html>
<head>
    <title>Sample hot load demo</title>
    <link href="/static/"/>
</head>
<body>
    <div id="content"></div>
    @*<script src="/static/app.js"></script>*@
    <script src="http://localhost:3000/static/app.js"></script>
</body>
</html>

As you can see I'm pointing to localhost:3000 instead of directly to disk, this is what makes everything above work. In production probably want to point to the file to disk, and that could probably be solved by tag helpers in ASP.NET 5, or using server side variables based on environment in any other version of ASP.NET.

Running everything

If you have cloned the repository and want to try it out you can now either run start from the Task Runner Explorer if you have the NPM Scripts Task Runner installed, or you can run npm run start from the command line in the root of the web project.

This will start the node server for you. When the node server is up and running you can start the ASP.NET application. Now you can start to interact with the application in the browser and then try to change the jsx or less files, save and see the changes appear in the browser with no refresh of the page.

Summary

React hot load looks to me like an awesome way to get fast feedback while doing web development with react. There was a little bit of hazzle to get it up and running on Windows but it is doable, and you probably only need to feel that pain once :). Let me know if you have any questions.

I've been using TeamCity for quite a while and trusted it to do the right thing when it comes to building. It has a lot of built in feature to do so, but the moment you start to define your build inside TeamCity you have painted yourself into a corner. I still think TeamCity is a great tooling triggering my builds, but I have come to realize that I should start to put the actual definition of the builds outside of TeamCity, or whatever CI tool I am using.

Why build scripts?

Many people argue that it works just fine with using TeamCity and other CI tools to define the way you build, and it is when you have a set of smaller projects I think. The moment when you need to do more things in your build it make sense to put it in a separate script file and here are some of my reasons:

• You got your actual build process under source control together with your code instead of defined in a CI tool
• It is easier to handle versioning
• The way you build on the server is identical to the way you build locally
• You will not be locked in to one CI tool, if you use TeamCity today it is quite easy to switch to AppVeyor tomorrow and TravisCI the day after that.

What is FAKE

FAKE is a domain specific language, dsl, for build tasks implemented in F#. That it is implemented in F# doesn't mean you can only build F# projects, in fact you should event be able to build java project if you would like to. F#, along with many functional programming languages, is terrific when you want to create a dsl, since functional programming languages often are more expressive and not as verbose as OO languages. This is not an introduction to FAKE, and the documentation is here if you want to check it out. Later in the post I will go through what my FAKE script look like to solve the problems I imagined.

The demo

The thing I wanted to try was this:

• Simple .NET Web app that includes some js building
• GitHub as source control
• AppVeyor as a CI engine to execute build
• Octopus Deploy to handle deployment
• Deploy to Azure Web App
• Most of it configured in FAKE

Illustration of the flow from code to deployment:

I think the end result is really good since it is the first time I used FAKEand AppVeyor. If you want to see the sample app and also the builds scripts they are in this repo at github: https://github.com/mastoj/FAKESimpleDemo

Appveyor

I could as well used TeamCity, but I thought I would try Appveyor out. The goal of AppVeyor is:

AppVeyor aims to give powerful Continuous Integration and Deployment tools to every .NET developer without the hassle of setting up and maintaining their own build server. - About AppVeyor

Their goal is the reason I think AppVeyor is interesting and I wanted to take it for a spin. The experience has been truly amazing with really fast build times and easy to get started, and another really positive aspect is that I don't have to think about maintaining the build server.

AppVeyor can do a lot more than I did, one might even argue that I should use AppVeyor instead of Octopus Deploy for deployment, but this is a proof of concept for a scenario were I still will have Octopus Deploy and I really like Octopus Deploy :).

When you configure AppVeyor you can do so in the UI or with a yml-file. Of course I choose to use a yml-file since I do want as much as I can in source control. The configuration file I created is minimal since I have FAKE that take care of the actual build process:

environment:
  version: 1.0.0
assembly_info:
  patch: false
build_script:
  build.cmd
test: off

This specifies a environment variable version and that AppVeyor should not run tests or patch the assmebly information file. To start the build the build.cmd default commmand should be used. I can still run test and patch the assembly information file, but that is something I want to do from FAKE and not define in AppVeyor.

I alos added some environment settings in the UI like this:

The reason I added them there and not in source control is because that is not I want to have on GitHub. The same would apply if I needed some other "secret" settings during build, then I would add that a a environment setting in the UI. I do the same thing if I use TeamCity.

Octopus Deploy

I'm not going to go through what Octopus Deploy and what you can do with it, all you nned to know is that it is a great tool for handling deployments to multiple different environments locally as well as in the cloud.

To deploy to an Azure Web App you need to create an account under Azure that is an Azure account. You create accounts in Environment->Accounts. To create an Azure account you will need a Management Certificate (.pfx) and your subscriptiong id. I don't remember which guide I followed to generate mine, but this might work: https://azure.microsoft.com/en-us/blog/obtaining-a-certificate-for-use-with-windows-azure-web-sites-waws/

When that is done you just add your project and creates a deployment step of type Deploy an Azure Web App and you are good to go. The step configuration should like something like:

FAKE

This is were all the magic happens. I will try to break down this file: https://github.com/mastoj/FAKESimpleDemo/blob/master/build.fsx and explain the different parts of it. I expect you to know the minimum basics of FAKE, that is, what is a target and how you set up a build process with the ==> operator.

Bootstrap

To get FAKE running easily I also have a bootstrap file build.cmd.

@echo off
cls
NuGet.exe "Install" "FAKE" "-OutputDirectory" "packages" "-ExcludeVersion"
NuGet.exe "Install" "OctopusTools" "-OutputDirectory" "packages" "-ExcludeVersion"
NuGet.exe "Install" "Node.js" "-OutputDirectory" "packages" "-ExcludeVersion"
NuGet.exe "Install" "Npm.js" "-OutputDirectory" "packages" "-ExcludeVersion"
"packages\FAKE\tools\Fake.exe" build.fsx %*

All this file does is installing the tools I need to build the project, and then calls the build script with the arguments provided.

I separated the build script into some modules to make it easier to follow and I thought I would go through this module by module.

Npm module

The demo application had a simple js-app (that alerted "Hello") to show that it is possible to build js-apps with FAKE. I would be stupid if I did not use the tools from the js-community to do the heavy lifting here, so that is exactly what I did. You can find the package.json here and the gulpfile.js here. I won't cover those since it is a different topic. When npm and gulp was configured all I needed to do was trigger it from FAKE and to do so I wrote a simple Npm wrapper. (I might extract this and try to submit a PR to FAKE later). The code is quite simple:

module Npm =
  open System

  let npmFileName =
    match isUnix with
      | true -> "/usr/local/bin/npm"
      | _ -> "./packages/Npm.js/tools/npm.cmd"

  type InstallArgs =
    | Standard
    | Forced

  type NpmCommand =
    | Install of InstallArgs
    | Run of string

  type NpmParams = {
    Src: string
    NpmFilePath: string
    WorkingDirectory: string
    Command: NpmCommand
    Timeout: TimeSpan
  }

  let npmParams = {
    Src = ""
    NpmFilePath = npmFileName
    Command = Install Standard
    WorkingDirectory = "."
    Timeout = TimeSpan.MaxValue
  }

  let parseInsallArgs = function
    | Standard -> ""
    | Forced -> " --force"

  let parse command =
    match command with
    | Install installArgs -> sprintf "install%s" (installArgs |> parseInsallArgs)
    | Run str -> sprintf "run %s" str

  let run npmParams =
    let npmPath = Path.GetFullPath(npmParams.NpmFilePath)
    let arguments = npmParams.Command |> parse
    let result = ExecProcess (
                  fun info ->
                    info.FileName <- npmPath
                    info.WorkingDirectory <- npmParams.WorkingDirectory
                    info.Arguments <- arguments
                  ) npmParams.Timeout
    if result <> 0 then failwith (sprintf "'npm %s' failed" arguments)

  let Npm f =
    npmParams |> f |> run

The Npm function is the important part. There I take the default arguments, pass it through f which adds changes to the arguments and then that is passed to run. The run method parse the Command property and execute Npm. It doesn't get simpler than that.

OctoHelpers module

There were two things I wanted to do with Octopus Deploy; create release and create deploy. This helper module is a wrapper around the Octo module in FAKE since there were a lot of similarities between the steps.

module OctoHelpers =
  let executeOcto command =
    let serverName = environVar "OCTO_SERVER"
    let apiKey = environVar "OCTO_KEY"
    let server = { Server = serverName; ApiKey = apiKey }
    Octo (fun octoParams ->
        { octoParams with
            ToolPath = "./packages/octopustools"
            Server   = server
            Command  = command }
    )

All that is going on here is that I read some things from environment variables that should be configured on AppVeyor (or the CI you use) and then execute an action against Octopus Deploy.

AppVeyorHelper module

I extracted the things that dealt with AppVeyor integration to a separate module to keep my targets clean (I get to the targets soon). It is actually one thing I'm doing on AppVeyor and that is publishing the artifacts to the nuget feed hosted by AppVeyor.

module AppVeyorHelpers =
  let execOnAppveyor arguments =
    let result =
      ExecProcess (fun info ->
        info.FileName <- "appveyor"
        info.Arguments <- arguments
        ) (System.TimeSpan.FromMinutes 2.0)
    if result <> 0 then failwith (sprintf "Failed to execute appveyor command: %s" arguments)
    trace "Published packages"

  let publishOnAppveyor folder =
    !! (folder + "*.nupkg")
    |> Seq.iter (fun artifact -> execOnAppveyor (sprintf "PushArtifact %s" artifact))

The publishOnAppveyor function takes a folder and finds all the nuget packages in it. After that it executes the appveyor command to publish every package to the feed.

Settings module

The module name isn't perfect, but what is. It contains all the variables that are used in the targets as well as two three helper functions:

module Settings =
  let buildDir = "./.build/"
  let packagingDir = buildDir + "FAKESimple.Web/_PublishedWebsites/FAKESimple.Web"
  let deployDir = "./.deploy/"
  let testDir = "./.test/"
  let projects = !! "src/**/*.csproj" -- "src/**/*.Tests.csproj"
  let testProjects = !! "src/**/*.Tests.csproj"
  let packages = !! "./**/packages.config"

  let getOutputDir proj =
    let folderName = Directory.GetParent(proj).Name
    sprintf "%s%s/" buildDir folderName

  let build proj =
    let outputDir = proj |> getOutputDir
    MSBuildRelease outputDir "ResolveReferences;Build" [proj] |> ignore

  let getVersion() =
    let buildCandidate = (environVar "APPVEYOR_BUILD_NUMBER")
    if buildCandidate = "" || buildCandidate = null then "1.0.0" else (sprintf "1.0.0.%s" buildCandidate)

The getOutputDir is used to get the name of the folder of a file, it says proj but it could be any file. I'm using it to create one output folder per project instead of everything ending up in the same folder or under bin/release as it usually does when building from VS. It is using the convention that the parent folder name of a project file is the name of the project. The build function is a wrapper to build one single project at a time to be able to specify the output folder per project. I don't bother to explain the getVersion function.

Targets module

This is where I define all the separate steps. When I have all my helpers settled it is quite straighforward:

module Targets =
  Target "Clean" (fun() ->
    CleanDirs [buildDir; deployDir; testDir]
  )

  Target "RestorePackages" (fun _ ->
    packages
    |> Seq.iter (RestorePackage (fun p -> {p with OutputPath = "./src/packages"}))
  )

  Target "Build" (fun() ->
    projects
    |> Seq.iter build
  )

  Target "Web" (fun _ ->
    Npm (fun p ->
      { p with
          Command = Install Standard
          WorkingDirectory = "./src/FAKESimple.Web/"
      })

    Npm (fun p ->
      { p with
          Command = (Run "build")
          WorkingDirectory = "./src/FAKESimple.Web/"
      })
  )

  Target "CopyWeb" (fun _ ->
    let targetDir = packagingDir @@ "dist"
    let sourceDir = "./src/FAKESimple.Web/dist"
    CopyDir targetDir sourceDir (fun x -> true)
  )

  Target "BuildTest" (fun() ->
    testProjects
    |> MSBuildDebug testDir "Build"
    |> ignore
  )

  Target "Test" (fun() ->
    !! (testDir + "/*.Tests.dll")
        |> xUnit2 (fun p ->
            {p with
                ShadowCopy = false;
                HtmlOutputPath = Some (testDir @@ "xunit.html");
                XmlOutputPath = Some (testDir @@ "xunit.xml");
            })
  )

  Target "Package" (fun _ ->
    trace "Packing the web"
    let version = getVersion()
    NuGet (fun p ->
          {p with
              Authors = ["Tomas Jansson"]
              Project = "FAKESimple.Web"
              Description = "Demoing FAKE"
              OutputPath = deployDir
              Summary = "Does this work"
              WorkingDir = packagingDir
              Version = version
              Publish = false })
              (packagingDir + "/FAKESimple.Web.nuspec")
  )

  Target "Publish" (fun _ ->
    match buildServer with
    | BuildServer.AppVeyor ->
        publishOnAppveyor deployDir
    | _ -> ()
  )

  Target "Create release" (fun _ ->
    let version = getVersion()
    let release = CreateRelease({ releaseOptions with Project = "FAKESimple.Web"; Version = version }, None)
    executeOcto release
  )

  Target "Deploy" (fun _ ->
    let version = getVersion()
    let deploy = DeployRelease(
                  { deployOptions with
                      Project = "FAKESimple.Web"
                      Version = version
                      DeployTo = "Prod"
                      WaitForDeployment = true})
    executeOcto deploy
  )

  Target "Default" (fun _ ->
    ()
  )

The responsibility of each target is as follows:

• Clean - removes all the output files so I get a clean build
• RestorePackages - gets all the packages from nuget
• Build - take my definition of project files and runs the build helper for each one
• Web - restore all the node modules and then builds the js-app using Npm
• CopyWeb - copy the js-app to the web application so it can be added to the package for deployment
• BuildTest - same as Build but for the test projects
• Test - execute the tests
• Package - packages the web application to a nuget package that I can use for deployment
• Publish - publishes all the packages to the nuget feed, in this case AppVeyor
• Create release - creates a release on Octopus Deploy
• Deploy - deploy the release created
• Default - empty default step that I can always use to run everything

There were many steps there, but I think it is nice to have that separation in place. I could probably combine some of the steps, but I like it as it is since it is easier to move things around if I want to.

The build process

The last part is to defined the dependencies between all the targets, and here it is:

"Clean"
==> "RestorePackages"
==> "Build"
==> "Web"
==> "CopyWeb"
==> "BuildTest"
==> "Test"
==> "Package"
==> "Publish"
==> "Create release"
==> "Deploy"
==> "Default"

RunTargetOrDefault "Default"

I most likely only want to run down to Test locally and need to make some adjustments to run everything locally, but it is doable. The important part is that I can create an actual artifact locally, and that I'm doing it the same way as I would on the build server.

Summary

This post became longer than I thought, but I hope you see the use of using FAKE. Once more I think F# shows that it is a good fit for many different problems, not just science and math. If I start a new .NET-project today I would definitely add FAKE as one of the first things. That gives me a reliable build that executes the same way on the server and locally as well as version control of the build process compared to having it all configured in the CI server.

If you find any improvements, please comment here or on GitHub. You should be able to clone the repo and just execute build.cmd Test to get started.

Constraints in F# is a really powerful feature, but it is also an area that I think is missing some clear documentation about how to use it. This will be a short write up for future me when I need it, and I also it will help some lost souls out there.

Inline functions

F# will most of the time handle the types for you, also generics when it can do so. A simple example:

let add x y = x + y

This will when it stands all by itself resolve to the have the type:

val add : x:int -> y:int -> int

That all the compiler can do when it is not provided any more information. If you instead write

let add x y = x + y
add "Hello " "world"

add will get the type

val add : x:string -> y:string -> string

since you on line two specify that you will use it with strings. What should you do if you want to add it with any arguments that supports the + operator? This is where the inline keyword is useful (it can also be used in some optimization scenarios). So let us define the function again and adding the inline keyword:

let inline add x y = x + y

The type for this version of add is a little bit more complicated:

val inline add :
  x: ^a -> y: ^b ->  ^c
    when ( ^a or  ^b) : (static member ( + ) :  ^a *  ^b ->  ^c)

What does this mean?

• x: ^a -> y: ^b -> ^c specify that the function takes two arguments and returns something. The argument has types ^a and ^b and the result have type ^c
• when ( ^a or ^b) : (static member ( + ) : ^a * ^b -> ^c) is adding a constraint on the types ^a, ^b and ^c. The constraint says that it must exist a static operator + that takes a tuple of type ^a * ^b and returns type ^c.

We got all that by adding the keyword inline, but why? When adding the inline keyword you basically say that whenever you see the function name somewhere in the code replace that name with this function definition. This basically gives you one new implementation of the function every time you use it. If you don't use inline you will only have one implementation and that is why you don't get it as generic as you would like. This is my simplified explanation, someone on the compiler team can probably explain it a lot more in detail.

Constraining on interfaces

I won't go on with you can constrain the types on interfaces because it quite simple and there is more interesting constraint in the next section. I'll just throw the code at you

type ISimpleInterface =
    abstract member Add: int -> int -> int

type SimpleClassA() =
    interface ISimpleInterface with
        member this.Add x y = x + y

type SimpleClassB() =
    interface ISimpleInterface with
        member this.Add x y = x + y

let doSimple<'T when 'T :> ISimpleInterface>  (x: 'T) = x.Add 5 5
let doSimple2 (x: ISimpleInterface) = x.Add 5 5

let a = new SimpleClassA()
let b = new SimpleClassB()
doSimple a
doSimple b

Here first define an interface and then two implementations of that interface. After that I defines two functions doSimple and doSimple2, which are basically identical. I prefer to use the second variant when possible.

If it walks like a duck and quack like a duck

Duck typing can be achieved in F# by using constraint. This is really powerful since you can write functions that take in arguments and as long as the types of the arguments support doesn't violate the constraint you can use them without the need of an interface. Why would you want to do this you might ask? The reason I started to look into this was actually because I needed it at work. I am using the excellent SqlClient type provider, and wanted to use the SqlProgrammabilityProvider to read and update multiple tables using the pipe operator. The problem is that the Update method is defined on the generated table type and not as a static Update method. A simplified version of what I had looks somewhat like this:

type SomeClass1() =
    member
       this.Add(a:int, b:int) = a + b

type SomeClass2() =
    member
        this.Add(a:int, b:int) = a + b

and I wanted an add method that could be applied to either SomeClass1 or SomeClass2 so I could write someClassInstance |> add 2 3. To achieve that you use member constraints like this:

type SomeClass1() =
    member
        this.Add(a:int, b:int) = a + b

type SomeClass2() =
    member
        this.Add(a:int, b:int) = a + b

let inline add (y:int) (z:int) (x: ^T when ^T : (member Add : int*int->int)) =
    (^T : (member Add : int*int->int) (x,y,z))

SomeClass1() |> add 2 3
SomeClass2() |> add 2 3

In the function definition I define that I the argument x must have an operator called Add that has type int*int->int. In the body of the function I specify that I will call the Add function on variable x with the input of y and z. It looks complicated until you get your head around it. It is also important to specify the method as inline. One thing that is good to know is that it doesn't work for curried functions even though you don't get a compile time error.

The case I had was a slightly more complicated since the Update method I wanted to use had some optional arguments. If you know that optional arguments is represented as Option types in F# the code isn't that surprising:

type SomeClass1Option() =
    member
        this.Add(?a:int, ?b:int) = 
            match a,b with
            | Some x, Some y -> x + y
            | _, _ -> 0

type SomeClass2Option() =
    member
        this.Add(?a:int, ?b:int) = 
            match a,b with
            | Some x, Some y -> x + y
            | _, _ -> 0

let inline add (y:int) (z:int) (x: ^T when ^T : (member Add : int option*int option->int)) =
    (^T : (member Add : int option*int option->int) (x,Some y,Some z))

SomeClass1Option() |> add 2 3
SomeClass2Option() |> add 2 3

Given all this I ended up writing an update method that looks like this:

let inline updateTable(table: ^T when ^T : (member Update : SqlConnection option*SqlTransaction option*int option -> int)) = 
    (^T : (member Update : SqlConnection option*SqlTransaction option*int option-> int) (table, None, None, None))

That method can be used with piping to call the Update method on any table generated with SqlProgrammabilityProvider with default arguments.

You can also write constraint on static operators, but I won't cover that. The documentation for constraints is found here: https://msdn.microsoft.com/en-us/library/dd233203.aspx.

As you can read in Topshelf F# api improved I started working on a demo, but it magically grown to include a change to the Topshelf.FSharp project. The good part with this is that to run Suave with Topshelf as a Windows service have never been easier than now. The example code will be made available official examples for Suave, I hope, but meanwhile you can find the code on github.

The code <tl;dr;>

open Suave
open Suave.Http.Successful
open Suave.Web 
open Suave.Http
open Suave.Http.Applicatives
open Suave.Http.Successful
open Topshelf
open System
open System.Threading

[<EntryPoint>]
let main argv = 
    printfn "%A" argv

    let cancellationTokenSource = ref None

    let home = choose [path "/" >>= GET >>= OK "Hello world"]
    let mind = choose [path "/mind" >>= GET >>= OK "Where is my mind?"]
    let app = choose [ home; mind ]

    let start hc = 
        let cts = new CancellationTokenSource()
        let token = cts.Token
        let config = { defaultConfig with cancellationToken = token}

        startWebServerAsync config app
        |> snd
        |> Async.StartAsTask 
        |> ignore

        cancellationTokenSource := Some cts
        true

    let stop hc = 
        match !cancellationTokenSource with
        | Some cts -> cts.Cancel()
        | None -> ()
        true

    Service.Default 
    |> display_name "ServiceDisplayName"
    |> instance_name "ServiceName"
    |> with_start start
    |> with_stop stop
    |> with_topshelf

What is going on?

I don't think the code need much explanation, but here are some lines. First we create a CancellationToken which we pass to the start function. The stop can then use the CancellationTokenSource to cancel the async operation that we start in the start function. Right before start we define our Suave app, it consist of two web parts, home and mind which are combined to one app in app. When the start and stop functions are defined it is trivial to use the new updated version of Topshelf.FSharp to run the suave application as a service.

My plan was never to improve the F# api for Topshelf, Topshelf.FSharp, but Henrik Feldt asked me to when he saw what I was doing when working with a Suave, which Henrik is a core contributor of, demo where I host the application in Topshelf. I implemented a simple Topshelf wrapper that had a nice, at least I think so, fluent api. The Topshelf.FSharp also had a fluent api, but it was somewhat more complicated so what I was asked to do was implement my concept in the existing Topshelf.FSharp project, and so I did (released as version 2.0.1 of the package).

The API

The already existing API was good, but there some things that could be improved. The major plus with the existing function was that it already had existing functions for basically do every possible configuration of the service, so all I had to do was to find a nicer way to improve the fluent part of the API. The goal I had in mind, and what I implemented in my demo was an API looking somewhat like this:

Service.Default
|> with_start start
|> with_recovery (ServiceRecovery.Default |> restart (min 10))
|> with_stop stop
|> run

It's really easy to follow and true to F#. There are many more functions you could execute before run to configure the service, but the most important ones are with_start, with_stop and run. The start and stop functions just take a single functions which are executed on start and stop and returns a bool and the run function is what executes the service and returns an int as expected. I won't cover any more details, but just look at the github repo if you want to know what configuration functions there is.

Under the hood

So how do one build this type of API on top of another more OO oriented framework and one answer to this is to use a kind of builder pattern. All the functions before run function is executed just creates a specification of the service and what should happen when run is executed. The specification I ended up implementing for Topshelf look like this:

type Service =
  { Start: HostControl -> bool
    Stop: HostControl -> bool
    HostConfiguration: (HostConfigurator -> HostConfigurator) list }
  static member Default =
      { Start = (fun _ -> true)
        Stop = (fun _ -> true)
        HostConfiguration = [] }

For a service to work you need a start and stop function, and that is what with_start and with_stop do. All the functions, except from run, take the Service type as the last parameter as well as returning a new instance of a Service making it possible to pipe the specification between all the configuration steps. The static Default member makes it easy to start the configuration. To configure the service all the configuration functions add a function to the list of HostConfiguration describing what should be done when the service is instantiated. This is done by a base function add_host_configuration_step which all the configuration functions partially applies like below:

let add_host_configuration_step step service =
    {service with HostConfiguration = step::service.HostConfiguration}

let enable_pause_and_continue =
    add_host_configuration_step (fun c -> c.EnablePauseAndContinue();c)

To start the service there are a couple of things we need to do, but first the code and then the description of it

let toAction1 f = new Action<_>(f)
let toFunc f = new Func<_>(f)

let service_control (start : HostControl -> bool) (stop : HostControl -> bool) () =
  { new ServiceControl with
    member x.Start hc =
      start hc
    member x.Stop hc =
      stop hc }

let create_service (hc:HostConfigurator) service_func =
  hc.Service<ServiceControl>(service_func |> toFunc) |> ignore

let run service =
  let hostFactoryConfigurator hc =
      let createdHc = service.HostConfiguration |> List.fold (fun chc x -> x chc) hc
      service_control service.Start service.Stop
      |> create_service createdHc

  hostFactoryConfigurator |> toAction1 |> HostFactory.Run |> int

First we need to figure out what the we need to run the service, and the Topshelf HostFactory.Run method takes an Action<HostConfigurator>. To create action from a F# lambda I have a simple helper function, toAction1. And to create the actual lambda that takes a HostConfigurator we just create a function that has a single parameter let hostFactoryConfigurator hc, and now we can send this function to toAction1 and we have what we need to run the service. To run the actual configuration in the hostFactoryConfigurator we just left fold over the HostConfiguration list with the first HostConfiguration as initial state and apply the functions. When we've done that we can just create our ServiceControl from the final HostConfigurator, after left fold, and pass in the start and stop functions and we're done.

Improvements

The api can be improved a little bit, but now it works in a nice way. The major improvements that can be added is validation and create separate types for start and stop so we don't mix them. The validation wasn't in the API when I started implemented the change so I didn't add them now either and I don't think they are crucial either.

At NDC Oslo 2015 I had the opportunity to speak, and it was a fun experience. My talk, F# as our day job by 2016, was part of the functional track and it was a popular track. I had well over 100 people in the audience for my talk even though it was the second last talk of the conference, and it is really nice to see a growing interest in the functional programming world. I stayed most of the time in the functional track room, and here are some of my takeaways.

We are becoming more functional!

I think this is the third or fourth time there is a functional track at NDC, if you count NDC London as well, and I have never seen this much people for the tracks as I did this year. It started of great with a packed room for Bryan Hunter's "Lean and Functional Programming" talk, a talk I really enjoyed. Even though the language you use is not the biggest problem, but it is one way that you can improve yourself, and choosing functional will most likely help with that.

Another talk that I really like was Yan Cui's "A tour of the language landscape". Important parts here was that you need to put in the hours, and it is enough with 20 hours dedicated learning, to get of a good start. Of course 20 hours want make you a master, but after that you will most likely be able to start producing.

If you want to see a general talk about why should start doing functional I recommend Venkat Subramaniam's "Learning from Haskell". He explained to us things like "C++ is not strongly typed, it is sadly typed", and why that is a problem. If you have a good strongly typed functional language with great type inference it will most likely feel as you are in a dynamic language, and this is the nature of languages like Haskell and also F#.

If you are a C# developer and want is interested in F# I do recommend the talk by Phillip Trelford (or Sean's dad) where he showed "F# for C# Developers". I can also pitch my own talk "F# as our day job by 2016" where I try to show some arguments to why I like F# more than C# and also some facts to back that up. That might be a good resource if you want to start using F# in your project.

If you're more into functional design I do recommend Scott Wlaschin's "Enterprise Tic-Tac-Toc -- A functional approach", which was entertaining and a lot of useful information, as always when Scott presents). The "Type-Driven Development" by Mark Seeman i also a great talk for this topic.

What will I try after the conference

Even though I have done some F# I haven't tried idiomatic web development with F# and that is something I definitely will try out after I saw Tomas Petricek demoing suave on stage. He basically implemented two applications and also deployed one of the to azure and heroku in about 45 minutes. Of course he had some small things prepared for building, but he basically implemented the whole backend in that time.

The next thing I'll looking at is elixir and Phoenix. The creator of elixir, José Valim, was at the conference and had a great talk about idioms for building distributed applications in elixir. The main reasons that is possible is because of the erlang vm which you use to run elixir. Elixir supposed to have nicer syntax and a great metaprogramming model to extend the language for your domain. The metaprogramming model was something Chris McCord, the author of Phoenix, demonstrated in one of his great technical talks. The other talk he had was an introduction to Phoenix which I also recommend.

Don't wait for the community, be the community!

My talk was almost the last talk of the conference it was supposed to be a "call to action" talk for functional developers in Norway. Now is the time to start doing functional programming if you haven't done it before. Of course it might slow you down for a period of time, but thinking like that will not make you in the long run. Improvement doesn't happen over night, you have to practice it and you will see that the new knowledge you get will arm you with new ways of thinking about problem.

If you don't have a community were you live, don't let that stop you instead you should be the community!. There are ton of material online and I promise you that there are other people where you live that are interested in functional programming in general and specifically F#. If it is a F# community you want to grow you can always reach out to the people at the F# foundation or people on twitter and you will get help.

A couple of weeks ago I had a presentation about ASP.NET 5 and MVC 6 at NNUG Oslo. The presentation wasn't recorded so I thought I just write some blog posts about it insted. This will be a serie of posts where I plan to go through the features that I demonstrated during the presentation, plus some more features that I didn't have time to cover. I'll start with the basic and show one thing at a time and then add features as we go along. So let's get started.

Post in this serie:

• The pipeline
• Adding MVC to an application
• Setting up frontend build (with grunt)
• IoC and dependency injection
• View Components
• Self-hosting the application
• Hosting your application in docker

Source code: https://github.com/mastoj/OneManBlog

The next logical step is container hosting

Microsoft stepping in and partnering up with Docker was great news for the future if you ask me. Hosting applications is a container is a nice mix between light weight hosting and isolation between the application. There exist some security issues with hosting in a container that a someone could take advantage of if the container is running on a host that is not isolated enough, but don't let us go into that discussion.

The goal of this post is to show you one way to host your ASP.NET 5 application inside a docker container. I will not go into details about how you should put this container into production, mainly because I need to figure out a nice way to do it myself first.

Docker Machine and Docker

To manage different "hosting machines" I use Docker Machine, not because I have many machines but because it is easy to manage machines with Docker Machine. Installation is straightforward and the documentation is good so I want go into all details about Docker Machine, all we're going to do is create virtual machine that we can use to host our docker containers. To create a machine on VirtualBox you run the following command:

$ docker-machine create --driver virtualbox dev

Note that it is recommended to use msysgit when running Docker Machine, on Windows I recommend using ConEmu (install from Chocolatey) and run bash inside ConEmu to use Docker Machine and Docker. To target this machine and start using Docker against it you run the following command:

$ eval "$(docker-machine env dev)"

The last thing before we start defining our docker image is to find the IP of the current machine so we can test it later, execute the following and note down the IP (it is most likely 192.168.99.100):

$ docker-machine ip

Updating the project

Before we create the container image definition file we need to update the project so we can host it in a linux container. We need one more dependency in our project.json file and one more command so we can start the application:

"dependencies": {
    "Microsoft.AspNet.Mvc": "6.0.0-beta4",
    "Microsoft.AspNet.Server.IIS": "1.0.0-beta4",
    "Microsoft.AspNet.StaticFiles": "1.0.0-beta4",
    "Microsoft.AspNet.Server.WebListener": "1.0.0-beta4",
    "Microsoft.AspNet.Hosting": "1.0.0-beta4",
    "Kestrel": "1.0.0-beta4"
},
"commands": {
    "web": "Microsoft.AspNet.Hosting --server Microsoft.AspNet.Server.WebListener --server.urls http://localhost:5000",
    "kestrel": "Microsoft.AspNet.Hosting --server Kestrel --server.urls http://localhost:5001"
},

The new part is the Kestrel package is a web server that runs on ASP.NET 5 applications on linux. To start a web server with Kestrel we add the kestrel command which we can use with dnx on linux.

Creating the Dockerfile

A docker image is sort of starting point for a container. You define images with Dockerfiles, which you can base on other images so you get a hiearchy of images basically. A running instance of an image is what is called a container, so they are almost the same thing but one is passive and one is active. We are going to base our image on the official ASP.NET image, but we will add node, grunt and bower so we can build the application when we create the image. The Dockerfile we have define look like:

FROM microsoft/aspnet:vs-1.0.0-beta4

COPY . /app
WORKDIR /app

RUN apt-get update -y && apt-get install --no-install-recommends -y -q \
    curl \
    python \
    build-essential \
    git \
    ca-certificates

RUN mkdir /nodejs && \
    curl http://nodejs.org/dist/v0.10.33/node-v0.10.33-linux-x64.tar.gz | \
    tar xvzf - -C /nodejs --strip-components=1

ENV PATH $PATH:/nodejs/bin

RUN npm install -g grunt-cli bower

RUN ["dnu", "restore"]
RUN ["npm", "install", "."]
RUN ["grunt", "default"]

EXPOSE 5001

ENTRYPOINT ["dnx", "./", "kestrel"]

Let us go through this line by line (almost):

• COPY . /app, I have put the Dockerfile in the same folder as the project.json file this command copies all the code on the host into the /app folder in the image.
• WORKDIR /app, just sets the current working directory for when we execute the rest of the commands.
• The next two RUN commands and the ENV command installs and add nodejs to the environment in the image.
• When we have nodejs installs we can install grunt and bower using npm with the RUN command in the image.
• RUN ["dnu", "restore"] installs all the packages.
• RUN ["npm", "install", "."] installs all the javascript packages.
• RUN ["grunt", "default"] execute the grunt build steps that I wrote about here.
• In our command we specified that we will use port 5001 for kestrel, so we use EXPOSE 5001 to expose that port from the image when we run it.
• The last row, ENTRYPOINT ["dnx", "./", "kestrel"], specifies that when we run this image (making it a container) we will execute dnx and in the current folder passing it the kestrel command.

No when we have the Dockerfile ready all we need to do is to create the image:

$ docker build -t onemanblog .

and then start a container with:

$ docker run -i -t -p 5001:5001 onemanblog

The -i and -t flags make the container run in interactive mode, which means we see if it crashes and it also listens to stdin, that is, we can press enter to stop the application. The -p 5001:5001 redirects the port 5001 from host to the container.

Summary

If you have followed all the steps above you should now have a running container which you can go to http://<ip from "docker-machine ip">:5001/ and you should see the same application as we previously ran on Windows.

A couple of weeks ago I had a presentation about ASP.NET 5 and MVC 6 at NNUG Oslo. The presentation wasn't recorded so I thought I just write some blog posts about it insted. This will be a serie of posts where I plan to go through the features that I demonstrated during the presentation, plus some more features that I didn't have time to cover. I'll start with the basic and show one thing at a time and then add features as we go along. So let's get started.

Post in this serie:

• The pipeline
• Adding MVC to an application
• Setting up frontend build (with grunt)
• IoC and dependency injection
• View Components
• Self-hosting the application
• Hosting your application in docker

Source code: https://github.com/mastoj/OneManBlog

Setting up self-hosting

In ASP.NET 5 you are more in charge of how you host the application. So far we've been using IIS (or IIS Express), but I thought I would show how easy it is to add self-hosting to the application. Self-hosting is great if you want to run it as a service instead of in IIS.

Updating project.json

The first step is to add two more packages we need to add:

• Microsoft.AspNet.Hosting - the hosting infrastructure
• Microsoft.AspNet.Server.WebListener - the self-hosted web server implementation for Windows

The dependencies node in the project.json file should now look like:

"dependencies": {
    "Microsoft.AspNet.Mvc": "6.0.0-beta4",
    "Microsoft.AspNet.Server.IIS": "1.0.0-beta4",
    "Microsoft.AspNet.StaticFiles": "1.0.0-beta4",
    "Microsoft.AspNet.Server.WebListener": "1.0.0-beta4",
    "Microsoft.AspNet.Hosting": "1.0.0-beta4"
},

The last thing we need to do is add a "command" to the commands node:

"commands": {
    "web": "Microsoft.AspNet.Hosting --server Microsoft.AspNet.Server.WebListener --server.urls http://localhost:5000"
},

After you added that you will get a new menu item under the debug button called web:

Choosing that menu item and start the application will start the self-hosted version instead of IIS Express.

DNVM, DNX and DNU

If you want to run this from the command line you first install DNVM. To install it you can follow the instructions here: https://github.com/aspnet/home#powershell. You use DNVM to manage which version of DNX and DNU you want to use. After DNVM you can run a command like:

dnvm use 1.0.0-beta4 -r coreclr -arch x64

This will select version 1.0.0-beta4 and the core runtime. When we have selected the version you can now use DNX to run the application from the command line with the following command:

dnx . web

Note that the command above only works is you are in the same folder as the project.json file and that web must match the name of the command in the project.json file. If you need to restore the packages you can run DNU before DNX:

dnu restore

A couple of weeks ago I had a presentation about ASP.NET 5 and MVC 6 at NNUG Oslo. The presentation wasn't recorded so I thought I just write some blog posts about it insted. This will be a serie of posts where I plan to go through the features that I demonstrated during the presentation, plus some more features that I didn't have time to cover. I'll start with the basic and show one thing at a time and then add features as we go along. So let's get started.

Post in this serie:

• The pipeline
• Adding MVC to an application
• Setting up frontend build (with grunt)
• IoC and dependency injection
• View Components
• Self-hosting the application
• Hosting your application in docker

Source code: https://github.com/mastoj/OneManBlog

What is view components?

Partial views in MVC has one limitation, which is that they doesn't have a controller for them. Not having controller makes it hard to have more complex logic associated with the view. View components changes that. Each view component consist of a view and a backing class, it's not a controller but almost. In this post we will take the post list which we created by injecting the "repository" into the index view in the last post and create a reuseable view component instead which we can use in multiple views.

Creating the view component

The purpose of this view component is to list all the posts for this simple blog, making it much easier to reuse in multiple views. An alternative would have been to use a partial view, but using a partial view would have forced us to get the data for the partial view in the controller for each of the views where we used the partial, but since a view component has a backing class that is not necessary.

The ViewComponent class

Create a folder called ViewComponents where you can have all your view components and add a PostListViewComponent class in the folder. The implementation of the class should look like this:

public class PostListViewComponent : ViewComponent
{
    private readonly Data _data;

    public PostListViewComponent(Data data)
    {
        _data = data;
    }

    public IViewComponentResult Invoke(string title)
    {
        ViewBag.Title = title;
        return View(_data.GetPosts());
    }
}

We are inheriting from the class ViewComponent to get some extra help, like the View method in the base class. The Invoke method is what will be called from the views that adds this view component and it is returning a simple view that show the posts. We are also injecting the Data class in the constructor which makes it possible to get the data we need in this class instead of it being sent to it like you do with a partial view. The view we return has a ViewBag as a regular view which we take advantage of, and the Invoke method can also have arguments which you can pass to the view component when you use it.

The View

The view is really straightforward:

@model IEnumerable<OneManBlog.Model.PostModel>
<div>
    <h3>@ViewBag.Title</h3>
    <ul>
        @foreach (var item in Model)
            {
            <li>@Html.ActionLink(item.Slug, "Index", "Post", new { slug = item.Slug })</li>
        }
    </ul>
</div>

I want get into any detail what this does, but the hard part is where to put it. To get this to work it must be in the folder Views\Shared\Components\PostList\ and the file must be called Default.cshtml. The important part is from Components and down, the first part just scope where it can be used. Since I want this to be used from all over the site I put it in Shared.

With these two files added my solution look like this:

Using the view component

To use this in any view all I have to do is add this line:

@Component.Invoke("PostList", "Blog posts")        

So my new Home\Index.cshtml container part now looks like this:

<div class="container">
    <h1>This is my new blog</h1>
    Hello from MVC NNUG! asdad sasdasdasd adasd asd sadsa

    @using (Html.BeginForm("Create", "Post", FormMethod.Post))
    {
        <div class="form-group">
            <label for="slug">Slug</label>
            <input type="text" name="slug" id="name" value="" class="form-control" />
        </div>
        <div class="form-group">
            <label for="content">Content</label>
            <textarea id="content" name="content" class="form-control"></textarea>
        </div>
        <input type="submit" value="Save" class="btn btn-default" />
    }
    @Component.Invoke("PostList", "Blog posts")        
</div>

and my new Post\Index.cshtml looks like this:

<div class="container">
    <h1>@Model.Slug</h1>
    <div>@Model.Content</div>
    @Component.Invoke("PostList", "Other blog posts")
</div>

That's all there is to it.

Summary

This is something I really missed in regular partial views and something that I think will be really useful creating more simple components that is reusable across a site. I look forward to try out this feature in a larger project and so how well it fits in, but so far I think it is really promising.