Use Case #3 - Publishing a new blog post

Publishing a new blog post use case involves the following

• A user can publish a new blog post by providing a title, body of the post and a set of tags.

• If the title already exists we need to let the user know about it.

• We also need to let him know if publish went well.

Though the requirement looks simple on paper, there are some complexities regarding organizing the code and orchestrating the entire operation.

Let’s dig up and see how we can solve it!

The Database Schema

As a first step, let’s add some new tables to our existing schema which has only the users table.

The first one is the posts table

1
2
3
4
5
6
7

CREATE TABLE posts(
  id SERIAL PRIMARY KEY,
  title VARCHAR(50) UNIQUE NOT NULL,
  body TEXT NOT NULL,
  published_at TIMESTAMP NOT NULL,
  author_id INTEGER NOT NULL REFERENCES users(id)
);

Then we need to have an another table for tags

1
2
3
4

CREATE TABLE tags(
  id SERIAL PRIMARY KEY,
  name VARCHAR(50) NOT NULL
);

And finally, a bridge table to associate posts and tags

1
2
3
4

CREATE TABLE posts_tags(
  tag_id INTEGER REFERENCES tags(id),
  post_id INTEGER REFERENCES posts(id)
);

Note: a blog post can have multiple tags, and a tag can have multiple blog posts associated with it

Model Definitions - Post and Tag

The next step is defining equivalent models for Post and Tag in golang.

Let’s create a new folder called tag and define the Tag model.

1
2
3
4
5
6
7

// tag/model.go
package tag

type Tag struct {
  ID   uint
  Name string
}

Then define the Post model in an another new folder post

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

// post/model.go
package post

import (
  "time"

  "github.com/tamizhvendan/gomidway/tag"
)

const (
  UniqueConstraintTitle = "posts_title_key"
)

type Post struct {
  ID          uint
  Title       string
  Body        string
  AuthorID    uint
  Tags        []tag.Tag `gorm:"many2many:posts_tags;"`
  PublishedAt time.Time
}

type TitleDuplicateError struct{}

func (e *TitleDuplicateError) Error() string {
  return "title already exists"
}

As we have seen in the previous blog post, the constant UniqueConstraintTitle and the TitleDuplicateError are required for doing unique constraint violation error check on the title column and to communicate it to the application respectively.

The important thing to notice in this model definition is the Tags field

1
2
3
4
5

type Post struct {
  // ...
  Tags        []tag.Tag `gorm:"many2many:posts_tags;"`
  // ..
}

The Tags field has a golang struct tag gorm defining the association type many2many and the name of the bridge table posts_tags.

Implementing new blog post use case

Now we have all the models required to enable publishing a new blog post, and it’s time to have to go at its implementation.

Let’s start by creating a new folder publish under post and add the scaffolding for the handler

As we have seen earlier, the folder structure represent the use case, and the handler orchestrate the use case

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

// post/publish/handler.go
package publish

import (
  "github.com/jinzhu/gorm"
)

type Request struct {
  Title    string
  Body     string
  AuthorID uint
  Tags     []string
}

type Response struct {
  PostId uint
}

func NewPost(db *gorm.DB, req *Request) (*Response, error) {
  // TODO
}

The implementation of publishing a new post involves the following steps

1. For all the Tags that are part of the request, we need to create an entry in the tags table. If it is already there, we don’t need to create.

2. The next step is creating a new entry in the posts table with the given details.

3. Finally, we need to associate the Tags with the newly created Post via the bridge table.

Since it involves multiple inserts on the database side, all the three steps should happen inside a transaction.

The first two steps can be executed in any order as they are independent of each other

Code Organization (aka Responsibility Separation)

We discussed a little bit about code organization in the last blog post. One important thing which can help us, in the long run is having proper separation of concern in the code base.

There are multiple ways we can separate the concern. In our case, we are organizing by use cases with handler driving the implementation. The handler may access the data layer if the use case requires.

To keep things simple, we are not discussing dependency injection in handler and data layer interaction here. I am planning to cover this in my future blog posts.

Back to our business, the data access logic of the three steps will be in their respective packages and the publish handler coordinate the entire use case logic.

Step 1: Create a Tag if not exists

The first step is creating a tag if it is not there in the database. For the both new tags and the existing tags we need to get its id from the database to associate it with the posts.

Gorm has a method called FirstOrCreate to help us to implement this step.

1
2
3
4
5
6
7
8
9
10
11
12
13

// tag/create.go
package tag

import "github.com/jinzhu/gorm"

func CreateIfNotExists(db *gorm.DB, tagName string) (*Tag, error) {
  var tag Tag
  res := db.FirstOrCreate(&tag, Tag{Name: tagName})
  if res.Error != nil {
    return nil, res.Error
  }
  return &tag, nil
}

The FirstOrCreate method populates the Id field of the tag.

Step 2: Creating a new Post

It is similar to creating a new user that we saw in the last blog post

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

// post/create.go
package post

import (
  "github.com/jinzhu/gorm"
  "github.com/tamizhvendan/gomidway/postgres"
)

func Create(db *gorm.DB, post *Post) (uint, error) {
  res := db.Create(post)
  if res.Error != nil {
    if postgres.IsUniqueConstraintError(res.Error, UniqueConstraintTitle) {
      return 0, &TitleDuplicateError{}
    }
    return 0, res.Error
  }
  return post.ID, nil
}

Step 3: Associating Tag with Post

The final step is associating the tag with the post in the database. Gorm has a decent support for Associations. The one that we needed from gorm to carry out the current step is its Append method.

Let’s define a constant in Post model which holds the AssociationTag

1
2
3
4
5
6
7

// post/model.go
// ...
const (
  // ...
  AssociationTags = "Tags"
)
// ...

Then add a new file tag.go in the post folder and implement the third step as below

1
2
3
4
5
6
7
8
9
10
11
12

// post/tag.go
package post

import (
	"github.com/jinzhu/gorm"
	"github.com/tamizhvendan/gomidway/tag"
)

func AddTag(db *gorm.DB, post *Post, tag *tag.Tag) error {
  res := db.Model(post).Association(AssociationTags).Append(tag)
  return res.Error
}

Publishing New Blog Post

Now we have all the individual database layer functions ready for all the three steps, and it’s time to focus on the implementation of publishing a new blog post.

We already have the scaffolding in place

1
2
3
4
5

// publish/handler.go
// ...
func NewPost(db *gorm.DB, req *Request) (*Response, error) {
  // TODO
}

As a first step, let’s begin a new transaction using gorm’s Begin method.

1
2
3
4
5
6
7

func NewPost(db *gorm.DB, req *Request) (*Response, error) {
  tx := db.Begin()
  if tx.Error != nil {
    return nil, tx.Error
  }
  // TODO
}

Then call the Create function in post package to create a new blog post

1
2
3
4
5
6
7
8
9
10
11
12
13
14

func NewPost(db *gorm.DB, req *Request) (*Response, error) {
  // ...
  newPost := &post.Post{
    AuthorID:    req.AuthorId,
    Title:       req.Title,
    Body:        req.Body,
    PublishedAt: time.Now().UTC(),
  }
  _, err := post.Create(tx, newPost)
  if err != nil {
    return nil, err
  }
  // TODO
}

Then for all the tags in the request, call the CreateIfNotExists function in tag package to get its respective Ids and associate it with the newly created post using the AddTag function in the post package.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

func NewPost(db *gorm.DB, req *Request) (*Response, error) {
  // ...
  for _, tagName := range req.Tags {
    t, err := tag.CreateIfNotExists(tx, tagName)
    if err != nil {
      tx.Rollback()
      return nil, err
    }
    err = post.AddTag(tx, newPost, t)
    if err != nil {
      tx.Rollback()
      return nil, err
    }
  }
  // TODO
}

A thing to note here is we are rolling back the transaction using the RollBack method in case of error.

The final step is committing the transaction and returning the newly created post id as response

1
2
3
4
5
6
7
8

func NewPost(db *gorm.DB, req *Request) (*Response, error) {
  // ...
  res := tx.Commit()
  if res.Error != nil {
    return nil, res.Error
  }
  return &Response{PostId: newPost.ID}, nil
}

Test Driving Publish New Blog Post

Let’s test drive our implementation from the main function with some hard coded value

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

// main.go
package main

import (
  // ...
  "github.com/tamizhvendan/gomidway/post"
  "github.com/tamizhvendan/gomidway/post/publish"
)
// ...
func main() {
  // ...
  publishPost(db)
}


func publishPost(db *gorm.DB) {
  res, err := publish.NewPost(db, &publish.Request{
    AuthorId: 1,
    Body:     "Golang rocks!",
    Title:    "My first gomidway post",
    Tags:     []string{"intro", "golang"},
  })
  if err != nil {
    if _, ok := err.(*post.TitleDuplicateError); ok {
      fmt.Println("Bad Request: ", err.Error())
      return
    }
    fmt.Println("Internal Server Error: ", err.Error())
    return
  }
  fmt.Println("Created: ", res.PostId)
}

if we run the program with these hard coded values, we will get the following output

1

Created: 1

if we rerun the program without changing anything, we will get the bad request error as expected

1

Bad Request: title already exists

Summary

In this blog post, we have seen how to perform, create operation of a model having many to many relationship using gorm. The source code is available in my GitHub repository.

Gorm does an excellent job as an ORM library, and we enjoyed using it in our projects.

Through this blog post series, I will be sharing our experiences on how we leveraged gorm to solve our client’s needs.

The Domain

In this blog post series, we will be implementing the data persistence side of a blogging platform similar to medium, called gomidway.

Part-1 Introduction

In this blog post, we will be working on defining the backend for signing up a new user and providing a provision for user login.

The User Model

Let’s start our modeling from the User table.

1
2
3
4
5
6

CREATE TABLE users(
  id SERIAL PRIMARY KEY,
  username VARCHAR(50) UNIQUE NOT NULL,
  email VARCHAR(255) UNIQUE NOT NULL,
  password_hash TEXT NOT NULL
);

Then define an equivalent model in golang.

1
2
3
4
5
6

type User struct {
  ID           uint `gorm:"primary_key"`
  Username     string
  Email        string
  PasswordHash string
}

Before taking the next steps, let me spend some time on explaining where to put this User struct

The Folder Structure

There are two common ways to organize the models in golang.

One approach is defining a folder called models and put all the models here

The another approach is an invert of this structure. In this design, we will have a separate folder for each model.

Both the approaches have pros and cons. Choosing one over the other is entirely opinionated, and my preference is the second one.

IMHO, the folder structure has to represent the domain and the code associated a domain model should coexist with proper separation of concern.

Software architectures are structures that support the use cases of the system - Ivar Jacobson

In this gorm blog post series, I will be following the domain based folder structure.

Use Case #1 - User Signup

The Signup use case of a user is defined as

• A user should sign up himself by providing his email, username, and password
• If the username or the email already exists, we need to let him now
• We also need to let him know if there is any error while persisting his signup details
• If signup succeeds, he should be getting a unique identifier in the system

To keep things simple and the focus of this series is on the data persistence side, we are not going to discuss/implement the HTTP portion of the application. Instead, we will be driving our implementation with some hard code values during the application bootstrap.

Defining the Signup handler

Like many terms in software engineering, the term handler has different meanings. So, let me start by explaining what I mean by a handler here.

A handler is a function that represents an use-case. It takes its dependency(ies) and its input(s) as parameters and returns the outcome of the use case.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

// user/signup/handler.go
package signup

type Request struct {
  Username string
  Email    string
  Password string
}

type Response struct {
  Id uint
}

func Signup(db *gorm.DB, req *Request) (*Response, error) {
  // ...
}

One important thing to notice here is, the Request and the Response are golang structs. How the request is being populated from user’s request (JSON Post / HTML form Post / command from a message queue) and how the response communicated to the user (JSON response / HTML view / event in the message queue) are left up to the application boundary.

In the Signup function, as a first step, we need to create the hash for the password using bcrypt and then create the new user.

1
2
3
4
5
6
7
8
9
10
11
12

func Signup(db *gorm.DB, req *Request) (*Response, error) {
  passwordHash, err := bcrypt.GenerateFromPassword([]byte(req.Password), bcrypt.DefaultCost)
  if err != nil {
    return nil, err
  }
  newUser := &user.User{
    Username:     req.Username,
    Email:        req.Email,
    PasswordHash: string(passwordHash),
  }
  // ????
}

The next step is persisting this newly created user

Adding User Create function

The create user function is straight forward. We just need to call the Create method in gorm

1
2
3
4
5
6
7
8
9
10

// user/create.go
package user

func Create(db *gorm.DB, user *User) (uint, error) {
  err := db.Create(user).Error
  if err != nil {
    return 0, err
  }
  return user.ID, nil
}

But the work is not done yet!

As per our use case, We need to let the handler to know if the username or the email already exists.

We already have unique constraints in place in the users table.

1
2
3
4
5
6
7
8
9
10
11
12

gomidway=# \d users
                                    Table "public.users"
    Column     |          Type          |                     Modifiers
---------------+------------------------+----------------------------------------------------
 id            | integer                | not null default nextval('users_id_seq'::regclass)
 username      | character varying(50)  | not null
 email         | character varying(255) | not null
 password_hash | text                   | not null
Indexes:
    "users_pkey" PRIMARY KEY, btree (id)
    "users_email_key" UNIQUE CONSTRAINT, btree (email)
    "users_username_key" UNIQUE CONSTRAINT, btree (username)

So, the Create method in gorm would return an error of type pq Error with the ErrorCode as "23505" for unique_violation, and the Constraint field will be having the unique constraint key name users_email_key and users_username_key for email and username duplicate error respectively.

Though this error does communicate what we wanted, it is very generic and what we want is something concrete to our use case.

To make it happen, let’s create a new folder postgres (aka package) and write a utility function IsUniqueConstraintError which checks whether the given error is a unique constraint error or not.

1
2
3
4
5
6
7

// postgres/pq.go
func IsUniqueConstraintError(err error, constraintName string) bool {
  if pqErr, ok := err.(*pq.Error); ok {
    return pqErr.Code == "23505" && pqErr.Constraint == constraintName
  }
  return false
}

and then in the model.go, where we have the User model, add the constraint names as constants.

1
2
3
4
5
6
7

// user/model.go
// ...
const (
  UniqueConstraintUsername = "users_username_key"
  UniqueConstraintEmail    = "users_email_key"
)
// ...

Finally, define the custom error types

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

// user/model.go
// ...
type UsernameDuplicateError struct {
  Username string
}

func (e *UsernameDuplicateError) Error() string {
  return fmt.Sprintf("Username '%s' already exists", e.Username)
}

type EmailDuplicateError struct {
  Email string
}

func (e *EmailDuplicateError) Error() string {
  return fmt.Sprintf("Email '%s' already exists", e.Email)
}

With this new helper function, constants and types in places, we can complete the create user function as follows.

1
2
3
4
5
6
7
8
9
10
11
12
13
14

// user/create.go
func Create(db *gorm.DB, user *User) (uint, error) {
  err := db.Create(user).Error
  if err != nil {
    if postgres.IsUniqueConstraintError(err, UniqueConstraintUsername) {
      return 0, &UsernameDuplicateError{Username: user.Username}
    }
    if postgres.IsUniqueConstraintError(err, UniqueConstraintEmail) {
      return 0, &EmailDuplicateError{Email: user.Email}
    }
    return 0, err
  }
  return user.ID, nil
}

On the handler side, we just pass the outcome of this create function to the outside(application boundary) layer

1
2
3
4
5
6
7
8
9
10

// user/signup/handler.go
// ...
func Signup(db *gorm.DB, req *Request) (*Response, error) {
  // ...
  id, err := user.Create(db, newUser)
  if err != nil {
    return nil, err
  }
  return &Response{Id: id}, err
}

Test Driving User Signup

As we discussed earlier, let’s test drive the implementation from the application bootstrap.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

// main.go
func panicOnError(err error) {
  if err != nil {
    panic(err)
  }
}

func main() {
  db, err := gorm.Open("postgres",
    `host=localhost 
      user=postgres password=test
      dbname=gomidway 
      sslmode=disable`)
  panicOnError(err)
  defer db.Close()

  signupUser(db)
}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

// main.go
// ...
func signupUser(db *gorm.DB) {
  res, err := signup.Signup(db, &signup.Request{
    Email:    "foo@bar.com",
    Username: "foo",
    Password: "foobar",
  })
  if err != nil {
    switch err.(type) {
    case *user.UsernameDuplicateError:
      fmt.Println("Bad Request: ", err.Error())
      return
    case *user.EmailDuplicateError:
      fmt.Println("Bad Request: ", err.Error())
      return
    default:
      fmt.Println("Internal Server Error: ", err.Error())
      return
    }
  }
  fmt.Println("Created: ", res.Id)
}

if we run the program for the first time after setting up the PostgreSQL database gomidway with the users table, we will get the following output

1

Created:  1

if we rerun the program again, we’ll see the bad request error

1

Bad Request:  Username 'foo' already exists

if we modify the username from foo to bar and run the program, we’ll again get a bad request error for the email address

1

Bad Request:  Email 'foo@bar.com' already exists

That’s it! We have completed the first use case of gomidway!!

Use Case #2 - User Login

The next use case that we are going to implement is user login. The requirement for login has been defined as

• if the user logs in with a different email address or with a different password, we need to show him appropriate errors

• if the email and the password matches, let the user know that he has logged in

Defining the Login handler

As we did for the signup, let’s start our implementation from defining the handler for login

1
2
3
4
5
6
7
8
9
10
11
12
13

// user/login/handler.go
type Request struct {
  Email    string
  Password string
}

type Response struct {
  User *user.User
}

func Login(db *gorm.DB, req *Request) (*Response, error) {
  // ...
}

To implement login, we need some help from the persistence. In other words, we have to find whether the user with the given email address exists in our application.

Adding User FindByEmail function

Let’s create a new file find.go in the user folder and define the FindByEmail function

1
2
3
4
5
6

// user/find.go
func FindByEmail(db *gorm.DB, email string) (*User, error) {
  var user User
  res := db.Find(&user, &User{Email: email})
  return &user, res.Error
}

That’s great. But how are we going to find if the email didn’t exist in the first place?

Thankfully, We don’t need to anything extra other than calling the RecordNotFound to figure this out!

Let’s define a custom error type EmailNotExistsError and return it if no records found.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

// user/find.go
type EmailNotExistsError struct{}

func (*EmailNotExistsError) Error() string {
  return "email not exists"
}

func FindByEmail(db *gorm.DB, email string) (*User, error) {
  var user User
  res := db.Find(&user, &User{Email: email})
  if res.RecordNotFound() {
    return nil, &EmailNotExistsError{}
  }
  return &user, nil
}

Now its time to turn our attention to Login handler to wire up the login functionality.

1
2
3
4
5
6
7
8
9

// user/login/handler.go
// ...
func Login(db *gorm.DB, req *Request) (*Response, error) {
  user, err := user.FindByEmail(db, req.Email)
  if err != nil {
    return nil, err
  }
  // ...
}

The next scenario that we need to handle is, compare the password in the request with the password hash. As we need to let the user know in case of password mismatch, let’s create a PasswordMismatchError in the login handler and return it during the mismatch.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

// user/login/handler.go
// ...
type PasswordMismatchError struct{}
func (e *PasswordMismatchError) Error() string {
  return "password didn't match"
}

func Login(db *gorm.DB, req *Request) (*Response, error) {
  // ...
  err = bcrypt.CompareHashAndPassword([]byte(user.PasswordHash), []byte(req.Password))
  if err != nil {
    return nil, &PasswordMismatchError{}
  }
  return &Response{User: user}, nil
}

With this, we are done with login handler implementation. Let’s test drive it!

Test Driving User Login

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

// main.go
// ...
func main() {
  // ...
  loginUser(db)
}
// ...
func loginUser(db *gorm.DB) {
  res, err := login.Login(db, &login.Request{
    Email: "foo@bar.com",
    Password: "foobar"})
  if err != nil {
    switch err.(type) {
    case *user.EmailNotExistsError:
      fmt.Println("Bad Request: ", err.Error())
      return
    case *login.PasswordMismatchError:
      fmt.Println("Bad Request: ", err.Error())
      return
    default:
      fmt.Println("Internal Server Error: ", err.Error())
      return
    }
  }
  fmt.Printf("Ok: User '%s' logged in", res.User.Username)
}

Summary

In this blog post, we have seen how we can use the create and find function in gorm along with the use case driven approach. The source code can be found in my GitHub repository.

Some Context

In my current project, we are using Postgres for persisting the application data. To make our life easier, we are using gorm to talk to Postgres from our golang code. Things were going well and we started rolling out new features without any challenges. One beautiful day, we came across an interesting requirement which gave us a run for the money.

The requirement is to store and retrieve an array of strings from Postgres!

It sounds simple on paper but while implementing it we found that it is not straightforward. Let me explain what the challenge was and how we solved it through a Task list example

The Database Side

Let’s assume that we have database mytasks with a table tasks to keep track of the tasks.

The tasks table has the following schema

1
2
3
4
5
6

CREATE TABLE tasks (
  id SERIAL PRIMARY KEY,
  name TEXT NOT NULL,
  is_completed BOOL NOT NULL,
  tags VARCHAR(10)[]
)

An important thing to note over here is that each task has an array of tags of type varchar(10).

The Golang Side

The equivalent model definition of the tasks table would look like the following in Golang

1
2
3
4
5
6

type Task struct {
  Id          uint
  Name        string
  IsCompleted bool
  Tags        []string
}

The Challenge

Everything is set to test drive the task creation.

Let’s see what happens when we try to create a new task!

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

package main
import (
  "fmt"

  "github.com/jinzhu/gorm"
  _ "github.com/jinzhu/gorm/dialects/postgres"
)

func panicOnError(err error) {
  if err != nil {
    panic(err)
  }
}

func CreateTask(db *gorm.DB, name string, tags []string) (uint, error) {
  newTask := &Task{Name: name, Tags: tags}
  result := db.Create(newTask)
  if result.Error != nil {
    return 0, result.Error
  }
  return newTask.Id, nil
}

func main() {
  db, err := gorm.Open("postgres",
    `host=localhost 
      user=postgres password=test
      dbname=mytasks 
      sslmode=disable`)
  panicOnError(err)
  defer db.Close()

  id, err := CreateTask(db, "test 123", []string{"personal", "test"})
  panicOnError(err)
  fmt.Printf("Task %d has been created\n", id)
}

When we run this program, it will panic with the following error message

1

panic: sql: converting Exec argument $3 type: unsupported type []string, a slice of string

As the error message says, the SQL driver doesn’t support []string. From the documentation, we can found that the SQL drivers only support the following values

1
2
3
4
5
6

int64
float64
bool
[]byte
string
time.Time

So, we can’t persist the task with the tags using this approach.

Golang’s interface in Action

As a first step towards the solution, let’s see how the plain SQL insert query provides the value for arrays in Postgres

1

INSERT INTO tasks(name,is_completed,tags) VALUES('buy milk',false,'{"home","delegate"}');

The clue here is the plain SQL expects the value for array as a string with the following format

1

'{ val1 delim val2 delim ... }'

double quotes around element values if they are empty strings, contain curly braces, delimiter characters, double quotes, backslashes, or white space, or match the word NULL. Double quotes and backslashes embedded in element values will be backslash-escaped. - Postgres Documentation

That’s great! All we need to do is convert the []string to string which follows the format specified above.

An easier approach would be changing the Tags field of the Task struct to string and do this conversion somewhere in the application code before persisting the task.

But it’s not a cleaner approach as the resulting code is not semantically correct!

Golang provides a neat solution to this problem through the Valuer interface

Types implementing Valuer interface are able to convert themselves to a driver Value.

That is we need to have a type representing the []string type and implement this interface to do the type conversion.

Like we did in the part-1 of this series, let’s make use of named types by creating a new type called StringSlice

1

type StringSlice []string

Then we need to do the type conversion in the Value method

1
2
3
4
5
6
7
8

func (stringSlice StringSlice) Value() (driver.Value, error) {
  var quotedStrings []string
  for _, str := range stringSlice {
    quotedStrings = append(quotedStrings, strconv.Quote(str))
  }
  value := fmt.Sprintf("{ %s }", strings.Join(quotedStrings, ","))
  return value, nil
}

Great!

With this new type in place, we can change the datatype of Tags field from []string to StringSlice in the Task struct.

If we rerun the program, it will work as expected!!

1

Task 1 has been created

Filter by tag

Let’s move to the query side of the problem.

We would like to get a list of tasks associated with a particular tag.

It’d be a straightforward function that uses the find method in gorm.

1
2
3
4
5
6
7
8

func GetTasksByTag(db *gorm.DB, tag string) ([]Task, error) {
  tasks := []Task{}
  result := db.Find(&tasks, "? = any(tags)", tag)
  if result.Error != nil {
    return nil, result.Error
  }
  return tasks, nil
}

Then we need to call it from our main function

1
2
3
4
5
6
7

// ...
func main() {
  // ...
  tasks, err := GetTasksByTag(db, "project-x")
  panicOnError(err)
  fmt.Println(tasks)
}

Unfortunately, if we run the program, it will panic with the following error message

1
2
3

panic: sql: Scan error on column index 3: unsupported Scan, storing driver.Value type []uint8 into
type *main.StringSlice; sql: Scan error on column index 3: unsupported Scan,
storing driver.Value type []uint8 into type *main.StringSlice

As the error message says, the SQL driver unable to scan (unmarshal) the data type byte slice ([]uint8) into our custom type StringSlice.

To fix this, we need to provide a mechanism to convert []uint8 to StringSlice which in turn will be used by the SQL driver while scanning.

Like the Valuer interface, Golang provides Scanner interface to do the data type conversion while scanning.

The signature of the Scanner interface returns an error and not the converted value.

1
2
3

type Scanner interface {
  Scan(src interface{}) error
}

So, it implies the implementor of this interface should have a pointer receiver (*StringSlice) which will mutate its value upon successful conversion.

1
2
3

func (stringSlice *StringSlice) Scan(src interface{}) error {
  // ...
}

In the implementation of this interface, we just need to convert the byte slice into a string slice by converting it to a string (Postgres representation of array value) first, and then to StringSlice

1

[]uint8 --> {home,delegate} --> []string{"home", "delegate"}

After successful conversion, we need to assign the converted value to the receiver (*stringSlice)

1
2
3
4
5
6
7
8
9
10
11
12

func (stringSlice *StringSlice) Scan(src interface{}) error {
  val, ok := src.([]byte)
  if !ok {
    return fmt.Errorf("unable to scan")
  }
  value := strings.TrimPrefix(string(val), "{")
  value = strings.TrimSuffix(value, "}")

  *stringSlice = strings.Split(value, ",")

  return nil
}

That’s it. If we run the program now, we can see the output as expected.

1
2

[{2 schedule meeting with the team false [project-x]}
  {3 prepare for client demo false [slides project-x]}]

Summary

In this blog post, we have seen how we can make use of Valuer and Scanner interfaces in golang to marshal and unmarshal our custom data type from the database.

The source code can be found in my GitHub repository

As a follow-up of this comment, I would like to share how we are using (my current project is also in golang!) the interfaces to keep our code clean and consistent through a series of three blog posts

This blog post series assumes that you are familiar with the basics of interfaces in golang. If would like to know what it brings to the table, I strongly recommend to check out this well-written article by Yan Cui.

Let me start the series with a solution that we have implemented a few days back.

Some Context

The product that we are building consists of a suite of web applications. To authenticate the users, these web applications talk to a centralized web application called “Identity Server”.

Upon receiving valid login credentials, the Identity Server generates a JSON Web Token(JWT) with the corresponding user claims and signs it using a public/private key pair using RSA.

The downstream web applications will then use this JWT to grant the access to their corresponding protected resources.

Let’s assume a simple JWT claims payload

1
2
3
4
5

{
  "sub": "1234567890",
  "name": "John D",
  "admin": true
}

The above payload uses one of the JWT standard claims, sub, to communicate the unique identifier of the user in the product for all the web applications. The name and admin are the custom claims.

As per the JWT spec, the sub claim is a case-sensitive string containing a StringOrURI value which is locally unique in the context of the issuer or be globally unique, but in our system, the unique identifier of a user is an unsigned integer(uint) type.

As we will be sharing the JWT with other third party applications, we made a call to stick to the JWT spec and converted the uint to string type while generating the token and did the reverse while authenticating the user using this token.

Update - After writing this blog post, I came to know from the comment that the use case of this blog post, unmarshalling a JSON string to uint type can be done by adding string to the json tag. Being said that, if you’d like to know about how to use an interface to solve it, the rest of the post would help.

Unmarshalling JWT - A Naive Approach

Before witnessing the golang interface in action, let’s see a naive implementation how we can unmarshal the claim and use.

The straightforward thing would be creating a struct matching the properties of the claim

1
2
3
4
5

type UserJwt struct {
  Sub    string
  Name   string
  Admin  bool
}

and unmarshalling using the json package in golang

1
2
3
4
5
6
7
8
9

claims := `
{
  "sub": "1234567890",
  "name": "John D",
  "admin": true
}`

var userJwt *UserJwt
err := json.Unmarshal([]byte(claims), &userJwt)

To convert the sub from string to uint, we can have a method Id on the UserJwt type.

1
2
3
4
5
6
7

func (u *UserJwt) Id() (uint, error) {
  v, err := strconv.ParseUint(u.Sub, 10, strconv.IntSize)
  if err != nil {
    return 0, err
  }
  return uint(v), nil
}

and use it after successful unmarshalling of the JWT claims.

1
2
3
4
5
6
7
8
9
10
11

// ...
err := json.Unmarshal([]byte(claims), &userJwt)
if err != nil {
  return err
}
id, err := userJwt.Id()
if err != nil {
  return err
}
// Do something with the id of type uint
fmt.Println(id)

That’s great. But did you see any code smell here?

Let me share what’s wrong with this approach,

Let’s say that we have the following JSON claim with sub has the value user/john instead of a string representing the unsigned integer identifier of the user.

1
2
3
4
5
6

claims := `
{
  "sub": "user/john",
  "name": "John D",
  "admin": true
}`

Unmarshalling this claim will work, and it won’t return any error

1
2
3
4
5

// ...
err := json.Unmarshal([]byte(claims), &userJwt)
if err != nil {
  return err
}

We can share the unmarshalled userJwt with the rest of the code to carry the business logic.

We will come to know that the claim has an invalid sub value only when we try to get the id of the user by calling the Id method

1
2
3
4

// ...
id, err := userJwt.Id()
// This will return the following error
// strconv.ParseUint: parsing "name/john": invalid syntax

If we didn’t call the Id method, this subtle bug slip silently into the product and someday will end up as a production issue!

In a nutshell, this approach is not robust, and the resulting code is not clean.

Using Interfaces - A better approach

Ideally, we want the json.Unmarshal function should return an error if the sub doesn’t contain a uint value as string.

To make it happen, we need to inform the json.Unmarshal function somehow to do the type conversion while unmarshalling and return an error if the conversion fails.

How to make it happen?

We can do this by using the Unmarshaler interface.

In our case, we can declare the UnmarshalJSON method with the UserJwt type and in the definition, we can do the type conversion. But that’d be an overkill as we need to do the unmarshalling of the other fields, Name, and Admin, which is already working well without any custom logic.

In other words, the effective way would be overriding the JSON unmarshalling behavior of Sub field alone by having the UnmarshalJSON method with uint type. But according to golang’s spec we can’t do it

You can only declare a method with a receiver whose type is defined in the same package as the method. You cannot declare a method with a receiver whose type is defined in another package (which includes the built-in types such as int).

To handle this kind of scenario, we can make use of the named types in golang and define a new type called Sub with an underlying type uint

1

type Sub uint

Then we can declare the UnmarshalJSON method with this Sub type.

1
2
3
4
5
6
7
8
9

func (s *Sub) UnmarshalJSON(b []byte) error {
  sub := strings.Replace(string(b), `"`, "", 2)
  v, err := strconv.ParseUint(sub, 10, strconv.IntSize)
  if err != nil {
    return err
  }
  *s = Sub(uint(v))
  return nil
}

Using the Replace function, we are getting rid of the double quotes in the actual JSON encoded value

With this new Sub type in place, We can rewrite the UserJwt by replacing the Sub field with the Id field of type Sub.

1
2
3
4
5

type UserJwt struct {
  Id    Sub `json:"sub"`
  Name  string
  Admin bool
}

The json tag with the value "sub" is required to map the sub key in the JSON claim with the Id.

Now if we try to unmarshal the invalid claim, the json.Unmarshal function will return an error

1
2
3
4
5
6
7
8
9
10

claims := `
{
  "sub": "user/john",
  "name": "John D",
  "admin": true
}`
var userJwt *UserJwt
err := json.Unmarshal([]byte(claims), &userJwt)
// This will return the following error
// strconv.ParseUint: parsing "name/john": invalid syntax

For a valid claim, we can now get the Id directly from the UserJwt Id field.

1
2
3
4
5
6
7
8
9
10

claims := `
{
  "sub": "1234567890",
  "name": "John D",
  "admin": true
}`
var userJwt *UserJwt
err := json.Unmarshal([]byte(claims), &userJwt)
// ...
fmt.Println(userJwt.Id)

That’s it! The code is in better shape now :)

Summary

In this blog post, we have seen how we can write cleaner code by leveraging the interfaces. The source code is available in my GitHub repository.

We built the entire platform using golang and event-driven microservices architecture. It was an exciting journey, and this blog post summarizes my experiences of using golang in this project.

The Learning Curve

Golang is a simple language to learn. Any developer who has some good experience in any programming language can pick it in a month. It took me two weeks to understand the language.

I’ve also learned a lot from the code review comments from the other developers of my team who are already familiar with the language.

The official golang site has done an excellent job in coming up two important learning resources for the beginners.

• A Tour of Go - Through a series of in-browser hands-on tutorials you can get a good feel of the language in few hours.
• Effective Go - It is one the best introduction as well as best practices documentation of a programming language that I have read. Right from package naming convention to how to do state sharing, this documentation will provide all it required to write an idiomatic golang code.

In addition to these resources, I recommend the Golang FAQs and the GoByExample for the beginners.

Go Routines, Channels and Select

In my perspective, go routines, channels and select are the sweet spots of golang. These abstractions are elegant and enable you to write concurrent programs without any hassles.

We have made substantial use of golang concurrency whenever it made sense, and the benefits were incredible. The blog post on Go Concurrency Patterns is an excellent read to get a feel of it.

Interfaces in golang

My favorite feature in golang is its support for interfaces.

Unlike C# or Java, you don’t need to specify the name of the interface when a type implements an interface. Because of this, the package in which the type present, doesn’t need to refer the package in which the interface has been declared.

Yan Cui has written a great blog post about its elegance.

The design for Go’s interface stems from the observation that patterns and abstractions only become apparent after we’ve seen it a few times. So rather than locking us in with abstractions at the start of a project when we’re at the point of our greatest ignorance, we can define these abstractions as and when they become apparent to us. - Yan Cui

The Go Proverbs

The Go Proverbs provides a set of principles and guidelines while developing software in golang.

Though certain principles (A little copying is better than a little dependency.) look weird, they have profound meaning behind it.

Like the saying, “You need to spend time crawling alone through shadows to truly appreciate what it is to stand in the sun.”, to appreciate these proverbs you need to spend a decent amount time in developing software in go.

Golang Standard Library

The golang standard library is futuristic. It has pretty much all that is required for solving the general problems efficiently.

The surprising factor for me is treating JSON as a first class citizen in the standard library.

Most of the languages rely on an open source library to deal with JSON. Having JSON handling as part of the standard library shows how much thought and care has been put in.

Declared and Not used Compiler Error

Another good thing that golang got it right is showing compiler errors for unused variables and packages.

The software which we write evolve along with its business requirements. Continuous refactoring, feature addition or deletion, changing the architecture to meet the new demands are some the stuff that we do during the evolution of the software. During these activities often we miss removing the unused code and the packages that we were using before.

Through this compiler error, golang forces the developer to write better code. The biggest advantage is while reading the code written by someone, you can be sure that the variables and the packages are always being used.

Also removing unused packages results in smaller binary size.

go Command (Command Line Interface) and go tools

The go command is an another well thought out feature in golang which comes by default with the installation.

Using go command, you can compile & build the code, download and install external packages, run the tests and lot more without relying on any other external tools. The go test command is incredibly fast and it made our job easier while doing Test-Driven development.

The gofmt command enabled our team to follow a consistent formatting of golang source files. Thanks to this awesome tool we hardly had code review comments on the format of the code.

The golang tools, especially goimports and gorename has been extremely useful

go install and Single executable binary

The command which I loved the most in golang is the go install command. It takes care of compiling the code and produces a stand alone executable binary. Due to caching of intermediary packages, it builds the executable binary incredibly fast.

To run this resulted binary file, we don’t have to install anything on the target machine (Like JVM, .NET).

It enabled us to share the microservice as an executable file with the other team members to create spikes, develop and locally test other dependent microservices.

Just like bundling the front-end assets into single javascript file and serving them via CDNs, we can even do continuous deployment of golang codebase by serving the binaries from a content provider like Amazon S3, if it makes sense!

We can also do cross-compilation of your golang code for different OSes and Architectures using go build command.

Open Source Libraries in golang

Golang has a thriving open source community, and it houses a vast collection of open source libraries and frameworks. Here is the list of some of the things that we have used heavily

• AMQP - To interact with RabbitMQ
• GORM - To perform database operations in PostgreSQL
• go-uuid - To generate and use UUIDs
• migrate - For database migrations
• envconfig - For managing configuration data from environment variables
• negroni - To write HTTP middlewares
• httprouter - For handling HTTP routes. We have used this along with golang’s standard HTTP handlers
• logrus - Simple and powerful library for logging
• testify - For unit test assertions

For most of the common things that you’d like to do in your project, you can always find an off the shelf open source library

Private methods, functions and struct fields

I was admired when I came to know that just by using a naming convention, I can specify public and private methods, functions and struct fields.

It is a smart feature in the language. If you want to expose them as public, start with an upper case letter and start with a lower case letter if you want to them to be used only inside a package or a struct.

Not So Good Features in Golang

So far in this blog post, I’ve shared the features that I liked very much in golang. In the rest of the post, I am about to share the not so good features in golang. The heading was intentionally started with “Not So Good” as the things that I am about to share are not bad ones according to the golang language design principles and the go proverbs that we have seen earlier.

IMHO there is no perfect programming language, and there is no silver bullet. All languages shine in certain areas and not so good in some other areas. At the end of the day, a programming language is just a tool which we use to solve the business problems. We need to pick an appropriate language that solves the problem in hand.

Disclaimer: The intention of this blog post is to share my experiences and provide honest feedback to the people who is evaluating golang.

The Golang way of handling errors

The imperative way of handling errors is one of the common complaint raised by many people.

1
2
3
4
5
6
7
8
9
10
11
12
13

_, err = foobar1()
if err != nil {
 return err
}
_, err = foobar2()
if err != nil {
 return err
}
_, err = foobar3()
if err != nil {
 return err
}
// and so on

Rob Pike addresses this concern in a detailed blog post, Errors are values and quotes a pattern using errWriter to avoid the repetition. It is a good design and elegantly wraps the nil check on the error. The downside of this approach is we need to write an extra wrapper for every new type to get rid of the repetitiveness. The wrapper approach may get complex if the foobar1, foobar2, and foobar3 are addressing different concerns.

For people who is coming from typed functional programming languages (F#, Haskell, Scala) background, like me, this may look little awkward to write it in this way. There is a proposal to include Result type in golang but I feel it may not be incorporated.

Treating nil as value

While other languages are moving away from dealing with null values and replacing it with option types, golang is taking a backward step by treating nil (equivalent for null in Go) as a valid value. So we need to be cautious while using the nil value. Lack of discipline may result in runtime exceptions (panic in go vocabulary)

This problem can be avoided to some degree by having a function/method to return two values, the actual value to be returned and the error, and ensuring that the actual value will never be nil, if the error is nil and vice versa. Unfortunately, this approach leads to the repetition concern that we just saw.

Absence of generics

This also an another typical critic in golang. There is an answer of for this question in golang FAQ saying, it is in the pipeline but my opinion is it may not be in the near future.

There are some workarounds like using an empty interface and casting it to appropriate types using type assertion.

The real concern is trying to do map, filter, reduce operations over collections. Again there are workarounds by using templating and reflection, but it has its own cost (performance and boilerplate code). After fiddling around with different options, We went back to the classic way of iterating through for loop.

Mutable variables and imperative coding

For developers who is coming from the functional programming background, the mutable variables and imperative way of writing code are speed breakers in the golang journey.

Though golang has some good support for doing functional programming, the presence of mutable variables and imperative way doing certain things, hinder the progress.

Summary

Apart from these few concerns, I’ve liked golang and enjoyed using it.

The language design principles of golang are my important takeaways. I strongly recommend watching the talks of Rob Pike on golang design; even you are not going to use golang.

I’d consider using golang in an another project in future for sure if it is an appropriate language for the problem at hand.

Google Authenticator is one of the popular application that implements two-factor authentication services. In this blog post, we are going to learn how to implement Two-factor authentication in web applications developed using suave

The idea presented here is a naive implementation of Two-factor authentication. The objective here is to demonstrate how to implement it in a functional programming language, F#. Things like TLS/HTTPS, preventing CSRF and other attacks are ignored for brevity.

This blog post is a part of fsharp advent calendar 2016.

Prerequisite

This blog post assumes that you are familiar with the concept of Two-factor authentication and Google Authenticator.

If you would like to know more about these, check out the below resources to get a picture of what it is all about.

• Two-step verification by Google
• How Google Authenticator works
• A Stack-Overflow Question

We are going to use Time-based One-time Password(TOTP) algorithm in this blog post

What we will be building

We are going to build a tiny web appliaction that has an inbuilt user account with the username foo and the password bar

After successful login, the user redirected to the Profile page where the user sees his name with a couple of buttons. One to enable Two-factor authentication and another one to log out

Upon clicking the Enable Two Factor Authentication button, the user redirected to the Enable Two Factor Authentication page where the user has to scan the QR Code with the Google Authenticator App (For Android or iPhone). Then he needs to enter the verification code to enable Two-factor authentication for his account.

Google Authenticator App

If the verification code matches, the updated Profile page will look like

Now if the user logout and login again, he will be prompted to enter the verification code

After entering the verification code from the Google Authenticator, the user will be redirected to his Profile page.

Getting Started

Create a new F# Console Project with the name Suave.TwoFactorAuth and use Paket to install the following dependencies.

paket.dependencies

1
2
3
4
5

nuget FSharp.Core
nuget Suave
nuget DotLiquid
nuget Suave.DotLiquid
nuget OtpSharp

Then reference them in the Suave.TwoFactorAuth project.

Suave.TwoFactorAuth/paket.references

1
2
3
4
5

FSharp.Core
Suave
DotLiquid
Suave.DotLiquid
OtpSharp

The OtpSharp is an .NET library that we will be using to generate keys and to verify the verification code from Google Authenticator app using the TOTP algorithm.

The reference to DotLiquid library is required to render the templates using Suave.DotLiquid

Initializing DotLiquid

To use DotLiquid to render the views in Suave, we need to set the templates directory explicitly.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

// Suave.TwoFactorAuth/Suave.TwoFactorAuth.fs
module Suave.TwoFactorAuth.Main

open Suave
open System.IO
open System.Reflection
open Suave.DotLiquid

let initializeDotLiquid () =
  let currentDirectory =
    let mainExeFileInfo =
      new FileInfo(Assembly.GetEntryAssembly().Location)
    mainExeFileInfo.Directory
  Path.Combine(currentDirectory.FullName, "views")
  |> setTemplatesDir

[<EntryPoint>]
let main argv =
  initializeDotLiquid ()
  0

In this sample application we are going to create a directory views. This views directory will contain the liquid templates of our appliaction

Serving the Login Page

Let’s start by serving the Login page.

Create a new directory with the name views in the Suave.TwoFactorAuth project and add a new liquid template page.liquid. This page.liquid is the master template for our application

After creating, change the ‘Copy to output’ property of the page.liquid file to ‘Copy if newer’ so that the view files copied to the build output directory.

This step is applicable for all the other view templates that we will be creating later

If you are using VS Code or atom editor, you need to do this property change manually by opening the Suave.TwoFactorAuth.fsproj file

1
2
3
4
5
6
7
8
9
10

<!-- .... -->
<ItemGroup>
  <Folder Include="views\" />
</ItemGroup>
<ItemGroup>
  <None Include="views\page.liquid">
    <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
  </None>
</ItemGroup>
<!-- .... -->

Then create a new template file login.liquid in the views directory

The login.liquid view extends the page.liquid view and fill the placeholders for head and content.

To display the error messages like Password didn’t match, login.liquid view bounded to the model of type string.

Now we have the view template for the login page ready and the next step is to render it upon receiving an HTTP request.

Create a new fsharp source file Login.fs and update it as below

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

module Suave.TwoFactorAuth.Login

open Suave
open Suave.DotLiquid
open Suave.Filters
open Suave.Operators

let loginPath = "/login"

let renderLoginView (request : HttpRequest) =
  let errMsg =
    match request.["err"] with
    | Some msg -> msg
    | _ -> ""
  page "login.liquid" errMsg

let loginWebPart =
  path loginPath >=> choose [
      GET >=> request renderLoginView]

As a good practice let’s create a new module Web which will be containing all the WebParts of the application

1
2
3
4
5
6
7

// Suave.TwoFactorAuth/Suave.Web.fs
open Suave
open Login

let app =
  choose [
    loginWebPart]

Then start the Web Server

1
2
3
4
5
6
7
8

// Suave.TwoFactorAuth/Suave.TwoFactorAuth.fs
// ...
open Web
// ...
let main argv =
  // ...
  startWebServer defaultConfig app
  0

Keeping all the Suave WebParts in a single place like we did in the Web.fs file, enable us to host Suave in Azure Functions or Asp.Net Core without doing any significant changes.

Handling User Login

To handle the login request from the user, we need to have some users in the application. Let’s hardcode a user account.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

// Suave.TwoFactorAuth/Suave.TwoFactorAuth.fs
module Suave.TwoFactorAuth.User

open System.Collections.Generic

type User = {
  Username : string
  Password : string
}

let private users = new Dictionary<string, User>()

users.Add("foo", {Username = "foo"; Password = "bar"})

let getUser username =
  match users.TryGetValue username with
  | true, user -> Some user
  | _ -> None

Post successful login, to serve the subsequent requests we need to identify the user who logged in. We can achieve it Suave using statefulForSession, which initializes a user state for a browsing session.

Let’s create some helper functions to do this.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

// Suave.TwoFactorAuth/Combinators.fs
module Suave.TwoFactorAuth.Combinators

open Suave.State.CookieStateStore
open Suave.Cookie
open Suave
open Suave.Operators
open Suave.Authentication

let sessionSet failureF key value =
  statefulForSession
  >=> context (fun ctx ->
                match HttpContext.state ctx with
                | Some state -> state.set key value
                | _ -> failureF
              )

let sessionGet failureF key successF =
  statefulForSession
  >=> context (fun ctx ->
                match HttpContext.state ctx with
                | Some store ->
                  match store.get key with
                  | Some value -> successF value
                  | _ -> failureF
                | _ -> failureF
  )

let clearSession =
  unsetPair SessionAuthCookie
    >=> unsetPair StateCookie

The sessionSet function takes a WebPart and a key value pair and tries to persist the value in the session state with the given key. If it fails, it calls the WebPart.

The sessionGet function takes a success WebPart Combinator, a failure WebPart, and a key. If retrieving the value from session state is successful it calls the success WebPart combinator with the retrieved value. In case of retrieval failure it calls the failure WebPart

The clearSession function clears the state. We will be using it while implementing log out

Now we have all the building blocks for handling user login request, and it’s time to start its implementation

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

// Suave.TwoFactorAuth/Login.fs
// ...
open Suave.Redirection
open Suave.Authentication
open Suave.Cookie

open Combinators
open User
// ...
let userSessionKey = "loggedUser"

let redirectToLogin = function
  | Some errMsg -> FOUND (sprintf "%s?err=%s" loginPath errMsg)
  | None -> FOUND loginPath

let loginSucess failureW redirectPath username =
  authenticated Cookie.CookieLife.Session false
    >=> sessionSet failureW userSessionKey username
    >=> FOUND redirectPath

let onLogin redirectPath (request : HttpRequest) =
  match request.["Username"], request.["Password"] with
  | Some username, Some password ->
    match getUser username with
    | Some user ->
      match user.Password = password with
      | true -> loginSucess never redirectPath username
      | _ -> redirectToLogin (Some "Password didn't match")
    | _ -> redirectToLogin (Some "Invalid username")
  | _ -> redirectToLogin (Some "Invalid request")

let secured webpart =
  let onFail = redirectToLogin (Some "sign-in to access")
  sessionGet onFail userSessionKey webpart

let loginWebPart redirectPath =
  path loginPath >=> choose [
      // ...
      POST >=> request (onLogin redirectPath)]

The key function to note here is secured that takes a WebPart. It calls this WebPart only if the user has logged in. If he didn’t the user will be redirected to the Login page

After successful login, we need to redirect the user to his profile page. Let’s create a profile.liquid a view template for the Profile page

To render this profile page let’s add some code

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

// Suave.TwoFactorAuth/Profile.fs
module Suave.TwoFactorAuth.Profile

open Suave.DotLiquid
open Suave.Redirection
open Suave.Filters
open Suave.Operators

open User
open Login

type ProfileViewModel = {
  Username: string
  SecretKey : string
  IsTwoFactorAuthEnabled : bool
}
with static member FromUser user =
        {
          SecretKey = ""
          IsTwoFactorAuthEnabled = false
          Username = user.Username
        }

let profilePath = "/profile"

let renderProfile notFoundPath username =
  match getUser username with
  | Some user ->
    user
    |> ProfileViewModel.FromUser
    |> page "profile.liquid"
  | _ -> FOUND notFoundPath

let profileWebPart notFoundPath =
  path profilePath >=> secured (renderProfile notFoundPath)

The labels IsTwoFactorAuthEnabled, SecretKey are just blank right now, and we will be seeing them in action while adding two-factor authentication

The notfound.liquid page that is going to our fancy 404 page

1
2

<!-- Suave.TwoFactorAuth/views/not_found.liquid -->
Not Found :(

The final step is to put these WebParts together

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

// Suave.TwoFactorAuth/Web.fs
// ...
open Profile
open Suave.DotLiquid
open Suave.Filters
open Suave.Operators

let notFoundPath = "/notfound"

let app =
  choose [
    loginWebPart profilePath
    profileWebPart notFoundPath
    path notFoundPath >=> page "not_found.liquid" ""
  ]

Handling Logout

Handling logout is a simpler task as we have all the infrastructure already in place.

1
2
3
4
5
6
7
8
9
10
11

// Suave.TwoFactorAuth/Web.fs
// ...
open Combinators
open Login
// ...

let app =
  choose [
    // ...
    path "/logout" >=> clearSession >=> redirectToLogin None
  ]

Enabling Two-factor Authentication

To enable Two-factor authentication, we need to change our User domain model first.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

// Suave.TwoFactorAuth/User.fs
// ...
type TwoFactorAuthentication =
| Enabled of SecretKey:string
| Disabled

type User = {
  // ...
  TwoFactorAuthentication : TwoFactorAuthentication
}
// ... 
// Let's assume TwoFactorAuthentication is disabled by default 
users.Add("foo", {Username = "foo"; Password = "bar"; TwoFactorAuthentication = Disabled})
// ...
let enableTwoFactorAuth username key =
  match getUser username with
  | Some user ->
    users.[username] <- {user with TwoFactorAuthentication = Enabled key}
  | _ -> ()

The next step is to define a liquid view template for the enable_two_factor page.

While enabling the Two-factor authentication, we need to generate a secret key for the user that will be required for both, one-time verification code generation as well as its verification.

So, in the enable_two_factor.liquid template we pass the generated SecretKey as a hidden input which will then be used for the verification of the code.

1

<input type="hidden" name="SecretKey" value="">

Now we need to render the enable_two_factor page in response to the HTTP GET request /enable_two_factor

Let’s create a new module GoogleAuthenticator to put the Two-factor authentication related functions together

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47

// Suave.TwoFactorAuth/GoogleAuthenticator.fs
module Suave.TwoFactorAuth.GoogleAuthenticator

open Suave
open Suave.Filters
open Suave.Operators
open Suave.DotLiquid
open Suave.Redirection
open OtpSharp
open Base32

open Login
open User

let enableTwoFactorAuthPath = "/enable_two_factor"

type EnableTwoFactorViewModel = {
  Key : string
  Url : string
  Err : string
}
with static member From username err =
      let secretKey = KeyGeneration.GenerateRandomKey(20)
      let appName = "SuaveRocks"
      let label = sprintf "%s (%s.com/%s)" appName appName username
      let keyUrl = KeyUrl.GetTotpUrl(secretKey, label)
      { Url = sprintf "https://qrcode.kaywa.com/img.php?s=4&d=%s&issuer=%s" keyUrl appName
        Key = Base32Encoder.Encode(secretKey)
        Err = err}

let renderEnableTwoFactorAuthView notFoundPath username ctx = async {
  match getUser username with
  | Some user ->
    let err =
      match ctx.request.["err"] with
      | Some err -> err
      | _ -> ""
    let vm = EnableTwoFactorViewModel.From username err
    return! page "enable_two_factor.liquid" vm ctx
  | _ -> return! redirect notFoundPath ctx
}

let googleAuthenticatorWebPart notFoundPath =
  choose [
    path enableTwoFactorAuthPath >=> choose [
      GET >=> secured (renderEnableTwoFactorAuthView notFoundPath)
    ]]

As we did in the login page, we are using the err query string in the request to pass the verification code mismatch errors.

We are leveraging the OtpSharp library to generate the URL that is in turn represented as a QR Code.

If you would like to how Google Authenticator interprets the generated key and the issuer name from the URL embedded in the QR Code, check out the UriFormat documentation.

The last step in rendering this page is adding the googleAuthenticatorWebPart in the Web module where we are putting all the WebParts together

1
2
3
4
5
6
7
8
9

// Suave.TwoFactorAuth/Web.fs
// ...
open GoogleAuthenticator

let app =
  choose [
    // ...
    googleAuthenticatorWebPart notFoundPath
  ]

While enabling Two-factor authentication, the user has to scan the QR-Code from his Google Authenticator app and he will be getting a one-time verification code like this upon adding

Then he will be entering this in the enable_two_factor page and click Enable.

Let’s handle this POST request.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

// Suave.TwoFactorAuth/GoogleAuthenticator.fs
// ...
let verifyOtp secretKey code =
  let otp = new Totp(Base32Encoder.Decode secretKey)
  otp.VerifyTotp(code, ref 0L, new VerificationWindow(2, 2))

let enableTwoFactorAuth redirectPath notFoundPath username ctx = async {
  match ctx.request.["SecretKey"], ctx.request.["Code"] with
  | Some secretKey, Some code ->
    match verifyOtp secretKey code with
    | true ->
      enableTwoFactorAuth username secretKey
      return! redirect redirectPath ctx
    | _ ->
      let redirectTo =
        sprintf "%s?err=code validation failed" enableTwoFactorAuthPath
      return! redirect redirectTo ctx
  | _ -> return! redirect notFoundPath ctx
}

let googleAuthenticatorWebPart redirectPath notFoundPath =
  choose [
    path enableTwoFactorAuthPath >=> choose [
      // ...
      POST >=> secured (enableTwoFactorAuth redirectPath notFoundPath)
    ]]

Thanks to the OtpSharp library for making our job simpler here. We just need to get the SecretKey, and the Code from the POST request and get it verified using OtpSharp’s VerifyTotp function.

If the verification is successful, we will be enabling the Two-factor authentication for the user in our in-memory backend using the enableTwoFactorAuth function and then redirect to the redirect path which in this case the Profile page.

1
2
3
4
5
6
7
8

// Suave.TwoFactorAuth/Web.fs
// ...

let app =
  choose [
    // ...
    googleAuthenticatorWebPart profilePath notFoundPath
  ]

Login With Two-factor Authentication

The last step is to prompt for the verification code whenever the Two-factor Authentication enabled user tries to log in and verify the one-time verification code before granting the access.

Let’s start by defining the liquid view template auth_code.liquid for getting the one-time verification code.

1
2
3
4
5
6
7
8

// Suave.TwoFactorAuth/GoogleAuthenticator.fs
// ...
let authCodePath = "/authcode"

let googleAuthenticatorWebPart redirectPath notFoundPath =
  choose [
    // ...
    path authCodePath >=> page "auth_code.liquid" ""]

After username and password verification, the user has to be redirected the auth_code page

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

// Suave.TwoFactorAuth/Login.fs
// ...
let authCodeSessionKey = "loginUser"

let onLogin redirectPath authCodePath (request : HttpRequest) =
  match request.["Username"], request.["Password"] with
  | Some username, Some password ->
    match getUser username with
    | Some user ->
      match user.Password = password, user.TwoFactorAuthentication with
      | true, Disabled -> loginSucess never redirectPath username
      | true, Enabled _ ->
          sessionSet never authCodeSessionKey username
            >=> FOUND authCodePath
      // ...

// ...

let loginWebPart redirectPath authCodePath =
  path loginPath >=> choose [
      // ...
      POST >=> request (onLogin redirectPath authCodePath)]

1
2
3
4
5
6
7

// Suave.TwoFactorAuth/Web.fs
// ...
let app =
  choose [
    loginWebPart profilePath authCodePath
    // ...
  ]

We are using a separate session key authCodeSessionKey to hold the username of the user.

The final step is verifying the one-time verification code (from the Google Authenticator app) entered by the user.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25