The Riemann Hypothesis is one of the most important unsolved problems in mathematics and the subject of one of my favourite books about maths: Prime Obsession.

A few years ago in Paris, I had the pleasure of seeing Sir Michael speak. He spoke so cheerfully about life and mathematics. There is a great interview with him on the Web of Stories YouTube channel.

Gmail

Does the job.

Lastpass

Everyone needs a password manager.

Pinboard (and the PinSwift iPhone app)

The best bookmark tracking tool. Simple, fast, powerful. The best mobile app for it is PinSwift.

Bear

A very pretty MarkDown editor for Mac and iPhone. I use Bear for all my notes.

Google Drive File Stream

Since I use GSuite, it makes sense to try to put everything there. Google Drive has improved enormously since they moved to a file streaming approach.

Google Tasks (and the GoTasks app)

I need simple task manager with support for multiple lists, hierarchies and available everywhere. Google Tasks is often overlooked in this regard. Check out the Google Tasks canvas. There is also a fantastic free iPhone app for it GoTasks.

Feedly

Feedly is the best RSS news feed reader, in my opinion. I keep up with a few development blogs, a few science blogs and the blogs of my friends. The mobile app is great too.

Xero

I’ve been using Xero for eight years now. Business accounting software done right.

Kindle

I like real books but don’t always lug them about with me, so often I buy both the hard copy and the kindle version (there shoudl be some sort of clever discount for this…). The Kindle app is where I spend my time when I have no connectivity (the London tube, airplanes, etc.)

Chess.com and Lichess.com

I waste hours playing online blitz chess. I’ve even written a couple of Chrome extensions: Pretty print your games and Analyse any chess.com game with lichess.

Chess is a mascohistic, character-building pastime. Progress is elusive. You play a couple of good games and you think you’re improving and then you get thrashed repeatedly by a 10 year old.

CodeRush

I’ve been using DevExpress CodeRush since 2005. Check out this video tutorial for a lightening tour of a lot of the features, and look at the DevExpress youtube channel for a load of other tutorials.

NCrunch

NCrunch provides continuous testing for Visual Studio. When I make any change to my code which breaks a unit test, the NCrunch risk status goes red a few seconds later, even without recompiling. I get immediate feedback for any breaking change, so long as I have a test for it. Not only does it encourages me and my team to write good tests, but it allows us to make new changes and refactor with confidence.

Redgate .NET Reflector Developer Bundle

When the going gets tough, the ANTS Performance Profiler and ANTS Memory Profiler have helped me find some of the hardest bugs I’ve come across.

Also, while there are several good .NET decompilers, many of them free, I got used to .NET Reflector which comes part of this bundle.

T4

I use code generation to automatically generate templates for unit tests for validation rules and T4 and the T4 Toolbox fit the bill nicely.

For a more advanced use case, see my post about automatically converting DevExpress v1 report scripts to check for compilation errors and allow for unit testing of report scripts.

AWS Toolkit

I always install the Amazon Web Services Toolkit to make it easy to spin up test servers in the Amazon cloud.

Next up

My essential web applications and iPhone apps.

A bit of preamble, my most powerful machine is a Windows-only desktop with lots of RAM and an SSD drive. I use it almost exclusively for development.

I also have a Macbook Pro which is configured to dual-boot Windows and MacOS. The Windows machine is more or less a mirror of my main development machine. The Mac is where I do all my document editing, Word, blogging, etc. I also use it for my occasional forays in to iPhone development or other non-Windows experiments.

When I upgrade my development machine, I reconfigure the previous system for Linux, currently Ubuntu.

Chocolatey

In Windows, Chocolatey is a very convenient way to install programs. Just go:

C:> choco install nodejs

Boxstarter

Even better, create a powershell script with all the applications you want to install and run it with boxstarter. Repeatable and reboot-resilient environment installations using chocolatey under the hood. For several years now, I have maintained a script for installing my entire development environment (and the build server) from scratch. Works whether it’s a physical or a virtual machine.

BeyondCompare

BeyondCompare is the best file-comparison tool I’ve found. Cross-platform too, I have it installed on my Mac as well.

Synergy

Synergy allows me to share one mouse and keyboard across all my machines. It works just like dual monitors, except when the mouse moves onto the next screen, you’ve actually changed computer. I don’t have multiple monitors any more, I just switch the input on my giant curved 34” Dell monitor via a keyboard shortcut.

7Zip

Nothing to say.

VS Code

First class support for .NET and C#, Typescript in particular, but VS Code is also extremely good with Javascript, Powershell, Python, Markdown, etc. Multiple cursor support. Cross-platform. Extensions. I’m writing this blog post in VS Code.

Visual Studio

My daily workhorse. I’ve spent more screen time here than anywhere.

Zotero

Keep track of academic papers. Download them for offline reading. Handle citations and bibliographies effortlessly from within Word. Zotero.

Next up

More on Visual Studio in the next post where I go through my essential extensions and tools.

TL;DR - Skip to the end.

Clues

It only seemed to happen when I was working while sitting up on the bed with the computer on my lap. It seemed to be related to the position of the computer or possibly the lid. The computer would sleep. I’d wake it and login again. Continue. Sometimes it would happen once. Sometimes three times in ten minutes. Annoying. I assumed some hardware defect.

Pajamas

Then over several days I cracked it. I realised it never happened when I was wearing pajamas. Very strange.

Reflections

I started thinking about what normally triggers sleep mode. Closing the lid. How does a Macbook know the lid is closed. There’s not really a clasp or anything so it must be a magnet. Perhaps it’s getting confused by my belt buckle or something. My phone?

No, my phone case! The flip-case of my phone has a magnet to keep the flap closed. In my jeans pocket it’s near enough to make the laptop think the lid has been closed. Duh.

TL;DR

Take your mobile phone out of your pocket!

Firstly, I attended the second London serverless meetup yesterday evening which was excellent and showed just how much enthusiasm there is for serverless architectures. Check out their new logo on the left. It was significant that each of the three speakers announced that they are actively hiring serverless developers.

Secondly, Stolz has contributed improvements to my sample project for integrating PHP into the serverless framework. It’s the purpose of this blog post to cover the changes.

The trick to getting AWS lambda to support PHP is to bundle in a PHP binary so that nodejs can call it with child_process.spawn(). In my first implementation, I used an Ubuntu docker base image to compile and produce the php binary. Unfortunately, this is not identical to the container that AWS Lambda uses and so sometimes the logs would contain errors such as:

1
2
3
4
5

START RequestId: 728dcddf-feaa-11e6-8346-2125e1c055d7 Version: $LATEST
2017-03-01 18:11:10.455 (+00:00)    728dcddf-feaa-11e6-8346-2125e1c055d7    stderr: ./php: /usr/lib64/libcurl.so.4: no version information available (required by ./php)

END RequestId: 728dcddf-feaa-11e6-8346-2125e1c055d7
REPORT RequestId: 728dcddf-feaa-11e6-8346-2125e1c055d7    Duration: 6000.08 ms    Billed Duration: 6000 ms    Memory Size: 1024 MB    Max Memory Used: 23 MB  

In my experience, these errors were often not fatal, but the correct approach is to build the php binary from a base image which is closer to the one lambda uses. So instead of my docker file starting with FROM ubuntu, it now starts with FROM amazonlinux. Also, with this image, I can use yum to install other dependencies like libpng-devel. So the new docker build script for producing the php binary looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

# Compile PHP with static linked dependencies
# to create a single running binary

FROM amazonlinux

ARG PHP_VERSION

RUN yum install \
    autoconf \
    automake \
    libtool \
    bison \
    re2c \
    libxml2-devel \
    openssl-devel \
    libpng-devel \
    libjpeg-devel \
    curl-devel -y

RUN curl -sL https://github.com/php/php-src/archive/$PHP_VERSION.tar.gz | tar -zxv

WORKDIR /php-src-$PHP_VERSION

RUN ./buildconf --force

RUN ./configure \
    --enable-static=yes \
    --enable-shared=no \
    --disable-all \
    --enable-json \
    --enable-libxml \
    --enable-mbstring \
    --enable-phar \
    --enable-soap \
    --enable-xml \
    --with-curl \
    --with-gd \
    --with-zlib \
    --with-openssl \
    --without-pear

RUN make -j 5

If you run this with

$ sh dockerfile.buildphp

It will use docker to overwrite the php binary which will get shipped when you deploy with sls deploy. And this time, there are no more libcurl errors. All the code is on Github.

The service I’m building connects runs a PHP function for pretty-printing chess games from the lichess online chess server. James Clarke has written a PHP function to do this using fpdf17.

The lichess exporter takes the game id of any game that has been played on the lichess server and produced a PDF output. Take for example, Game 8 of the current World Championship which is here. When I open the resulting file, I see this:

In this blog post I’ll describe how I turned this into a serverless service. The goal is to create:

• Add an endpoint which takes the game id as a parameter
• Run the PHP function via an AWS lambda function
• Return the result as a stream

Prerequisites

First check everything we need is installed.

$ serverless --version
1.2.1
$ node --version
v7.1.0

Initial setup

$ mkdir serverless-lichess-to-pdf
$ cd serverless-lichess-to-pdf
$ sls install --url https://github.com/ZeroSharp/serverless-php

Next copy in the source from https://github.com/clarkerubber/lichessPDFExporter.

You can check it works by running the following.

$ php main.php COQChpzH > COQChpzH.pdf

What’s going on here? The php binary (from the serverless-php project) is running main.php (from the lichess-pdf-exporter project) with argument COQChpzH (which corresponds to a chess game on the lichess server. The main.php function downloads the game from the lichess API and passes it through the fpdf17 library to create a pdf stream which is written out to the COQChpzH.pdf file.

Lessons learned

I learned a few things while trying to get this project working. The basic plan is to modify handler.js so that it return the output of the call described above. Turns out there are quite a few gotchas along the way.

Lesson 1 - Defining a path parameter

I want my API to look like this:

http://.../serverless-lichess-to-pdf/export/{gameid}

I could not find an example in the serverless docs for getting a parameter that is passed in the URL.

Turns out your serverless.yml file should look like this:

1
2
3
4
5
6
7

functions:
  exportToPdf:
    handler: handler.exportToPdf
    events:
      - http:
          path: export/{gameid}
          method: get

Then, in your handler.js you can retrieve the parameter with:

1
2
3
4

module.exports.exportToPdf = (event, context, callback) => {
  var gameid = event.pathParameters.gameid;
  // etc...
}

Lesson 2 - API Gateway does not support binary data

I was hoping I could just do something like this:

1
2
3
4
5
6
7
8

// this does NOT work
const response = {
    statusCode: 200,
    body: outputFromPhpCall,
    content-type: "application/pdf"
};

return callback(null, response);

At present, you cannot return a binary file. Amazon have just (November 2016) released support for binary types in API Gateway but it’s currently an open issue in the Serverless Framework.

Lesson 3 - You can redirect the response to an S3 bucket

So instead of returning the binary output, I can write the output to an S3 bucket and return a 302 redirection to the S3 resource. Like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

// body contains the output from the PHP call
const params = {
    Bucket: bucket,
    Key: key,
    ACL: 'public-read-write',
    Body: body,
    ContentType: 'application/pdf'
};

// Save the pdf file to S3    
s3.putObject(params, function(err, data) {
if (err)
{
    return callback(new Error(`Failed to put s3 object: ${err}`));
}

// respond with a 302 redirect to the PDF file
const response = {
    statusCode: 302,
    headers: {
        location : `https://s3-eu-west-1.amazonaws.com/${bucket}/${key}`
    }
};

return callback(null, response);

Lesson 4 - You can automatically delete S3 objects after a number of days

Each S3 bucket has optional lifecycle rules where you can specify that files are automatically removed after a time period. I wanted to set this up within the serverless.yml resources section, but the syntax for the lifecycle rules were not very obvious and I could not find any examples online. The following seems to work:

1
2
3
4
5
6
7
8
9
10
11

resources:
  Resources:
    PackageStorage:
      Type: AWS::S3::Bucket
      Properties:
        AccessControl: PublicRead
        BucketName: ${self:custom.exportToPdfBucket}
        LifecycleConfiguration:
          Rules:
            - ExpirationInDays: 1
              Status: Enabled

It’s all working now

You can check it out by visiting this link.

The source code is on Github.

I also wrote a Chrome extension which injects the link into the lichess page.

Prerequisites

First check everything we need is installed.

$ serverless --version
1.1.0
$ node --version
v7.1.0

Install the sample PHP function

Install my sample Hello function from my github repository.

$ sls install --url https://github.com/ZeroSharp/serverless-php

1
2

Serverless: Downloading and installing "serverless-php"…
Serverless: Successfully installed "serverless-php".

The code

$ cd serverless-php

Let’s have a look at the serverless.yml file.

1
2
3
4
5
6
7
8
9
10
11
12
13
14

service: serverless-php

provider:
  name: aws
  runtime: nodejs4.3
  # region: eu-west-1

functions:
  hello:
    handler: handler.hello
    events:
      - http:
          path: hello
          method: get

Now look at the php function index.php that we’d like our lambda to call.

1
2
3
4
5
6

<?php

# $argv will contain the event object. You can output its contents like this if you like
#var_export($argv, true);

printf('Go Serverless v1.0! Your PHP function executed successfully!');

And the handler.js for the hello function looks as follows. It defines a simple lambda which calls the PHP binary, logs any errors and returns the result.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

'use strict';

var child_process = require('child_process');

module.exports.hello = (event, context, callback) => {

  var strToReturn = '';

  var php = './php';

  // workaround to get 'sls invoke local' to work
  if (typeof process.env.PWD !== "undefined") {
    php = 'php';
  }

  var proc = child_process.spawn(php, [ "index.php", JSON.stringify(event), { stdio: 'inherit' } ]);

  proc.stdout.on('data', function (data) {
    var dataStr = data.toString()
    // console.log('stdout: ' + dataStr);
    strToReturn += dataStr
  });

  // this ensures any error messages raised by the PHP function end up in the logs
  proc.stderr.on('data', function (data) {
    console.log(`stderr: ${data}`);
  });

  proc.on('close', function(code) {
    if(code !== 0) {
      return callback(new Error(`Process exited with non-zero status code ${code}`));
    }

    const response = {
      statusCode: 200,
      body: JSON.stringify({
        message: strToReturn,
        //input: event,
      }),
    };

    callback(null, response);
  });
};

Included is the PHP binary to bundle with our serverless function.

(You may need to compile it yourself with different options. See below for help on how to do this.)

Check it works from your shell.

$ php index.php

1

Go Serverless v1.0! Your PHP function executed successfully!

Run it locally through the Serverless Framework.

$ sls invoke local --function hello

1
2
3
4
5
6

Serverless: Your function ran successfully.

{
    "statusCode": 200,
    "body": "{\"message\":\"Go Serverless v1.0! Your PHP function executed successfully!\"}"
}

Looks good. Let’s deploy.

$ sls deploy

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

Serverless: Packaging service…
Serverless: Uploading CloudFormation file to S3…
Serverless: Uploading service .zip file to S3…
Serverless: Updating Stack…
Serverless: Checking Stack update progress…
..........
Serverless: Stack update finished…

Service Information
service: serverless-php
stage: dev
region: eu-west-1
api keys:
  None
endpoints:
  GET - https://c1w0hct166.execute-api.eu-west-1.amazonaws.com/dev/hello
functions:
  serverless-php-dev-hello: arn:aws:lambda:eu-west-1:962613113552:function:serverless-php-dev-hello

Run the remote function via Serverless.

$ sls invoke --function hello

1
2
3
4

{
    "statusCode": 200,
    "body": "{\"message\":\"Go Serverless v1.0! Your PHP function executed successfully!\",\"input\":{}}"
}

Visit the endpoint in your browser.

1
2
3

{
    "message": "Go Serverless v1.0! Your PHP function executed successfully!"
}

Nice. It’s all working.

Rebuilding the PHP binary

Depending on the PHP function you need to run, it may be necessary to rebuild the php binary with different flags and dependencies. You can do this best with docker.

$ docker --version
Docker version 1.12.3, build 6b644ec

Modify dockerfile.buildphp as necessary.

Then run:

$ sh buildphp.sh

This will build a new PHP binary and copy it to the project root. You can immediately deploy for testing with:

$ sls deploy

Thanks

Shout out to Danny Linden whose code got me started on this.

The problem

In the XAF MainDemo, lets make Departments read-only for the User role.

1
2
3

userRole.AddTypePermissionsRecursively<Department>(SecurityOperations.Create, SecurityPermissionState.Deny);
userRole.AddTypePermissionsRecursively<Department>(SecurityOperations.Write, SecurityPermissionState.Deny);
userRole.AddTypePermissionsRecursively<Department>(SecurityOperations.Delete, SecurityPermissionState.Deny);

Then start the web application, login as John and navigate to the Departments list view. There is a column selection box, but it serves no purpose. There are no actions that depend on a grid selection.

Without the SelectionColumnVisibilityController

The fix

Here is a controller which calculates whether there are any available actions which require one or more rows to be selected. If there are none, the selection box will not appear.

Add the following controller to the MainDemo.Module.Web project. It hides the selection box if there are no actions which depend on a grid selection.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141

using System;
using DevExpress.ExpressApp;
using DevExpress.ExpressApp.Actions;
using DevExpress.ExpressApp.Editors;
using DevExpress.ExpressApp.SystemModule;
using DevExpress.Web;
using System.Linq;

namespace MainDemo.Module.Web.Controllers
{
    public class SelectionColumnVisibilityController : ViewController
    {
        public SelectionColumnVisibilityController()
        {
            TargetViewType = ViewType.ListView;
        }

        private bool IsSelectionColumnVisible()
        {
            bool isSelectionColumnRequired = false;
            // remove checkbox if there are no available actions
            foreach (Controller controller in Frame.Controllers)
            {
                if (!controller.Active)
                    continue;

                if (controller.Actions.Count == 0)
                    continue;

                bool allowEdit = true;
                if ((Frame is NestedFrame) && (((NestedFrame)Frame).ViewItem is PropertyEditor))
                    allowEdit = (bool)((PropertyEditor)((NestedFrame)Frame).ViewItem).AllowEdit;

                foreach (ActionBase action in controller.Actions)
                {
                    if (action.SelectionDependencyType == SelectionDependencyType.RequireMultipleObjects)
                    {
                        if (action.Active || IsActionInactiveBySelectionContext(action))
                        {
                            if (action.Enabled || IsActionDisabledBySelectionContext(action))
                            {
                                isSelectionColumnRequired = true;
                                break;
                            }
                        }
                    }
                }
                if (isSelectionColumnRequired)
                    break;
            }
            return isSelectionColumnRequired;
        }

        private bool IsActionInactiveBySelectionContext(ActionBase action)
        {
            if (action.Active)
                return true;
            else
            {
                foreach (string item in action.Active.GetKeys())
                {
                    if (item == ActionBase.RequireMultipleObjectsContext || item == ActionBase.RequireSingleObjectContext)
                        continue;
                    if (!action.Active[item])
                        return false;
                }
                return true;
            }
        }

        private bool IsActionDisabledBySelectionContext(ActionBase action)
        {
            if (action.Enabled)
                return true;
            else
            {
                foreach (string item in action.Enabled.GetKeys())
                {
                    if (item == ActionBase.RequireMultipleObjectsContext ||
                        item == ActionBase.RequireSingleObjectContext ||
                        item == ActionsCriteriaViewController.EnabledByCriteriaKey)
                        continue;
                    if (!action.Enabled[item])
                        return false;
                }
                return true;
            }
        }

        protected override void OnViewControlsCreated()
        {
            base.OnViewControlsCreated();
            ASPxGridView grid = ((ListView)this.View).Editor.Control as ASPxGridView;
            if (grid != null)
            {
                grid.Load += grid_Load;
                grid.DataBound += grid_DataBound;
            }
        }

        protected override void OnDeactivated()
        {
            base.OnDeactivated();
            ASPxGridView grid = ((ListView)this.View).Editor.Control as ASPxGridView;
            if (grid != null)
            {
                grid.DataBound -= grid_DataBound;
                grid.Load -= grid_Load;
            }
        }

        void grid_Load(object sender, EventArgs e)
        {
            SetSelectionColumnVisibility(sender, e);
        }

        void grid_DataBound(object sender, EventArgs e)
        {
            SetSelectionColumnVisibility(sender, e);
        }

        private void SetSelectionColumnVisibility(object sender, EventArgs e)
        {
            bool isSelectionColumnVisible = IsSelectionColumnVisible();
            if (!isSelectionColumnVisible)
            {
                var grid = (ASPxGridView)sender;
                var selectionBoxColumn =
                    grid.Columns
                        .OfType<GridViewCommandColumn>()
                        .Where(x => x.ShowSelectCheckbox)
                        .FirstOrDefault();

                if (selectionBoxColumn != null)
                {
                    selectionBoxColumn.Visible = false;
                }
            }
        }
    }
}

Run the application again and see the difference. Now the grid looks like this. Notice, there is no longer a selection box on the row.

By the way, this is how it looks with old-style XAF web apps.

Without the SelectionColumnVisibilityController

With the SelectionColumnVisibilityController

One way is just to mark the test with the [Ignore] attribute.

1
2
3
4
5
6

[Test]
[Ignore] // TODO: Fix this test before the next release!
public void Test()
{
    // Some failing test code...
}

After the weekend, everyone forgets about the ignored tests and they never get fixed.

Instead, I like to do this.

1
2
3
4
5
6
7

[Test]
public void Test()
{
    if (DateTime.Now < new DateTime(2016, 10, 17))
        Assert.Ignore("Temporarily ignored until October 17.");
    // Some failing test code...
}

This is a fairly rare occurrence for my team, so the above approach is sufficient and works with all test frameworks. But if you want to go further Richard Slater shows how to create an NUnit attribute.

This is working very well for our developers. Maintaining the installation script takes a bit of effort but the benefits are worth it.

Recently, the chocolatey developers have been making things more secure and recent versions now require a checksum with any downloaded package. For now, there seems to be some difficulty using Install-ChocolateyVsixPackage within BoxStarter scripts.

If I try to run a script with

1
2
3

Install-ChocolateyVsixPackage
  StopOnFirstBuildError
  https://visualstudiogallery.msdn.microsoft.com/91aaa139-5d3c-43a7-b39f-369196a84fa5/file/44205/7/StopOnFirstBuildError.vsix

I get the following error message:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

 WARNING: Missing package checksums are not allowed (by default for HTTP/FTP,
  HTTPS when feature 'allowEmptyChecksumsSecure' is disabled) for
  safety and security reasons. Although we strongly advise against it,
  if you need this functionality, please set the feature
  'allowEmptyChecksums' ('choco feature enable -n
  allowEmptyChecksums')
 There were errors attempting to retrieve the vsix from https://visualstudiogallery.msdn.microsoft.com/91aaa139-5d3c-43a
  or pass in the option '--allow-empty-checksums'.
 7-b39f-369196a84fa5/file/44205/7/StopOnFirstBuildError.vsix. The error message was 'Empty checksums are no longer
 allowed by default for non-secure sources. Please ask the maintainer to add checksums to this package. In the meantime 
 if you need this package to work correctly, please enable the feature allowEmptyChecksums or provide the runtime
 switch '--allowEmptyChecksums'. We strongly advise against allowing empty checksums for HTTP/FTP sources.'.
 At C:\ProgramData\chocolatey\helpers\functions\Install-ChocolateyVsixPackage.ps1:173 char:13

etc...

I tried the suggested allowEmptyChecksums but it was still raising an error. After some experimentation it seems that the combination of (temporarily) disabling the checksumFiles feature and enabling allowEmptyChecksums works. Here is the relevant section of my boxstarter script.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

# temporarily enable/disable features to bypass checksums
choco feature disable -n=checksumFiles
choco feature enable -n=allowEmptyChecksums
try {
    # VS extensions for all versions of Visual Studio
    Install-ChocolateyVsixPackage StopOnFirstBuildError https://visualstudiogallery.msdn.microsoft.com/91aaa139-5d3c-43a7-b39f-369196a84fa5/file/44205/7/StopOnFirstBuildError.vsix
    Install-ChocolateyVsixPackage VisualHG https://visualhg.codeplex.com/downloads/get/1475782

    # VS extensions for VS2015
    Install-ChocolateyVsixPackage WebEssentials2015 https://visualstudiogallery.msdn.microsoft.com/ee6e6d8c-c837-41fb-886a-6b50ae2d06a2/file/146119/19/WebEssentials2015.vsix
    Install-ChocolateyVsixPackage WebExtensionPack https://visualstudiogallery.msdn.microsoft.com/f3b504c6-0095-42f1-a989-51d5fc2a8459/file/186606/23/Web%20Extension%20Pack%20v1.5.50.vsix
    Install-ChocolateyVsixPackage T4Toolbox2015 https://visualstudiogallery.msdn.microsoft.com/34b6d489-afbc-4d7b-82c3-dded2b726dbc/file/165481/2/T4Toolbox.14.0.0.71.vsix

    # CodeRush for Roslyn Preview
    Install-ChocolateyVsixPackage CodeRushForRoslyn https://visualstudiogallery.msdn.microsoft.com/8a8390ae-1f71-4659-9d8d-5dd56fd8a72e/file/163212/15/DevExpress.CodeRush.Roslyn-16.1.6.vsix
}
finally {
    choco feature enable -n=checksumFiles
    choco feature disable -n=allowEmptyChecksums
}

And then everything works as expected.

In XAF, this behaviour prevents the application from working correctly. For instance, it becomes impossible to confirm a deletion.

The following controller provides a work around. It injects some javascript into the page wrapping the call to confirm(). The new code detects when the Prevent this page from creating additional dialogs checkbox has been checked and returns true instead. The confirmation window still does not appear, but the XAF application works as if the user had pressed confirm instead of cancel.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77

using DevExpress.ExpressApp;
using DevExpress.ExpressApp.Web;
using DevExpress.Web.Internal;

namespace NetModule.Web.Controllers
{
    public class HandleDisabledConfirmationsController : Controller
    {
        public static bool IsChrome
        {
            get
            {
                return RenderUtils.Browser.IsChrome;
            }
        }

        public static bool IsFirefox
        {
            get
            {
                return RenderUtils.Browser.IsFirefox;
            }
        }

        protected override void OnFrameAssigned()
        {
            base.OnFrameAssigned();
            if (IsChrome || IsFirefox)
            {
                WebWindow window = Frame as WebWindow;
                if (window != null)
                {
                    window.CustomRegisterTemplateDependentScripts += window_CustomRegisterTemplateDependentScripts;
                }
            }
        }

        private void window_CustomRegisterTemplateDependentScripts(object sender, CustomRegisterTemplateDependentScriptsEventArgs e)
        {
            // wrapper for 'confirm'
            WebWindow window = (WebWindow)sender;

            // Detect the user has checked the 'prevent this page from creating additional dialogs'.
            // In which case assume all confirmations are accepted, rather than the default rejected
            var confirmWrapper = @"
<script>
window.nativeConfirm = window.confirm;
window.confirm = function(message)
{
    var timeBefore = new Date();
    var confirmBool = nativeConfirm(message);
    var timeAfter = new Date();
    if ((timeAfter - timeBefore) < 350)
    {
        confirmBool = true;
    }
    return confirmBool;
}
</script>";

            e.Page.ClientScript.RegisterClientScriptBlock(GetType(), "WrapConfirmations", confirmWrapper);
        }

        protected override void OnDeactivated()
        {
            if (IsChrome || IsFirefox)
            {
                WebWindow window = Frame as WebWindow;
                if (window != null)
                {
                    window.CustomRegisterTemplateDependentScripts -= window_CustomRegisterTemplateDependentScripts;
                }
                base.OnDeactivated();
            }
        }
    }
}

The controller works by timing the milliseconds to close the confirmation window. If it is less than 350 milliseconds we can assume the confirmation window never opened owing to the checkbox. In this scenario, it returns true (confirm) rather than false (cancel) in order for XAF to function correctly.

Somewhere along the way I came across a Chrome extension written by Allan Rempel which would do this in one step. Unfortunately it stopped working when chess.com upgraded its site and Allan does not seem to be maintaining it any longer. I decided to jump in and try to fix it and maybe add a couple of features myself.

The extension

When a chess.com game is finished, you can click on a little knight icon and jump straight to the analysis of the game in Lichess.

It takes a few seconds to process but then you can see all your mistakes and blunders.

You can play through the game and experiment with different variations. Chess.com’s new v3 has greatly improved its own post-game analysis but it still lacks many of the features of the Lichess one.

Try it out! The extension is available here.

How to modify an existing extension

First thing was to find the code of the existing broken extension. A bit of googling lead me to:

$ cd ~/Library/Application\ Support/Google/Chrome/Default/Extensions/
$ ls

total 0
drwx------+ 30 ra  staff  1020 29 Feb 20:39 .
drwx------+ 66 ra  staff  2244 29 Feb 20:56 ..
drwx------+  2 ra  staff    68 29 Feb 20:39 Temp
drwx------+  3 ra  staff   102 14 Sep  2012 allibancfngcpjmjnnboebggipbaalhn
drwx------+  3 ra  staff   102 28 Mar  2013 anfemphbgknehemmbbajjcmakfijiohp
drwx------+  3 ra  staff   102 25 Oct 08:15 apdfllckaahabafndbhieahigkjlhalf
drwx------+  3 ra  staff   102  5 Mar  2015 bepbmhgboaologfdajaanbcjmnhjmhfn
drwx------+  3 ra  staff   102  5 Jun  2014 bfbameneiokkgbdmiekhjnmfkcnldhhm
drwx------+  3 ra  staff   102 22 Feb 20:09 bhjlkimpkkgkmfjlcfngmakenalgleap
drwx------+  3 ra  staff   102 24 Sep 20:33 blpcfgokakmgnkcojhhkbfbldkacnbeo
drwx------+  3 ra  staff   102 29 Jul  2013 bmihblnpomgpjkfddepdpdafhhepdbek
drwx------+  3 ra  staff   102 13 Sep  2012 boeajhmfdjldchidhphikilcgdacljfm
... etc.

Each extension is in a not very helpfully named folder. If you open chrome://extensions/, you can find the id of any installed extension and work out which directory corresponds to which extension.

So then I dug around in the folder and found ext/background.js where the bulk of the code lies. There is also a manifest.json file which is specifies various parameters such as what permissions are required and which websites will be affected by the extension. I copied all the important files into a development folder.

Loading the extension from the development folder.

• Navigate to chrome://extensions
• Expand the developer dropdown menu and click “Load Unpacked Extension”
• Navigate to development folder

Assuming there are no errors, the extension should load into your browser. Don’t forget to disable any conflicting versions you’ve installed from the Chrome Web Store.

I used the Chrome’s built-in Developer Tools and peppered the code with console.log() to explore how it worked. There are plenty of good resources on the Chrome developer site. I managed to fix the broken parts and added support for analysis from chess.com’s game archive. Then I posted the link on reddit/r/chess.

The source code for the extension is on GitHub.

In the previous posts, the PasswordGenerator always returned ‘Password’. Instead each date should corresponds to a new unique password. We’ll make use of the Crypto-js node package and we’ll see that the AWS lambda copes just fine.

Installing a node package

Pull in the crypto-js package into the serverless component.

$ cd nodejscomponent/
$ npm install crypto-js --save
crypto-js@3.1.6 node_modules/crypto-js

Now we need the typescript definitions. Watch out there are two different TypeScript typings called cryptojs and crypto-js. The first one is more complete.

$ typings install cryptojs --ambient --save
? Found cryptojs typings for DefinitelyTyped. Continue? Yes
Installing cryptojs@~3.1.2 (DefinitelyTyped)...

cryptojs
└── (No dependencies)

I’m not sure why, but there’s no export in the typings file for cryptojs. Add the following to the bottom of the cryptojs.d.ts file.

1
2
3
4

  ...
declare module "crypto-js" {
    export = CryptoJS;
}

Improved PasswordGenerator

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

/// <reference path="../typings/main.d.ts" />
import CryptoJS = require("crypto-js");

export function checkPotd(password : string) : boolean
{
    return new PasswordGenerator().check(password);
}

export class PasswordGenerator
{
  generate(date: Date) : string
  {
        // Get the current date as a YYYYMMDD string
        var yyyy = date.getFullYear().toString();
        var mm = (date.getMonth()+1).toString(); // getMonth() is zero-based
        var dd  = date.getDate().toString();
        var plain = `${yyyy}${mm}${dd}`;

        // Using AES CTR with 32 byte key and iv ensures the encrypted string is not too long
        // See http://stackoverflow.com/a/13298019/1077279

        var key = CryptoJS.enc.Hex.parse('108c786594543687891374723e809ec5e475a8361f7ad82df04e91ba2c139321');
        // Use a different initialization vector each time by using the date as part of the vector
        var iv  = CryptoJS.enc.Hex.parse(plain + '3a8fe4440be1e113a271574f379d70a76c3477aaff036d1e83fcd4b9');
        var options = { mode: CryptoJS.mode.CTR, padding: CryptoJS.pad.NoPadding, iv: iv };

        var encrypted = CryptoJS.AES.encrypt(plain, key, options);

      return encrypted.ciphertext.toString();
  }    
  
  check(password : string) : boolean
  {
        // check the value matches today's password of the day
      return password == this.generate(new Date());
  }
}

Run the tests

We expect the tests to fail now since we are no longer returning the same password.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

$ npm test
> @0.0.1 pretest /Users/ra/Projects/Coprocess/serverlessPotd/nodejscomponent
> tsc

> @0.0.1 test /Users/ra/Projects/Coprocess/serverlessPotd/nodejscomponent
> mocha ./lib/test

  Generator
    #generate
      1) should generate the password
    #check
      ✓ should return false when the password is incorrect
    #check
      2) should return true when the password is correct

  1 passing (16ms)
  2 failing

  1) Generator #generate should generate the password:
     Error: Expected 'Password' but was bb4bde4d76b055
      at Context.<anonymous> (lib/test/passwordOfTheDayTest.js:12:23)

  2) Generator #check should return true when the password is correct:
     Error: Expected 'true' but was false
      at Context.<anonymous> (lib/test/passwordOfTheDayTest.js:28:23)

npm ERR! Test failed.  See above for more details.

Update and improve the tests

It’s getting a little more complicated so let’s pull in chai which is a pretty assertions library.

1
2
3
4
5

$ npm install chai --save-dev
chai@3.5.0 node_modules/chai
├── assertion-error@1.0.1
├── type-detect@1.0.0
└── deep-eql@0.1.3 (type-detect@0.1.1)

And the Typescript definitions for chai.

1
2
3
4
5
6

$ typings install chai --save --ambient
? Found chai typings for DefinitelyTyped. Continue? Yes
Installing chai@~3.4.0 (DefinitelyTyped)...

chai
└── (No dependencies)

Now let’s flesh out the tests for the PasswordGenerator class.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58

/// <reference path="../../typings/main.d.ts" />
import PasswordOfTheDay = require("../passwordOfTheDay");
import Chai = require("chai");

// Tell chai that we'll be using the "should" style assertions.
Chai.should();

describe("Generator", () => {
    var subject : PasswordOfTheDay.PasswordGenerator;

    beforeEach(function () {
        subject = new PasswordOfTheDay.PasswordGenerator();
    });

    describe("#generate", () => {
        it("should generate the password when the date is 24th July 2010", () => {
            var date : Date = new Date(2010, 6, 24);
            var password : string = subject.generate(date);

            password.should.equal("92ab1ff89bf9af");
        });
    });

    describe("#generate", () => {
        it("should generate a different password when the date is 25th July 2010", () => {
            var date : Date = new Date(2010, 6, 25);
            var password : string = subject.generate(date);

            password.should.equal("26a394b21800f1");
        });
    });
    describe("#check", () => {
        it("should return false when the password is incorrect", () => {
            var password : string = "garbage";
            var result : boolean = subject.check(password);

            result.should.be.false;
        });
    });

    describe("#check", () => {
        it("should return false when the password is null", () => {
            var password : string = null;
            var result : boolean = subject.check(password);

            result.should.be.false;
        });
    });

    describe("#check", () => {
        it("should return true when the password is correct", () => {
            var password : string = subject.generate(new Date());
            var result : boolean = subject.check(password);

            result.should.be.true;
        });
    });
});

Run our tests:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

$ npm test
> @0.0.1 pretest /Users/ra/Projects/Coprocess/serverlessPotd/nodejscomponent
> tsc

> @0.0.1 test /Users/ra/Projects/Coprocess/serverlessPotd/nodejscomponent
> mocha ./lib/test

  Generator
    #generate
      ✓ should generate the password when the date is 24th July 2010
    #generate
      ✓ should generate a different password when the date is 25th July 2010
    #check
      ✓ should return false when the password is incorrect
    #check
      ✓ should return false when the password is null
    #check
      ✓ should return true when the password is correct


  5 passing (20ms)

Deploy

$ serverless dash deploy

Now when we visit the endpoint with the correct password for today’s date which happens to be 89366e6199f3.

https://...amazonaws.com/dev/potd/check?password=89366e6199f3

1
2
3

{
    message: true
}

Mission accomplished! Notice that using a node package did not require any special steps on the AWS side. In fact we have not had to login to AWS since the very beginning when we created an IAM user for the project. And yet we’ve managed to build and deploy a cheap and scalable cloud-based service.

The source code is on GitHub. Note the default .gitignore file skips the admin.env file which contains the (sensitive) AWS keys in it so don’t forget to add your own.

That wraps up my series on building a small, but real-world Serverless application. In a future post, I’d like to look at providing a secured ‘generate’ service to allow authorized users to get today’s password. Stay tuned.

New version 0.3.1

Edit: since the original version of this post, a new version 0.3.1 of Serverless was released. I have updated the tutorial below to reflect the newer version. Also, TSD has been deprecated in favour of Typings so I’ve updated to use Typings instead. All parts have been updated for the latest version of the framework 0.3.1.

The Password of the Day Generator class

First up we need a class to generate and check the password of the day. For the moment, let’s pretend the password of the day is always the string “Password”. Put the following typescript class in nodejscomponent/src.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

export function checkPotd(password : string) : boolean
{
    return new PasswordGenerator().check(password);
}

export class PasswordGenerator
{
  generate(date: Date) : string
  {
        // generate today's password
      return "Password";
  }    
  
  check(password : string) : boolean
  {
        // check the value matches today's password of the day
      return password == this.generate(new Date());
  }
}

Now add a mocha test for it.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

/// <reference path="../../typings/main.d.ts" />
import PasswordOfTheDay = require("../passwordOfTheDay");

describe("Generator", () => {
    var subject : PasswordOfTheDay.PasswordGenerator;

    beforeEach(function () {
        subject = new PasswordOfTheDay.PasswordGenerator();
    });

    describe("#generate", () => {
        it("should generate the password", () => {
            var result : string = subject.generate(new Date(2010, 6, 24));
            if (result !== "Password") {
                throw new Error("Expected 'Password' but was " + result);
            }
        });
    });

    describe("#check", () => {
        it("should return false when the password is incorrect", () => {
            var result : boolean = subject.check("garbage");
            if (result !== false) {
                throw new Error("Expected 'false' but was " + result);
            }
        });
    });

    describe("#check", () => {
        it("should return true when the password is correct", () => {
            var result : boolean = subject.check("Password");
            if (result !== true) {
                throw new Error("Expected 'true' but was " + result);
            }
        });
    });

});

Now compile everything.

$ cd nodejscomponent
$ tsc

You will now find that there is a corresponding javascript file in the lib folder

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

function checkPotd(password) {
    return new PasswordGenerator().check(password);
}
exports.checkPotd = checkPotd;
var PasswordGenerator = (function () {
    function PasswordGenerator() {
    }
    PasswordGenerator.prototype.generate = function (date) {
        // generate today's password
        return "Password";
    };
    PasswordGenerator.prototype.check = function (password) {
        // check the value matches today's password of the day
        return password == this.generate(new Date());
    };
    return PasswordGenerator;
})();
exports.PasswordGenerator = PasswordGenerator;

And likewise for the mocha test in lib/test. Now to run those tests:

$ npm test

> @0.0.1 pretest /Users/ra/Projects/Coprocess/serverlessPotd/nodejscomponent
> tsc

> @0.0.1 test /Users/ra/Projects/Coprocess/serverlessPotd/nodejscomponent
> mocha ./lib/test

Generator
    #generate
    ✓ should generate the password
    #check
    ✓ should return false when the password is incorrect
    #check
    ✓ should return true when the password is correct

3 passing (10ms)

Nice. Next, modify the main entry point of the component index.js.

1
2
3
4
5
6
7
8
9
10
11
12

// Dependencies
var PasswordOfTheDay = require('./passwordOfTheDay');

module.exports.respond = function(event, cb) {

  var result = PasswordOfTheDay.checkPotd(event.password);
  var response = {
    message: result
  };

  return cb(null, response);
};

Notice how we make use of event.password which is the parameter we configured in part 3 in the s_function.json file.

Let’s deploy!

$ serverless dash deploy
_______                             __
|   _   .-----.----.--.--.-----.----|  .-----.-----.-----.
|   |___|  -__|   _|  |  |  -__|   _|  |  -__|__ --|__ --|
|____   |_____|__|  \___/|_____|__| |__|_____|_____|_____|
|   |   |             The Serverless Application Framework
|       |                           serverless.com, v0.3.1
`-------'

Use the <up>, <down>, <pageup>, <pagedown>, <home>, and <end> keys to navigate.
Press <enter> to select/deselect, or <space> to select/deselect and move down.
Press <ctrl> + <enter> to immediately deploy selected.


Serverless: Select the assets you wish to deploy:
    nodejscomponent - potd - check
    function - nodejscomponent/potd/check
    endpoint - nodejscomponent/potd/check@potd/check~GET
    - - - - -
> Deploy

Serverless: Deploying functions in "dev" to the following regions: eu-west-1  
Serverless: ------------------------  
Serverless: Successfully deployed functions in "dev" to the following regions:   
Serverless: eu-west-1 ------------------------  
Serverless:   nodejscomponent/potd/check: arn:aws:lambda:eu-west-1:962613113552:function:serverlessPotd-nodejscomponent-potd-check:dev  

And lets visit that URI

https://rhnjv4ms2b.execute-api.eu-west-1.amazonaws.com/development/potd/check?password=nonsense

1
2
3

{
    message: false
}

https://rhnjv4ms2b.execute-api.eu-west-1.amazonaws.com/development/potd/check?password=Password

1
2
3

{
    message: true
}

Rock and roll. A working password checker running on Lambda in the Amazon cloud.

Next up - we’ll extend the PasswordGenerator class to pull in a node package and generate a better password.

The source code so far is on GitHub. Note the default .gitignore file skips the admin.env file which contains the (sensitive) AWS keys in it so don’t forget to add your own.

In this post, we’ll discuss how a Serverless function actually works.

The guts of a serverless function

When we visited the deployed endpoint at the end of part 1, it correctly returned some JSON content.

1
2
3

{
    message: "Your Serverless function ran successfully!"
}

Where does this message come from? Look at index.js in the component’s lib folder.

1
2
3
4
5
6
7
8
9
10
11
12

/**
 * Lib
 */

module.exports.respond = function(event, cb) {

  var response = {
    message: "Your Serverless function ran successfully!"
  };

  return cb(null, response);
};

And it’s the handler.js file in the function’s subfolder which calls it.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

'use strict';

/**
 * Serverless Module: Lambda Handler
 * - Your lambda functions should be a thin wrapper around your own separate
 * modules, to keep your code testable, reusable and AWS independent
 * - 'serverless-helpers-js' module is required for Serverless ENV var support.  Hopefully, AWS will add ENV support to Lambda soon :)
 */

// Require Serverless ENV vars
var ServerlessHelpers = require('serverless-helpers-js').loadEnv();

// Require Logic
var lib = require('../../lib');

// Lambda Handler
module.exports.handler = function(event, context) {

  lib.respond(event, function(error, response) {
    return context.done(error, response);
  });
};

In our case we’re coding a password checking function. The URI will look something like this:

http://something.amazonaws.com/development/potd/check?password=P455w0rd

We’ll modify lib/index.js to retrieve the value from the query parameter password and return true if the password is correct and false otherwise. But first we need to set up the parameter.

Configuring the function parameter

In each function’s directory, there is a file named s-function.json which allows you to specify the details of the function call. Add a section to the requestTemplates as follows:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

{
  "name": "check",
  "handler": "potd/check/handler.handler",
  "timeout": 6,
  "memorySize": 1024,
  "custom": {
    "excludePatterns": [],
    "envVars": []
  },
  "endpoints": [
    {
      "path": "potd/check",
      "method": "GET",
      "authorizationType": "none",
      "apiKeyRequired": false,
      "requestParameters": {},
      "requestTemplates": {
+       "application/json": {
+           "password": "$input.params('password')"
+       }
      },
      "responses": {
        "400": {
          "statusCode": "400"
        },
        "default": {
          "statusCode": "200",
          "responseParameters": {},
          "responseModels": {},
          "responseTemplates": {
            "application/json": ""
          }
        }
      }
    }
  ]
}

Note that the s-function.json file is also where you can configure if the service accepts POST or PUT or DELETE requests. Here we are only interested in GET.

You can easily tailor the requestTemplates in this file to extract whatever parameters you need in your lambda function.

Retrieving the parameter value

Now back in index.js you will find that the function’s event parameter has a property password which is set to the value of the querystring parameter.

1
2
3
4
5
6
7
8
9

module.exports.respond = function(event, cb) {

  var parameterValue = event.password; // the querystring parameter
  var response = {
    message: parameterValue
  };

  return cb(null, response);
};

Redeploy.

 $ serverless dash deploy

Visit the URI.

http://something.amazonaws.com/development/potd/check?password=P455w0rd

The response is:

1
2
3

{
    message: "P455w0rd"
}

Ready for implementation

We now have all the pieces we need. Serverless, Typescript, Mocha and AWS. In the next post I’ll show how to wire up everything get it working.

This is the second part of a series about the Serverless framework. Read the first part to get up and running.

First I’ll describe the webservice I’m building. Then we’ll configure our environment for Typescript and Mocha testing.

Poor man’s dual factor authentication via a password of the day

I’m the technical lead for an enterprise application which is in use by about 100 large multinational corporates. As part of the installation process, we ask for a registration code which is based on the date. The customer has to call us to get the password of the day. This gives us an opportunity to engage with the customer and also gives us little more control. It’s a simple form of dual factor authorization where one of the factors requires a phone call.

In the old days, the routine for checking the validity of the password was part of the source code, but we’ve since moved the checking function to a web service.

It’s my goal to replace this ‘password of the day’ check function with a Serverless module. The service will take a password as input and check that it matches the password of the day.

It’s a tiny, simple, rarely-used web service but AWS lambda is still a great fit for it. Although lambda can scale if necessary, in this case it’s about not having the hassle of administering a server.

Mocha and TypeScript

New version 0.3.1

Edit: since the original version of this post, a new version 0.3.1 of Serverless was released. I have updated the tutorial below to reflect the newer version. Also, TSD has been deprecated in favour of Typings so I’ve updated to use Typings instead.

Let’s do things properly and set up a testing framework.

Make sure you’re in the component folder.

$ cd nodejscomponent

Then we’ll install Mocha.

$ npm install mocha --save-dev
mocha@2.4.5 node_modules/mocha
├── escape-string-regexp@1.0.2
├── commander@2.3.0
├── diff@1.4.0
├── supports-color@1.2.0
├── growl@1.8.1
├── debug@2.2.0 (ms@0.7.1)
├── mkdirp@0.5.1 (minimist@0.0.8)
├── jade@0.26.3 (commander@0.6.1, mkdirp@0.3.0)
└── glob@3.2.3 (inherits@2.0.1, graceful-fs@2.0.3, minimatch@0.2.14)

Next we’ll install TypeScript. Of course you can use plain javascript if you prefer. My background is C#: I make fewer dumb mistakes with TypeScript. It looks like there is a typescript plugin in the pipeline which will make typescript integration even easier in the future, but for now:

$ npm install typescript --save
typescript@1.7.5 node_modules/typescript

$ npm install typings -g
/usr/local/bin/typings -> /usr/local/lib/node_modules/typings/dist/bin/typings.js
typings@0.6.6 /usr/local/lib/node_modules/typings
├── array-uniq@1.0.2
├── elegant-spinner@1.0.1
├── thenify@3.2.0
├── popsicle-status@1.0.1
... etc ...

Initialise typings.

$ typings init
-> written typings.json

Now we add the type definitions for Mocha.

$ typings install mocha --ambient --save
? Found mocha typings for DefinitelyTyped. Continue? Yes
Installing mocha@~2.2.5 (DefinitelyTyped)...

mocha
└── (No dependencies)

We need a config file for the TypeScript compiler. This goes in the same module directory (back/modules/potd in my case).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

{
    "compilerOptions": {
        "module": "commonjs",
        "target": "es5",
        "noImplicitAny": true,
        "sourceMap": false,
        "declaration": false,
        "outDir": "lib"
    },
    "exclude": [
        "node_modules",
        "typings/browser",
        "typings/browser.d.ts"
    ]
}

Now make a subdirectory for our TypeScript source files.

$ mkdir src
$ mkdir src/test

Next up we need to modify the package.json file to add a scripts section. Only the "scripts" section needs changing.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

{
  "name": "potd",
  "version": "0.0.1",
  "description": "Dependencies for a Password of the day Serverless Module",
  "author": "Robert Anderson",
  "license": "MIT",
  "private": true,
  "repository": {
    "type": "git",
    "url": "git://github.com/"
  },
  "keywords": [],
  "devDependencies": {
    "mocha": "^2.3.4"
  },
  "dependencies": {
    "serverless-helpers-js": "~0.0.3",
    "typescript": "^1.7.5"
  },
  "scripts": {
    "prepublish": "tsc",
    "pretest": "tsc",
    "test": "mocha ./lib/test"
  }
}

We’ve finished setting up everything for TypeScript and Mocha. Whenever you run tsc, any TypeScript files in /src will get compiled to javascript in /lib. And running npm test will compile and then run any Mocha tests in /lib/test.

The source code so far is on GitHub. Note the default .gitignore file skips the admin.env file which contains the (sensitive) AWS keys in it so don’t forget to add your own.

In the next post we’ll create a TypeScript class for the guts of the lambda function which checks the password of the day, along with some corresponding Mocha tests, also written in TypeScript.

Edit: since the original version of this post, a new version 0.3.1 of Serverless was released. I have updated the tutorial below to reflect the newer version.

I was in the middle of a blog post about the JAWS framework and before I had finished it changed its name to the Serverless framework. It is a very clever way to build apps without worrying about provisioning server or whether it will scale. This is because it uses Amazon Web Services and in particular the Amazon lambda compute service. It’s currently in beta.

Follow the instructions for setting up an administrative IAM user for use with the framework.

Make sure you have node and npm installed. You need node 4.0 or greater.

$ node -v
v4.2.3
$ npm -v
2.14.7

Install the Serverless framework.

$ npm install serverless -g 

Create a new project

$ serverless project create

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

 _______                             __
|   _   .-----.----.--.--.-----.----|  .-----.-----.-----.
|   |___|  -__|   _|  |  |  -__|   _|  |  -__|__ --|__ --|
|____   |_____|__|  \___/|_____|__| |__|_____|_____|_____|
|   |   |             The Serverless Application Framework
|       |                           serverless.com, v0.3.1
`-------'

Serverless: Initializing Serverless Project...  
Serverless: Enter a name for this project:  (serverless-vyedql) serverlessPotd
Serverless: Enter a universally unique project bucket name:  (serverless-vyedql-4jboce.com) potd-zerosharp.com
Serverless: Enter an email to use for AWS alarms:  (me@serverless-vyedql.com) potd@nosredna.com
Serverless: Select a region for your project: 
    us-east-1
    us-west-2
  > eu-west-1
    ap-northeast-1
Serverless: Select an AWS profile for your project: 
  > default
Serverless: Creating stage "dev"...  
Serverless: Creating region "eu-west-1" in stage "dev"...  
Serverless: Creating your project bucket on S3: serverless.eu-west-1.potd-zerosharp.com...  
Serverless: Deploying resources to stage "dev" in region "eu-west-1" via Cloudformation (~3 minutes)...  
Serverless: Successfully deployed "dev" resources to "eu-west-1"  
Serverless: Successfully created region "eu-west-1" within stage "dev"  
Serverless: Successfully created stage "dev"  
Serverless: Successfully initialized project "serverlessPotd"  

It takes about 3 minutes to setup the necessary CloudFormation stack for your project. Change directory to the newly created project.

$ cd serverlessPotd

Create a new component.

$ serverless component create

1
2
3
4
5
6
7
8
9
10
11

Serverless: Enter a name for your new component:  (nodejscomponent) 
Serverless: Enter a name for your component's first module:  (resource) potd
Serverless: Enter a name for your module's first function:  (show) check
Serverless: Successfully created function: "check"  
Serverless: Successfully created new serverless module "potd" inside the component "nodejscomponent"  
Serverless: Installing "serverless-helpers" for this component via NPM...  
Serverless: -----------------  
serverless-helpers-js@0.0.3 node_modules/serverless-helpers-js
└── dotenv@1.2.0
Serverless: -----------------  
Serverless: Successfully created new serverless component: nodejscomponent  

This has created the javascript code for a basic lambda function which we can immediately deploy.

$ serverless dash deploy

At the prompt select both the function and the endpoint and then select Deploy.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

 _______                             __
|   _   .-----.----.--.--.-----.----|  .-----.-----.-----.
|   |___|  -__|   _|  |  |  -__|   _|  |  -__|__ --|__ --|
|____   |_____|__|  \___/|_____|__| |__|_____|_____|_____|
|   |   |             The Serverless Application Framework
|       |                           serverless.com, v0.3.1
`-------'

Use the <up>, <down>, <pageup>, <pagedown>, <home>, and <end> keys to navigate.
Press <enter> to select/deselect, or <space> to select/deselect and move down.
Press <ctrl> + <enter> to immediately deploy selected.


Serverless: Select the assets you wish to deploy:
    nodejscomponent - potd - check
      function - nodejscomponent/potd/check
      endpoint - nodejscomponent/potd/check@potd/check~GET
    - - - - -
  > Deploy

Serverless: Deploying functions in "dev" to the following regions: eu-west-1  
Serverless: ------------------------  
Serverless: Successfully deployed functions in "dev" to the following regions:   
Serverless: eu-west-1 ------------------------  
Serverless:   nodejscomponent/potd/check: arn:aws:lambda:eu-west-1:962613113552:function:serverlessPotd-nodejscomponent-potd-check:dev  

Serverless: Deploying endpoints in "dev" to the following regions: eu-west-1  
Serverless: Successfully deployed endpoints in "dev" to the following regions:  
Serverless: eu-west-1 ------------------------  
Serverless:   GET - potd/check - https://rhnjv4ms2b.execute-api.eu-west-1.amazonaws.com/dev/potd/check  

Now open a browser and navigate to the URL in the last line. You should see the following JSON response.

1
2
3

{
    message: "Your Serverless function ran successfully!"
}

Now that’s already fantastic. With a handful of commands we have deployed an arbitrary javascript function to a URL endpoint very cheaply and with automatic scaling. We never had to consider instance size or memory or operating system.

And it’s extremely extensible too - thanks to the other AWS services, we can easily make it secure (with Amazon Cognito) kick off emails (SES), store files (S3), add persistence (DynamoDB), etc.

In the next post, I’ll be applying it to a real life scenario to replace an existing web service.

Batch editing

Another feature I’m excited about is the support for batch editing within the web application grids.

Let’s see what happens when combined with the validation rules. What happens if I edit two rows but only one has a validation problem - does the whole batch get rejected? Or just the row with the problem?

First I modified the edit mode of the Tasks grid to Batch via the model.

Then I added a new RuleRequiredField validation rule via the model.

I started up the MainDemo application and navigated to the Tasks view and tried to delete the subject of multiple rows at the same time.

I never even got to press Save because of another new 15.2 feature: inplace validation! The rules are being validated without a round trip to the server!

So let’s try another way. Inplace Validation does not work with all rule types, so I deleted my RuleRequiredField from the model and instead added a new RuleFromBoolProperty to the DemoTask object as follows:

1
2
3
4
5
6
7
8
9
10
11
12
13

[MemberDesignTimeVisibility(false)]
[RuleFromBoolProperty("SubjectIsRequired", 
    DefaultContexts.Save, 
    "Subject is required.", 
    UsedProperties = "Subject", 
    SkipNullOrEmptyValues = false)]
public bool IsSubjectRequired
{
    get
    {
        return !String.IsNullOrWhiteSpace(Subject);
    }
}

Now we go back to the list view for Task and try to set multiple subjects to empty.

Fantastic! This time the broken rules appear for each row and it is quite clear which message belongs to which object.

So the batch editing makes use of inplace validation when it can, but handles more complex validation rules well too. This is an excellent combination because the inpalce validation will help to make the client seem very quick and responsive.

Conclusion

This concludes my review of DevExpress 2015.2. We’ve looked in some detail at two of the most impressive new features - the way reports are serialised and the improvements to client-side validation. These changes help with performance and ease of maintenance and I’m very happy to see that the DevExpress team has focused on these areas. Looking forward to the 2016 releases!

There are already some good blog posts about the changes:

• Michael Bogaerts
• Gustavo Marzioni
• DevExpress What’s New

Rather than repeat general overviews provided in these, this two-part blog post is more of a ‘deep dive’. In particular I’ll be looking at two of the new features in the expressApp Framework (XAF). Today I’ll cover the new XML serialisation in the report designer. Tomorrow’s post will examine the new batch editing features.

Reports

Reporting is definitely one of the areas where XAF has progressed the most in recent versions. There is now an in-browser report designer (since 14.2) and an alternative implementation of the reports (reports v2, since 13.2).

In this release, I was a little worried that the support for Reports v1 would be deprecated, since our production system has over 100 custom reports and we have not yet looked for an easy way to migrate these to v2. I’m happy to report that v1 reports are still very much present.

However there are also a number of new features in v2 Reports, not least of which is the ability to store the report’s layout in XML. There’s currently not a lot of documentation about the XML serialization, so let’s dig in and see what we can discover.

XML serialization deep dive

First I ran the demo and ran the Copy Predefined Report on the Contacts Report.

Then I ran the report designer and added a dummy OnBeforePrint() script to one of the table’s cells.

1
2
3

private void xrTableCell8_BeforePrint(object sender, System.Drawing.Printing.PrintEventArgs e) {
  var x = 23;
}

Then, in the MainDemo’s Updater.cs file, I placed the following code.

1
2
3
4
5
6
7

var reportData = ObjectSpace.FindObject<ReportDataV2>(new BinaryOperator("DisplayName", "Contacts Report") & new BinaryOperator("IsPredefined", new OperandValue(false)));
if (reportData != null)
{
    var report = ReportDataProvider.ReportsStorage.LoadReport(reportData);
    report.SaveLayoutToXml(@"C:\Temp\ContactsReport.xml");
    report.SaveLayout(@"C:\Temp\ContactsReport.repx");
}

This creates two output files. One of them contains the familiar .repx format. The other contains the newer .xml serialization.

The contents of the XML file is displayed below and is 77 lines long.

By comparison, the ContactReport.repx is 408 lines long and much harder to read.

As you can see the XML file is much smaller and simpler than the .repx file. At first I didn’t believe it contained all the necessary information, so I started up the MainDemo WinForms application, created a blank new report and imported the layout and the layout looks correct and the preview loads with data as expected.

Scripts

What about scripts? Are they serialized in the xml version? You bet. In the XML export you can see this has been serialised near the top of the file.

1
2
3

ScriptsSource=
   "
private void xrTableCell8_BeforePrint(object sender, System.Drawing.Printing.PrintEventArgs e)
   {
  var x = 23;
}
"

Setup

In order to use this new XML serialization of report layouts, you must set the ReportsV2 module to use it.

1

reportsModuleV2.ReportStoreMode = ReportStoreModes.XML

This will most likely invalidate any reports which have already been serialized to the database. There are some notes on how to fix this here.

Default for new projects

What about new projects? I created a new solution and chose DevExpress 15.2 XAF Solution Wizard as the type of solution and added the reports module. Now when I navigate to the WinApplication.Designer.cs file, I find:

1
2
3
4
5

//
// reportsModuleV2
//
this.reportsModuleV2.ReportDataType = typeof(DevExpress.Persistent.BaseImpl.ReportDataV2);
this.reportsModuleV2.ReportStoreMode = DevExpress.ExpressApp.ReportsV2.ReportStoreModes.XML;

So the default storage method for new projects is now XML.

Conclusion

The XML serialisation looks like a considerable upgrade to the mechanism for storing, loading and saving reports. Now I just need to find a good way of converting my existing v1 reports… Perhaps a future blog post.

Coming up

Tomorrow I’ll be looking at the new batch editing in grids in more detail.