Sometimes there is a situation where there is a client application (SPA, or native mobile app for instance), which needs to access an API (resource). Let's say that API requires that users be authenticated using an OAuth service. In this case the user (resource owner) will login with an authentication service via the client app. If successful the authentication service will issue an access token, and the client application can use this in an Authentication header to access the resource.

So far so good. This is standard behaviour of something like an Authorization Code grant type. Now let's say that the API needs to call another API, ok we should use a Client Credentials grant type. This is what we would usually use for server to server calls, as the server can be trusted so we can just use a shared secret. But what if API one needs to all API two using the resource owners claims? We could just proxy the Bearer Authorization header through and API two would be none the wiser, but this is dishonest, as we probably want to apply security policies to API two, and give API one the scopes to access certain endpoints.

We can solve this issue by using a three legged authorization flow. In this flow the user will authenticate as normal and the client application will access API one with the usual header. When API one needs to access API two, API one can take the bearer token and swap it for a new one which will identify API one correctly to API two, but will also contain the resource owners claims. This is done by using a custom grant type in Identity Server 4.

If you just want the code it's on GitHub

Setting up Identity Server 4

In order to start using this custom grant, first we need to set up some resources and clients in Identity Server 4.

Resources

We will have two protected resources in this example. Each resource will have scopes depending on what we can do with them. For this example let's give each resource one scope.

API One will look like:

{
    "name": "apione",
    "scopes" : [
        "apione-full"
    ]
}

API Two will look like:

{
    "name": "apitwo",
    "scopes" : [
        "apitwo-readonly"
    ]
}

Clients

Now we have the resources defined that we want to access, let's set up some clients. For our example we will need two.

The first will be for our native or web client application

{
    "client_id": "native-client",
    "allowed_grant_types": [
        "authorization_code"
    ],
    "allowed_scopes": [
        "apione-full"
    ]
}

The second will be for Api One

{
    "client_id": "apione",
    "allowed_grant_types": [
        "delegation"
    ],
    "allowed_scopes": [
        "apitwo-readonly"
    ],
    "client_secrects" : [
        "sdkfhsdfhsdhfshfskdhf"
    ]
}

From the config above we can see that there is pretty standard configuration for the native client. It's configured with an authorization code grant type, meaning the resource owner will be redirected to the authentication server login page and be prompted for their credentials, also it's only allowed to ask for the scope apione-full. The apione client on the other hand is configured to use a custom grant type. We also configure it with a secret so that it can identify itself.

Custom Grant Type

We've configured API one to only be allowed to use a grant type of delegate. This is a custom grant type and we could have called it anything we liked. Because it's custom, Identity Server will not know how to process it out of the box. We need to write some code, and configure it to know what to do with this grant type. Identity Server 4 allows us to implement custom grant type handlers by implementing the IExtensionGrantValidator interface. An example of what our delegate grant type handler might look like this:

public class DelegationGrantValidator : IExtensionGrantValidator
{
    private readonly ITokenValidator _validator;

    public DelegationGrantValidator(ITokenValidator validator)
    {
        _validator = validator;
    }

    public string GrantType => "delegation";

    public async Task ValidateAsync(ExtensionGrantValidationContext context)
    {
        var userToken = context.Request.Raw.Get("token");

        if (string.IsNullOrEmpty(userToken))
        {
            context.Result = new GrantValidationResult(TokenRequestErrors.InvalidGrant);
            return;
        }

        var result = await _validator.ValidateAccessTokenAsync(userToken);
        if (result.IsError)
        {
            context.Result = new GrantValidationResult(TokenRequestErrors.InvalidGrant);
            return;
        }

        var sub = result.Claims.FirstOrDefault(c => c.Type == "sub")?.Value;

        if (string.IsNullOrEmpty(sub))
        {
            context.Result = new GrantValidationResult(TokenRequestErrors.InvalidGrant);
            return;
        }

        context.Result = new GrantValidationResult(sub, "delegation");
    }
}

This example is taken from the docs.

Here we can see that we will read the value of token from the request payload, and pass that into the validator. This value is given to us by whatever code will be calling the authentication server, and is expected to be the resource owners access token. It's passed into the validation function and provided it's valid the code will then read the sub claim out of it and return a new validation result for that subject. In this case the sub will be the resource owner.

Calling API Two

In order for API One to authenticate and receive it's own access token for API Two, it will need to call the authentication server with the resource owners access token. The code looks something like this:

public async Task<TokenResponse> DelegateAsync()
{
    var userToken = Request.Headers["Authorization"][0].Substring("Bearer ".Length);
    var payload = new
    {
        token = userToken
    };

    var discoClient = new DiscoveryClient("https://authentication.example.com");
    var disco = await discoClient.GetAsync();

    // create token client
    var client = new TokenClient(disco.TokenEndpoint, "apione", "secret");

    // send custom grant to token endpoint, return response
    return await client.RequestCustomGrantAsync("delegation", "apitwo-readonly", payload);
}

This example is based on code taken from the docs.

In this method we get the resource owners token from the Bearer token in the Authorization header, and add it to the payload we send to the authentication server to get the API one access token. The token client is created with the client ID and secret for the apione client, we then hit the token endpoint of the authentication server with the payload, along with the grant type (delegation) and the scopes we need (apitwo-readonly).

If authentication is successful then the response will be a TokenResponse object, and we use the value in the AccessToken property to set the Authorization header for requests to API Two.

What does this all look like? let's look at the contents of both the access tokens.

First the resource owners access token:

[
    {
        "type": "aud",
        "value": "apione"
    },
    {
        "type": "client_id",
        "value": "apione-client"
    },
    {
        "type": "sub",
        "value": "2e4b6ea5-85bc-4e53-a252-fecb163128dd"
    },
    {
        "type": "scope",
        "value": "apione-full"
    },
    {
        "type": "amr",
        "value": "pwd"
    }
]

Here we can see some of the claims in the access token issued to the resource owner when they authenticated in the native application. We can see the client ID, audience, subject, and allowed scopes. We can also see the authentication method (amr) was password.

Now let's look at the access token which API One got in order to access API Two on behalf of the resource owner:

[
    {
        "type": "aud",
        "value": "apitwo"
    },
    {
        "type": "client_id",
        "value": "apione"
    },
    {
        "type": "sub",
        "value": "2e4b6ea5-85bc-4e53-a252-fecb163128dd"
    },
    {
        "type": "scope",
        "value": "apitwo-readonly"
    },
    {
        "type": "amr",
        "value": "delegation"
    }
]

We can now see that the token is identifying as API One (client_id), it has the correct scope in order to carry out the required requests, the subject is still identifying as the resource owner, and we can see the authentication method is delegation.

Summary

By looking at the example access tokens above we can see that this method is far more honest. When API One is making requests to API Two on behalf of the resource owner, the access token now identifies the request correctly as being from API One, and the scope is validated and set correctly. This allows us to implement the correct security policies in both the API's, and the native client application doesn't need to worry about knowing anything about API Two.

A working example of the client, API one, and API two can be found in the repo.

Building and running a ASP.Net Core web application in a Docker container is pretty straight forward. We can use use the images provided by Microsoft. There is one image for building, and one for running. We can also use Dockers new multi stage build feature to make things even easier.

Let’s take a look at a Dockerfile.

FROM microsoft/aspnetcore-build:2 As build
WORKDIR /app
COPY ./app/WebApp.csproj .
RUN ["dotnet", "restore"]
COPY ./src .
RUN dotnet publish -c Release -o ./output

FROM microsoft/aspnetcore:2
WORKDIR /app
COPY --from=build /app/output .
ENTRYPOINT ["dotnet", "WebApp.dll"]

Breaking the file down it works as follows:

• We use the base image from DockerHub. The ‘build’ bit means it’s been made with the SDK and contains all the bits we need to compile our application. The ‘As build’ part is the multi stage build feature, being able to assign a name to the stage.
• We set a working directory within the container and copy our project file into the container.
• We now run a NuGet restore using the dotnet command line. This will read the project the file we copied and pull down any dependencies
• Now we copy over the rest of the files and use the dotnet command line to build and publish to the output directory. We could also run our tests here.
• Now we pull down a different image from DockerHub. This time it’s a more lightweight image which just contains a runtime environment.
• We again set the work directory, and now we copy the application we compiled in the previous stage, from the output directory (where we published to) into the runtime container.
• Finally we set an entrypoint which will be the dotnet cli, running the assembly we compiled.

In the build step you notice we have seperate restore and publish commands. Why do we restore then build, when building will also perform a restore? Well the reason is so the Docker can cache layers with the packages when there are no changes, to help speed up subsequent builds. For more information about how Docker will cache layers see the docs

Now we have our Docker file in the root of the application, we can now build the image:

docker build -t WebApp .

If it’s the first time you have used the base images, Docker will pull them down which may take some time, depending on your connection speed. The next time you build, the images will be cached locally, so will be much quicker.

Now run a container:

docker run -it --rm -p 8080:80 WebApp

If all is good you should now be browse to http://localhost:8080 and see your site.

When you’re building out a new API or service, it’s sometimes difficult to know exactly what sort of server you’re going to need to run it on. You probably have a rough idea of what your MVP is, but you don’t really know what features you will need to add in the future.

You might not really know what sort of traffic you will be dealing with either, both in terms of number of requests or load pattern. In order to get your API live, you need to take an educated guess, and size for your estimated peak load.

I’ve recently gone through a sizing exercise for one of our new APIs. Initially we had the problems described above, so went with what we knew in terms of instance sizing, we played it safe and opted to deploy on instances which were sized to easily handle our estimated peak load. We also decided that our pre-production environments would be deployed on the same configuration so that we could get a ‘like live’ QA system whilst we initially built the application.

The problem with this is that once you get live and start slowly ramping up live traffic, you get a better idea of your traffic patterns, you also start to think about resilience. In AWS, we need to be thinking about splitting our servers across availability zones and maybe across regions too.

We saw that our servers were over-sized for our average load, and having the same specification of servers across 3 pre-production environments was costing us money that we didn’t really need to spend. Our production boxes were also all in the same availability zone, so losing that would mean no service. As well as server sizing we also had other things to consider, such as the ability to easily scale as we added more traffic to our new API. As our service was already in AWS we decided to take a look at what AWS could offer in terms of scaling solutions.

Elastic Beanstalk

The Amazon documentation describes Elastic Beanstalk as:

With Elastic Beanstalk, you can quickly deploy and manage applications in the AWS Cloud without worrying about the infrastructure that runs those applications. AWS Elastic Beanstalk reduces management complexity without restricting choice or control. You simply upload your application, and Elastic Beanstalk automatically handles the details of capacity provisioning, load balancing, scaling, and application health monitoring.

Elastic Beanstalk would allow us to define an environment containing EC2 instance details, Elastic Load Balancers, EBS, Security groups, and auto scaling groups, all in a single config file. In order to change any part of the infrastructure we could just make a change to the config file, save it S3, and then apply it to an environment and by using auto scaling groups triggered by Cloud Watch alarms we could also scale the application up and down based on traffic.

Configuring and Creating an Environment

Setting up an application in Elastic Beanstalk is pretty straight forward. The first step is to define the application, then within an application you define environments. These are typically things like QA, UAT, and Production. Each environment can have a completely different configuration if required. Once you have an environment configured and up and running (usually with a sample application) you can go ahead and publish your own application to it and you’re done. Now it’s a case of changing the various configuration options to fine tune your environment. You may need to consider things such as instance types, VPC configurations, security groups, monitoring and auto scaling.

Automating Provisioning and Deployment

Being able to work with the AWS web UI is all well and good and pretty straightforward, but the aim here is automate every aspect of this process. We need to automate the creation and updating of environment configuration, EC2 instance configuration, and application deployment.

To do this we use a combination of two command line tools. The first one being the standard AWS command line tool, and the other being the AWS EB command line tool. The EB tool is made, as the name suggests, for Elastic Beanstalk, and it will allow you carry out the main operations you need but we found it was a little restrictive in certain areas when it comes to some of the tasks you need to perform when automating your deployments, that’s where the AWS command line tool comes in.

Let’s take a look at the broad steps we need to automate to get from having nothing but an empty Elastic Beanstalk application (with no environments) to a functioning API:

• Upload a configuration file to describe an environment
• Create (or update) an environment
• Publish a specific application version to Elastic Beanstalk
• Deploy a specific application version to an environment

The way we tackled this was to create a bash script for each aspect of the process. The scripts take the various parameters they need from the command line, which can be given to it by the build automation server. The bash scripts just use a combination of the command line tools mentioned above to perform the various tasks.

One area of the AWS documentation on all this that I found to be a bit light was exactly how you can define the various configuration options for the environment. In the end we used a bit of a cheaty way to get the configuration Yaml file that we can then upload to S3 and use to define the environment.
The EB tool has a config command, this will do a couple of things, but one of the things it lets you do is to edit the active config in a text editor. This basically exposes the format of the file, and all the options you can change. I’ve included an example below:

Another aspect of the environment creation command to pay close attention to is the CNAME parameter. This will be used to point to the correct load balancer. As the things like EC2 instances and Elastic Load Balancers are immutable, it would be a pain to have to change the URL of your API in everything that calls it every time you make a change that swaps out the load balancer. This CNAME is the constant that holds it all together. Once you define it for the environment you are given a URL that points to you environment. Once you have this you can then point your DNS name to the Beanstalk address. The whole thing then becomes completely transparent to consumers.

It also allows for blue/green deploys where you can stand up a completely separate environment. Lets say your current live traffic is using an environment called 'green'. You could choose to create a new environment, which is exactly the same as 'green' but called 'blue'. You could then deploy a new version of the application to 'blue', test it against the Beanstalk URL, then once you are happy it’s working, you can perform a swap using the EB tool to the 'green' environment. This will swap the CNAME records of the two environments, meaning your live traffic is now going to the 'blue' environment. This may be useful if you want to deploy your application without ever taking any instances out of the load balancer.

What’s next?

We are using Elastic Beanstalk for one of our API’s in production now, and are still making slight changes to our auto scaling configurations as we start to ramp up traffic to it. We can monitor the metrics in Cloud Watch, and change the scaling parameters as we go, using our build pipelines to push the changes out, with no impact on the live traffic.
We are also starting to build a new worker service that will also use Elastic Beanstalk, but will scale based on a custom trigger which should give us significant performance improvements over our existing versions.

This article was originally published here

I recently came across this error whilst developing stored procedures in oracle. The stored procedure will be built in Oracle but marked as invalid. Trying a re-compile will give you the above error.

The problem appears to be with Windows CRLF characters on line breaks. Oracle does not treat this as white space, instead it sees it as an empty string. In order to get round this problem, convert the CRLF characters to LF characters and Oracle should be happy.

I wrote a post a while ago dealing with an error that VB can throw when dealing with an LDAP connection. Because this post has proved popular with people searching for the error code on Google, I thought I’d put together a quick post on using LDAP in VB.Net.

Simple Directory Search

To perform a simple query you can use the following syntax:

Dim results as SearchResultCollection

Using dir As New DirectoryEntry(connectionPath, principle, credentials, AuthenticationTypes.ServerBind)
	Dim s As New DirectorySearcher
	s.SearchRoot = dir
        s.Filter = (&(objectClass=groupOfUniqueNames)(cn=techSupport))
        s.SearchScope = SearchScope.Subtree
        results = s.FindAll
End Using

• First we are declaring a variable to hold our search results.
• Next we open up a connection to the LDAP directory. This is done by instantiating a DirectoryEntry object. The DirectoryEntry object requires the connection path in the form of a URL LDAP://MyServer:10389/OU=subdir,O=parent, the principle uid=myuser,ou=system, credentials mypassword, and the method of authentication (this will depend upon your LDAP directory).
• In order to search the directory we need to use the DirectorySearcher Object. We assign our DirectoryEntry object to the SearchRoot property, this will tell the DirectorySearcher where to base its search. The Filter property needs the details of what we are searching for, in this example we are looking for an entry whose ObjectClass is groupOfUniqueNames and CN is techSupport. The ampersand in front of the query tells the searcher this is an AND operation. The SearchScope property tells the searcher that we want to search the whole sub tree from the SearchRoot down.
• Lastly we return the results.

This is a very basic example of searching the directory. The directory is searchable on any property that an object may have, e.g. you may want to find all entries whose are inetOrgPerson and have a surname of smith. In that example your filter would be (&(objectClass=inetOrgPerson)(sn=Smith))

Performing An Update

You can perform an update on a directory entry just like you can in a database. When performing an update it is important to remember an entry can have multiple properties of the same name. In other words the properties are in fact arrays. The following is an example of an update.

Public Sub UpdateUser(ByVal username as String, ByVal firstName as String, ByVal surname as String)

        Dim myUserDN As String = String.Format("{0}uid={1},{2}", _settings.LDAPUrl, username, _settings.UsersDN)
	Using u As New DirectoryEntry(myUserDN, _settings.Principle, _settings.Credentials, AuthenticationTypes.ServerBind)
	    AddUpdateProperty(u, "cn", firstName)
            AddUpdateProperty(u, "sn", surname)
 	    u.CommitChanges()
	End Using
End Sub

Private Sub AddUpdateProperty(ByVal r As DirectoryEntry, ByVal propName As String, ByVal value As Object)
        If r.Properties(propName).Count = 0 Then
            r.Properties(propName).Add(value)
        Else
            r.Properties(propName).Item(0) = value
	End If
End Sub

The above code shows two sub routines. Lets look at the first one, UpdateUser:

• First we construct the URL to the object we want. We supply the LDAP URL along with the path to the entry we want.
• Next we attempt to connect straight to our entry using the DirectoryEntry object. If the object is not found this line will throw a DirectoryServicesCOMException exception.
• Now we can go ahead and update the properties we need to change. This is done via a helper function to handle the fact the entry could contain none, one or many of the property.
• Lastly we call the CommitChanges method on the Directory Entry.

Performing An Insert

Performing an insert on the directory is very similar to performing an update. The code below shows how we can insert a new user type entry.

Using users As New DirectoryEntry(userDNRoot, _settings.Principle, _settings.Credentials, AuthenticationTypes.ServerBind)
    Using newUser As DirectoryEntry = users.Children.Add(String.Format("uid={0}", username), "person")

        ' add user required properties
        AddUpdateProperty(newUser, "cn", forename)
        AddUpdateProperty(newUser, "sn", surname)
        newUser.CommitChanges()
    End Using
End Using

Lets look at the code:

• We need to connect to the LDAP directory. This time we connect to the root DN of where we want to add our new entry. Think of this as XML, we are going to add a child entry to the root.
• Once we have a connection, we can use the Children.Add methods to create a new DirectoryEntry object. As a parameter we pass in the name of the entry.
• Using the same helper function we saw above in the update functionality we create the properties on the object (note that this is a shortened list).
• Once we have assigned all the properties we go ahead and commit changes.