Yet another developer blog

Exploring Neon as a Serverless Postgres Alternative for .NET Applications on Azure - Part 1 (Simple ASP.NET Core on App Service)

noreply@blogger.com (Tomasz Pęczek) — Mon, 10 Feb 2025 21:26:00 +0000

Neon, a serverless Postgres platform, was recently brought to my attention. It comes with the promise of some interesting features like scale-to-zero, on-demand autoscaling, point-in-time restore and time travel with up to 30 days retention, instant read replicas, and probably the most unique, branching (which allows you to quickly branch your schema and data to create isolated development, test, or other purpose environments).

That's a lot of stuff to play with, but before I jumped in, I wanted to see how Neon integrated with my tech stack of choice - .NET, Azure, and GitHub. I decided to start with something simple - an ASP.NET Core application hosted on Azure App Service. Since I like to build things from the ground up, my first step was infrastructure.

Deploying the Infrastructure

Neon has three deployment options that might be of interest to me:

Create a Neon project hosted in the AWS region
Create a Neon project hosted in the Azure region
Deploy a Neon organization as an Azure Native ISV Service

The first two options are fully managed by Neon, you grab the connection details from the Neon console and start hacking. But I wanted to explore the third option because it provides the tightest integration from an Azure perspective (including SSO and unified billing), and that's what the organizations I work with are usually looking for.

As a strong IaC advocate, I also didn't want to deploy Neon through the Azure Portal or CLI, I wanted to use Bicep. Thanks to the native Azure integration, the Neon organization comes with its own resource type - Neon.Postgres/organizations.

resource neonOrganization 'Neon.Postgres/organizations@2024-08-01-preview' = {
  name: 'neon-net-applications-on-azure'
  location: location
  properties: {
    companyDetails: { }
    marketplaceDetails: {
      subscriptionId: subscription().id
      offerDetails: {
        publisherId: 'neon1722366567200'
        offerId: 'neon_serverless_postgres_azure_prod'
        planId: 'neon_serverless_postgres_azure_prod_free'
        termUnit: 'P1M'
        termId: 'gmz7xq9ge3py'
      }
    }
    partnerOrganizationProperties: {
      organizationName: 'net-applications-on-azure'
    }
    userDetails: {
      upn: 'userId@domainName'
    }
  }
}

You may ask, how did I get the values for the properties? Well, they are not documented (yet, I'm assuming) and I had to resort to reverse engineering (inspecting the template of the manually deployed resource).

The next step is to create a project. Everything in Neon (branches, databases, etc.) lives inside a project. Neon projects don't have an Azure resource representation, so I had to change the tool. I could create a project through the UI, but since I still wanted to have repeatability (something I could later reuse in a GitHub Actions workflow later), I decided to use Neon CLI. I still had to visit the UI (thanks to SSO I could just click on a link available on the resource Overview blade) to get myself an API key.

export NEON_API_KEY=<neon_api_key>

neon projects create \
    --name simple-asp-net-core-on-app-service \
    --region-id azure-westus3 \
    --output json

The output includes connection details for the created database. I don't want to manage secrets manually if I don't have to, so I decided to quickly create a key vault and put them there.

The last missing elements of my intended infrastructure were a managed identity, a container registry, an app service plan, and an app service. Below is the final diagram.

Seeding the Database

With all the infrastructure in place, I could start building my ASP.NET Core application. I wanted something simple, something that just displayed data, as my goal here was to see if Neon could be a drop-in replacement for Posgres from a code perspective. But to display data, you have to have data. So I needed to seed the database. I decided to take the simplest approach I could think of - using the built-in seeding capabilities of the Entity Framework Core. I had two options to choose from: UseSeeding/UseAsyncSeeding method or model managed data. I chose the latter and I quickly created a database context with two sets and a bunch of entities to add.

public class StarWarsDbContext : DbContext
{
    public DbSet<Character> Characters { get; private set; }

    public DbSet<Planet> Planets { get; private set; }

    public StarWarsDbContext(DbContextOptions<StarWarsDbContext> options)
    : base(options)
    { }

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        modelBuilder.Entity<Planet>(builder =>
        {
            builder.Property(x => x.Name).IsRequired();
            builder.HasData(
                new Planet { Id = 1, Name = "Tatooine", ... },
                ...
            );
        });

        modelBuilder.Entity<Character>(builder =>
        {
            builder.Property(x => x.Name).IsRequired();
            builder.HasData(
                new Character { Id = 1, Name = "Luke Skywalker", ... },
                ...
            );
        });
    }
}

Now I needed to register this database context as a service in my application. For the provider I used Npgsql.EntityFrameworkCore.PostgreSQLin the spirit of the drop-in replacement approach. With the connection string in the key vault and a managed identity in place to authenticate against that key vault, I could use DefaultAzureCredential and SecretClient to configure the provider and restrict my application settings to the key vault URI (yes, I choose this option even in demos).

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddDbContext<StarWarsDbContext>(options =>
{
    var keyVaultSecrettClient = new SecretClient(
        new Uri(builder.Configuration["KEY_VAULT_URI"]),
        new DefaultAzureCredential()
    );
    options.UseNpgsql(keyVaultSecrettClient.GetSecret("neon-connection-string").Value.Value);
});

...

The last thing to do here was to trigger the model creation code. This only needs to happen once, and as this is a demo, it can be ugly. I used an old trick of getting the database context as part of the startup and calling EnsureCreated (please don't do this in serious applications).

...

var app = builder.Build();

using (var serviceScope = app.Services.GetRequiredService<IServiceScopeFactory>().CreateScope())
{
    var context = serviceScope.ServiceProvider.GetRequiredService<StarWarsDbContext>();
    context.Database.EnsureCreated();
}

...

All the pieces are now in place. It's time to wrap things up.

Completing and Deploying the Application

For the application to be complete, it needed some UI. Since I didn't have an idea for anything special, I did something I used to do a lot in the past - a jqGrid based table that was quickly set up with the help of my old project Lib.AspNetCore.Mvc.JqGrid (I've basically copied some stuff from its demo).

As you've probably guessed from the infrastructure, my intention from the start was to deploy this application as a container. So my last step was to add a docker file, build, push, and voilà. I was able to navigate to the application, browse and sort the data.

Everything worked as expected and I consider my first experiment with Neon a success 😊.

Thoughts

They say to never publish part one if you don't have part two ready. I don't have part two ready, but I really think I'll write one, I just don't know when 😉. That's because Neon really got me interested.

In this post I wanted to check out the basics and set the stage for digging deeper. While writing it, I've also created a repository where you can find ready-to-deploy infrastructure and application. I've created GitHub Actions workflows for deploying a Neon organization, creating a Neon project, and deploying the solution itself. All you need to do to play with it is clone and provide credentials 🙂.

Monitoring C# Azure Functions in the Isolated Worker Model - Infrastructure & Configuration Deep Dive

noreply@blogger.com (Tomasz Pęczek) — Tue, 26 Nov 2024 12:20:00 +0000

Not so long ago my colleague reached out with a question "Are there any known issues with ITelemetryInitializer in Azure Functions?". This question started a discussion about the monitoring configuration for C# Azure Functions in the isolated worker model. At some point in that discussion I stated "Alright, you've motivated me, I'll make a sample". When I sat down to do that, I started wondering which scenario should I cover and my conclusion was that there are several things I should go through... So here I am.

Setting the Scene

Before I start describing how we can monitor an isolated worker model C# Azure Function, allow me to introduce you to the one we are going to use for this purpose. I want to start with as basic setup as possible. This is why the initial Program.cs will contain only four lines.

var host = new HostBuilder()
    .ConfigureFunctionsWebApplication()
    .Build();

host.Run();

The host.json will also be minimal.

{
    "version": "2.0",
    "logging": {
        "logLevel": {
            "default": "Information"
        }
    }
}

For the function itself, I've decided to go for the Fibonacci sequence implementation as it can easily generate a ton of logs.

public class FibonacciSequence
{
    private readonly ILogger<FibonacciSequence> _logger;

    public FibonacciSequence(ILogger<FibonacciSequence> logger)
    {
        _logger = logger;
    }

    [Function(nameof(FibonacciSequence))]
    public async Task<HttpResponseData> Run(
        [HttpTrigger(AuthorizationLevel.Anonymous, "get", Route = "fib/{index:int}")]
        HttpRequestData request,
        int index)
    {
        _logger.LogInformation(
            $"{nameof(FibonacciSequence)} function triggered for {{index}}.",
            index
        );

        var response = request.CreateResponse(HttpStatusCode.OK);
        response.Headers.Add("Content-Type", "text/plain; charset=utf-8");

        await response.WriteStringAsync(FibonacciSequenceRecursive(index).ToString());

        return response;
    }

    private int FibonacciSequenceRecursive(int index)
    {
        int fibonacciNumber = 0;

        _logger.LogInformation("Calculating Fibonacci sequence for {index}.", index);

        if (index <= 0)
        {
            fibonacciNumber = 0;
        }
        else if (index <= 1)
        {
            fibonacciNumber = 1;
        }
        else
        {
            fibonacciNumber = 
              FibonacciSequenceRecursive(index - 1)
              + FibonacciSequenceRecursive(index - 2);
        }

        _logger.LogInformation(
            "Calculated Fibonacci sequence for {index}: {fibonacciNumber}.",
            index,
            fibonacciNumber
        );

        return fibonacciNumber;
    }
}

This is a recursive implementation which in our case has the added benefit of being crashable on demand 😉.

Now we can start capturing the signals this function will produce after deployment.

Simple and Limited Option for Specific Scenarios - File System Logging

What we can capture in a minimal deployment scenario is logs coming from our function. What do I understand by a minimal deployment scenario? The bare minimum that Azure Function requires is a storage account, app service plan, and function app. The function can push all its logs into that storage account. Is this something I would recommend for a production scenario? Certainly not. It's only logs (and in production you will want metrics) and works reasonably only for Azure Functions deployed on Windows (for production I would suggest Linux due to better cold start performance or lower pricing for dedicated plan). But it may be the right option for some development scenarios (when you want to run some work, download the logs and analyze them locally). So, how to achieve this? Let's start with some Bicep snippets for the required infrastructure. First, we need to deploy a storage account.

resource storageAccount 'Microsoft.Storage/storageAccounts@2023-05-01' = {
  name: 'stwebjobs${uniqueString(resourceGroup().id)}'
  location: resourceGroup().location
  sku: {
    name: 'Standard_LRS'
  }
  kind: 'Storage'
}

We also need an app service plan. It must be a Windows one (the below snippet creates a Windows Consumption Plan).

resource appServicePlan 'Microsoft.Web/serverfarms@2024-04-01' = {
  name: 'plan-monitored-function'
  location: resourceGroup().location
  sku: {
    name: 'Y1'
    tier: 'Dynamic'
  }
  properties: {
    computeMode: 'Dynamic'
  }
}

And the actual service doing the work, the function app.

resource functionApp 'Microsoft.Web/sites@2024-04-01' = {
  name: 'func-monitored-function'
  location: resourceGroup().location
  kind: 'functionapp'
  properties: {
    serverFarmId: appServicePlan.id
    siteConfig: {
      netFrameworkVersion: 'v8.0'
      appSettings: [
        {
          name: 'AzureWebJobsStorage'
          value: 'DefaultEndpointsProtocol=https;AccountName=${storageAccount.name};EndpointSuffix=${environment().suffixes.storage};AccountKey=${storageAccount.listKeys().keys[0].value}'
        }
        {
          name: 'WEBSITE_CONTENTAZUREFILECONNECTIONSTRING'
          value: 'DefaultEndpointsProtocol=https;AccountName=${storageAccount.name};EndpointSuffix=${environment().suffixes.storage};AccountKey=${storageAccount.listKeys().keys[0].value}'
        }
        {
          name: 'WEBSITE_CONTENTSHARE'
          value: 'func-monitored-function'
        }
        {
          name: 'FUNCTIONS_EXTENSION_VERSION'
          value: '~4'
        }
        {
          name: 'FUNCTIONS_WORKER_RUNTIME'
          value: 'dotnet-isolated'
        }
      ]
    }
    httpsOnly: true
  }
}

A couple of words about this function app. The kind: 'functionapp' indicates that this is a Windows function app that creates the requirement of setting netFrameworkVersion to desired .NET version. To make this an isolated worker model function, the FUNCTIONS_EXTENSION_VERSION is set to ~4 and FUNCTIONS_WORKER_RUNTIME to dotnet-isolated. When it comes to all the settings referencing the storage account, your attention should go to WEBSITE_CONTENTAZUREFILECONNECTIONSTRING and WEBSITE_CONTENTSHARE - those indicate the file share that will be created for this function app in the storage account (this is where the logs will go).

After deployment, the resulting infrastructure should be as below.

What remains is configuring the file system logging in the host.json file of our function. The default behavior is to generate log files only when the function is being debugged using the Azure portal. We want them to be generated always.

{
    "version": "2.0",
    "logging": {
        "fileLoggingMode": "always",
        "logLevel": {
            "default": "Information"
        }
    }
}

When you deploy the code and execute some requests, you will be able to find the log files in the defined file share (the path is /LogFiles/Application/Functions/Host/).

2024-11-17T15:33:39.339 [Information] Executing 'Functions.FibonacciSequence' ...
2024-11-17T15:33:39.522 [Information] FibonacciSequence function triggered for 6.
2024-11-17T15:33:39.526 [Information] Calculating Fibonacci sequence for 6.
2024-11-17T15:33:39.526 [Information] Calculating Fibonacci sequence for 5.
...
2024-11-17T15:33:39.527 [Information] Calculated Fibonacci sequence for 5: 5.
...
2024-11-17T15:33:39.528 [Information] Calculated Fibonacci sequence for 4: 3.
2024-11-17T15:33:39.528 [Information] Calculated Fibonacci sequence for 6: 8.
2024-11-17T15:33:39.566 [Information] Executed 'Functions.FibonacciSequence' ...

For Production Scenarios - Entra ID Protected Application Insights

As I said, I wouldn't use file system logging for production. Very often it's not even enough for development. This is why Application Insights are considered a default these days. But before we deploy one, we can use the fact that we no longer new Windows and change the deployment to be a Linux based one. First I'm going to change the app service plan to a Linux Consumption Plan.

resource appServicePlan 'Microsoft.Web/serverfarms@2024-04-01' = {
  ...
  kind: 'linux'
  properties: {
    reserved: true
  }
}

With a different app service plan, the function app can be changed to a Linux one, which means changing the kind to functionapp,linux and replacing netFrameworkVersion with the corresponding linuxFxVersion.

resource functionApp 'Microsoft.Web/sites@2024-04-01' = {
  ...
  kind: 'functionapp,linux'
  properties: {
    serverFarmId: appServicePlan.id
    siteConfig: {
      linuxFxVersion: 'DOTNET-ISOLATED|8.0'
      ...
    }
    httpsOnly: true
  }
}

There is also a good chance that you no longer need the file share (although the documentation itself is inconsistent whether it's needed when running Linux functions on the Elastic Premium plan) so the WEBSITE_CONTENTAZUREFILECONNECTIONSTRING and WEBSITE_CONTENTSHARE can simply be removed. No requirement for the file share creates one more improvement opportunity - we can drop credentials for the blob storage (AzureWebJobsStorage) as this connection can use managed identity. I'm going to use a user-assigned managed identity because I often prefer them and they usually cause more trouble to set up 😉. To do so, we need to create one and grant it the Storage Blob Data Owner role for the storage account.

resource managedIdentity 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = {
  name: 'id-monitored-function'
  location: resourceGroup().location
}

resource storageBlobDataOwnerRoleDefinition 'Microsoft.Authorization/roleDefinitions@2022-04-01' existing = {
  name: 'b7e6dc6d-f1e8-4753-8033-0f276bb0955b' // Storage Blob Data Owner
  scope: subscription()
}

resource storageBlobDataOwnerRoleAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  scope: storageAccount
  name: guid(storageAccount.id, managedIdentity.id, storageBlobDataOwnerRoleDefinition.id)
  properties: {
    roleDefinitionId: storageBlobDataOwnerRoleDefinition.id
    principalId: managedIdentity.properties.principalId
    principalType: 'ServicePrincipal'
  }
}

The change to the function app is only about assigning the managed identity and replacing AzureWebJobsStorage with AzureWebJobsStorage__accountName.

resource functionApp 'Microsoft.Web/sites@2024-04-01' = {
  ...

  ...
  properties: {
    ...
    siteConfig: {
      ...
      appSettings: [
        {
          name: 'AzureWebJobsStorage__accountName'
          value: storageAccount.name
        }
        ...
      ]
    }
    httpsOnly: true
  }
}

Enough detouring (although it wasn't without a purpose 😜) - it's time to deploy Application Insights. As the classic Application Insights are retired, we are going to create a workspace-based instance.

resource logAnalyticsWorkspace 'Microsoft.OperationalInsights/workspaces@2023-09-01' = {
  name: 'log-monitored-function'
  location: resourceGroup().location
  properties: {
    sku: { 
      name: 'PerGB2018' 
    }
  }
}

resource applicationInsights 'Microsoft.Insights/components@2020-02-02' = {
  name: 'appi-monitored-function'
  location: resourceGroup().location
  kind: 'web'
  properties: {
    Application_Type: 'web'
    WorkspaceResourceId: logAnalyticsWorkspace.id
    DisableLocalAuth: true
  }
}

You might have noticed that I've set DisableLocalAuth to true. This is a security improvement. It enforces authentication by Entra ID for ingestion and as a result, makes InstrumentationKey a resource identifier instead of a secret. This is nice and we can easily handle it because we already have managed identity in place (I told you it had a purpose 😊). All we need to do is grant the Monitoring Metrics Publisher role to our managed identity.

resource monitoringMetricsPublisherRoleDefinition 'Microsoft.Authorization/roleDefinitions@2022-04-01' existing = {
  name: '3913510d-42f4-4e42-8a64-420c390055eb' // Monitoring Metrics Publisher
  scope: subscription()
}

resource monitoringMetricsPublisherRoleAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  scope: applicationInsights
  name: guid(applicationInsights.id, managedIdentity.id, monitoringMetricsPublisherRoleDefinition.id)
  properties: {
    roleDefinitionId: monitoringMetricsPublisherRoleDefinition.id
    principalId: managedIdentity.properties.principalId
    principalType: 'ServicePrincipal'
  }
}

Adding two application settings definitions to our function app will tie the services together.

resource functionApp 'Microsoft.Web/sites@2024-04-01' = {
  ...
  properties: {
    ...
    siteConfig: {
      ...
      appSettings: [
        ...
        {
          name: 'APPLICATIONINSIGHTS_CONNECTION_STRING'
          value: applicationInsights.properties.ConnectionString
        }
        {
          name: 'APPLICATIONINSIGHTS_AUTHENTICATION_STRING'
          value: 'ClientId=${managedIdentity.properties.clientId};Authorization=AAD'
        }
        ...
      ]
    }
    httpsOnly: true
  }
}

Are we done with the infrastructure? Not really. There is one more useful thing that is often forgotten - enabling storage logs. There are some important function app data in there so it would be nice to monitor it.

resource storageAccountBlobService 'Microsoft.Storage/storageAccounts/blobServices@2023-05-01' existing = {
  name: 'default'
  parent: storageAccount
}

resource storageAccountDiagnosticSettings 'Microsoft.Insights/diagnosticSettings@2021-05-01-preview' = {
  name: '${storageAccount.name}-diagnostic'
  scope: storageAccountBlobService
  properties: {
    workspaceId: logAnalyticsWorkspace.id
    logs: [
      {
        category: 'StorageWrite'
        enabled: true
      }
    ]
    metrics: [
      {
        category: 'Transaction'
        enabled: true
      }
    ]
  }
}

Now we are done and our infrastructure should look like below.

Once we deploy our function and make some requests, thanks to the codeless monitoring by the host and relaying worker logs through the host, we can find the traces in Application Insights.

You may be asking what I mean by "relaying worker logs through the host"? You probably remember that in the case of C# Azure Functions in the isolated worker model, we have two processes: functions host and isolated worker. Azure Functions wants to be helpful and by default, it sends the logs from the worker process to the functions host process, which then sends them to Application Insights.

This is nice, but may not be exactly what you want. You may want to split the logs so you can treat them separately (for example by configuring different default log levels for the host and the worker). To achieve that you must explicitly set up Application Insights integration in the worker code.

var host = new HostBuilder()
    .ConfigureFunctionsWebApplication()
    .ConfigureServices(services => {
        services.AddApplicationInsightsTelemetryWorkerService();
        services.ConfigureFunctionsApplicationInsights();
    })
    .Build();

host.Run();

The telemetry from the worker can now be controlled in the code, while from the host through the host.json. But if you deploy this, you will find no telemetry from the worker in Application Insights. You can still see your logs in the Log stream. You will also see a lot of those:

Azure.Identity: Service request failed.
Status: 400 (Bad Request)

Content:
{"statusCode":400,"message":"Unable to load the proper Managed Identity.","correlationId":"..."}

That's Application Insights SDK not being able to use the user-assigned managed identity. Azure Functions runtime uses the APPLICATIONINSIGHTS_AUTHENTICATION_STRING setting to provide a user-assigned managed identity OAuth token to the Application Insights but none of that (at the time of writing this) happens when we set up the integration explicitly. That said, we can do it ourselves. The parser for the setting is available in the Microsoft.Azure.WebJobs.Logging.ApplicationInsights so we can mimic the host implementation.

using TokenCredentialOptions =
    Microsoft.Azure.WebJobs.Logging.ApplicationInsights.TokenCredentialOptions;

var host = new HostBuilder()
    ...
    .ConfigureServices(services => {
        services.Configure<TelemetryConfiguration>(config =>
        {
            string? authenticationString =
                Environment.GetEnvironmentVariable("APPLICATIONINSIGHTS_AUTHENTICATION_STRING");

            if (!String.IsNullOrEmpty(authenticationString))
            {
                var tokenCredentialOptions = TokenCredentialOptions.ParseAuthenticationString(
                    authenticationString
                );
                config.SetAzureTokenCredential(
                    new ManagedIdentityCredential(tokenCredentialOptions.ClientId)
                );
            }
        });
        services.AddApplicationInsightsTelemetryWorkerService();
        services.ConfigureFunctionsApplicationInsights();
    })
    .Build();

host.Run();

The telemetry should now reach the Application Insights without a problem.

Be cautious. As Application Insights SDK now controls emitting the logs you must be aware of its opinions. It so happens that it tries to optimize by default and adds a logging filter that sets the minimum log level to Warning. You may want to get rid of that (be certain to make it after ConfigureFunctionsApplicationInsights).

...

var host = new HostBuilder()
    ...
    .ConfigureServices(services => {
        ...
        services.ConfigureFunctionsApplicationInsights();
        services.Configure<LoggerFilterOptions>(options =>
        {
            LoggerFilterRule? sdkRule = options.Rules.FirstOrDefault(rule =>
                rule.ProviderName == typeof(Microsoft.Extensions.Logging.ApplicationInsights.ApplicationInsightsLoggerProvider).FullName
            );

            if (sdkRule is not null)
            {
                options.Rules.Remove(sdkRule);
            }
        });
    })
    .Build();

host.Run();

Modifying Application Map in Application Insights

The default application map is not impressive - it will simply show your function app by its resource name.

As function apps rarely exist in isolation, we often want to present them in a more meaningful way on the map by providing a cloud role. The simplest way to do this is through the WEBSITE_CLOUD_ROLENAME setting.

resource functionApp 'Microsoft.Web/sites@2024-04-01' = {
  ...
  properties: {
    ...
    siteConfig: {
      ...
      appSettings: [
        ...
        {
          name: 'WEBSITE_CLOUD_ROLENAME'
          value: 'InstrumentedFunctions'
        }
        ...
      ]
    }
    ...
  }
}

This will impact both the host and the work. With explicit Application Insights integration, we can separate the two (if there is such a need) and change the cloud role for the worker. This is done in the usual way, by registering an ITelemetryInitializer. The only important detail is the place of the registration - it needs to be after ConfigureFunctionsApplicationInsights as Azure Functions are adding their own initializers.

public class IsolatedWorkerTelemetryInitializer : ITelemetryInitializer
{
    public void Initialize(ITelemetry telemetry)
    {
        telemetry.Context.Cloud.RoleName = "InstrumentedFunctionsIsolatedWorker";
    }
}

...

var host = new HostBuilder()
    ...
    .ConfigureServices(services => {
        ...
        services.ConfigureFunctionsApplicationInsights();
        services.AddSingleton<ITelemetryInitializer, IsolatedWorkerTelemetryInitializer>();
        services.Configure<LoggerFilterOptions>(options =>
        {
            ...
        });
    })
    .Build();

host.Run();

The resulting map will look as below.

Monitoring for Scaling Decisions

This is currently in preview, but you can enable emitting scale controller logs (for a chance to understand the scaling decisions). This is done with a single setting (SCALE_CONTROLLER_LOGGING_ENABLED) that takes a value in the : format. The possible values for destination are Blob and AppInsights, while the verbosity can be None, Verbose, and Warning. The Verbose seems to be the most useful one when you are looking for understanding as it's supposed to provide reason for changes in the worker count, and information about the triggers that impact those decisions.

resource functionApp 'Microsoft.Web/sites@2024-04-01' = {
  ...
  properties: {
    ...
    siteConfig: {
      ...
      appSettings: [
        ...
        {
          name: 'SCALE_CONTROLLER_LOGGING_ENABLED'
          value: 'AppInsights:Verbose'
        }
        ...
      ]
    }
    ...
  }
}

Is This Exhaustive?

Certainly not 🙂. There are other things that you can fiddle with. You can have a very granular configuration of different log levels for different categories. You can fine-tune aggregation and sampling. When you are playing with all those settings, please remember that you're looking for the right balance between the gathered details and costs. My advice is not to configure monitoring for the worst-case scenario (when you need every detail) as that often isn't financially sustainable. Rather aim for a lower amount of information and change the settings to gather more if needed (a small hint - you can override host.json settings through application settings without deployment).

ASP.NET Core 9 and IAsyncEnumerable - Async Streaming JSON and NDJSON From Blazor WebAssembly

noreply@blogger.com (Tomasz Pęczek) — Tue, 24 Sep 2024 08:57:00 +0000

I've been exploring working with asynchronous streaming data sources over HTTP for quite some time on this blog. I've been playing with async streamed JSON and NDJSON. I've been streaming and receiving streams in ASP.NET Core and console applications. But there is one scenario I haven't touched yet - streaming from Blazor WebAssembly. Why? Simply it wasn't possible.

Streaming objects from Blazor WebAssembly requires two things. First is the support for streaming upload in browsers Fetch API. The Fetch API did support streams for response bodies for quite some time (my first post using it is from 2019) but non-experimental support for streams as request bodies come to Chrome, Edge, and Safari in 2022 (and it is yet to come to Firefox as far as I can tell). Still, it's been available for two years and I haven't explored it yet? Well, I did, more than a year ago, with NDJSON and pure Javascript, where I used the ASP.NET Core endpoint I created long ago. But here we are talking about Blazor WebAssembly, which brings us to the second requirement - the browser API needs to be surfaced in Blazor. This is finally happening with .NET 9 and now I can fully explore it.

Async Streaming NDJSON From Blazor WebAssembly

I've decided to start with NDJSON because I already have my own building blocks for it (that mentioned ASP.NET Core endpoint, and NdjsonAsyncEnumerableContent coming from one of my packages). If things wouldn't work I would be able to easily debug them.

I've quickly put together a typical code using HttpClient. I just had to make sure I've enabled streaming upload by calling SetBrowserRequestStreamingEnabled on the request instance.

private async Task PostWeatherForecastsNdjsonStream()
{
    ...

    try
    {
        ....

        HttpRequestMessage request = new HttpRequestMessage(
            HttpMethod.Post, "api/WeatherForecasts/stream");
        request.Content = new NdjsonAsyncEnumerableContent<WeatherForecast>(
            StreamWeatherForecastsAsync());
        request.SetBrowserRequestStreamingEnabled(true);

        using HttpResponseMessage response = await Http.SendAsync(request, cancellationToken);

        response.EnsureSuccessStatusCode();
    }
    finally
    {
        ...
    }
}

It worked! No additional changes and no issues to resolve. I could see the items being logged by my ASP.NET Core service as they were streamed.

After this initial success, I could move to a potentially more challenging scenario - async streaming JSON.

Async Streaming JSON From Blazor WebAssembly

In theory, async streaming JSON should also just work. .NET has the support for IAsyncEnumerable built into JsonSerializer.SerializeAsync so JsonContent should just use it. Well, there is no better way than to try, so I've quickly changed the code.

private async Task PostWeatherForecastsJsonStream()
{
    ...

    try
    {
        ...

        HttpRequestMessage request = new HttpRequestMessage(
            HttpMethod.Post, "api/WeatherForecasts/stream");
        request.Content = JsonContent.Create(StreamWeatherForecastsAsync());
        request.SetBrowserRequestStreamingEnabled(true);

        using HttpResponseMessage response = await Http.SendAsync(request, cancellationToken);

        response.EnsureSuccessStatusCode();
    }
    finally
    {
        ...
    }
}

To my surprise, it also worked! I couldn't test it against my ASP.NET Core service because it didn't support it, but I quickly created a dumb endpoint that would simply dump whatever was incoming.

Now, why was I surprised? Because this is in fact a platform difference - this behavior seems to be unique for the "browser" platform. I did attempt the same in a desktop application earlier and HttpClient would buffer the request - I had to create a simple HttpContent that would wrap the underlying stream and flush on write. I think I like the Blazor WebAssembly behavior better and I'm happy that I didn't have to do the same here.

Ok, but if I can't handle the incoming stream nicely in ASP.NET Core, it's not really that useful. So I've tried to achieve that as well.

Receiving Async Streamed JSON in ASP.NET Core

Receiving async streamed JSON is a little bit more tricky. JsonSerializer has a dedicated method (DeserializeAsyncEnumerable) to deserialize an incoming JSON stream into an IAsyncEnumerable instance. The built-in SystemTextJsonInputFormatter doesn't use that method, it simply uses JsonSerializer.DeserializeAsync with request body as an input. That shouldn't be a surprise, it is the standard way to deserialize incoming JSON. To support async streamed JSON, a custom formatter is required. What is more, to be sure that this custom formatter will be used, it can't be just added - it has to replace the built-in one. But that would mean that the custom formatter has to have all the functionality of the built-in one (if we want to support not only async streamed JSON). Certainly, not something I wanted to reimplement, so I've decided to simply inherit from SystemTextJsonInputFormatter.

internal class SystemTextStreamedJsonInputFormatter : SystemTextJsonInputFormatter
{
    ...

    public override Task<InputFormatterResult> ReadRequestBodyAsync(InputFormatterContext context)
    {
        return base.ReadRequestBodyAsync(context);
    }
}

Now, for the actual functionality, the logic has to branch based on the type of the model as we want to use DeserializeAsyncEnumerable only if it's IAsyncEnumerable.

internal class SystemTextStreamedJsonInputFormatter : SystemTextJsonInputFormatter
{
    ...

    public override Task<InputFormatterResult> ReadRequestBodyAsync(InputFormatterContext context)
    {
        if (context.ModelType.GetGenericTypeDefinition() == typeof(IAsyncEnumerable<>))
        {
            ...
        }

        return base.ReadRequestBodyAsync(context);
    }
}

That's not the end of challenges (and reflection). JsonSerializer.DeserializeAsyncEnumerable is a generic method. That means we have to get the actual type of values, based on it construct the right method, and call it.

internal class SystemTextStreamedJsonInputFormatter : SystemTextJsonInputFormatter
{
    ...

    public override Task<InputFormatterResult> ReadRequestBodyAsync(InputFormatterContext context)
    {
        if (context.ModelType.GetGenericTypeDefinition() == typeof(IAsyncEnumerable<>))
        {
            MethodInfo deserializeAsyncEnumerableMethod = typeof(JsonSerializer)
                .GetMethod(
                    nameof(JsonSerializer.DeserializeAsyncEnumerable),
                    [typeof(Stream), typeof(JsonSerializerOptions), typeof(CancellationToken)])
                .MakeGenericMethod(context.ModelType.GetGenericArguments()[0]);

            return Task.FromResult(InputFormatterResult.Success(
                deserializeAsyncEnumerableMethod.Invoke(null, [
                    context.HttpContext.Request.Body,
                    SerializerOptions,
                    context.HttpContext.RequestAborted
                ])
            ));
        }

        return base.ReadRequestBodyAsync(context);
    }
}

The implementation above is intentionally ugly, so it doesn't hide anything and can be understood more easily. You can find a cleaner one, which also performs some constructed methods caching, in the demo repository.

Now that we have the formatter, we can create a setup for MvcOptions that will replace the SystemTextJsonInputFormatter with a custom implementation.

internal class SystemTextStreamedJsonMvcOptionsSetup : IConfigureOptions<MvcOptions>
{
    private readonly IOptions<JsonOptions>? _jsonOptions;
    private readonly ILogger<SystemTextJsonInputFormatter> _inputFormatterLogger;

    public SystemTextStreamedJsonMvcOptionsSetup(
        IOptions<JsonOptions>? jsonOptions,
        ILogger<SystemTextJsonInputFormatter> inputFormatterLogger)
    {
        _jsonOptions = jsonOptions;
        _inputFormatterLogger = inputFormatterLogger;
    }

    public void Configure(MvcOptions options)
    {
        options.InputFormatters.RemoveType<SystemTextJsonInputFormatter>();
        options.InputFormatters.Add(
            new SystemTextStreamedJsonInputFormatter(_jsonOptions?.Value, _inputFormatterLogger)
        );
    }
}

A small convenience method to register that setup.

internal static class SystemTextStreamedJsonMvcBuilderExtensions
{
    public static IMvcBuilder AddStreamedJson(this IMvcBuilder builder)
    {
        ...

        builder.Services.AddSingleton
            <IConfigureOptions<MvcOptions>, SystemTextStreamedJsonMvcOptionsSetup>();

        return builder;
    }
}

And we can configure the application to use it.

public class Startup
{
    public void ConfigureServices(IServiceCollection services)
    {
        services.AddControllers()
            ...
            .AddStreamedJson();

        ...
    }

    ...
}

It gets the job done. As the formatter for NDJSON expects a different content type (application/x-ndjson) the content negotiation also works and I could asynchronously stream NDJSON and JSON to the same endpoint. Honestly, I think it's cool.

Afterthoughts

One question that arises at this point is "Which one to use?". I prefer NDJSON here. Why? Both require custom formatters and both formatters must use reflection. But in the case of NDJSON that is isolated only to the requests coming with a dedicated content type. To support async streamed JSON, the initial model type check has to happen on every request with JSON-related content type. Also I don't feel great with replacing the built-in formatter like that. On top of that, NDJSON enables clean cancellation as the endpoint is receiving separate JSON objects. In the case of JSON that is a single JSON collection of objects that will not be terminated properly and it will trip the deserialization.

But this is just my opinion. You can grab the demo and play with it, or use one of my NDJON packages and experiment with it yourself. That's the best way to choose the right approach for your scenario.

Azure Functions Extensibility - Extensions and Isolated Worker Model

noreply@blogger.com (Tomasz Pęczek) — Tue, 05 Mar 2024 09:22:00 +0000

I've been exploring the subject of Azure Functions extensibility on this blog for quite some time. I've touched on subjects directly related to creating extensions and their anatomy, but also some peripheral ones.

I have always written from the perspective of the in-process model, but since 2020 there has been a continued evolution when it comes to the preferred model for .NET-based function apps. The isolated worker model, first introduced with .NET 5, has been gaining parity and becoming the leading vehicle for making new .NET versions available with Azure Functions. In August 2023 Microsoft announced the intention for .NET 8 to be the last LTS release to receive in-process model support. So the question comes, does it invalidate all that knowledge about Azure Functions extensibility? The short answer is no. But before I go into details, I need to cover the common ground.

Isolated Worker Model in a Nutshell

.NET was always a little bit special in Azure Functions. It shouldn't be a surprise. After all, it's Microsoft technology and there was a desire for the integration to be efficient and powerful. So even when Azure Functions v2 brought the separation between the host process and language worker process, .NET-based function apps were running in the host process. This had performance benefits (no communication between processes) and allowed .NET functions apps to leverage the full capabilities of the host, but started to become a bottleneck when the pace of changes in the .NET ecosystem accelerated. There were more and more conflicts between the assemblies that developers wanted to use in the apps and the ones used by the host. There was a delay in making new .NET versions available because the host had to be updated. Also, there were things that the app couldn't do because it was coupled with the host. Limitations like those were reasons for bringing Azure Functions .NET Worker to life.

At the same time, Microsoft didn't want to take away all the benefits that .NET developers had when working with Azure Functions. The design had to take performance and developers' experience into account. So how does Azure Functions .NET Worker work? In simplification, it's an ASP.NET Core application that receives inputs and provides outputs to the host over gRPC (which is more performant than HTTP primitives used in the case of custom handlers)

The request and response payloads are also pretty well hidden. Developers have been given a new binding model with required attributes available through *.Azure.Functions.Worker.Extensions.* packages. But if the actual bindings activity happens in the host, what do those new packages provide? And what is their relation with the *.Azure.WebJobs.Extensions.* packages?

Worker Extensions and WebJobs Extensions

The well-hidden truth is that the worker extension packages are just a bridge to the in-process extension packages. It means that if you want to create a new extension or understand how an existing one works, you should start with an extension for the in-process model. The worker extensions are mapped to the in-process ones through an assembly-level attribute, which takes the name of the package and version to be used as parameters.

[assembly: ExtensionInformation("RethinkDb.Azure.WebJobs.Extensions", "0.6.0")]

The integration is quite seamless. During the build, the Azure Functions tooling will use NuGet to install the needed in-process extension package, it doesn't have to be referenced. Of course that has its drawbacks (tight coupling to a specific version and more challenges during debugging). So, the final layout of the packages can be represented as below.

What ensures the cooperation between those two packages running in two different processes are the binding attributes.

Binding Attributes

In the case of the in-process model extensions we have two types of attributes - one for bindings and one for trigger. In the case of the isolated worker model, there are three - for input binding, for output binding, and for trigger.

public class RethinkDbInputAttribute : InputBindingAttribute
{
    ...
}

public sealed class RethinkDbOutputAttribute : OutputBindingAttribute
{
    ...
}

public sealed class RethinkDbTriggerAttribute : TriggerBindingAttribute
{
    ...
}

The isolated worker model attributes are used in two ways. One is for developers, who use them to decorate their functions and provide needed settings. The other is for the worker, which uses them as data transfer objects. They are being serialized and transferred as metadata. On the host side, they are being deserialized to the corresponding in-process model extension attribute. The input and output attributes will be deserialized to the binding attribute, and the trigger will be deserialized to the trigger. This means that we need to ensure that the names of properties which we want to support are matching.

Implementing the attributes and decorating the functions with them is all we need to make it work. This will give us support for POCOs as values (the host and worker will take care of serialization, transfer over gRPC, and deserialization). But what if we want something more than POCO?

Beyond POCO Inputs With Converters

It's quite common for in-process extensions to support binding data provided using types from specific service SDKs (for example CosmosClient in the case of Azure Cosmos DB). That kind of binding data is not supported out-of-the-box by isolated worker extensions as they can't be serialized and transferred. But there is a way for isolated worker extensions to go beyond POCOs - input converters.

Input converters are classes that implement the IInputConverter interface. This interface defines a single method, which is supposed to return a conversation result. Conversation result can be one of the following:

Unhandled (the converter did not act on the input)
Succeeded (conversion was successful and the result is included)
Failed

The converter should check if it's being used with an extension it supports (the name that has been used for isolated extensions registration will be provided as part of the model binding data) and if the incoming content is in supported format. The converter can also be decorated with multiple SupportedTargetType attributes to narrow its scope.

Below is a sample template for an input converter.

[SupportsDeferredBinding]
[SupportedTargetType(typeof(...))]
[SupportedTargetType(typeof(...))]
internal class RethinkDbConverter : IInputConverter
{
    private const string RETHINKDB_EXTENSION_NAME = "RethinkDB";
    private const string JSON_CONTENT_TYPE = "application/json";

    ...

    public RethinkDbConverter(...)
    {
        ...
    }

    public async ValueTask<ConversionResult> ConvertAsync(ConverterContext context)
    {
        ModelBindingData modelBindingData = context?.Source as ModelBindingData;

        if (modelBindingData is null)
        {
            return ConversionResult.Unhandled();
        }

        try
        {
            if (modelBindingData.Source is not RETHINKDB_EXTENSION_NAME)
            {
                throw new InvalidOperationException($"Unexpected binding source.");
            }

            if (modelBindingData.ContentType is not JSON_CONTENT_TYPE)
            {
                throw new InvalidOperationException($"Unexpected content-type.");
            }

            object result = context.TargetType switch
            {
                // Here you can use modelBindingData.Content,
                // any injected services, etc.
                // to prepare the value.
                ...
            };


            return ConversionResult.Success(result);
        }
        catch (Exception ex)
        {
            return ConversionResult.Failed(ex);
        }
    }
}

Input converters can be applied to input and trigger binding attributes by simply decorating them with an attribute (we should also define the fallback behavior policy).

[InputConverter(typeof(RethinkDbConverter))]
[ConverterFallbackBehavior(ConverterFallbackBehavior.Default)]
public class RethinkDbInputAttribute : InputBindingAttribute
{
    ...
}

Adding input converters to the extensions moves some of the logic from host to worker. It may mean that the worker will be now establishing connections to the services or performing other operations. This will most likely create a need to register some dependencies, read configuration, and so on. Such things are best done at a function startup.

Participating in the Function App Startup

Extensions for the isolated worker model can implement a startup hook. It can be done by creating a public class with a parameterless constructor, that derives from WorkerExtensionStartup. This class also has to be registered through an assembly-level attribute. Now we can override the Configure method and register services and middlewares. The mechanic is quite similar to its equivalent for in-process extension.

[assembly: WorkerExtensionStartup(typeof(RethinkDbExtensionStartup))]

namespace Microsoft.Azure.Functions.Worker
{
    public class RethinkDbExtensionStartup : WorkerExtensionStartup
    {
        public override void Configure(IFunctionsWorkerApplicationBuilder applicationBuilder)
        {
            if (applicationBuilder == null)
            {
                throw new ArgumentNullException(nameof(applicationBuilder));
            }

            ...
        }
    }
}

The Conclusion

The isolated worker model doesn't invalidate what we know about the Azure Functions extensions, on the contrary, it builds another layer on top of that knowledge. Sadly, there are limitations in the supported data types for bindings which come from the serialization and transfer of data between the host and the worker. Still, in my opinion, the benefits of the new model can outweigh those limitations.

If you are looking for a working sample to learn and explore, you can find one here.

Azure Functions Extensibility - Runtime Scaling

noreply@blogger.com (Tomasz Pęczek) — Thu, 22 Feb 2024 12:28:00 +0000

In the past, I've written posts that explored the core aspects of creating Azure Functions extensions:

In this post, I want to dive into how extensions implement support for the Azure Functions scaling model.

The Azure Functions scaling model (as one can expect) has evolved over the years. The first model is incremental scaling handled by the scale controller (a dedicated Azure Functions component). The scale controller monitors the rate of incoming events and makes magical (following the rule that every sophisticated enough technology is indistinguishable from magic) decisions about whether to add or remove a worker. The workers count can change only by one and at a specific rate. This mechanism has some limitations.

The first problem is the inability to scale in network-isolated deployments. In such a scenario, the scale controller doesn't have access to the services and can't monitor events - only the function app (and as a consequence the extensions) could. As a result, toward the end of 2019, the SDK provided support for extensions to vote in scale in and scale out decisions.

The second problem is the scaling process clarity and rate. Change by one worker at a time driven by complex heuristics is not perfect. This led to the introduction of target-based scaling at the beginning of 2023. Extensions become able to implement their own algorithms for requesting a specific number of workers and scale up by four instances at a time.

So, how do those two mechanisms work and can be implemented? To answer this question I'm going to describe them from two perspectives. One will be how it is being done by a well-established extension - Azure Cosmos DB bindings for Azure Functions. The other will be a sample implementation in my extension that I've used previously while writing on Azure Functions extensibility - RethinkDB bindings for Azure Functions.

But first things first. Before we can talk about the actual logic driving the scaling behaviors, we need the right SDK and the right classes in place.

Runtime and Target-Based Scaling Support in Azure WebJobs SDK

As I've mentioned above, support for extensions to participate in the scaling model has been introduced gradually to the Azure WebJobs SDK. At the same time, the team makes an effort to introduce changes in a backward-compatible manner. Support for voting in scaling decisions came in version 3.0.14 and for target-based scaling in 3.0.36, but you can have an extension that is using version 3.0.0 and it will work on the latest Azure Functions runtime. You can also have an extension that uses the latest SDK and doesn't implement the scalability mechanisms. You need to know that they are there and choose to utilize them.

This is reflected in the official extensions as well. The Azure Cosmos DB extension has implemented support for runtime scaling in version 3.0.5 and target-based scaling in 4.1.0 (you can find some tables that are gathering version numbers, for example in the section about virtual network triggers). This means, that if your function app is using lower versions of extensions, it won't benefit from these capabilities.

So, regardless if you're implementing your extension or you're just using an existing one, the versions of dependencies matter. However, if you're implementing your extension, you will need some classes.

Classes Needed to Implement Scaling Support

The Azure Functions scaling model revolves around triggers. After all, it's their responsibility to handle the influx of events. As you may know (for example from my previous post 😉), the heart of a trigger is a listener. This is where all the heavy lifting is being done and this is where we can inform the runtime that our trigger supports scalability features. We can do so by implementing two interfaces: IScaleMonitorProvider and ITargetScalerProvider. They are respectively tied to runtime scaling and target-based scaling. If they are implemented, the runtime will use the properties they define to obtain the actual logic implementations.

internal class RethinkDbTriggerListener : IListener, IScaleMonitorProvider, ITargetScalerProvider
{
    ...

    private readonly IScaleMonitor<RethinkDbTriggerMetrics> _rethinkDbScaleMonitor;
    private readonly ITargetScaler _rethinkDbTargetScaler;

    ...

    public IScaleMonitor GetMonitor()
    {
        return _rethinkDbScaleMonitor;
    }

    public ITargetScaler GetTargetScaler()
    {
        return _rethinkDbTargetScaler;
    }
}

From the snippets above you can notice one more class - RethinkDbTriggerMetrics. It derives from the ScaleMetrics class and is used for capturing the values on which the decisions are being made.

internal class RethinkDbTriggerMetrics : ScaleMetrics
{
    ...
}

The IScaleMonitor implementation contributes to the runtime scaling part of the scaling model. Its responsibilities are to provide snapshots of the metrics and to vote in the scaling decisions.

internal class RethinkDbScaleMonitor : IScaleMonitor<RethinkDbTriggerMetrics>
{
    public ScaleMonitorDescriptor Descriptor => ...;

    public Task<RethinkDbTriggerMetrics> GetMetricsAsync()
    {
        ...
    }

    Task<ScaleMetrics> IScaleMonitor.GetMetricsAsync()
    {
        ...
    }

    public ScaleStatus GetScaleStatus(ScaleStatusContext<RethinkDbTriggerMetrics> context)
    {
        ...
    }

    public ScaleStatus GetScaleStatus(ScaleStatusContext context)
    {
        ...
    }
}

The ITargetScaler implementation is responsible for the target-based scaling. It needs to focus only on one thing - calculating the desired worker count.

internal class RethinkDbTargetScaler : ITargetScaler
{
    public TargetScalerDescriptor TargetScalerDescriptor => ...;

    public Task<TargetScalerResult> GetScaleResultAsync(TargetScalerContext context)
    {
        ...
    }
}

As both of these implementations are tightly tied with a specific trigger, it is typical to instantiate them in the listener constructor.

internal class RethinkDbTriggerListener : IListener, IScaleMonitorProvider, ITargetScalerProvider
{
    ...

    public RethinkDbTriggerListener(...)
    {
        ...

        _rethinkDbScaleMonitor = new RethinkDbScaleMonitor(...);
        _rethinkDbTargetScaler = new RethinkDbTargetScaler(...);
    }

    ...
}

We are almost ready to start discussing the details of scaling logic, but before we do that, we need to talk about gathering metrics. The decisions need to be based on something.

Gathering Metrics

Yes, the decisions need to be based on something and the quality of the decisions will depend on the quality of metrics you can gather. So a lot depends on how well the service, for which the extension is being created, is prepared to be monitored.

Azure Cosmos DB is well prepared. The Azure Cosmos DB bindings for Azure Functions implement the trigger on top of the Azure Cosmos DB change feed processor and use the change feed estimator to gather the metrics. This gives access to quite an accurate estimate of the remaining work. As an additional metric, the extension is gathers the number of leases for the container that the trigger is observing.

RethinkDB is not so well prepared. It seems like change feed provides only one metric - buffered items count.

internal class RethinkDbTriggerMetrics : ScaleMetrics
{
    public int BufferedItemsCount { get; set; }
}

Also, the metric can only be gathered while iterating the change feed. This forces an intermediary between the listener and the scale monitor (well you could use the scale monitor directly, but it seemed ugly to me).

internal class RethinkDbMetricsProvider
{
    public int CurrentBufferedItemsCount { get; set; }

    public RethinkDbTriggerMetrics GetMetrics()
    {
        return new RethinkDbTriggerMetrics { BufferedItemsCount = CurrentBufferedItemsCount };
    }
}

Now the same instance of this intermediary can be used by the listener to provide the value.

internal class RethinkDbTriggerListener : IListener, IScaleMonitorProvider, ITargetScalerProvider
{
    ...

    private readonly RethinkDbMetricsProvider _rethinkDbMetricsProvider;

    ...

    public RethinkDbTriggerListener(...)
    {
        ...

        _rethinkDbMetricsProvider = new RethinkDbMetricsProvider();
        _rethinkDbScaleMonitor = new RethinkDbScaleMonitor(..., _rethinkDbMetricsProvider);

        ...
    }

    ...

    private async Task ListenAsync(CancellationToken listenerStoppingToken)
    {
        ...

        while (!listenerStoppingToken.IsCancellationRequested
               && (await changefeed.MoveNextAsync(listenerStoppingToken)))
        {
            _rethinkDbMetricsProvider.CurrentBufferedItemsCount = changefeed.BufferedItems.Count;
            ...
        }

        ...
    }
}

And the scale monitor to read it.

internal class RethinkDbScaleMonitor : IScaleMonitor<RethinkDbTriggerMetrics>
{
    ...

    private readonly RethinkDbMetricsProvider _rethinkDbMetricsProvider;

    ...

    public RethinkDbScaleMonitor(..., RethinkDbMetricsProvider rethinkDbMetricsProvider)
    {
        ...

        _rethinkDbMetricsProvider = rethinkDbMetricsProvider;

        ...
    }

    public Task<RethinkDbTriggerMetrics> GetMetricsAsync()
    {
        return Task.FromResult(_rethinkDbMetricsProvider.GetMetrics());
    }

    Task<ScaleMetrics> IScaleMonitor.GetMetricsAsync()
    {
        return Task.FromResult((ScaleMetrics)_rethinkDbMetricsProvider.GetMetrics());
    }

    ...
}

We have the metrics now, it is finally time to make some decisions.

Voting to Scale In or Scale Out

An extension can cast one of three votes: to scale out, to scale in, or neutral. This is where the intimate knowledge of the service that the extension is created for comes into play.

As I've already written, the Azure Cosmos DB change feed processor is well prepared to gather metrics about it. It is also well prepared to be consumed in a scalable manner. It can distribute the work among multiple compute instances, by balancing the number of leases owned by each compute instance. It will also adjust the number of leases based on throughput and storage. This is why the Azure Cosmos DB bindings are tracking the number of leases as one of the metrics - it's an upper limit for the number of workers. So the extension utilizes all this knowledge and employs the following algorithm to cast a vote:

If there are no metrics gathered yet, cast a neutral vote.
If the current number of leases is greater than the number of workers, cast a scale-in vote.
If there are less than five metrics samples, cast a neutral vote.
If the ratio of workers to remaining items is less than 1 per 1000, cast a scale-out vote.
If there are constantly items waiting to be processed and the number of workers is smaller than the number of leases, cast a scale-out vote.
If the trigger source has been empty for a while, cast a scale-in vote.
If there has been a continuous increase across the last five samples in items to be processed, cast a scale-out vote.
If there has been a continuous decrease across the last five samples in items to be processed, cast a scale-in vote.
If none of the above has happened, cast a neutral vote.

RethinkDB is once again lacking here. It looks like its change feed is not meant to be processed in parallel at all. This leads to an interesting edge case, where we never want to scale beyond a single instance.

internal class RethinkDbScaleMonitor : IScaleMonitor<RethinkDbTriggerMetrics>
{
    ...

    public ScaleStatus GetScaleStatus(ScaleStatusContext<RethinkDbTriggerMetrics> context)
    {
        return GetScaleStatus(
            context.WorkerCount,
            context.Metrics?.ToArray()
        );
    }

    public ScaleStatus GetScaleStatus(ScaleStatusContext context)
    {
        return GetScaleStatus(
            context.WorkerCount, 
            context.Metrics?.Cast<RethinkDbTriggerMetrics>().ToArray()
        );
    }

    private ScaleStatus GetScaleStatus(int workerCount, RethinkDbTriggerMetrics[] metrics)
    {
        ScaleStatus status = new ScaleStatus
        {
            Vote = ScaleVote.None
        };

        // RethinkDB change feed is not meant to be processed in paraller.
        if (workerCount > 1)
        {
            status.Vote = ScaleVote.ScaleIn;

            return status;
        }

        return status;
    }
}

Requesting Desired Number of Workers

Being able to tell if you want more workers or less is great, being able to tell how many workers you want is even better. Of course, there is no promise the extension will get the requested number (even in the case of target-based scaling the scaling happens with a maximum rate of four instances at a time), but it's better than increasing and decreasing by one instance. Extensions can also use this mechanism to participate in dynamic concurrency.

This is exactly what the Azure Cosmos DB extension is doing. It divides the number of remaining items by the value of the MaxItemsPerInvocation trigger setting (the default is 100). The result is capped by the number of leases and that's the desired number of workers.

We already know that in the case of RethinkDB, it's even simpler - we always want just one worker.

internal class RethinkDbTargetScaler : ITargetScaler
{
    ...

    public Task<TargetScalerResult> GetScaleResultAsync(TargetScalerContext context)
    {
        return Task.FromResult(new TargetScalerResult
        {
            TargetWorkerCount = 1
        });
    }
}

That's it. The runtime scaling implementation is now complete.

Consequences of Runtime Scaling Design

Creating extensions for Azure Functions is not something commonly done. Still, there is a value in understanding their anatomy and how they work as it is often relevant also when you are creating function apps.

The unit of scale for Azure Functions is the entire functions app. At the same time, the scaling votes and the desired number of workers are delivered at a trigger level. This means that you need to be careful when creating function apps with multiple triggers. If the triggers will be aiming for completely different scaling decisions it may lead to undesired scenarios. For example, one trigger may constantly want to scale in, while another will want to scale out. As you could notice throughout this post, some triggers have hard limits on how far they can scale to not waste resources (even two Azure Cosmos DB triggers may have a different upper limit because the lease containers they are attached to will have different numbers of leases available). This all should be taken into account while designing the function app and trying to foresee how it will scale.

Experimenting With .NET & WebAssembly - Running .NET Based Slight Application On WASM/WASI Node Pool in AKS

noreply@blogger.com (Tomasz Pęczek) — Tue, 09 Jan 2024 11:09:00 +0000

A year ago I wrote about running a .NET based Spin application on a WASI node pool in AKS. Since then the support for WebAssembly in AKS hasn't changed. We still have the same preview, supporting the same versions of two ContainerD shims: Spin and Slight (a.k.a. SpiderLightning). So why am I coming back to the subject? The broader context has evolved.

With WASI preview 2, the ecosystem is embracing the component model and standardized APIs. When I was experimenting with Spin, I leveraged WAGI (WebAssembly Gateway Interface) which allowed me to be ignorant about the Wasm runtime context. Now I want to change that, cross the barrier and dig into direct interoperability between .NET and Wasm.

Also, regarding the mentioned APIs, one of the emerging ones is wasi-cloud-core which aims to provide a generic way for WASI applications to interact with services. This proposal is not yet standardized but it has an experimental host implementation which happens to be Slight. By running a .NET based Slight application I want to get a taste of what that API might bring.

Last but not least, .NET 8 has brought a new way of building .NET based Wasm applications with a wasi-experimental workload. I want to build something with it and see where the .NET support for WASI is heading.

So, this "experiment" has multiple angles and brings together a bunch of different things. How am I going to start? In the usual way, by creating a project.

Creating a .NET 8 `wasi-wasm` Project

The wasi-experimental workload is optional, so we need to install it before we can create a project.

dotnet workload install wasi-experimental

It also doesn't bundle the WASI SDK (the WASI SDK for .NET 7 did), so we have to install it ourselves. The releases of WASI SDK available on GitHub contain binaries for different platforms. All you need to do is download the one appropriate for yours, extract it, and create the WASI_SDK_PATH environment variable pointing to the output. The version I'll be using here is 20.0.

With the prerequisites in place, we can create the project.

dotnet new wasiconsole -o Demo.Wasm.Slight

Now, if you run dotnet build and inspect the output folder, you can notice that it contains Demo.Wasm.Slight.dll, other managed DLLs, and dotnet.wasm. This is the default output, where the dotnet.wasm is responsible for loading the Mono runtime and then loading the functionality from DLLs. This is not what we want. We want a single file. To achieve that we need to modify the project file by adding the WasmSingleFileBundle property (in my opinion this should be the default).

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    ...
    <WasmSingleFileBundle>true</WasmSingleFileBundle>
  </PropertyGroup>
</Project>

If you run dotnet build after this modification, you will find Demo.Wasm.Slight.wasm in the output. Exactly what we want.

But, before we can start implementing the actual application, we need to work on the glue between the .NET and WebAssembly so we can interact with the APIs provided by Slight from C#.

From WebAssembly IDL To C#

The imports and exports in WASI APIs are described in Wasm Interface Type (WIT) format. This format is an IDL which is a foundation behind tooling for the WebAssembly Component Model.

WIT aims at being a developer-friendly format, but writing bindings by hand is not something that developers expect. This is where wit-bindgen comes into the picture. It's a (still young) binding generator for languages that are compiled into WebAssembly. It currently supports languages like Rust, C/C++, or Java. The C# support is being actively worked on and we can expect that at some point getting C# bindings will be as easy as running a single command (or even simpler as Steve Sanderson is already experimenting with making it part of the toolchain) but for now, it's too limited and we will have to approach things differently. What we can use is the C/C++ support.

There is one more challenge on our way. The current version of wit-bindgen is meant for WASI preview 2. Meanwhile, a lot of existing WIT definitions and WASI tooling around native languages are using WASI preview 1. This is exactly the case when it comes to the Slight implementation available in the AKS preview. To handle that we need an old (and I mean old) version of wit-bindgen. I'm using version v0.2.0. Once you install it, you can generate C/C++ bindings for the desired imports and exports. Slight in version 0.1.0 provides a couple of capabilities, but I've decided to start with just one, the HTTP Server. That means we need imports for http.wit and exports for http-handler.wit.

wit-bindgen c --import ./wit/http.wit --out-dir ./native/
wit-bindgen c --export ./wit/http-handler.wit --out-dir ./native/

Once we have the C/C++ bindings, we can configure the build to include those files in the arguments passed to Clang. For this purpose, we can use the .targets file.

<Project>
  <ItemGroup>
    <_WasmNativeFileForLinking Include="$(MSBuildThisFileDirectory)\..\native\*.c" />
  </ItemGroup>
</Project>

We also need to implement the C# part of the interop. For the main part, it's like any other interop you might have ever done. You can use generators or ask for help from a friendly AI assistant and it will get you through the code for types and functions. However, there is one specific catch. The DllImportAttribute requires passing a library name for the P/Invoke generator, but we have no library. The solution is as simple as surprising (at least to me), we can provide the name of any library that the P/Invoke generator knows about.

internal static class HttpServer
{
    // Any library name that P/Invoke generator knows
    private const string LIBRARY_NAME = "libSystem.Native";

    [DllImport(LIBRARY_NAME)]
    internal static extern unsafe void http_server_serve(WasiString address,
        uint httpRouterIndex, out WasiExpected<uint> ret0);

    [DllImport(LIBRARY_NAME)]
    internal static extern unsafe void http_server_stop(uint httpServerIndex,
        out WasiExpected<uint> ret0);
}

With all the glue in place, we can start implementating the HTTP server capability.

Implementing the Slight HTTP Server Capability

In order for a Slight application to start accepting HTTP requests, the application needs to call the http_server_serve function to which it must provide the address it wants to listen on and the index of a router that defines the supported routes. I've decided to roll out to simplest implementation I could think of to start testing things out - the HttpRouter and HttpServer classes which only allow for calling the serve function (no support for routing).

internal class HttpRouter
{
    private uint _index;

    public uint Index => _index;

    private HttpRouter(uint index)
    {
        _index = index;
    }

    public static HttpRouter Create()
    {
        http_router_new(out WasiExpected<uint> expected);

        if (expected.IsError)
        {
            throw new Exception(expected.Error?.ErrorWithDescription.ToString());
        }

        return new HttpRouter(expected.Result.Value);
    }
}

internal static class HttpServer
{
    private static uint? _index;

    public static void Serve(string address)
    {
        if (_index.HasValue)
        {
            throw new Exception("The server is already running!");
        }

        HttpRouter router = HttpRouter.Create();
        http_server_serve(
            WasiString.FromString(address),
            router.Index,
            out WasiExpected<uint> expected
        );

        if (expected.IsError)
        {
            throw new Exception(expected.Error?.ErrorWithDescription.ToString());
        }

        _index = expected.Result;
    }
}

This allowed me to implement a simplistic application, a one-liner.

HttpServer.Serve("0.0.0.0:80");

It worked! Every request results in 404, because there is no routing, but it worked. So, how to add support for routing?

The http_router_* functions for defining routes expect two strings - one for the route and one for the handler. This suggests that the handler should be an exported symbol that Slight will be able to call. I went through the bindings exported for http-handler.wit and I've found a function that is being exported as handle-http. That function seems to be what we are looking for. It performs transformations to/from the request and response objects and calls a http_handler_handle_http function which has only a definition. So it looks like the http_handler_handle_http implementation is a place for the application logic. To test this theory, I've started by implementing a simple route registration method.

internal class HttpRouter
{
    ...

    private static readonly WasiString REQUEST_HANDLER = WasiString.FromString("handle-http");

    ...

    public HttpRouter RegisterRoute(HttpMethod method, string route)
    {
        WasiExpected<uint> expected;

        switch (method)
        {
            case HttpMethod.GET:
                http_router_get(_index, WasiString.FromString(route), REQUEST_HANDLER,
                    out expected);
                break;
            case HttpMethod.PUT:
                http_router_put(_index, WasiString.FromString(route), REQUEST_HANDLER,
                    out expected);
                break;
            case HttpMethod.POST:
                http_router_post(_index, WasiString.FromString(route), REQUEST_HANDLER,
                    out expected);
                break;
            case HttpMethod.DELETE:
                http_router_delete(_index, WasiString.FromString(route), REQUEST_HANDLER,
                    out expected);
                break;
            default:
                throw new NotSupportedException($"Method {method} is not supported.");
        }

        if (expected.IsError)
        {
            throw new Exception(expected.Error?.ErrorWithDescription.ToString());
        }

        return new HttpRouter(expected.Result.Value)
    }

    ...
}

Next I've registered some catch-all routes and implemented the HandleRequest method as the .NET handler. It would still return a 404, but it would be mine 404.

internal static class HttpServer
{
    ...

    public static void Serve(string address)
    {
        ...

        HttpRouter router = HttpRouter.Create()
                            .RegisterRoute(HttpMethod.GET, "/")
                            .RegisterRoute(HttpMethod.GET, "/*");
        http_server_serve(
            WasiString.FromString(address),
            router.Index,
            out WasiExpected<uint> expected
        );

        ...
    }

    private static unsafe void HandleRequest(ref HttpRequest request,
        out WasiExpected<HttpResponse> result)
    {
        HttpResponse response = new HttpResponse(404);
        response.SetBody($"Handler Not Found ({request.Method} {request.Uri.AbsolutePath})");

        result = new WasiExpected<HttpResponse>(response);
    }

    ...
}

It's time for some C. I went through the Mono WASI C driver and found two functions that looked the right tools for the job: lookup_dotnet_method and mono_wasm_invoke_method_ref. The implementation didn't seem overly complicated.

#include <string.h>
#include <wasm/driver.h>
#include "http-handler.h"

MonoMethod* handle_request_method;

void mono_wasm_invoke_method_ref(MonoMethod* method, MonoObject** this_arg_in,
                                 void* params[], MonoObject** _out_exc, MonoObject** out_result);

void http_handler_handle_http(http_handler_request_t* req,
                              http_handler_expected_response_error_t* ret0)
{
    if (!handle_request_method)
    {
        handle_request_method = lookup_dotnet_method(
            "Demo.Wasm.Slight",
            "Demo.Wasm.Slight",
            "HttpServer",
            "HandleRequest",
        -1);
    }

    void* method_params[] = { req, ret0 };
    MonoObject* exception;
    MonoObject* result;
    mono_wasm_invoke_method_ref(handle_request_method, NULL, method_params, &exception, &result);
}

But it didn't work. The thrown exception suggested that the Mono runtime wasn't loaded. I went back to studying Mono to learn how it is being loaded. What I've learned is that during compilation a _start() function is being generated. This function performs the steps necessary to load the Mono runtime and wraps the entry point to the .NET code. I could call it, but this would mean going through the Main method and retriggering HttpServer.Serve, which was doomed to fail. I needed to go a level lower. By reading the code of the _start() function I've learned that it calls the mono_wasm_load_runtime function. Maybe I could as well?

...

int mono_runtime_loaded = 0;

...

void http_handler_handle_http(http_handler_request_t* req,
                              http_handler_expected_response_error_t* ret0)
{
    if (!mono_runtime_loaded) {
        mono_wasm_load_runtime("", 0);

        mono_runtime_loaded = 1;
    }

    ...
}

Now it worked. But I wasn't out of the woods yet. What I've just learned meant that to provide dedicated handlers for routes I couldn't rely on registering dedicated methods as part of the Main method flow. I could only register the routes and the handlers needed to be discoverable later, in a new context, with static HandleRequest as the entry point. My thoughts went in the direction of a poor man's attribute-based routing, so I've started with an attribute for decorating handlers.

internal class HttpHandlerAttribute: Attribute
{
    public HttpMethod Method { get; }

    public string Route { get; }

    public HttpHandlerAttribute(HttpMethod method, string route)
    {
        Method = method;
        Route = route;
    }

    ...
}

A poor man's implementation of an attribute-based routing must have an ugly part and it is reflection. To register the routes (and later match them with handlers), the types must be scanned for methods with the attributes. In a production solution, it would be necessary to narrow the scan scope but as this is just a small demo I've decided to keep it simple and scan the whole assembly for static, public and non-public, methods decorated with the attribute. The code supports adding multiple attributes to a single method just because it's simpler than putting proper protections in place. As you can probably guess, I did override the Equals and GetHashCode implementations in the attribute to ensure it behaves nicely as a dictionary key.

internal class HttpRouter
{
    ...

    private static readonly Type HTTP_HANDLER_ATTRIBUTE_TYPE = typeof(HttpHandlerAttribute);

    private static Dictionary<HttpHandlerAttribute, MethodInfo>? _routes;

    ...

    private static void DiscoverRoutes()
    {
        if (_routes is null)
        {
            _routes = new Dictionary<HttpHandlerAttribute, MethodInfo>();

            foreach (Type type in Assembly.GetExecutingAssembly().GetTypes())
            {
                foreach(MethodInfo method in type.GetMethods(BindingFlags.Static |
                                                             BindingFlags.Public |
                                                             BindingFlags.NonPublic))
                {
                    foreach (object attribute in method.GetCustomAttributes(
                             HTTP_HANDLER_ATTRIBUTE_TYPE, false))
                    {
                        _routes.Add((HttpHandlerAttribute)attribute, method);
                    }
                }
            }
        }
    }

    ...
}

With the reflection stuff (mostly) out of the way, I could implement a method that can be called to register all discovered routes and a method to invoke a handler for a route. This implementation is not "safe". I don't do any checks on the reflected MethodInfo to ensure that the method has a proper signature. After all, I can only hurt myself here.

internal class HttpRouter
{
    ...

    internal HttpRouter RegisterRoutes()
    {
        DiscoverRoutes();

        HttpRouter router = this;
        foreach (KeyValuePair<HttpHandlerAttribute, MethodInfo> route in _routes)
        {
            router = router.RegisterRoute(route.Key.Method, route.Key.Route);
        }

        return router;
    }

    internal static HttpResponse? InvokeRouteHandler(HttpRequest request)
    {
        DiscoverRoutes();

        HttpHandlerAttribute attribute = new HttpHandlerAttribute(request.Method,
                                                                  request.Uri.AbsolutePath);
        MethodInfo handler = _routes.GetValueOrDefault(attribute);

        return (handler is null) ? null : (HttpResponse)handler.Invoke(null,
                                                                     new object[] { request });
    }

    ...
}

What remained was small modifications to the HttpServer to use the new methods.

internal static class HttpServer
{
    ...

    public static void Serve(string address)
    {
        ...

        HttpRouter router = HttpRouter.Create()
                            .RegisterRoutes();
        http_server_serve(
            WasiString.FromString(address),
            router.Index,
            out WasiExpected<uint> expected
        );

        ...
    }

    private static unsafe void HandleRequest(ref HttpRequest request,
        out WasiExpected<HttpResponse> result)
    {
        HttpResponse? response = HttpRouter.InvokeRouteHandler(request);

        if (!response.HasValue)
        {
            response = new HttpResponse(404);
            response.Value.SetBody(
                $"Handler Not Found ({request.Method} {request.Uri.AbsolutePath})"
            );
        }

        result = new WasiExpected<HttpResponse>(response.Value);
    }

    ...
}

To test this out, I've created two simple handlers.

[HttpHandler(HttpMethod.GET, "/hello")]
internal static HttpResponse HandleHello(HttpRequest request)
{
    HttpResponse response = new HttpResponse(200);
    response.SetHeaders(new[] { KeyValuePair.Create("Content-Type", "text/plain") });
    response.SetBody($"Hello from Demo.Wasm.Slight!");

    return response;
}

[HttpHandler(HttpMethod.GET, "/goodbye")]
internal static HttpResponse HandleGoodbye(HttpRequest request)
{
    HttpResponse response = new HttpResponse(200);
    response.SetHeaders(new[] { KeyValuePair.Create("Content-Type", "text/plain") });
    response.SetBody($"Goodbye from Demo.Wasm.Slight!");

    return response;
}

That's it. Certainly not complete, certainly not optimal, most likely buggy, and potentially leaking memory. But it works and can be deployed to the cloud.

Running a Slight Application in WASM/WASI Node Pool

To deploy our Slight application to the cloud, we need an AKS Cluster with a WASM/WASI node pool. The process of setting it up hasn't changed since my previous post and you can find all the necessary steps there. Here we can start with dockerizing our application.

As we are dockerizing a Wasm application, the final image should be from scratch. In the case of a Slight application, it should contain two elements: app.wasm and slightfile.toml. The slightfile.toml is a configuration file and its main purpose is to define and provide options for the capabilities needed by the application. In our case, that's just the HTTP Server capability.

specversion = "0.1"

[[capability]]
name = "http"

The app.wasm file is our application. It should have this exact name and be placed at the root of the image. To be able to publish the application, the build stage in our Dockerfile must install the same prerequisites as we did for local development (WASI SDK and wasi-experimental workload).

FROM mcr.microsoft.com/dotnet/sdk:8.0 AS build

RUN curl https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-20/wasi-sdk-20.0-linux.tar.gz -L --output wasi-sdk-20.0-linux.tar.gz
RUN tar -C /usr/local/lib -xvf wasi-sdk-20.0-linux.tar.gz
ENV WASI_SDK_PATH=/usr/local/lib/wasi-sdk-20.0

RUN dotnet workload install wasi-experimental

WORKDIR /src
COPY . .
RUN dotnet publish --configuration Release

FROM scratch

COPY --from=build /src/bin/Release/net8.0/wasi-wasm/AppBundle/Demo.Wasm.Slight.wasm ./app.wasm
COPY --from=build /src/slightfile.toml .

With the infrastructure in place and the image pushed to the container registry, all that is needed is a deployment manifest for Kubernetes resources. It is the same as it was for a Spin application, the only difference is the kubernetes.azure.com/wasmtime-slight-v1 node selector.

apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
  name: "wasmtime-slight-v1"
handler: "slight"
scheduling:
  nodeSelector:
    "kubernetes.azure.com/wasmtime-slight-v1": "true"
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: slight-with-dotnet-8
spec:
  replicas: 1
  selector:
    matchLabels:
      app: slight-with-dotnet-8
  template:
    metadata:
      labels:
        app: slight-with-dotnet-8
    spec:
      runtimeClassName: wasmtime-slight-v1
      containers:
        - name: slight-with-dotnet-8
          image: crdotnetwasi.azurecr.io/slight-with-dotnet-8:latest
          command: ["/"]
---
apiVersion: v1
kind: Service
metadata:
  name: slight-with-dotnet-8
spec:
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
  selector:
    app: slight-with-dotnet-8
  type: LoadBalancer

After applying the above manifest, you can get the service IP with kubectl get svc or from the Azure Portal and execute some requests.

WebAssembly, Cloud, .NET, and Future?

Everything I toyed with here is either a preview or experimental, but it's a glimpse into where WebAssembly in the Cloud is heading.

If I dare to make a prediction, I think that when the wasi-cloud-core proposal reaches a mature enough phase, we will see it supported on AKS (probably it will replace the Spin and Slight available in the current preview). The support for WASI in .NET will also continue to evolve and we will see a non-experimental SDK once the specification gets stable.

For now, we can keep experimenting and exploring. If you want to kick-start your own exploration with what I've described in this post, the source code is here.

Postscript

Yes, I know that this post is already a little bit long, but I wanted to mention one more thing.

When I was wrapping this post, I read the "Extending WebAssembly to the Cloud with .NET" and learned that Steve Sanderson has also built some Slight samples. Despite that, I've decided to publish this post. I had two main reasons for that. First,I dare to think that there is valuable knowledge in this dump of thoughts of mine. Second, those samples take a slightly different direction than mine, and I believe that the fact that you can arrive at different destinations from the same origin is one of the beautiful aspects of software development - something worth sharing.

Azure Functions Integration Testing With Testcontainers

noreply@blogger.com (Tomasz Pęczek) — Tue, 31 Oct 2023 12:03:00 +0000

As developers, we want reassurance that our code functions as expected. We also want to be given that reassurance as fast as possible. This is why we are writing automated tests. We also desire for those tests to be easy to run on our machine or in the worst-case scenario on the build agent as part of the continuous integration pipeline. This is something that can be challenging to achieve when it comes to Azure Functions.

Depending on the language we are using for our Azure Functions, the challenges can be different. Let's take .NET (I guess I haven't surprised anyone with that choice) as an example. In the case of .NET, we can write any unit tests we want (I will deliberately avoid trying to define what is that unit). But the moment we try to move to integration tests, things get tricky. If our Azure Function is using the in-process model, we have an option of crafting a system under test based on the WebJobs host which will be good enough for some scenarios. If our Azure Function is using the isolated worker model there are only two options: accept that our tests will integrate only to a certain level and implement Test Doubles or wait for the Azure Functions team to implement a test worker. This is all far from perfect.

To work around at least some of the above limitations, I've adopted with my teams a different approach for Azure Functions integration testing - we've started using Testcontainers. Testcontainers is a framework for defining through code throwaway, lightweight instances of containers, to be used in test context.

We initially adopted this approach for .NET Azure Functions, but I know that teams creating Azure Functions in different languages also started using it. This is possible because the approach is agnostic to the language in which the functions are written (and the tests can be written using any language/framework supported by Testcontainers).

In this post, I want to share with you the core parts of this approach. It starts with creating a Dockerfile for your Azure Functions.

Creating a Dockerfile for an Azure Functions Container Image

You may already have a Dockerfile for your Azure Functions (for example if you decided to host them in Azure Container Apps or Kubernetes). From my experience, that's usually not the case. That means you need to create a Dockerfile. There are two options for doing that. You can use Azure Functions Core Tools and call func init with the --docker-only option, or you can create the Dockerfile manually. The Dockerfile is different for every language, so until you gain experience I suggest using the command. Once you are familiar with the structure, you will very likely end up with a modified template that you will be reusing with small adjustments. The one below is my example for .NET Azure Functions using the isolated worker model.

FROM mcr.microsoft.com/dotnet/sdk:7.0 AS installer-env
ARG RESOURCE_REAPER_SESSION_ID="00000000-0000-0000-0000-000000000000"
LABEL "org.testcontainers.resource-reaper-session"=$RESOURCE_REAPER_SESSION_ID

WORKDIR /src
COPY function-app/ ./function-app/

RUN dotnet publish function-app \
    --output /home/site/wwwroot

FROM mcr.microsoft.com/azure-functions/dotnet-isolated:4-dotnet-isolated7.0

ENV AzureWebJobsScriptRoot=/home/site/wwwroot \
    AzureFunctionsJobHost__Logging__Console__IsEnabled=true

COPY --from=installer-env ["/home/site/wwwroot", "/home/site/wwwroot"]

What's probably puzzling you right now is that label based on the provided argument. This is something very specific to Testcontainers. The above Dockerfile is for a multi-stage build, so it will generate intermediate layers. Testcontainers has a concept of Resource Reaper which job is to remove Docker resources once they are no longer needed. This label is needed for the Resource Reaper to be able to track those intermediate layers.

Once we have the Dockerfile we can create the test context.

Creating a Container Instance in Test Context

The way you create the test context depends on the testing framework you are going to use and the isolation strategy you want for that context. My framework of choice is xUnit. When it comes to the isolation strategy, it depends 😉. That said, the one I'm using most often is test class. For xUnit that translates to class fixture. You can probably guess that there are also requirements when it comes to the context lifetime management. After all, we will be spinning containers and that takes time. That's why the class fixture must implement IAsyncLifetime to provide support for asynchronous operations.

public class AzureFunctionsTestcontainersFixture : IAsyncLifetime
{
    ...

    public AzureFunctionsTestcontainersFixture()
    { 
        ...
    }

    public async Task InitializeAsync()
    {
        ...
    }

    public async Task DisposeAsync()
    {
        ...
    }
}

There are a couple of things that we need to do here. The first is creating an image based on our Dockerfile. For this purpose, we can use ImageFromDockerfileBuilder. The minimum we need to provide is the location of the Dockerfile (directory and file name). Testcontainers provides us with some handy helpers for getting the solution, project, or Git directory. We also want to set that RESOURCE_REAPER_SESSION_ID argument.

public class AzureFunctionsTestcontainersFixture : IAsyncLifetime
{
    private readonly IFutureDockerImage _azureFunctionsDockerImage;

    public AzureFunctionsTestcontainersFixture()
    {
        _azureFunctionsDockerImage = new ImageFromDockerfileBuilder()
            .WithDockerfileDirectory(CommonDirectoryPath.GetSolutionDirectory(), String.Empty)
            .WithDockerfile("AzureFunctions-Testcontainers.Dockerfile")
            .WithBuildArgument(
                 "RESOURCE_REAPER_SESSION_ID",
                 ResourceReaper.DefaultSessionId.ToString("D"))
            .Build();
    }

    public async Task InitializeAsync()
    {
        await _azureFunctionsDockerImage.CreateAsync();

        ...
    }

    ...
}

With the image in place, we can create a container instance. This will require reference to the image, port binding, and wait strategy. Port binding is something that Testcontainers can almost completely handle for us. We just need to tell which port to bind and the host port can be assigned randomly. The wait strategy is quite important. This is how the framework knows that the container instance is available. We have a lot of options here: port availability, specific message in log, command completion, file existence, successful request, or HEALTHCHECK. What works great for Azure Functions is a successful request to its default page.

public class AzureFunctionsTestcontainersFixture : IAsyncLifetime
{
    private readonly IFutureDockerImage _azureFunctionsDockerImage;

    public IContainer AzureFunctionsContainerInstance { get; private set; }

    ...

    public async Task InitializeAsync()
    {
        await _azureFunctionsDockerImage.CreateAsync();

        AzureFunctionsContainerInstance = new ContainerBuilder()
            .WithImage(_azureFunctionsDockerImage)
            .WithPortBinding(80, true)
            .WithWaitStrategy(
                Wait.ForUnixContainer()
                .UntilHttpRequestIsSucceeded(r => r.ForPort(80)))
            .Build();
        await AzureFunctionsContainerInstance.StartAsync();
    }

    ...
}

The last missing part is the cleanup. We should nicely dispose the container instance and the image.

public class AzureFunctionsTestcontainersFixture : IAsyncLifetime
{
    private readonly IFutureDockerImage _azureFunctionsDockerImage;

    public IContainer AzureFunctionsContainerInstance { get; private set; }

    ...

    public async Task DisposeAsync()
    {
        await AzureFunctionsContainerInstance.DisposeAsync();

        await _azureFunctionsDockerImage.DisposeAsync();
    }
}

Now we are ready to write some tests.

Implementing Integration Tests

At this point, we can start testing our function. We need a test class using our class fixture.

public class AzureFunctionsTests : IClassFixture<AzureFunctionsTestcontainersFixture>
{
    private readonly AzureFunctionsTestcontainersFixture _azureFunctionsTestcontainersFixture;

    public AzureFunctions(AzureFunctionsTestcontainersFixture azureFunctionsTestcontainersFixture)
    {
        _azureFunctionsTestcontainersFixture = azureFunctionsTestcontainersFixture;
    }

    ...
}

Now for the test itself, let's assume that the function has an HTTP trigger. To build the URL of our function we can use the Hostname provided by the container instance and acquire the host port by calling .GetMappedPublicPort. This means that the test only needs to create an instance of HttpClient, make a request, and assert the desired aspects of the response. The simplest test I could think of was to check for a status code indicating success.

public class AzureFunctionsTests : IClassFixture<AzureFunctionsTestcontainersFixture>
{
    private readonly AzureFunctionsTestcontainersFixture _azureFunctionsTestcontainersFixture;

    ...

    [Fact]
    public async Task Function_Request_ReturnsResponseWithSuccessStatusCode()
    {
        HttpClient httpClient = new HttpClient();
        var requestUri = new UriBuilder(
            Uri.UriSchemeHttp,
            _azureFunctionsTestcontainersFixture.AzureFunctionsContainerInstance.Hostname,
            _azureFunctionsTestcontainersFixture.AzureFunctionsContainerInstance.GetMappedPublicPort(80),
            "api/function"
        ).Uri;

        HttpResponseMessage response = await httpClient.GetAsync(requestUri);

        Assert.True(response.IsSuccessStatusCode);
    }
}

And Voila. This will run on your machine (assuming you have Docker) and in any CI/CD environment which build agents have Docker pre-installed (for example Azure DevOps or GitHub).

Adding Dependencies

What I've shown you so far covers the scope of the function itself. This is already beneficial because it allows for verifying if dependencies are registered properly or if the middleware pipeline behaves as expected. But Azure Functions rarely exist in a vacuum. There are almost always dependencies and Testcontainers can help us with those dependencies as well. There is a wide set of preconfigured implementations that we can add to our test context. A good example can be storage. In the majority of cases, storage is required to run the function itself. For local development, Azure Functions are using the Azurite emulator and we can do the same with Testcontainers as it is available as a ready-to-use module. To add it to the context you just need to reference the proper NuGet package and add a couple of lines of code.

public class AzureFunctionsTestcontainersFixture : IAsyncLifetime
{
    ...

    public AzuriteContainer AzuriteContainerInstance { get; private set; }

    ...

    public async Task InitializeAsync()
    {
        AzuriteContainerInstance = new AzuriteBuilder().Build();
        await AzuriteContainerInstance.StartAsync();

        ...
    }

    public async Task DisposeAsync()
    {
        ...

        await AzuriteContainerInstance.DisposeAsync();
    }
}

We also need to point Azure Functions to use this Azurite container by setting the AzureWebJobsStorage parameter.

public class AzureFunctionsTestcontainersFixture : IAsyncLifetime
{
    ...

    public async Task InitializeAsync()
    {
        ...

        AzureFunctionsContainerInstance = new ContainerBuilder()
            ...
            .WithEnvironment("AzureWebJobsStorage", AzuriteContainerInstance.GetConnectionString())
            ...
            .Build();

        ...
    }

    ...
}

That's it. Having Azurite in place also enables testing functions that use triggers and bindings based on Azure Storage. There are also ready-to-use modules for Redis, Azure Cosmos DB, Azure SQL Edge, MS SQL, Kafka, or RabbitMQ. So there is quite good out-of-the-box coverage for potential Azure Functions dependencies. Some other dependencies can be covered by creating containers yourself (for example with an unofficial Azure Event Grid simulator). That said, some dependencies can be only satisfied by a real thing (at least for now).

A Powerful Tool in Your Toolbox

Is Testcontainers a solution for every integration problem - no. Should Testcontainers be your default choice when thinking about integration tests - also no. But it is a very powerful tool and you should be familiar with it, so you can use it when appropriate.

Revisiting Various Change Feeds Consumption in .NET

noreply@blogger.com (Tomasz Pęczek) — Tue, 10 Oct 2023 11:54:00 +0000

The first time I wrote about change feeds consumption from .NET (ASP.NET Core to be more precise) was back in 2018 in the context of RethinkDB. It was always a very powerful concept. Having access to an ordered flow of information about changes to items is a low-entry enabler for various event-driven, stream processing, or data movement scenarios. As a result, over the years, this capability (with various name variations around the words change, stream, and feed) has found its way to many databases and sometimes even other storage services. The list includes (but is not limited to) MongoDB, RavenDB, Cosmos DB, DynamoDB or Azure Blob Storage (in preview).

As I was cleaning up and updating a demo application that shows how to consume and expose various change feeds from ASP.NET Core, I decided to write down some notes to refresh the content from my previous posts.

IAsyncEnumerable as Universal Change Feed Abstraction

When I started working with change feeds over 5 years ago, I initially didn't put them behind any abstraction. I like to think that I was smart and avoided premature generalization. The abstraction came after a couple of months when I could clearly see that I was implementing the same concepts through similar components in different projects where teams were using RethinkDB, MongoDB, or Cosmos DB. The abstraction that I started advocating back then looked usually like this.

public interface IChangeFeed<T>
{
    T CurrentChange { get; }

    Task<bool> MoveNextAsync(CancellationToken cancelToken = default(CancellationToken));
}

In retrospect, I'm happy with this abstraction, because around two or more years later, when those teams and projects started to adopt C# 8 and .NET Core 3 (or later versions), refactoring all those implementations was a lot easier. C# 8 has brought async streams, a natural programming model for asynchronous streaming data sources. Asynchronous streaming data source is exactly what change feeds are and modeling them through IAsyncEnumerable results in nice and clean consumption patterns. This is why currently I advocate for using IAsyncEnumerable as a universal change feed abstraction. The trick to properly using that abstraction is defining the right change representation to be returned. That should depend on change feed capabilities and actual needs in a given context. Not all change feeds are the same. Some of them can provide information on all operations performed on an item, and some only on a subset. Some can provide old and new value, and some only the old. Your representation of change should consider all that. In the samples ahead I'm avoiding this problem by reducing the change representation to the changed version of the item.

Azure Cosmos DB Change Feed

Azure Cosmos DB change feed is the second (after the RethinkDB one) I've been writing about in the past. It's also the one which consumption has seen the most evolution through time.

The first consumption model was quite complicated. It required going through partition key ranges, building document change feed queries for them, and then obtaining enumerators. This whole process required managing its state, which resulted in non-trivial code. It's good that it has been deprecated as part of Azure Cosmos DB .NET SDK V2, and it's going out of support in August 2024.

Azure Cosmos DB .NET SDK V3 has brought the second consumption model based on change feed processor. The whole inner workings of consuming the change feed have been enclosed within a single class, which reduced the amount of code required. But change feed processor has its oddities. It requires an additional container - a lease container that deals with previously described state management. This is beneficial in complex scenarios as it allows for coordinated processing by multiple workers, but becomes an unnecessary complication for simple scenarios. It also provides only a push-based programming model. The consumer must provide a delegate to receive changes. Once again this is great for certain scenarios, but leads to awkward implementation when you want to abstract change feed as a stream.

The story doesn't end there, version 3.20.0 of Azure Cosmos DB .NET SDK has introduced the third consumption model based on change feed iterator. It provides a pull-based alternative to change feed processor for scenarios where it's more appropriate. With the change feed iterator the control over the pace of consuming the changes is given back to the consumer. State management is also optional, but it's the consumer's responsibility to persist continuation tokens if necessary. Additionally, the change feed iterator brings the option of obtaining a change feed for a specific partition key.

The below snippet shows a very simple consumer implementation of the change feed iterator model - no state management, just starting the consumption from a certain point in time and waiting one second before polling for new changes.

public async IAsyncEnumerable<T> FetchFeed(
    [EnumeratorCancellation] CancellationToken cancellationToken = default)
{
    FeedIterator<T> changeFeedIterator = _container.GetChangeFeedIterator<T>(
        ChangeFeedStartFrom.Time(DateTime.UtcNow),
        ChangeFeedMode.LatestVersion
    );

    while (changeFeedIterator.HasMoreResults && !cancellationToken.IsCancellationRequested)
    {
        FeedResponse<T> changeFeedResponse = await changeFeedIterator
            .ReadNextAsync(cancellationToken);

        if (changeFeedResponse.StatusCode == HttpStatusCode.NotModified)
        {
            await Task.Delay(TimeSpan.FromSeconds(1), cancellationToken);
        }
        else
        {
            foreach (T item in changeFeedResponse)
            {
                yield return item;
            }
        }
    }
}

MongoDB Change Feed

MongoDB is probably the most popular NoSQL choice among the teams that I've been working with, which doesn't use cloud PaaS databases for their needs. Among its many features, it has quite powerful change feed (a.k.a. Change Streams) capability.

The incoming change information can cover a wide spectrum of operations which can come from a single collection, database, or entire deployment. If the operation relates to a document, the change feed can provide the current version, the previous version, and the delta. There is also support for resume tokens which can be used to manage state if needed.

One unintuitive thing when it comes to MongoDB change feed is that it's only available when you are running a replica set or a sharded cluster. This doesn't mean that you have to run a cluster. You can run a single instance as a replica set (even in a container), you just need the right configuration (you will find a workflow that handles such a deployment to Azure Container Instances in the demo repository).

The consumption of MongoDB change feed is available through the Watch and WatchAsync methods available on IMongoCollection, IMongoDatabase, and IMongoClient instances. The below snippet watches a single collection and configures the change feed to return the current version of the document. You can also provide a pipeline definition when calling Watch or WatchAsync to filter the change feed (for example to monitor only specific operation types).

public async IAsyncEnumerable<T> FetchFeed(
    [EnumeratorCancellation]CancellationToken cancellationToken = default)
{
    IAsyncCursor<ChangeStreamDocument<T>> changefeed = await _collection.WatchAsync(
        new ChangeStreamOptions { FullDocument = ChangeStreamFullDocumentOption.UpdateLookup },
        cancellationToken: cancellationToken
    );

    while (!cancellationToken.IsCancellationRequested)
    {
        while (await changefeed.MoveNextAsync(cancellationToken))
        {
            IEnumerator<ChangeStreamDocument<T>>  changefeedCurrentEnumerator = changefeed
                .Current.GetEnumerator();

            while (changefeedCurrentEnumerator.MoveNext())
            {
                if (changefeedCurrentEnumerator.Current.OperationType
                    == ChangeStreamOperationType.Insert)
                {
                    yield return changefeedCurrentEnumerator.Current.FullDocument;
                }

                ...
            }
        }

        await Task.Delay(_moveNextDelay, cancellationToken);
    }
}

Azure Blob Storage Change Feed

Azure Blob Storage is the odd one on this list because it's an object storage, not a database. Its change feed provides information about changes to blobs and blobs metadata in an entire storage account. Under the hood the change feed is implemented as a special container (yes it's visible, yes you can take a look) which is being created once you enable it. As it is a container you should consider the configuration of the retention period as it will affect your costs.

There is one more important aspect of Azure Blob Storage change feed when considering its usage - latency. It's pretty slow. It can take minutes for changes to appear.

From the consumption perspective, it follows the enumerator approach. You can obtain the enumerator by calling BlobChangeFeedClient.GetChangesAsync. The enumerator is not infinite, it will return the changes currently available and once you process them you have to poll for new ones. This makes managing the continuation tokens required even for a local state. What is unique is that you can request changes within a specified time window.

The change feed supports six events in the latest schema version. In addition to expected ones like created or deleted, there are some interesting ones like tier changed. The information never contains the item, which shouldn't be surprising as in the context of object storage this would be quite risky.

The below snippet streams the change feed by locally managing the continuation token and for changes that represent blob creation, it downloads the current version of the item.

public async IAsyncEnumerable<T> FetchFeed(
    [EnumeratorCancellation]CancellationToken cancellationToken = default)
{
    string? continuationToken = null;

    TokenCredential azureCredential = new DefaultAzureCredential();

    BlobServiceClient blobServiceClient = new BlobServiceClient(_serviceUri, azureCredential);
    BlobChangeFeedClient changeFeedClient = _blobServiceClient.GetChangeFeedClient();

    while (!cancellationToken.IsCancellationRequested)
    {
        IAsyncEnumerator<Page<BlobChangeFeedEvent>> changeFeedEnumerator = changeFeedClient
            .GetChangesAsync(continuationToken)
            .AsPages()
            .GetAsyncEnumerator();

        while (await changeFeedEnumerator.MoveNextAsync())
        {
            foreach (BlobChangeFeedEvent changeFeedEvent in changeFeedEnumerator.Current.Values)
            {
                if ((changeFeedEvent.EventType == BlobChangeFeedEventType.BlobCreated)
                    && changeFeedEvent.Subject.StartsWith($"/blobServices/default/containers/{_container}"))
                {
                    BlobClient createdBlobClient = new BlobClient(
                        changeFeedEvent.EventData.Uri,
                        azureCredential);

                    if (await createdBlobClient.ExistsAsync())
                    {
                        MemoryStream blobContentStream =
                            new MemoryStream((int)changeFeedEvent.EventData.ContentLength);
                        await createdBlobClient.DownloadToAsync(blobContentStream);
                        blobContentStream.Seek(0, SeekOrigin.Begin);

                        yield return JsonSerializer.Deserialize<T>(blobContentStream);
                    }
                }
            }

            continuationToken = changeFeedEnumerator.Current.ContinuationToken;
        }

        await Task.Delay(TimeSpan.FromSeconds(1), cancellationToken);
    }
}

There Is More

The above samples are in no way exhaustive. They don't show all the features of given change feeds and they don't show all the change feeds out there. But they are a good start, this is why I've been evolving them for the past five years.

Deploying a Dapr Sidecar to Azure Container Instances

noreply@blogger.com (Tomasz Pęczek) — Tue, 05 Sep 2023 11:08:00 +0000

Containers have become one of the main, if not the main, ways to modularize, isolate, encapsulate, and package applications in the cloud. The sidecar pattern allows for taking this even further by allowing the separation of functionalities like monitoring, logging, or configuration from the business logic. This is why I recommend that the teams who are adopting containers adopt sidecars as well. One of my preferred suggestions is Dapr which can bring early value by providing abstractions for message broker integration, encryption, observability, secret management, state management, or configuration management.

To my surprise, many conversations starting around adopting sidecars quickly deviate to "we should set up a Kubernetes cluster". It's almost like there are only two options out there - you either run a standalone container or you need Kubernetes for anything more complicated. This is not the case. There are multiple ways to run containers and you should choose the one which is most suitable for your current context. Many of those options will give you more sophisticated features like sidecars or init containers while your business logic is still in a single container. Sidecars give here an additional benefit of enabling later evolution to more complex container hosting options without requirements for code changes.

In the case of Azure, such a service that enables adopting sidecars at an early stage is Azure Container Instances.

Quick Reminder - Azure Container Instances Can Host Container Groups

Azure Container Instances provides a managed approach for running containers in a serverless manner, without orchestration. What I've learned is that a common misconception is that Azure Container Instances can host only a single container. That is not exactly the truth, Azure Container Instances can host a container group (if you're using Linux containers 😉).

A container group is a collection of containers scheduled on the same host and sharing lifecycle, resources, or local network. The container group has a single public IP address, but the publicly exposed ports can forward to ports exposed on different containers. At the same time, all the containers within the group can reach each other via localhost. This is what enables the sidecar pattern.

How to create a container group? There are three options:

With ARM/Bicep
With Azure CLI by using YAML file
With Azure CLI by using Docker compose file

I'll go with Bicep here. The Microsoft.ContainerInstance namespace contains only a single type which is containerGroups. This means that from ARM/Bicep perspective there is no difference if you are deploying a standalone container or a container group - there is a containers list available as part of the resource properties where you specify the containers.

resource containerGroup 'Microsoft.ContainerInstance/containerGroups@2023-05-01' = {
  name: CONTAINER_GROUP
  location: LOCATION
  ...
  properties: {
    sku: 'Standard'
    osType: 'Linux'
    ...
    containers: [
      ...
    ]
    ...
  }
}

How about a specific example? I've mentioned that Dapr is one of my preferred sidecars, so I'm going to use it here.

Running Dapr in Self-Hosted Mode Within a Container Group

Dapr has several hosting options. It can be self-hosted with Docker, Podman, or without containers. It can be hosted in Kubernetes with first-class integration. It's also available as a serverless offering - part of Azure Container Apps. The option interesting us in the context of Azure Container Instances is self-hosted with Docker, but from that list, you could pick up how Dapr enables easy evolution from Azure Container Instances to Azure Container Apps, Azure Kubernetes Services or non-Azure Kubernetes clusters.

But before we will be ready to deploy the container group, we need some infrastructure around it. We should start with a resource group, container registry and managed identity.

az group create -l $LOCATION -g $RESOURCE_GROUP
az acr create -n $CONTAINER_REGISTRY -g $RESOURCE_GROUP --sku Basic
az identity create -n $MANAGED_IDENTITY -g $RESOURCE_GROUP

We will be using the managed identity for role-based access control where possible, so we should reference it as the identity of the container group in our Bicep template.

resource managedIdentity 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' existing = {
  name: MANAGED_IDENTITY
}

resource containerGroup 'Microsoft.ContainerInstance/containerGroups@2023-05-01' = {
  ...
  identity: {
    type: 'UserAssigned'
    userAssignedIdentities: {
      '${managedIdentity.id}': {}
    }
  }
  properties: {
    ...
  }
}

The Dapr sidecar requires a components directory. It's a folder that will contain YAML files with components definitions. To provide that folder to the Dapr sidecar container, we have to mount it as a volume. Azure Container Instances supports mounting an Azure file share as a volume, so we have to create one.

az storage account create -n $STORAGE_ACCOUNT -g $RESOURCE_GROUP --sku Standard_LRS
az storage share create -n daprcomponents --account-name $STORAGE_ACCOUNT

The created Azure file share needs to be added to the list of volumes that can be mounted by containers in the group. Sadly, the integration between Azure Container Instances and Azure file share doesn't support role-based access control, an access key has to be used.

...

resource storageAccount 'Microsoft.Storage/storageAccounts@2022-09-01' existing = {
  name: STORAGE_ACCOUNT
}

resource containerGroup 'Microsoft.ContainerInstance/containerGroups@2023-05-01' = {
  ...
  properties: {
    ...
    volumes: [
      {
        name: 'daprcomponentsvolume'
        azureFile: {
          shareName: 'daprcomponents'
          storageAccountKey: storageAccount.listKeys().keys[0].value
          storageAccountName: storageAccount.name
          readOnly: true
        }
      }
    ]
    ...
  }
}

We also need to assign the AcrPull role to the managed identity so it can access the container registry.

az role assignment create --assignee $MANAGED_IDENTITY_OBJECT_ID \
    --role AcrPull \
    --scope "/subscriptions/$SUBSCRIPTION_ID/resourcegroups/$RESOURCE_GROUP/providers/Microsoft.ContainerRegistry/registries/$CONTAINER_REGISTRY"

I'm skipping the creation of the image for the application with the business logic, pushing it to the container registry, adding its definition to the containers list, and exposing needed ports from the container group - I want to focus on the Dapr sidecar.

In this example, I will be grabbing the daprd image from the Docker Registry.

The startup command for the sidecar is ./daprd. We need to provide a --resources-path parameter which needs to point to the path where the daprcomponentsvolume will be mounted. I'm also providing the --app-id parameter. This parameter is mostly used for service invocation (it won't be the case here and I'm not providing --app-port) but Dapr is using it also in different scenarios (for example as partition key for some state stores).

Two ports need to be exposed from this container (not publicly): 3500 is the default HTTP endpoint port and 50001 is the default gRPC endpoint port. There is an option to change both ports through configuration if they need to be taken by some other container.

resource containerGroup 'Microsoft.ContainerInstance/containerGroups@2023-05-01' = {
  ...
  properties: {
    ...
    containers: [
      ...
      {
        name: 'dapr-sidecar'
        properties: {
          image: 'daprio/daprd:1.10.9'
          command: [ './daprd', '--app-id', 'APPLICATION_ID', '--resources-path', './components']
          volumeMounts: [
            {
              name: 'daprcomponentsvolume'
              mountPath: './components'
              readOnly: true
            }
          ]
          ports: [
            { 
              port: 3500
              protocol: 'TCP'
            }
            { 
              port: 50001
              protocol: 'TCP'
            }
          ]
          ...
        }
      }
    ]
    ...
  }
}

I've omitted the resources definition for brevity.

Now the Bicep template can be deployed.

az deployment group create -g $RESOURCE_GROUP -f container-group-with-dapr-sidecar.bicep

The below diagram visualizes the final state after the deployment.

Configuring a Dapr Component

We have a running Dapr sidecar, but we have yet to make it truly useful. To be able to use APIs provided by Dapr, we have to provide the mentioned earlier components definitions which will provide implementation for those APIs. As we already have a storage account as part of our infrastructure, a state store component seems like a good choice. Dapr supports quite an extensive list of stores, out of which two are based on Azure Storage: Azure Blob Storage and Azure Table Storage. Let's use the Azure Table Storage one.

First I'm going to create a table. This is not a required step, the component can do it for us, but let's assume we want to seed some data manually before the deployment.

Second, the more important operation is granting needed permissions to the storage account. Dapr has very good support for authenticating to Azure which includes managed identities and role-based access control, so I'm just going to assign the Storage Table Data Reader role to our managed identity for the scope of the storage account.

az storage table create -n $TABLE_NAME --account-name $STORAGE_ACCOUNT
az role assignment create --assignee $MANAGED_IDENTITY_OBJECT_ID \
    --role "Storage Table Data Contributor" \
    --scope "/subscriptions/$SUBSCRIPTION_ID/resourcegroups/$RESOURCE_GROUP/providers/Microsoft.Storage/storageAccounts/$STORAGE_ACCOUNT"

The last thing we need is the component definition. The component type we want is state.azure.tablestorage. The name is what we will be using when making calls with a Dapr client. As we are going to use managed identity for authenticating, we should provide accountName, tableName, and azureClientId as metadata. I'm additionally setting skipCreateTable because I created the table earlier and the component will fail on an attempt to create it once again.

apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
  name: state.table.<TABLE_NAME>
spec:
  type: state.azure.tablestorage
  version: v1
  metadata:
  - name: accountName
    value: <STORAGE_ACCOUNT>
  - name: tableName
    value: <TABLE_NAME>
  - name: azureClientId
    value: <Client ID of MANAGED_IDENTITY>
  - name: skipCreateTable
    value: true

The file with the definition needs to be uploaded to the file share which is mounted as the components directory. The Azure Container Instances need to be restarted for the component to be loaded. We can quickly verify if it has been done by taking a look at logs.

time="2023-08-31T21:25:22.5325911Z"
level=info
msg="component loaded. name: state.table.<TABLE_NAME>, type: state.azure.tablestorage/v1"
app_id=APPLICATION_ID
instance=SandboxHost-638291138933285823
scope=dapr.runtime
type=log
ver=1.10.9

Now you can start managing your state with a Dapr client for your language of choice or with HTTP API if one doesn't exist.

The Power of Abstraction, Decoupling, and Flexibility

As you can see, the needed increase in complexity (when compared to a standalone container hosted in Azure Container Instances) is not that significant. At the same time, the gain is. Dapr allows us to abstract all the capabilities it provides in the form of building blocks. It also decouples the capabilities provided by building blocks from the components providing implementation. We can change Azure Table Storage to Azure Cosmos DB if it better suits our solution, or to AWS DynamoDB if we need to deploy the same application to AWS. We also now have the flexibility of evolving our solution when the time comes to use a more sophisticated container offering - we just need to take Dapr with us.

DevOps Practices for Azure Infrastructure - Continuous Operations & Continuous Monitoring

noreply@blogger.com (Tomasz Pęczek) — Tue, 01 Aug 2023 10:50:00 +0000

This series on implementing DevOps practices for Azure infrastructure is nearing its conclusion. The last part remaining is completing the operations side of the loop.

This brings focus to the last two practices on our list:

Continuous Planning
Continuous Integration
Continuous Delivery
Continuous Deployment
Continuous Testing
Continuous Operations
Continuous Monitoring

Continuous Operations & Continuous Monitoring

The Continuous Operations and Continuous Monitoring practices are closely tied together. They jointly serve the goal of ensuring the overall reliability, resiliency, and security of solutions. The majority of capabilities supporting that goal are within the scope of Continuous Operations practice and cover aspects like compliance enforcement, cost management, proactive maintenance, security posture management, and intelligence-driven responses to operational and security events. That said, most of those capabilities can't be achieved without capabilities coming from Continuous Monitoring practice. There can be no cost management without cost tracking. There is no way to have proactive maintenance and intelligence-driven responses without gathering observability signals, configuring alerts, and building dashboards.

Organizations usually have the capabilities covered by Continuous Operations and Continuous Monitoring already established, but often they are not aligned with DevOps cultural philosophies. This means that implementing those practices is often about addressing gaps around automation, collaboration, continuous feedback, and continuous improvement.

But before we start addressing those gaps, it's worth making sure that the capabilities have been established on the right foundations, as Azure provides a wide range of services to support us here:

Azure Policy for compliance enforcement.
Azure Monitor with its insights, visualization, analytics, and response stack for gathering observability signals, configuring alerts, and building dashboards.
Microsoft Defender for Cloud for workload protection and security posture management.
Azure Sentinel for security information and event management (SIEM) as well as security orchestration, automation, and response (SOAR).
Azure Automation and Azure Logic Apps for automating event-based intelligence-driven responses and orchestrating proactive maintenance.
Microsoft Cost Management and Billing for cost tracking and management.

With the right foundations in place, we can focus on aspects that make the difference between "being DevOps" and "not being DevOps". The most crucial one is ensuring that everyone has access to information on how the part they are responsible for is behaving in production.

All Teams Need Observability Signals

As you may remember from the post on Continuous Delivery and Continuous Deployment, at certain sizes solutions often start moving from centralized ownership to being owned by multiple independent applications teams and an environment team. This dynamics needs to be reflected in monitoring architecture as well. A single, centralized monitoring service, although needed by the environment team, may not be sufficient. This is why mature monitoring implementations utilize resource-context observability signals and granular insights, visualization, and analytics workspaces from which the signals are later centrally aggregated. This approach enables every application team to have direct access to its signals, configure alerts and build dashboards, while the environment team still has visibility into the whole picture.

This approach also enables democratization when it comes to the tools itself. The native observability stack in Azure is provided by Azure Monitor, but it no longer means that application teams are fully limited to Application Insights. If they prefer they can use Prometheus and Grafana for metrics (which is great when they need to be more cloud agnostic and looking at adopting Open Telemetry).

Of course, such a democratized monitoring architecture cannot be left without governance. There need to be rules around observability signals granularity, retention, and archiving data to cool-tier storage. Otherwise, we can be very unpleasantly surprised by the cost of our monitoring implementation.

Automated responses should also be exporting proper context information to the respective tools - because part of automated response should be creating a proper item in the collaboration tool to ensure continuous feedback. What item should that be? That depends on the event category.

Operational Events Should Create Issues

From the infrastructure perspective, there are usually two main types of operational events that are potentially interesting:

Resources events like creation, deletion, or modification
Alerts defined in Azure Monitor

The usage of resource events often covers adding special tags, granting permissions to special groups, or reacting to delete/create/update operation fails.

Alerts are usually raised when the measured state of the system deviates from what is considered a baseline. To name just a few examples, this can mean networking issues, an erroneously stopped VM, or a resource reaching its capacity.

The remediation for every resource event or alert can be different. In some cases, the remediation can be fully automated (restarting a VM, truncating tables in a database, or increasing RUs for Azure Cosmos DB). In some cases, all that is needed is just a notification to deal with the problem at the earliest convenience (failure to delete a resource). There are also those cases that require waking up an engineer immediately (networking issues).

In the first post of this series, I wrote that the cornerstone of implementing DevOps practices for Azure infrastructure is infrastructure as code and the Git ecosystem used for collaboration. This means that regardless if the remediation is fully automated or an engineer needs to be engaged, part of the process should be issue creation (if the remediation has been already performed that issue can be closed and exist just for tracking purposes). In the stack I've chosen for this series, the Git ecosystem is GitHub. Integrating GitHub issue creation into the response workflow is not a huge challenge, because there is a ready-to-use GitHub connector for Azure Logic Apps. So, if we consider alerts, this means that we can build an automated response flow by using Azure Monitor Alerts, Azure Monitor Action Group, and Azure Logic Apps.

Almost identical flow can be built for the resources events if we use Azure Event Grid in place of Azure Monitor (as Azure Event Grid supports resource groups and subscriptions as sources).

This is the approach that should be applied to ensure collaboration and continuous feedback when it comes to operational events, how about security events?

Security Events Should Create Vulnerabilities

Security events have a specific lifecycle that falls under the responsibility of the organization's Security Operations Center (SOC). It's the SOC that uses available observability signals, CVE alerting platforms like OpenCVE, and other tools to detect, investigate, and remediate threats. In the case of Azure, Azure Sentinel is the one-stop shop to build and automate this responsibility.

That said, SOC usually deals with the immediate remediation of a threat. For example, SOC operator or automation may determine that to mitigate a threat a specific resource needs to be isolated because a new CVE has been disclosed. The only action performed will be isolation - the responsibility for mitigating the CVE is with the application or environment team. In such cases, the SOC operator or automation should report the specific vulnerability with context and findings in the collaboration tool. When using GitHub as the Git ecosystem for collaboration, a great way to report such vulnerabilities may be through security advisories.

Security advisories facilitate the process of reporting, discussing, and fixing vulnerabilities. Creating security advisories requires admin or security manager role within the repository, so the integration must be designed properly to avoid excessive permissions within the organization. My approach is to create a GitHub App. GitHub Apps use OAuth 2.0 and can act on behalf of a user, which in this case will be SOC operator or automation. To make the creation of security advisories available directly from Azure Sentinel, I expose a webhook from the GitHub App which can be called by a Playbook.

Providing automated tools which don't require context switching from the SOC perspective removes roadblocks, which is crucial for the adoption of collaboration and continuous feedback between otherwise disconnected teams. This is the true spirit of DevOps.

Infrastructure Drift Detection

There is one capability in the context of Continuous Monitoring and Continuous Operations, which is very specific to infrastructure - detecting drift.

As I have shown through the series, if we want to implement DevOps practices for Azure infrastructure, the infrastructure should be changed only through modifying and deploying its code. The repository should be the single source of truth. But sometimes, when there is pressure, stress, or time constraints (for example when solving a critical issue) engineers do take shortcuts and modify the infrastructure directly. It's not that big of an issue if such an engineer will later reflect the changes in the infrastructure code. But humans are humans and they sometimes forget. This can cause the environment to drift from its source of truth and creates potential risks from applied change being reverted after the deployment to deployment failures. This is why detecting drift is important.

Infrastructure drift detection is a complex problem. Depending on chosen stack there are different tools you can use to make it as sophisticated as you need. Here, as an example, I'm going to show a mechanism that can be set up quickly based on the stack I've already used throughout this series. It's far from perfect, but it's a good start. It's using the what-if command, which I've already been using for creating previews of changes as part of Continuous Integration implementation.

az deployment group what-if \
    --resource-group rg-devops-practices-sample-application-prod \
    --template-file applications/sample-application/application.bicep \
    --mode Complete \
    --no-pretty-print

You may notice two differences between the usage of what-if for previews and drift detection.

The first difference is the Complete deployment mode. The difference between Incremental (the default) and Complete deployment modes is that in the case of the second resources that exist in the resource group but aren't specified in the template will be deleted instead of ignored.

The second difference is the output format. For the previews, I wanted something human-readable, but here I prefer something which will be easy to process programmatically. Providing the --no-pretty-print switch changes the output format to JSON. Below you can see a snippet of it.

{
  "changes": [
    {
      "after": null,
      "before": {
        "name": "kvsampleapplication",
        ...
      },
      "changeType": "Delete",
      ...
    },
    {
      "after": {
        "name": "id-sampleapplication-gd3f7mnjwpuyu",
        ...
      },
      "before": {
        "name": "id-sampleapplication-gd3f7mnjwpuyu",
        ...
      },
      "changeType": "NoChange",
      ...
    },
    ...
  ],
  "error": null,
  "status": "Succeeded"
}

Our attention should focus on the changeType property. It provides information on what will happen with the resource after the deployment. The possible values are: Create, Delete, Ignore, NoChange, Modify, and Deploy. Create, Delete, and NoChange are self-explanatory. The Ignore value should not be present in the case of Complete deployment mode unless limits (number of nested templates or expanding time) have been reached - in such case, it will mean that the resource hasn't been evaluated. Modify and Deploy are tricky. They mean that the properties of the resource will be changed after the deployment. Unfortunately, the Resource Manager is not perfect here and those two can give false positive predictions. This is why this technique is far from perfect - the only drift that can be reliably detected are missing resources or resources which shouldn't exist. But, as I said, it's a good start as we can quickly create a GitHub Actions workflow that will be performing the detection. Let's start by checking out the deployed tag and connecting to Azure.

...

env:
  TAG: sample-application-v1.0.0

jobs:
  drift-detection:
    runs-on: ubuntu-latest
    permissions:
      id-token: write
      contents: read
    steps:
    - name: Checkout
      uses: actions/checkout@v3
      with:
        ref: ${{ env.TAG }}
    - name: Azure Login
      uses: azure/login@v1
      with:
        ...
  ...

The next step is to run a script that will call what-if and process the results to create an array of detected changes.

...

env:
  ...
  RESOURCE_GROUP: 'rg-devops-practices-sample-application-prod'

jobs:
  drift-detection:
    ...
    steps:
    ...
    - name: Detect infrastructure drift
      shell: pwsh
      run: |
        $issues = @()

        $drift = az deployment group what-if `
          --resource-group $env:RESOURCE_GROUP `
          --template-file applications/sample-application/application.bicep `
          --mode Complete `
          --no-pretty-print | ConvertFrom-Json

        foreach ($change in $drift.Changes)
        {
          switch ($change.changeType)
          {
            'Create'
            {
              $issues += @{
                ResourceName = $change.after.name
                Description = 'Defined resource doesn''t exist'
              }
            }
            'Delete'
            {
              $issues += @{
                ResourceName = $change.before.name
                Description = 'Undefined resource exists'
              }
            }
          }
        }

        'DRIFT_ISSUES<> $env:GITHUB_ENV
        $issues | ConvertTo-Json -AsArray >> $env:GITHUB_ENV
        'EOF' >> $env:GITHUB_ENV
  ...

Having all the changes gathered, we can use the proven script action to create an issue for every detected change.

...

jobs:
  drift-detection:
    ...
    permissions:
      ...
      issues: write
    steps:
    ...
    - name: Report detected infrastructure drift
      uses: actions/github-script@v6
      with:
        script: |
          const issues = JSON.parse(process.env.DRIFT_ISSUES);
          for (const issue of issues) {
            github.rest.issues.create({
              owner: context.repo.owner,
              repo: context.repo.repo,
              title: '[DRIFT DETECTED] ' + issue.Description + ' (' + issue.ResourceName + ')'
            });
          }
  ...

We can have this action running regularly. It will be creating nice issues like the one in the screenshot and will give us some start in drift detection.

The Journey Never Ends

With Continuous Operations and Continuous Monitoring practices, we have closed the loop.

But the nature of a loop is that an end is also the beginning. The implementation of DevOps is never "done". It's a direct consequence of its core cultural philosophies: continuous feedback and continuous improvement. Regardless of how your initial implementation will look, you should constantly evaluate it in the context of the ecosystem around and evolve. This will mean modifying the implementation of already established practices, but also implementing new complementary ones (like Continuous Learning or Continuous Documentation).

The goal of this series was to draw the overall picture and provide examples that will bring that picture to life. The accompanying repository contains working workflows that can kickstart your journey.

DevOps Practices for Azure Infrastructure - Continuous Testing

noreply@blogger.com (Tomasz Pęczek) — Tue, 11 Jul 2023 12:11:00 +0000

So far, as part of my series on implementing DevOps practices for Azure infrastructure, I've walked through Continuous Integration, Continuous Delivery, and Continuous Deployment. In many conversations I had around implementing DevOps I've heard an opinion that once you have CI/CD (or CI/CD/CD) you have DevOps. That's not true. DevOps is about a continuous loop of feedback, automation, collaboration, and improvement. As you can see in the picture below, those three practices give only about half of that loop and cover mostly the development side.

This is why there are more practices on the list:

To complete the loop and speak about complete DevOps implementation, it's time to start implementing practices that provide feedback from the deployed environment to the teams and automate operations concerns. In this post, I'm going to discuss Continuous Testing.

The goal of Continuous Testing is to ensure quality at different stages of the development life cycle. It's a practice that applies to both sides of the loop. We have already encountered it as part of the Continuous Integration practice. It's sometimes present as part of Continuous Delivery (for example running specific tests when versions of modules referenced from the environment repository are being updated) and it should be present as part of Continuous Deployment and later in the form of after-deployment tests. The after-deployment tests are what I want to focus on.

Discussing tests often revolves around discussing two aspects: tools to be used for implementation and types of tests to be implemented. Those are two main ingredients used to create a test strategy (of course a mature test strategy covers much more, but it's a discussion for a different occasion). Let's first consider the tools.

There are no real special requirements when choosing tools for infrastructure tests. As long as the stack allows calling APIs it should be sufficient. Very often the applications teams are using the same tools for testing the infrastructure tied to the application which they are using for testing the application itself. The environment teams, on the other hand, are looking for specific tools which fit the ecosystem they are familiar with. A popular choice when it comes to Azure is Pester, a test framework for Powershell. I'm going to use it for examples here.

What types of tests should you consider implementing? There are two which I consider a must-have - smoke tests and negative tests.

Smoke Tests

Smoke tests should be the first tests to verify the deployment. Their goal is to quickly provide feedback on crucial functions of the system without delving into finer details. Their implementation should be fast and simple. A typical smoke test is a verification if a host is responsive, which in Pester is just a couple of lines:

param(
  [Parameter(Mandatory)]
  [ValidateNotNullOrEmpty()]
  [string] $HostName
)

Describe 'Application Host' {
    It 'Serves pages over HTTPS' {
      $request = [System.Net.WebRequest]::Create("https://$HostName/")
      $request.AllowAutoRedirect = $false
      $request.GetResponse().StatusCode |
        Should -Be 200 -Because "It's responsive"
    }
}

Notice that we are not trying to determine if the hosted application is healthy beyond getting a successful status code - we are testing infrastructure and the application hasn't been deployed yet.

Running smoke tests should be the first job in our post-deployment workflow. GitHub-hosted runners come with Pester, which means that running the tests is just two lines of Powershell.

jobs:
  smoke-tests:
    runs-on: ubuntu-latest
    steps:
    - name: Checkout
      uses: actions/checkout@v3
    - name: Run Pester Tests
      shell: pwsh
      run: |
        $container = New-PesterContainer `
          -Path 'applications/sample-application/tests/smoke-tests.ps1' `
          -Data @{ HostName = '${{ env.APPLICATION_HOST_NAME }}' }
        Invoke-Pester -Container $container -CI
  ...

So, running the tests is not a challenge. But running the tests is not the goal by itself. The goal is to properly react when tests fail. What should we do when smoke tests fail? There are two options we can choose from: roll back or roll forward. For smoke tests, we should almost always aim for rolling back. After all a crucial function of our system is not working and reverting it to the previous stable version is usually the quickest way to fix this. Of course roll back may not always be possible and then you are left with roll forward as the only option. Still, you should aim for this to be an edge case.

The situation with negative tests is a little bit different.

Negative Tests

While smoke tests are providing feedback if crucial functions of the system are working as expected, negative tests are there to provide feedback on how the system will behave in invalid scenarios. A good example can be unencrypted requests over HTTP. They are not secure and we want to disable them at the host level by configuring a redirect. A negative test to verify that can look like below.

param(
  [Parameter(Mandatory)]
  [ValidateNotNullOrEmpty()]
  [string] $HostName
)

Describe 'Application Host' {
    It 'Does not serves pages over HTTP' {
      $request = [System.Net.WebRequest]::Create("http://$HostName/")
      $request.AllowAutoRedirect = $false
      $request.GetResponse().StatusCode | 
        Should -Be  301 -Because "Redirect is forced"
    }
}

As negative tests should be independent of smoke tests and still considered important, the typical approach is to run them in parallel with smoke tests.

jobs:
  smoke-tests:
    ...
  negative-tests:
    runs-on: ubuntu-latest
    steps:
    - name: Checkout
      uses: actions/checkout@v3
    - name: Run Pester Tests
      shell: pwsh
      run: |
        $container = New-PesterContainer `
          -Path 'applications/sample-application/tests/negative-tests.ps1' `
          -Data @{ HostName = '${{ env.APPLICATION_HOST_NAME }}' }
        Invoke-Pester -Container $container -CI
  ...

There is often a discussion about how to decide if something is a negative test or a smoke test. The distinction should be based on the impact. Taking our two examples:

Host not being responsive is catastrophic, we can't provide any service for our users.
Host responding to HTTP requests is something we can live with for a moment. There is secondary protection in the application code and in our industry context, it's only a recommendation, not a requirement.

Of course, context matters and what is a negative test in one situation might be a critical smoke test in another. The key aspect is that negative tests failing don't have to mean rolling back. The system can still provide a valuable service for users. This is why the strategy in case of negative tests is often to roll forward, fix the issue as soon as possible, and perform another deployment.

Other After-Deployment Tests

Smoke and negative tests are crucial, but they are only scratching the surface to provide initial feedback as soon as possible. They should be followed by different types of tests which go into previously ignored finer details. Depending on the needs you should implement functional, integration, or other types of tests.

You also shouldn't limit running tests only to the deployment moment. Infrastructure is much more fragile than application code, so you should continuously run at least the key tests to ensure that everything is working as expected. You should also adopt health checks (yes they are usually considered part of monitoring, but sophisticated health checks are often complex tests) to notice when something becomes unavailable or starts to misbehave. You can also go a step further and continuously test how your system will behave when something goes down by adopting chaos engineering.

Chaos Engineering

Chaos engineering is testing through experimenting. You can think of it as a form of exploratory testing. It's about discovering and building confidence in system resilience by exploring its reactions to infrastructure failures.

The level of 'chaotic-ness' can be very different. Chaos Monkey, probably the most famous chaos engineering tool, randomly terminates virtual machines and containers, but there are more structured approaches. The methodical approach to chaos engineering starts by defining a steady state, a measurable set of system characteristics that indicates normal behavior. That steady state is a base of a hypothesis that the system will continue in the same state after the experiment. To prove or disprove that hypothesis, an experiment is designed. The design of the experiment should include faults and their targets. Once the design is complete, the experiment is executed by injecting the faults into the environment and capturing the output state. The output state is being verified against the steady state. If the hypothesis has been disproven, the output state should be used for learning and improvement of the system. If the hypothesis has been proven, it's time to design a new experiment.

Despite being around for several years, the tooling ecosystem for chaos engineering wasn't growing as rapidly as one could wish for. That was until 2020 when AWS announced AWS Fault Injection Simulator. About a year later Microsoft followed by announcing a public preview of Azure Chaos Studio. Adopting chaos engineering through a managed service has become an option.

What does Azure Chaos Studio offer? Currently (still in preview) it provides ~30 faults and actions which can be applied to ~10 targets. What is interesting is that Azure Chaos Studio has two types of faults: service-direct and agent-based. The service-direct run directly against resources, while agent-based enable in-guest failures on virtual machines (for example high CPU).

How to adopt Azure Chaos Studio? The service provides capabilities to create experiments through ARM or Bicep. There is also REST API which can be used to create, manage, and run experiments. Those capabilities can be used to implement an architecture similar to the following, with continuous experiment execution (1, 2, 3, and 4).

Not There Yet

With Continuous Testing we have moved a little bit toward the right side of the loop, as part of this practice starts to provide us with feedback from the living system that we can use in our cycle of improvement. Still, there is a significant portion missing.

There are practices that I haven't touched yet, which are focusing on the missing part - Continuous Operations and Continuous Monitoring. It's quite likely that they are already present in your organization, just not providing feedback to the loop. This is the journey further and I intend to go there in the next post.

You can find samples for some of the aspects I'm discussing here on GitHub.

DevOps Practices for Azure Infrastructure - Continuous Delivery & Continuous Deployment

noreply@blogger.com (Tomasz Pęczek) — Tue, 27 Jun 2023 07:06:00 +0000

In my previous post, I started the journey of implementing DevOps practices for infrastructure. I've proposed implementation for Continuous Integration practice, which covers the Create and Verify stages of the DevOps pipeline.

But Continuous Integration is just the first of several practices which should be implemented for a complete pipeline:

Continuous Planning
Continuous Integration
Continuous Delivery
Continuous Deployment
Continuous Testing
Continuous Operations
Continuous Monitoring

In this post, I want to focus on Continuous Delivery and Continuous Deployment practices which are there to pick up where Continuous Integration has finished and continue through the Package and Release stages of the pipeline.

Continuous Delivery vs. Continuous Deployment

Quite often, when I'm discussing Software Development Life Cycle with teams, there is confusion around Continuous Delivery and Continuous Deployment. Teams will often say that they are doing CI/CD and when I ask about the CD part the terms Continuous Delivery and Continuous Deployment are being used interchangeably. A lot of marketing "What is DevOps" articles also don't help by confusing the terms. So what is the difference?

In short, Continuous Delivery is about making artifacts ready for deployment and Continuous Deployment is about actually deploying them. That seems to be quite a clear separation, so why the confusion? Because in the real world, they often blend. In an ideal scenario, when the Continuous Integration workflow is finished, the deployment workflow can kick off automatically and get the changes to the production. In such a scenario, the Continuous Delivery may not be there, and if it is there it will be considered an implicit part of Continuous Deployment. This is where the terms are often misused - the separation is not clear. Continuous Delivery exists in explicit form only when there is some kind of handover or different step between "packaging" and "deployment".

Why am I discussing this here? Because when it comes to infrastructure, especially for solutions of considerable size, there often is a need for separated Continuous Delivery and Continuous Deployment. Where does this need come from? From different responsibilities. For large solutions, there is infrastructure responsible for the overall environment and infrastructure tied to specific applications. That means multiple teams are owning different parts of the infrastructure and working on it independently. But from a governance and security perspective, there is often a desire to treat the entire infrastructure as one. Properly implemented Continuous Delivery and Continuous Deployment can solve this conflict, but before I move to discuss the practices I need to extend the context by discussing the repositories structure for such solutions.

Structuring Repositories

How repositories of your solutions are structured has an impact on the implementation of your DevOps practices. Very often projects start small with a monorepo structure.

There is nothing wrong with monorepo structure. It can be the only structure you will ever need. The main benefit of monorepo is that it's simple. All your code lives in one place, you can iterate fast, it's easy to govern and you can implement just a single set of DevOps practices. But there is a point at which those advantages are becoming limitations. This point comes when the solutions grow to consist of multiple applications owned by different teams. Sooner or later those teams start to ask for some level of independence. They want to have a little bit different governance rules (which better suit their culture) and they don't want to be blocked by work being done by other teams. Sometimes just the size of monorepo becomes a productivity issue. This is where the decoupling of the monorepo starts. Usually, the first step is that new applications are being created in their own repositories. Later, the existing ones are moved out from the monorepo. The outcome is multiple, independent repositories.

But, this structure has some problems of its own. There is no longer a single source of truth that would represent the entire solution. Establishing governance rules which are required for the entire solution is harder. There are multiple sources of deployments which means access to the environment from multiple places, which means increased security risk. There is a need for balance between those aspects and teams needs in areas of flexibility and productivity. A good option for such a balance is having applications repositories and a dedicated environment repository.

The environment repository is the single source of truth. It's also the place to apply required governance rules and the only source of deployments. It can also hold the shared infrastructure. The applications repositories are owned by the applications teams and contain the source code for the application as well as the infrastructure code tied to it. This is the structure we will focus on because this is the structure that requires Continuous Delivery and Continuous Deployment. The applications teams should implement Continuous Delivery for packaging the infrastructure to be used by the environment repository, while the team responsible for the environment repository should implement Continuous Deployment. Let's start with Continuous Delivery.

Continuous Delivery for Applications Infrastructure

The first idea for Continuous Delivery implementation can be simply copying the infrastructure code to the environment repository. The allure of this approach is that the environment repository will contain the complete code of the infrastructure removing any kind of context switching. The problem is that now the same artifacts live in two places and the sad truth is that when the same artifacts live in two places sooner or later something is going to get messed up. So, instead of copying the infrastructure code a better approach is to establish links from the environment module to the applications modules.

Options for linking from the environment module to applications modules strongly depend on chosen infrastructure as code tooling. Some tools support a wide variety of sources for the links starting with linking directly to git repositories (so Continuous Delivery can be as simple as creating a tag and updating a reference in the environment module). In rare cases, when there is no support for linking by the tooling, you can always use git submodules.

In the case of Bicep, there is one interesting option - using Azure Container Registry. This option can be attractive for two reasons. One is the possibility to create a private, isolated registry. The other is treating infrastructure the same way we would treat containers (so if you are using containers, both are treated similarly).

The publishing of bicep files is available through the az bicep publish command. We can create a workflow around this command. A good trigger for this workflow may be the creation of a tag. We can even extract the version from the tag name which we will later use to publish the module.

name: Continuous Delivery (Sample Application)
on:
  push:
    tags:
    - "sample-application-v[0-9]+.[0-9]+.[0-9]+"

...

jobs:
  publish-infrastructure-to-registry:
    runs-on: ubuntu-latest
    steps:
    - name: Extract application version from tag
      run: |
        echo "APPLICATION_VERSION=${GITHUB_REF/refs\/tags\/sample-application-v/}" >> $GITHUB_ENV
    ...

Now all that needs to be done is checking-out the repository, connecting to Azure, and pushing the module.

...

env:
  INFRASTRUCTURE_REGISTRY: 'crinfrastructuremodules'

jobs:
  publish-infrastructure-to-registry:
    runs-on: ubuntu-latest
    permissions:
      id-token: write
      contents: read
    steps:
    ...
    - name: Checkout
      uses: actions/checkout@v3
    - name: Azure Login
      uses: azure/login@v1
      with:
        client-id: ${{ secrets.AZURE_CLIENT_ID }}
        tenant-id: ${{ secrets.AZURE_TENANT_ID }}
        subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
    - name: Publish application Bicep to infrastructure registry
      run: |
        bicep publish  \
          applications/sample-application/application.bicep  \
          --target br:${INFRASTRUCTURE_REGISTRY}.azurecr.io/infrastructure/applications/sample-application:${APPLICATION_VERSION}

The linking itself is to be done in the environment infrastructure Bicep file. The module syntax allows the module path to be either a local file or a file in a registry. This is the part that applications teams will be contributing to the environment repository - the module definition.

...

resource sampleApplicationResourceGroupReference 'Microsoft.Resources/resourceGroups@2022-09-01' = {
  name: 'rg-devops-practices-sample-application-prod'
  location: environmentLocation
}

module sampleApplicationResourceGroupModule 'br:crinfrastructuremodules.azurecr.io/infrastructure/applications/sample-application:1.0.0' = {
  name: 'rg-devops-practices-sample-application-rg'
  scope: resourceGroup(sampleApplicationResourceGroupReference.name)
}

...

Now the Continuous Deployment practice for the environment can be implemented.

Continuous Deployment for Environment Infrastructure

There are two deployment strategies that you may have heard of in the context of deploying infrastructure: push-based deployment and pull-based deployment.

The push-based deployment is what one could call a classic approach to deployment. You implement a workflow that pushes the changes to the environment. That workflow is usually triggered as a result of changes to the code.

The pull-based deployment strategy is the newer approach. It introduces an operator in place of the workflow. The operator monitors the repository and the environment and reconciles any differences to maintain the infrastructure as described in the environment repository. That means it will not only react to changes done to the code but also changes applied directly to the environment protecting it from drifting (at least in theory).

The pull-based deployment strategy has found the most adoption in the Kubernetes space with two ready-to-use operators (Flux and Argo CD). When it comes to general Azure infrastructure, the push-based strategy is still the way to go, although there is a way to have a pull-based deployment for Azure resources that are tied to applications hosted in Kubernetes. Azure Service Operator for Kubernetes provides Custom Resource Definitions for deploying Azure resources, enabling a unified experience for application teams.

In the scope of this post, I'm going to stick with a typical push-based deployment, which means checking-out the repository, connecting to Azure, and deploying infrastructure based on the Bicep file.

name: Continuous Deployment (Environment)
...

jobs:
  deploy-environment:
    runs-on: ubuntu-latest
    permissions:
      id-token: write
      contents: read
    steps:
    - name: Checkout
      uses: actions/checkout@v3
    - name: Azure Login
      uses: azure/login@v1
      with:
        client-id: ${{ secrets.AZURE_CLIENT_ID }}
        tenant-id: ${{ secrets.AZURE_TENANT_ID }}
        subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
    - name: Deploy Environment
      uses: azure/arm-deploy@v1
      with:
        scope: 'subscription'
        region: 'westeurope'
        template: 'environment/environment.bicep'

The Journey Continues

The same as with the previous post, this one also just scratches the surface. There are many variations possible, depending on your needs. It can also by further automated - for example the Continuous Delivery implementation can be automatically creating a pull request to the environment repository. Part of the DevOps culture is continuous improvement and that also means improving the practices implementations itself.

Our journey is also not over yet, there are a couple more practices I would like to explore in the next posts, so the loop is complete.

If you want to play with the workflows, they are sitting on GitHub.

DevOps Practices for Azure Infrastructure - Continuous Integration

noreply@blogger.com (Tomasz Pęczek) — Mon, 12 Jun 2023 13:59:00 +0000

The generally adopted definition of DevOps methodology says that it's a combination of cultural philosophies, practices, and tools that increases an organization’s ability to deliver solutions. That's very broad. So broad, that initial adoption in the case of many organizations has focused on applying it only to application code. This has led to the naming of several additional methodologies to either further blend DevOps with additional areas or focus on previously neglected aspects: DevSecOps, FinOps, DataOps, GitOps, or MLOps. But, regardless of the flavor, the core remains the same. This is why, although I'm writing about DevOps in the context of infrastructure, I have avoided using GitOps in the title.

If you look for a definition of GitOps, you may find statements like "GitOps is a process of automating IT infrastructure using infrastructure as code and software development best practices" or "GitOps is a subset of DevOps". In reality, the term has been strongly tied to a specific ecosystem and specific tools. I don't want to fight with those associations. Instead, I want to focus on the essence - applying DevOps practices to infrastructure.

DevOps Practices

DevOps practices are a way to bring DevOps cultural philosophies (collaboration, automation, continuous feedback, continuous improvement, etc.) to life. They are used to implement all the stages of the DevOps pipeline:

You may find slightly different lists of those practices, this is the one I prefer:

In this post, I want to focus on Continuous Integration.

Continuous Integration Practice

On some occasions, I've heard an opinion that Continuous Integration is about merging the changes. The truth is that correctly implemented Continuous Integration practice covers the Create and Verify stages of the DevOps pipeline.

It shouldn't be a surprise that the cornerstone of the Create stage in the case of infrastructure is infrastructure as code, the tooling used for development, and the Git ecosystem used for collaboration. This is something that is already widely adopted with many options to choose from:

Terraform, Bicep, or Pulumi just to name some popular infrastructure as code options.
GitHub, Azure DevOps, or GitLab as potential Git ecosystems.
VS Code, Neovim, or JetBrains Fleet as possible development environments.

The above list is in no way exhaustive. I also don't aim at discussing the superiority of one tool over another. That said, discussing the Verify stage, which is the more challenging part of Continuous Integration practice, will be better done with specific examples. This is why I must choose a stack and I'm going to choose Bicep (as I'm writing this in the context of Azure) and GitHub (because it has some nice features which will make my life easier).

So, once we have our infrastructure as code created, what should we consider as verification from the perspective of Continuous Integration? In the beginning, I quoted a statement saying that GitOps is about using software development best practices in the process of automating infrastructure. What would be the first thing one would do with an application code to verify it? Most likely build it.

Building and Linting for Infrastructure Code

Building or compiling application code is the first step of the Verify stage. In the software development context, it's sometimes thought of as a way to generate the binaries (and it is), but it's also verifying if the code is syntactically correct. In the context of IaC, it means checking for the correct use of language keywords and that resources are defined according to the requirements for their type. This is something that IaC tooling should always support out of the box. Bicep provides this capability through az bicep build command, which we can simply run as a step in a workflow.

...

jobs:
  build-and-lint:
    runs-on: ubuntu-latest
    steps:
    - name: Checkout
      uses: actions/checkout@v3
    - name: Build and lint Bicep
      run: |
        az bicep build --file applications/sample-application/application.bicep
  ...

The az bicep build command also performs a second activity, which is closely tied to building/compiling - it runs linter over the template. The goal of linting is to help enforce best practices and coding standards based on defined rules. Best practices and coding standards are something that sometimes needs to be tailored to a specific team and organization, this is why Bicep allows for the configuration of rules severity through the bicepconfig.json file. Possible options are Error, Warning, Info, and Off. By default, the majority of rules are set to either Warning or Off. The typical adjustment which I almost always do is bumping No unused parameters to Error and enabling Use recent API versions (as it is Off by default).

{
    "analyzers": {
        "core": {
            "enabled": true,
            "rules": {
                ...
                "no-unused-params": {
                    "level": "error"
                },
                ...
                "use-recent-api-versions": {
                     "level": "warning"
                },
            }
        }
    }
}

The bicepconfig.json file should be committed to the repository, which will ensure that the local development environment will pick up the same configuration. This includes VS Code (if the Bicep extension is installed), enabling immediate feedback for engineers (in the spirit of DevOps cultural philosophies). Of course, engineers can ignore that feedback or simply use tooling which doesn't provide it, but then the Build and lint Bicep step of the integration workflow will catch it and give them that feedback.

If everything is correct, the workflow should move to the next phase, which doesn't mean we should be done with looking at the code itself. Following the software development best practices, the next phase should be static analysis.

Static Analysis for Infrastructure Code

Application code is usually scanned with tools like SonarQube, Veracode, Snyk, or GitHub's own CodeQL to detect potential vulnerabilities or bad patterns. The same should be done for infrastructure code and there are ready-to-use tools for that like KICS or Checkov. They are both designed to detect security vulnerabilities, compliance issues, and misconfigurations in our IaC. They both come with a huge set of configurable rules and the capability to create your own.

I prefer KICS, especially the way it can be integrated with GitHub. Checkmarx, the company behind KICS, provides a ready-to-use action. The support for Bicep is "indirect" - KICS supports ARM so the analysis has to be done after the build step. There is also small preparation needed as the directory for output should be created. Still, adding KICS-based static analysis to the workflow is only about 10 lines.

...

jobs:
  build-lint-and-static-analysis:
    runs-on: ubuntu-latest
    steps:
    ...
    - name: Create static analysis results folder
      run: |
        mkdir -p static-analysis-results
    - name: Perform KICS static analysis
      id: kics
      uses: checkmarx/kics-github-action@v1.6.3
      with:
        path: 'applications/sample-application/'
        fail_on: 'high,medium'
        output_path: 'static-analysis-results'
        output_formats: 'json,sarif'
  ...

The above analysis step will fail if any issues with severity high or medium are detected. Similarly to the build step, the feedback will be provided through workflow output.

But KICS integration is even more powerful than that. As you may have noticed I've configured output formats from the analysis to be JSON and SARIF. SARIF is a standardized format for sharing static analysis results and it can be used to integrate with the code scanning feature of GitHub Advanced Security. Once again we can use an existing action (this time provided by GitHub) to upload the SARIF file. The only tricky part is to put a proper condition on the upload step so the results are pushed also when the analysis step fails due to the severity of detected issues.

...

jobs:
  build-lint-and-static-analysis:
    runs-on: ubuntu-latest
    permissions:
      actions: read
      contents: read
      security-events: write
    steps:
    ...
    - name: Upload KICS static analysis results
      if: always() && (steps.kics.outcome == 'success' || steps.kics.outcome == 'failure')
      uses: github/codeql-action/upload-sarif@v2
      with:
        sarif_file: 'static-analysis-results/results.sarif'
  ...

Thanks to this, the issues will be available in the Code Scanning section of the repository Security tab. This will provide alerts for those issues, the ability to triage them, and audit for taken actions.

Now we can say that we have looked at the code enough as part of the integration workflow. In the case of software development, we would probably run now some unit tests. In the case of infrastructure, the equivalent at this stage is testing if the template will deploy successfully.

Preflight Validation for Infrastructure Code

We have verified that the template will build probably and we have removed all important vulnerabilities and misconfigurations. Sadly, this doesn't guarantee that the template will deploy. There may be some policies or conditions on the environment, which are not reflected in any of the checks. To make sure that the template will deploy, we need to perform a preflight validation against the environment. This capability is provided differently by different ecosystems, in the case of Bicep and ARM it comes as Validate deployment mode. This means that we can add another job to our workflow which will establish a connection to Azure and test the deployment.

...

jobs:
  ...
  preflight-validation:
    needs: build-lint-and-static-analysis
    runs-on: ubuntu-latest
    permissions:
      id-token: write
      contents: read
    steps:
    - name: Checkout
      uses: actions/checkout@v3
    - name: Azure Login
      uses: azure/login@v1
      with:
        client-id: ${{ secrets.AZURE_CLIENT_ID }}
        tenant-id: ${{ secrets.AZURE_TENANT_ID }}
        subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
    - name: Perform preflight validation
      uses: azure/arm-deploy@v1
      with:
        scope: 'resourcegroup'
        resourceGroupName: 'rg-devops-practices-sample-application-sandbox'
        template: 'applications/sample-application/application.bicep'
        deploymentMode: 'Validate'
        failOnStdErr: false
  ...

This will catch issues like duplicated storage account names (or simple cases where the name is too long) without actually deploying anything.

What's next? Well, there is one common software development practice that we haven't touched yet - pull requests and code reviews. This subject recently caused some heated discussions. There are opinions that if you have an explicit code review step in your process it's not true Continuous Integration. There are also opinions that code reviews are perfectly fine. My opinion is that it's part of team culture. If your team has asynchronous culture, then doing code reviews through pull requests may be the correct way. If your team is collocated or strongly collaborates online, using pair or mob programming instead of code reviews may be the best. We can also detach the discussion around pull requests from the discussion around code reviews. I know teams that are relying on pair programming in place of code reviews but still use pull requests (automatically closed) for tracking purposes. And when we are talking pull requests in the context of infrastructure code, there is one challenge - it's hard to understand the actual change just by looking at code diff (especially after some time). This is why generating a preview of changes as part of the integration workflow can be extremely beneficial.

Preview of Infrastructure Changes

Infrastructure as code tooling usually provides a method to generate a preview - Terraform has the plan, Pulumi has the preview, and Bicep/ARM has the what-if. From the perspective of the integration workflow we are not thinking about running those commands locally but as part of the workflow. And this time we are not interested in results being available as part of the workflow output, we are looking for adding them as more context to the pull request. To be able to do that we first must capture the results. A good method is writing the results to an environment variable.

...

jobs:
  ...
  preview:
    needs: preflight-validation
    runs-on: ubuntu-latest
    permissions:
      id-token: write
      contents: read
    steps:
    - name: Checkout
      uses: actions/checkout@v3
    - name: Azure Login
      uses: azure/login@v1
      with:
        client-id: ${{ secrets.AZURE_CLIENT_ID }}
        tenant-id: ${{ secrets.AZURE_TENANT_ID }}
        subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
    - name: Prepare preview
      run: |
        echo 'DEPLOYMENT_WHAT_IF<<EOF' >> $GITHUB_ENV
        az deployment group what-if \
          --resource-group rg-devops-practices-sample-application-sandbox \
          --template-file applications/sample-application/application.bicep \
          --result-format ResourceIdOnly >> $GITHUB_ENV
        echo 'EOF' >> $GITHUB_ENV
  ...

Once we have the results, we can add them to the pull request. My preferred approach is to create a comment. GitHub provides us with script action which allows us to use a pre-authenticated GitHub API client. The issue number and all other necessary information will be available through the context object (if we are using the right trigger).

...

jobs:
  ...
  preview:
    needs: preflight-validation
    runs-on: ubuntu-latest
    permissions:
      id-token: write
      contents: read
      pull-requests: write
    steps:
    ...
    - name:  Create preview comment
      uses: actions/github-script@v6
      with:
        script: |
          github.rest.issues.createComment({
            issue_number: context.issue.number,
            owner: context.repo.owner,
            repo: context.repo.repo,
            body: process.env.DEPLOYMENT_WHAT_IF
          })
  ...

As a result of this job, we will get a nice comment describing in a more human-readable form the changes which deploying the template would cause at this very moment.

We may want even more. Changes are not the only valuable context we may be interested in. The second important information is how deploying the changes will impact the costs.

Cost Estimation of Infrastructure Changes

ThoughtWorks has been recommending run cost as an architecture fitness function since 2019, and there is more than one infrastructure cost estimation tool available to us. The two which are worth mentioning are Infracost (for Terraform) and Azure Cost Estimator (for Bicep/ARM and recently also Terraform). As I'm using Bicep in this article, I'm going to focus on Azure Cost Estimator.

Azure Cost Estimator is still a young tool, yet it's already quite powerful. At the moment of writing this, it supports ~86 resource types. What is very important, it's capable of generating usage base consumption for some resources if you provide usage patterns through metadata in the template. The only tricky part can be integrating it into the workflow. The project repository provides a reusable workflow, but this may not be desired (or even allowed) method in many organizations. This is why I'll walk you through the integration step by step.

The first step is getting the binaries and installing them. If you are using self-hosted runners this can be part of runner setup. You can also download and install the binaries from some central location as part of the workflow itself. Below I'm doing exactly that from the official project releases.

...

jobs:
  ...
  cost-estimation:
    needs: preflight-validation
    runs-on: ubuntu-latest
    steps:
    - name: Checkout
      uses: actions/checkout@v3
    - name: Download Azure Cost Estimator
      id: download-ace
      uses: robinraju/release-downloader@v1.7
      with:
        repository: "TheCloudTheory/arm-estimator"
        tag: "1.2"
        fileName: "ace-linux-x64.zip"
    - name: Install Azure Cost Estimator
      run: |
        unzip ace-linux-x64.zip
        chmod +x ./azure-cost-estimator
  ...

With the binaries in place, we can use the same pattern as in the case of preview to run the tool, grab the results into an environment variable, and create a comment.

...

jobs:
  ...
  cost-estimation:
    needs: preflight-validation
    runs-on: ubuntu-latest
    permissions:
      id-token: write
      contents: read
      pull-requests: write
    steps:
    ...
    - name: Azure Login
      uses: azure/login@v1
      with:
        client-id: ${{ secrets.AZURE_CLIENT_ID }}
        tenant-id: ${{ secrets.AZURE_TENANT_ID }}
        subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
    - name: Prepare cost estimation
      run: |
        echo 'COST_ESTIMATION<<EOF' >> $GITHUB_ENV
        azure-cost-estimator applications/sample-application/application.bicep \
          ${{ secrets.AZURE_SUBSCRIPTION_ID } \
          rg-devops-practices-sample-application-sandbox \
          --stdout --disableDetailedMetrics >> $GITHUB_ENV
        echo 'EOF' >> $GITHUB_ENV
    - name:  Create pull request comment
      uses: actions/github-script@v6
      with:
        script: |
          github.rest.issues.createComment({
            issue_number: context.issue.number,
            owner: context.repo.owner,
            repo: context.repo.repo,
            body: process.env.COST_ESTIMATION
          })
  ...

This will give us a lot of additional, valuable context in the pull request.

The Beginning of the Implementation Journey

This post describes just the beginning of the DevOps practices implementation journey.

It gives a hint about the entire technology ecosystem, but the remaining practices have many interesting aspects to dive into. I do intend to continue walking through them with proposed implementations, just to make your journey easier. I've also created a repository that contains samples with different parts of implementation (available in different branches with results in different closed pull requests). You can review it to find all the relevant information. You can also create a fork and have your own playground.

Server Timing API Meets Isolated Worker Process Azure Functions - Custom Middleware and Dependency Injection

noreply@blogger.com (Tomasz Pęczek) — Wed, 15 Mar 2023 10:56:00 +0000

Server Timing API has piqued my interest back in 2017. I've always been a promoter of a data-driven approach to non-functional requirements. Also, I always warned the teams I worked with that it's very easy to not see the forest for the trees. Server Timing API was bringing a convenient way to communicate backend performance information to developer tools in the browser. It enabled access to back-end and front-end performance data in one place and within the context of actual interaction with the application. I've experimented with the technology together with a couple of teams where a culture of fronted and backend engineers working close together was high. The results were great, which pushed me to create a small library to simplify the onboarding of Server Timing API in ASP.NET Core applications. I've been using that library with multiple teams through the years and judging by the downloads number I wasn't the only one. There were even some contributions to the library.

Some time ago, an issue was raised asking if the library could also support Azure Functions using isolated worker process mode. I couldn't think of a good reason why not, it was a great idea. Of course, I couldn't add the support directly to the existing library. Yes, the isolated worker process mode of Azure Functions shares a lot of concepts with ASP.NET Core, but the technicalities are different. So, I've decided to create a separate library. While doing so, I've also decided to put some notes around those concepts into a blog post in hope that someone might find them useful in the future.

So, first things first, what is isolated worker process mode and why we are talking about it?

Azure Functions Execution Modes

There are two execution modes in Azure Functions: in-process and isolated worker process. The in-process mode means that the function code is running in the same process as the host. This is the approach that has been taken for .NET functions from the beginning (while functions in other languages were running in a separate process since version 2). This enabled Azure Functions to provide unique benefits for .NET functions (like rich bindings and direct access to SDKs) but at a price. The .NET functions could only use the same .NET version as the host. Dependency conflicts were also common. This is why Azure Functions has fully embraced the isolated worker process mode for .NET functions in version 4 and now developers have a choice of which mode they want to use. Sometimes this choice is simple (if you want to use non-LTS versions of .NET, the isolated worker process is your only option), sometimes it is more nuanced (for example isolated worker process functions have slightly longer cold start). You can take a look at the full list of differences here.

When considering simplifying the onboarding of Server Timing API, the isolated worker process mode is the only option as it supports a crucial feature - custom middleware registration.

Custom Middleware

The ability to register a custom middleware is crucial for enabling capabilities like Server Timing API because it allows for injecting logic into the invocation pipeline.

In isolated worker process Azure Functions, the invocation pipeline is represented by FunctionExecutionDelegate. Although it would be possible to work with FunctionExecutionDelegate directly (by wrapping it with parent invocations), Azure Functions provides a convenient extension method UseMiddleware() which enables registering inline or factory-based middleware. What is missing in comparison to ASP.NET Core is the convention-based middleware. This might be surprising at first as the convention-based approach is probably the most popular one in ASP.NET Core. So, for those of you who are not familiar with the factory-based approach, it requires the middleware class to implement a specific interface. In the case of Azure Functions, it's IFunctionsWorkerMiddleware (in ASP.NET Core it's IMiddleware). The factory-based middleware is prepared to be registered with different lifetimes, so the Invoke method takes not only the context but also the delegate representing the next middleware in the pipeline as a parameter. Similarly to ASP.NET Core, we are being given the option to run code before and after functions execute, by wrapping it around the call to the next middleware delegate.

internal class ServerTimingMiddleware : IFunctionsWorkerMiddleware
{
    public async Task Invoke(FunctionContext context, FunctionExecutionDelegate next)
    {
        // Pre-function execution

        await next(context);

        // Post-function execution
    }
}

The aforementioned UseMiddleware() extension method should be called inside the ConfigureFunctionsWorkerDefaults method as part of the host preparation steps. This method registers the middleware as a singleton (so it has the same lifetime as convention-based middleware in ASP.NET Core). It can be registered with different lifetimes, but it has to be done manually which includes wrapping invocation of FunctionExecutionDelegate. For the ones interested I recommend checking the UseMiddleware() source code for inspiration.

var host = new HostBuilder()
    .ConfigureFunctionsWorkerDefaults(workerApplication =>
    {
        // Register middleware with the worker
        workerApplication.UseMiddleware();
    })
    .Build();

host.Run();

All the valuable information about the invoked function and the invocation itself can be accessed through FunctionContext class. There are also some extension methods available for it, which make it easier to work with that class. One such extension method is GetHttpResponseData() which will return an instance of HttpResponseData if the function has been invoked by an HTTP trigger. This is where the HTTP response can be modified, for example by adding headers related to Server Timing API.

internal class ServerTimingMiddleware : IFunctionsWorkerMiddleware
{
    public async Task Invoke(FunctionContext context, FunctionExecutionDelegate next)
    {
        // Pre-function execution

        await next(context);

        // Post-function execution
        HttpResponseData? response = context.GetHttpResponseData();
        if (response is not null)
        {
            response.Headers.Add(
                "Server-Timing",
                "cache;dur=300;desc=\"Cache\",sql;dur=900;desc=\"Sql Server\",fs;dur=600;desc=\"FileSystem\",cpu;dur=1230;desc=\"Total CPU\""
            );
        }
    }
}

To make this functional, the values for the header need to be gathered during the invocation, which means that there needs to be a shared service between the function and the middleware. It's time to bring the dependency injection into the picture.

Dependency Injection

The support for dependency injection in isolated worker process Azure Functions is exactly what you can expect if you have been working with modern .NET. It's based on Microsoft.Extensions.DependencyInjection and supports all lifetimes options. The option which might require clarification is the scoped lifetime. In Azure Functions, it matches a function execution lifetime, which is exactly what is needed for gathering values in the context of a single invocation.

var host = new HostBuilder()
    ...
    .ConfigureServices(s =>
    {
        s.AddScoped();
    })
    .Build();

host.Run();

Functions that are using dependency injection must be implemented as instance methods. When using instance methods, each invocation will create a new instance of the function class. That means that all parameters passed into the constructor of the function class are scoped to that invocation. This makes usage of constructor-based dependency injection safe for services with scoped lifetime.

public class ServerTimingFunctions
{
    private readonly IServerTiming _serverTiming;

    public ServerTimingFunctions(IServerTiming serverTiming)
    {
        _serverTiming = serverTiming;
    }

    [Function("basic")]
    public HttpResponseData Basic([HttpTrigger(AuthorizationLevel.Anonymous, "get")] HttpRequestData request)
    {

        var response = request.CreateResponse(HttpStatusCode.OK);
        response.Headers.Add("Content-Type", "text/plain; charset=utf-8");

        _serverTiming.Metrics.Add(new ServerTimingMetric("cache", 300, "Cache"));
        _serverTiming.Metrics.Add(new ServerTimingMetric("sql", 900, "Sql Server"));
        _serverTiming.Metrics.Add(new ServerTimingMetric("fs", 600, "FileSystem"));
        _serverTiming.Metrics.Add(new ServerTimingMetric("cpu", 1230, "Total CPU"));

        response.WriteString("-- Demo.Azure.Functions.Worker.ServerTiming --");

        return response;
    }
}

The above statement is not true for the middleware. As I've already mentioned, the UseMiddleware() method registers the middleware as a singleton. So, even though middleware is being resolved for every invocation separately, it is always the same instance. This means that constructor-based dependency injection is safe only for services with a singleton lifetime. To properly use a service with scoped or transient lifetime we need to use the service locator approach. An invocation-scoped service locator is available for us under FunctionContext.InstanceServices property.

internal class ServerTimingMiddleware : IFunctionsWorkerMiddleware
{
    public async Task Invoke(FunctionContext context, FunctionExecutionDelegate next)
    {
        ...

        // Post-function execution
        InvocationResult invocationResult = context.GetInvocationResult();

        HttpResponseData? response = invocationResult.Value as HttpResponseData;
        if (response is not null)
        {
            IServerTiming serverTiming = context.InstanceServices.GetRequiredService();
            response.Headers.Add("Server-Timing", String.Join(",", serverTiming.Metrics));
        }
    }
}

It Works! (And You Can Use It)

This way, by combining support for middleware and dependency injection, I've established the core functionality of my small library. It's out there on NuGet, so if you want to use Server Timing to communicate performance information to your Azure Functions based API consumers you are welcome to use it. If you want to dig a little bit into the code (or maybe you have some suggestions or improvements in mind) it lives in the same repository as the ASP.NET Core one.

Experimenting With .NET & WebAssembly - Running .NET Based Spin Application On WASI Node Pool in AKS

noreply@blogger.com (Tomasz Pęczek) — Tue, 20 Dec 2022 14:17:00 +0000

I'm quite an enthusiast of WebAssembly beyond the browser. It's already made its way into edge computing with WasmEdge, Cloudflare Workers, or EdgeWorkers. It's also made its way into cloud computing with dedicated clouds like wasmCloud or Fermyon Cloud. So it shouldn't be a surprise that large cloud vendors are starting to experiment with bringing WASM to their platforms as well. In the case of Azure (my cloud of choice), it's running WASM workloads on WASI node pools in Azure Kubernetes Service. This is great because since Steve Sanderson showed an experimental WASI SDK for .NET Core back in March, I was looking for a good context to play with it too.

I took my first look at WASM/WASI node pools for AKS a couple of months ago. Back then the feature was based on Krustlet but I've quickly learned that the team is moving away from this approach and the feature doesn't work with the current version of the AKS control plane (it's a preview, it has that right). I've decided to wait. Time has passed, Deis Labs has evolved its tooling for running WebAssembly in Kubernetes from Krustlet to ContainerD shims, and the WASM/WASI node pools for AKS feature has embraced it. I've decided to take a look at it again.

The current implementation of WASM/WASI node pools provides support for two ContainerD shims: Spin and SpiderLightning. Both, Spin and Slight (alternative name for SpiderLightning) provide structure and interfaces for building distributed event-driven applications built from WebAssembly components. After inspecting both of them, I've decided to go with Spin for two reasons:

Spin is a framework for building applications in Fermyon Cloud. That meant a potentially stronger ecosystem and community. Also whatever I would learn, would have a broader application (not only WASM/WASI node pools for AKS).
Spin has (an alpha but still) a .NET SDK.

Your Scientists Were So Preoccupied With Whether They Could, They Didn't Stop to Think if They Should

When Steve Sanderson revealed the experimental WASI SDK for .NET Core, he showed that you can use it to run an ASP.NET Core server in a browser. He also clearly stated you absolutely shouldn't do that. Thinking about compiling .NET to WebAssembly and running it in AKS can make one wonder if it is the same case. After all, we can just run .NET in a container. Well, I believe it makes sense. WebAssembly apps have several advantages over containers:

WebAssembly apps are smaller than containers. In general, size is an Achilles' heel of .NET, but, even for the sample application, I've used here the WASM version is about ten times smaller than a container based on dotnet/runtime:7.0 (18.81 MB vs 190 MB).
WebAssembly apps start faster and execute faster than containers. This is something I haven't measured myself yet, but this paper seems to make quite a strong case for it.
WebAssembly apps are more secure than containers. This one is a killer aspect for me. Containers are not secure by default and significant effort has to be put to secure them. WebAssembly sandbox is secure by default.

This is why I believe exploring this truly makes sense.

But before we go further I want to highlight one thing - almost everything I'm using in this post is currently either an alpha or in preview. It's early and subject to change.

Configuring Azure CLI and Azure Subscription to Support WASI Node Pools

Working with preview features in Azure requires some preparation. The first step is registering the feature in your subscription

az feature register \
    --namespace Microsoft.ContainerService \
    --name WasmNodePoolPreview

The registration takes some time, you can query the features list to see if it has been completed.

az feature list \
    --query "[?contains(name, 'Microsoft.ContainerService/WasmNodePoolPreview')].{Name:name,State:properties.state}" \
    -o table

Once it's completed, the resource provider for AKS must be refreshed to pick it up.

az provider register \
    --namespace Microsoft.ContainerService

The subscription part is now ready, but to be able to use the feature you also have to add the preview extension to Azure CLI (WASM/WASI node pools can't be created from the Azure Portal).

az extension add \
    --name aks-preview \
    --upgrade

This is everything we need to start having fun with WASM/WASI node pools.

Creating an AKS Cluster

A WASM/WASI node pool can't be used for a system node pool. This means that before we create one, we have to create a cluster with a system node pool. Something like on the diagram below should be enough.

If you are familiar with spinning up an AKS cluster you can jump directly to the next section.

If you are looking for something to copy and paste, the below commands will create a resource group, container registry, and cluster with a single node in the system node pool.

az group create \
    -l ${LOCATION} \
    -g ${RESOURCE_GROUP}

az acr create \
    -n ${CONTAINER_REGISTRY} \
    -g ${RESOURCE_GROUP} \
    --sku Basic

az aks create \
    -n ${AKS_CLUSTER} \
    -g ${RESOURCE_GROUP} \
    -c 1 \
    --generate-ssh-keys \
    --attach-acr ${CONTAINER_REGISTRY}

Adding a WASM/WASI Node Pool to the AKS Cluster

A WASM/WASI node pool can be added to the cluster as any other node pool, with az aks nodepool add command. The part which makes it special is the workload-runtime parameter which takes a value of WasmWasi.

az aks nodepool add \
    -n ${WASI_NODE_POLL} \
    -g ${RESOURCE_GROUP} \
    -c 1 \
    --cluster-name ${AKS_CLUSTER} \
    --workload-runtime WasmWasi

The updated diagram representing the deployment looks like this.

You can inspect the WASM/WASI node pool by running kubectl get nodes and kubectl describe node commands.

With the infrastructure in place, it's time to build a Spin application.

Building a Spin Application With .NET 7

A Spin application has a pretty straightforward structure:

A Spin application manifest (spin.toml file).
One or more WebAssembly components.

The WebAssembly components are nothing else than event handlers, while the application manifest defines where there are located and maps them to triggers. The application mentions two triggers: HTTP and Redis. In the case of HTTP, you map components directly to routes.

So, first we need a component that will serve as a handler. In the introduction, I've written that one of the reasons why I have chosen Spin was the availability of .NET SDK. Sadly, when I tried to build an application using it, the application failed to start. The reason for that was that Spin SDK has too many features. Among other things it allows for making outbound HTTP requests which require wasi-outbound-http::request module, which is not present in WASM/WASI node pool (which makes sense as it's experimental and predicted to die once WASI networking APIs are stable).

Luckily, a Spin application supports fallback to WAGI. WAGI stands for WebAssembly Gateway Interface and is an implementation of CGI (now that's a blast from the past). It enables writing the WASM component as a "command line" application that handles HTTP requests by reading its properties from environment variables and writing responses to the standard output. This means we should start by creating a new .NET console application.

dotnet new console -o Demo.Wasm.Spin

Next we need to add a reference to Wasi.Sdk` package.

dotnet add package Wasi.Sdk --prerelease

It's time for the code. The bare required minimum for WAGI is outputting a Content-Type header and an empty line that separates headers from body. If you want to include a body, it goes after that empty line.

using System.Runtime.InteropServices;

Console.WriteLine("Content-Type: text/plain");
Console.WriteLine();
Console.WriteLine("-- Demo.Wasm.Spin --");

With the component ready, it's time for the application manifest. The one below defines an application using the HTTP trigger and mapping the component to a top-level wildcard route (so it will catch all requests). The executor is how the fallback to WAGI is specified.

spin_version = "1"
authors = ["Tomasz Peczek <tpeczek@gmail.com>"]
description = "Basic Spin application with .NET 7"
name = "spin-with-dotnet-7"
trigger = { type = "http", base = "/" }
version = "1.0.0"

[[component]]
id = "demo-wasm-spin"
source = "Demo.Wasm.Spin/bin/Release/net7.0/Demo.Wasm.Spin.wasm"
[component.trigger]
route = "/..."
executor = { type = "wagi" }

The last missing part is a Dockerfile which will allow us to build an image for deployment.

FROM mcr.microsoft.com/dotnet/sdk:7.0 AS build

WORKDIR /src
COPY . .
RUN dotnet build -c Release

FROM scratch
COPY --from=build /src/bin/Release/net7.0/Demo.Wasm.Spin.wasm ./bin/Release/net7.0/Demo.Wasm.Spin.wasm
COPY --from=build /src/spin.toml .

To run the image on WASM/WASI node pool it needs to be built and pushed to the container registry.

az acr login -n ${CONTAINER_REGISTRY}
docker build . -t ${CONTAINER_REGISTRY}.azurecr.io/spin-with-dotnet-7:latest
docker push ${CONTAINER_REGISTRY}.azurecr.io/spin-with-dotnet-7:latest

Running a Spin Application in WASM/WASI Node Pool

To run the Spin application we need to create proper resources in our AKS cluster. First is RuntimeClass which serves as a selection mechanism, so the Pods run on the WASM/WASI node pool. There are two node selectors related to WASM/WASI node pool kubernetes.azure.com/wasmtime-spin-v1 and kubernetes.azure.com/wasmtime-slight-v1, with spin and slight being their respective handlers. In our case, we only care about creating a RuntimeClass for kubernetes.azure.com/wasmtime-spin-v1.

apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
  name: "wasmtime-spin-v1"
handler: "spin"
scheduling:
  nodeSelector:
    "kubernetes.azure.com/wasmtime-spin-v1": "true"

With the RuntimeClass in place, we can define a Deployment.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: spin-with-dotnet-7
spec:
  replicas: 1
  selector:
    matchLabels:
      app: spin-with-dotnet-7
  template:
    metadata:
      labels:
        app: spin-with-dotnet-7
    spec:
      runtimeClassName: wasmtime-spin-v1
      containers:
        - name: spin-with-dotnet-7
          image: crdotnetwasi.azurecr.io/spin-with-dotnet-7:latest
          command: ["/"]

Last part is exposing our Spin application to the world. As this is just a demo I've decided to expose it directly as a Service of type LoadBalancer.

apiVersion: v1
kind: Service
metadata:
  name: spin-with-dotnet-7
spec:
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
  selector:
    app: spin-with-dotnet-7
  type: LoadBalancer

Now we can run kubectl apply and after a moment kubectl get svc to retrieve the IP address of the Service. You can paste that address into a browser and voilà.

That Was Fun!

Yes, that was really fun. All the stuff used here is still early bits, but it already shows possibilities. I intend to observe this space closely and possibly revisit it whenever some updates happen.

If you want to play with a ready-to-use demo, it's available on GitHub with a workflow ready to deploy it to Azure.

Micro Frontends in Action With ASP.NET Core - Universal Rendering With Blazor WebAssembly Based Web Components

noreply@blogger.com (Tomasz Pęczek) — Tue, 25 Oct 2022 12:24:00 +0000

In the last two posts of this series on implementing the Micro Frontends in Action samples in ASP.NET Core, I've focused on Blazor WebAssembly based Web Components as a way to achieve client-side composition. As a result, we have well-encapsulated frontend parts which can communicate with each other and the page. But there is a problem with the client-side rendered fragments, they appear after a delay. While the page loads, the user sees an empty placeholder. This is for sure a bad user experience, but it has even more serious consequences, those fragments may not be visible to search engine crawlers. In the case of something like a buy button, it is very important. So, how to deal with this problem? A possible answer is universal rendering.

Server-Side Routing via YARP in Azure Container Apps
Composition via YARP Transforms and Server-Side Includes (SSI)
Composition via Blazor WebAssembly Based Web Components
Communication Patterns for Blazor WebAssembly Based Web Components
Universal Rendering With Blazor WebAssembly Based Web Components

What Is Universal Rendering?

Universal rendering is about combining server-side and client-side rendering in a way that enables having a single codebase for both purposes. The typical approach is to handle the initial HTML rendering on the server with help of the server-side composition and then, when the page is loaded in the browser, seamlessly rerender the fragments on the client side. The initial rendering should only generate the static markup, while the rerender brings the full functionality. When done properly, this allows for a fast First Contentful Paint while maintaining encapsulation.

The biggest challenge is usually the single codebase, which in this case means rendering Blazor WebAssembly based Web Components on the server.

Server-Side Rendering for Blazor WebAssembly Based Web Components

There is no standard approach to rendering Web Components on the server. Usually, that requires some creative solutions. But Blazor WebAssembly based Web Components are different because on the server they are Razor components and ASP.NET Core provides support for prerendering Razor components. This support comes in form of a Component Tag Helper. But, before we get to it, we need to modify the Checkout service so it can return the rendered HTML. This is where the choice of hosted deployment with ASP.NET Core will be beneficial. We can modify the hosting application to support Blazor WebAssembly and controllers with views.

...

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddControllersWithViews();

var app = builder.Build();

...

app.UseBlazorFrameworkFiles();
app.UseStaticFiles();

app.UseRouting();

app.MapControllerRoute(
    name: "checkout-fragments",
    pattern: "fragment/buy/{sku}/{edition}",
    defaults: new { controller = "Fragments", action = "Buy" }
);

app.Run();

...

The controller for the defined route doesn't need any sophisticated logic, it only needs to pass the parameters to the view. For simplicity, I've decided to go with a dictionary as a model.

public class FragmentsController : Controller
{
    public IActionResult Buy(string sku, string edition)
    {
        IDictionary<string, string> model = new Dictionar<string, string>
        {
            { "Sku", sku },
            { "Edition", edition }
        };

        return View("Buy", model);
    }
}

The only remaining thing is the view which will be using the Component Tag Helper. In general, two pieces of information should be provided to this tag helper: the type of the component and the render mode. There are multiple render modes that render different markers to be used for later bootstrapping, but here we want to use the Static mode which renders only static HTML.

In addition to the component type and render mode, the Component Tag Helper also enables providing values for any component parameters with a param-{ParameterName} syntax. This is how we will pass the values from the model.

@using Demo.AspNetCore.MicroFrontendsInAction.Checkout.Frontend.Components
@addTagHelper *, Microsoft.AspNetCore.Mvc.TagHelpers
@model IDictionary<string, string>

<component type="typeof(BuyButton)" render-mode="Static" param-Sku="@(Model["Sku"])" param-Edition="@(Model["Edition"])" />

If we start the Checkout service and use a browser to navigate to the controller route, we will see an exception complaining about the absence of IBroadcastChannelService. At runtime Razor components are classes and ASP.NET Core will need to satisfy the dependencies while creating an instance. Sadly there is no support for optional dependencies. The options are either a workaround based on injecting IServiceProvider or making sure that the needed dependency is registered. I believe the latter to be more elegant.

...

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddBroadcastChannel();
builder.Services.AddControllersWithViews();

var app = builder.Build();

...

After this change, navigating to the controller route will display HTML, but in the case of the BuyButton, it is not exactly what we want. The BuyButton component contains the markup for a popup which is displayed upon clicking the button. The issue is, that the popup is hidden only with CSS. This is fine for the Web Component scenario (where the styles are already loaded when the component is being rendered) but not desired for this one. This is why I've decided to put a condition around the popup markup.

...

<button type="button" @ref="_buttonElement" @onclick="OnButtonClick">
    buy for @(String.IsNullOrWhiteSpace(Sku) || String.IsNullOrWhiteSpace(Edition)  ? "???" : _prices[Sku][Edition])
</button>
@if (_confirmationVisible)
{
    <div class="confirmation confirmation-visible">
        ...
    </div>
}

...

Now the HTML returned by the controller contains only the button markup.

Combining Server-Side and Client-Side Rendering

The Checkout service is now able to provide static HTML representing the BuyButton fragment, based on a single codebase. In the case of micro frontends that's not everything that is needed for universal rendering. The static HTML needs to be composed into the page before it's served. In this series, I've explored a single server-side composition technique based on YARP Transforms and Server-Side Includes), so I've decided to reuse it. First, I've copied the code for the body transform from the previous sample. Then, I modified the routing in the proxy to transform the request coming to the Decide service. The same as previously, I've created a dedicated route for static content so it doesn't go through the transform unnecessarily.

...

var routes = new[]
{
    ...
    new RouteConfig {
        RouteId = Constants.ROOT_ROUTE_ID,
        ClusterId = Constants.DECIDE_CLUSTER_ID,
        Match = new RouteMatch { Path = "/" },
        Metadata = SsiTransformProvider.SsiEnabledMetadata
    },
    (new RouteConfig {
        RouteId = Constants.DECIDE_ROUTE_ID + "-static",
        ClusterId = Constants.DECIDE_CLUSTER_ID,
        Match = new RouteMatch { Path = Constants.DECIDE_ROUTE_PREFIX + "/static/{**catch-all}" }
    }).WithTransformPathRemovePrefix(Constants.DECIDE_ROUTE_PREFIX),
    (new RouteConfig {
        RouteId = Constants.DECIDE_ROUTE_ID,
        ClusterId = Constants.DECIDE_CLUSTER_ID,
        Match = new RouteMatch { Path = Constants.DECIDE_ROUTE_PREFIX + "/{**catch-all}" },
        Metadata = SsiTransformProvider.SsiEnabledMetadata
    }).WithTransformPathRemovePrefix(Constants.DECIDE_ROUTE_PREFIX),
    ...
};

...

builder.Services.AddReverseProxy()
    .LoadFromMemory(routes, clusters);

...

Now I could modify the markup returned by the Decide service by placing the SSI directives inside the tag representing the Custom Element.

<html>
  ...
  <body class="decide_layout">
    ...
    <div class="decide_details">
      <checkout-buy sku="porsche" edition="standard">
        <!--#include virtual="/checkout/fragment/buy/porsche/standard" -->
      </checkout-buy>
    </div>
    ...
  </body>
</html>

This way the proxy can inject the static HTML into the markup while serving the initial response and once the JavaScript for Web Components is loaded they will be rerendered. We have achieved universal rendering.

What About Progressive Enhancements?

You might have noticed that there is a problem hiding in this solution. It's deceiving the users. The page looks like it's fully loaded but it's not interactive. There is a delay (until the JavaScript is loaded) before clicking the BuyButton has any effect. This is where progressive enhancements come into play.

I will not go into this subject further here, but one possible approach could be wrapping the button inside a form when the Checkout service is rendering static HTML.

@using Demo.AspNetCore.MicroFrontendsInAction.Checkout.Frontend.Components
@addTagHelper *, Microsoft.AspNetCore.Mvc.TagHelpers
@model IDictionary<string, string>

<form asp-controller="Checkout" asp-action="Buy" method="post">
    <input type="hidden" name="sku" valeu="@(Model["Sku"])">
    <input type="hidden" name="edition" valeu="@(Model["Edition"])">
    <component type="typeof(BuyButton)" render-mode="Static" param-Sku="@(Model["Sku"])" param-Edition="@(Model["Edition"])" />
</form>

Of course, that's not all the needed changes. The button would have to be rendered with submit type and the Checkout service needs to handle the POST request, redirect back to the product page, and manage the cart in the background.

If you are interested in doing that exercise, the sample code with universal rendering that you can use as a starter is available on GitHub.

Commits Promotion Between GitHub and Azure Databricks

noreply@blogger.com (Tomasz Pęczek) — Tue, 11 Oct 2022 12:30:00 +0000

One of the projects I'm currently working on is utilizing Azure Databricks for its machine learning component. The machine learning engineers working on the project wanted to use external IDEs for the development. Unfortunately, using external IDEs doesn't remove all needs for developing or testing directly in Azure Databricks. As we wanted our GitHub repository to be the only source of truth, we had to establish a commits promotion approach that would enable that.

Azure Databricks has support for Git integration, so we've decided to start by using it to integrate Azure Databricks with GitHub.

Configuring GitHub Credentials in Azure Databricks

The first step in setting up Git integration with Azure Databricks is credentials configuration. This is something that every engineer needs to do independently, to enable syncing workspace with a specific branch. It requires the following actions:

Login to GitHub, click the profile picture and go to Settings and then Developer settings at the bottom.
On the Settings / Developer settings switch to Personal access tokens and click Generate new token.
Fill in the form:
- Provide a recognizable Note for the token.
- Set the Expiration corresponding to the expected time of work on the project.
- Select the repo scope.
Click Generate token and copy the generated string.
Launch the Azure Databricks workspace.
Click the workspace name in the top right corner and then click the User Settings.
On the Git Integration tab select GitHub, provide your username, paste the copied token, and click Save.

Once the credentials to GitHub have been configured, the next step is the creation of an Azure Databricks Repo.

Creating Azure Databricks Repo Based on GitHub Repository

An Azure Databricks Repo is a clone of your remote Git repository (in this case GitHub repository) which can be managed through Azure Databricks UI. The creation process also happens through UI:

Launch the Azure Databricks workspace.
From the left menu choose Repos and then click Add Repo.
Fill in the form:
- Check the Create repo by cloning a Git repository.
- Select GitHub as Git provider.
- Provide the Git repository URL.
- The Repository name will auto-populate, but you can modify it to your liking.
Click Submit.

And it's done. You can now select a branch next to the newly created Azure Databricks Repo. If you wish you can click the down arrow next to the repo/branch name and create a notebook, folder, or file. If the notebook you want to develop in has been already in the cloned repository, you can just select it and start developing.

Promoting Commits From Azure Databricks Repo to GitHub Repository

As I've already mentioned, Azure Databricks Repo is managed through the UI. The Git dialog is accessible through the down arrow next to the repo/branch name or directly from the notebook through a button placed next to the name of the notebook (the label of the button is the current Git branch name). From the Git dialog, you can commit and push changes to the GitHub repository.

If you are interested in other manual operations, like pulling changes or resolving merge conflicts, they are well described in the documentation. I'm not going to describe their details here, because those are the operations we wanted to avoid by performing the majority of development in external IDEs and automating commits promotion from GitHub to Azure Databricks Repo.

Promoting Commits From GitHub Repository to Azure Databricks Repo

There are two ways to to manage Azure Databricks Repos programmatically: Repos API and Repos CLI. As GitHub-hosted runners doesn't come with preinstalled Databricks CLI, we've decided to go with Repos API and PowerShell.

We wanted a GitHub Actions workflow which would run on every push and update all Azure Databricks Repos mapped to the branch to which the push has happened. After going through API endpoints we came up with following flow.

Before we could start the implementation there was one more missing aspect - authentication.

Azure Databricks can use an Azure AD service principal as an identity for an automated tool or a CI/CD process. Creation of a service principal and adding it to an Azure Databricks workspace is a multistep process, which is quite well described in the documentation. After going through it, you should be able to create the following actions secrets for your repository:

AZURE_SP_CLIENT_ID - Application (client) ID for the service principal.
AZURE_SP_TENANT_ID - Directory (tenant) ID for the service principal.
AZURE_SP_CLIENT_SECRET - Client secret for the service principal.
AZURE_DATABRICKS_WORKSPACE_INSTANCE_NAME - The Azure Databricks workspace instance name.

With help of the first three of those secrets and the Microsoft identity platform REST API, we can obtain an Azure AD access token for the service principal. The request we need to make looks like this.

https://login.microsoftonline.com/<AZURE_SP_TENANT_ID>/oauth2/v2.0/token
Content-Type: application/x-www-form-urlencoded

client_id=<AZURE_SP_CLIENT_ID>&grant_type=client_credentials&scope=2ff814a6-3304-4ab8-85cb-cd0e6f879c1d%2F.default&client_secret=<AZURE_SP_CLIENT_SECRET>

The magical scope value (the URL-encoded 2ff814a6-3304-4ab8-85cb-cd0e6f879c1d/.default) is a programmatic identifier for Azure Databricks. The response to this request is a JSON object which contains the Azure AD access token in the access_token field. The PowerShell script to make the request and retrieve the token can look like the one below (assuming that the secrets have been put into environment variables).

$azureAdAccessTokenUri = "https://login.microsoftonline.com/$env:AZURE_SP_TENANT_ID/oauth2/v2.0/token"
$azureAdAccessTokenHeaders = @{ "Content-Type" = "application/x-www-form-urlencoded" }
$azureAdAccessTokenBody = "client_id=$env:AZURE_SP_CLIENT_ID&grant_type=client_credentials&scope=2ff814a6-3304-4ab8-85cb-cd0e6f879c1d%2F.default&client_secret=$env:AZURE_SP_CLIENT_SECRET"

$azureAdAccessTokenResponse = Invoke-RestMethod -Method POST -Uri $azureAdAccessTokenUri -Headers $azureAdAccessTokenHeaders -Body $azureAdAccessTokenBody
$azureAdAccessToken = $azureAdAccessTokenResponse.access_token

Having the token, we can start making requests against Repos API. The first request we want to make in our flow is for getting the repos.

$azureDatabricksReposUri = "https://$env:AZURE_DATABRICKS_WORKSPACE_INSTANCE_NAME/api/2.0/repos"
$azureDatabricksReposHeaders = @{ Authorization = "Bearer $azureAdAccessToken" }

$azureDatabricksReposResponse = Invoke-RestMethod -Method GET -Uri $azureDatabricksReposUri -Headers $azureDatabricksReposHeaders

The $azureDatabricksReposHeaders will be used for subsequent requests as well, because we assume that the access token shouldn't expire before all repos are updated (the default expiration time is ~60 minutes). There is one more assumption here - that there are no more than twenty repos. The results from the /repos endpoint are paginated (with twenty being the page size) which the above script ignores. If there are more than twenty repos, the script needs to be adjusted to handle that.

Once we have all the repos we can iterate through them and update those which have matching URL (in case different repositories than the current one has also been mapped) and branch (so we don't perform unnecessary updates).

$githubRepositoryUrl = $env:GITHUB_REPOSITORY_URL.replace("git://","https://")

foreach ($azureDatabricksRepo in $azureDatabricksReposResponse.repos)
{
    if (($azureDatabricksRepo.url -eq $githubRepositoryUrl) -and ($azureDatabricksRepo.branch -eq $env:GITHUB_BRANCH_NAME))
    {
    $azureDatabricksRepoId = $azureDatabricksRepo.id;
    $azureDatabricksRepoUri  = "https://$env:AZURE_DATABRICKS_WORKSPACE_INSTANCE_NAME/api/2.0/repos/$azureDatabricksRepoId"
    $updateAzureDatabricksRepoBody = @{ "branch" = $azureDatabricksRepo.branch }

    Invoke-RestMethod -Method PATCH -Uri $azureDatabricksRepoUri -Headers $azureDatabricksReposHeaders -Body ($updateAzureDatabricksRepoBody|ConvertTo-Json)
    }
}

The GITHUB_REPOSITORY_URL and GITHUB_BRANCH_NAME are being injected into environment variables from github context of the action.

That's all the logic we need, you can find the complete workflow here. Sadly, at least in our case, it has thrown the following error on the first run.

{"error_code":"PERMISSION_DENIED","message":"Missing Git | provider credentials. Go to User Settings > Git Integration to | add your personal access token."}

The error does make sense. After all, from the perspective of Azure Databricks, the service principal is a user and we have never configured GitHub credentials for that user. This raised two questions.

The first question was, which GitHub user should those credentials represent? This is where the concept of a GitHub machine user comes into play. A GitHub machine user is a GitHub personal account, separate from the GitHub personal accounts of engineers/developers in your organization. It should be created against a dedicated email provided by your IT department and used only for automation scenarios.

The second question was, how to configure the credentials. You can't launch the Azure Databricks workspace as the service principal user and do it through the UI. Luckily, Azure Databricks provides Git Credentials API which can be used for this task. You can use Postman (or any other tool of your preference) to first make the described above request for Azure AD access token, and then make the below request to configure the credentials.

https://<WORKSPACE_INSTANCE_NAME>/api/2.0/git-credentials
Content-Type: application/json

{
   "personal_access_token": "<GitHub Machine User Personal Access Token>",
   "git_username": "<GitHub Machine User Username>",
   "git_provider": "GitHub"
}

After this operation, the GitHub Actions workflow started working as expected.

What This Is Not

This is not CI/CD for Azure Databricks. This is just a process supporting daily development in the Azure Databricks context. If you are looking for CI/CD approaches to Azure Databricks, you can take a look here.

Micro Frontends in Action With ASP.NET Core - Communication Patterns for Blazor WebAssembly Based Web Components

noreply@blogger.com (Tomasz Pęczek) — Mon, 12 Sep 2022 19:11:00 +0000

I'm continuing my series on implementing the Micro Frontends in Action samples in ASP.NET Core, and I'm continuing the subject of Blazor WebAssembly based Web Components. In the previous post, the project has been expanded with a new service that provides its fronted fragment as a Custom Element power by Blazor WebAssembly. In this post, I will explore how Custom Elements can communicate with other frontend parts.

Server-Side Routing via YARP in Azure Container Apps
Composition via YARP Transforms and Server-Side Includes (SSI)
Composition via Blazor WebAssembly Based Web Components
Communication Patterns for Blazor WebAssembly Based Web Components
Universal Rendering With Blazor WebAssembly Based Web Components

There are three communication scenarios I would like to explore: passing information from page to Custom Element (parent to child), passing information from Custom Element to page (child to parent), and passing information between Custom Elements (child to child). Let's go through them one by one.

Page to Custom Element

When it comes to passing information from page to Custom Element, there is a standard approach that every web developer will expect. If I want to disable a button, I set an attribute. If I want to change the text on a button, I set an attribute. In general, if I want to change the state of an element, I set an attribute. The same expectation applies to Custom Elements. How to achieve that?

As mentioned in the previous post, the ES6 class, which represents a Custom Element, can implement a set of lifecycle methods. One of these methods is attributeChangedCallback. It will be invoked each time an attribute from a specified list is added, removed, or its value is changed. The list of the attributes which will result in invoking the attributeChangedCallback is defined by a value returned from observedAttributes static get method.

So, in the case of Custom Elements implemented in JavaScript, one has to implement the observedAttributes to return an array of attributes that can modify the state of the Custom Element and implement the attributeChangedCallback to modify that state. Once again, you will be happy to know that all this work has already been done in the case of Blazor WebAssembly. The Microsoft.AspNetCore.Components.CustomElements package, which wraps Blazor components as Custom Elements handles that. It provides an implementation of observedAttributes which returns all the properties marked as parameters, and an implementation of attributeChangedCallback which will update parameters values and give the component a chance to rerender. That makes the implementation quite simple.

I've added a new property named Edition to the BuyButton component, which I created in the previous post. The new property impacts the price depending if the client has chosen a standard or platinum edition. I've also marked the new property as a parameter.

<button type="button" @onclick="OnButtonClick">
    buy for @(String.IsNullOrWhiteSpace(Sku) || String.IsNullOrWhiteSpace(Edition)  ? "???" : _prices[Sku][Edition])
</button>
...

@code {
    private IDictionary<string, Dictionary<string, int>> _prices = new Dictionary<string, Dictionary<string, int>>
    {
        { "porsche", new Dictionary<string, int> { { "standard", 66 }, { "platinum", 966 } } },
        { "fendt", new Dictionary<string, int> { { "standard", 54 }, { "platinum", 945 } }  },
        { "eicher", new Dictionary<string, int> { { "standard", 58 }, { "platinum", 958 } }  }
    };

    [Parameter]
    public string? Sku { get; set; }

    [Parameter]
    public string? Edition { get; set; }

    ...
}

This should be all from the component perspective. The rest should be only about using the attribute representing the property. First, I've added it to the markup served by the Decide service with the default value. I've also added a checkbox that allows choosing the edition.

<html>
    ...
    <body class="decide_layout">
        ...
        <div class="decide_details">
            <label class="decide_editions">
                <p>Material Upgrade?</p>
                <input type="checkbox" name="edition" value="platinum" />
                <span>Platinum<br />Edition</span>
                <img src="https://mi-fr.org/img/porsche_platinum.svg" />
            </label>
            <checkout-buy sku="porsche" edition="standard"></checkout-buy>
        </div>
        ...
    </body>
</html>

Then I implemented an event handler for the change event of that checkbox, where depending on its state, I would change the value of the edition attribute on the custom element.

(function() {
    ...
    const editionsInput = document.querySelector(".decide_editions input");
    ...
    const buyButton = document.querySelector("checkout-buy");

    ...

    editionsInput.addEventListener("change", e => {
        const edition = e.target.checked ? "platinum" : "standard";
        buyButton.setAttribute("edition", edition);
        ...
    });
})();

It worked without any issues. Checking and unchecking the checkbox would result in nicely displaying different prices on the button.

Custom Element to Page

The situation with passing information from Custom Element to the page is similar to passing information from page to Custom Element - there is an expected standard mechanism: events. If something important has occurred internally in the Custom Element and the external world should know about it, Custom Element should raise an event to which whoever is interested can subscribe.

How to raise a JavaScript event from Blazor? This requires calling a JavaScript function which will wrap a call to dispatchEvent. Why can't dispatchEvent be called directly? That's because Blazor requires function identifier to be relative to the global scope, while dispatchEvent needs to be called on an instance of an element. This raises another challenge. Our wrapper function will require a reference to the Custom Element. Blazor supports capturing references to elements to pass them to JavaScript. The @ref attribute can be included in HTML element markup, resulting in a reference being stored in the variable it is pointing to. This means that the reference to the Custom Element itself can't be passed directly, but a reference to its child element can.

I've written a wrapper function that takes the reference to the button element (but it could be any direct child of the Custom Element) as a parameter and then calls dispatchEvent on its parent.

window.checkout = (function () {
    return {
        dispatchItemAddedEvent: function (checkoutBuyChildElement) {
            checkoutBuyChildElement.parentElement.dispatchEvent(new CustomEvent("checkout:item_added"));
        }
    };
})();

I wanted the event to be raised when the button has been clicked, so I've modified the OnButtonClick to use injected IJSRuntime to call my JavaScript function. In the below code, you can also see the @ref attribute in action and how I'm passing that element reference to the wrapper function.

@using Microsoft.JSInterop

@inject IJSRuntime JS

<button type="button" @ref="_buttonElement" @onclick="OnButtonClick">
    buy for @(String.IsNullOrWhiteSpace(Sku) || String.IsNullOrWhiteSpace(Edition)  ? "???" : _prices[Sku][Edition])
</button>
...

@code {
    private ElementReference _buttonElement;

    ...

    private async Task OnButtonClick(MouseEventArgs e)
    {
        ...

        await JS.InvokeVoidAsync("checkout.dispatchItemAddedEvent", _buttonElement);
    }

    ...
}

For the whole thing to work, I had to reference the JavaScript from the Decide service markup so that the wrapper function could be called.

<html>
    ...
    <body class="decide_layout">
        ...
        <script src="/checkout/static/components.js"></script>
        <script src="/checkout/_content/Microsoft.AspNetCore.Components.CustomElements/BlazorCustomElements.js"></script>
        ...
    </body>
</html>

Now I could subscribe to the checkout:item_added event and add some bells and whistles whenever it's raised.

(function() {
    ...
    const productElement = document.querySelector(".decide_product");
    const buyButton = document.querySelector("checkout-buy");

    ...

    buyButton.addEventListener("checkout:item_added", e => {
        productElement.classList.add("decide_product--confirm");
    });

    ...
})();

Custom Element to Custom Element

Passing information between Custom Elements is where things get interesting. That is because there is no direct relation between Custom Elements. Let's assume that the Checkout service exposes a second Custom Element which provides a cart representation. The checkout button and mini-cart don't have to be used together. There might be a scenario where only one of them is present, or there might be scenarios where there are rendered by independent parents.

Of course, everything is happening in the browser's context, so there is always an option to search through the entire DOM tree. This is an approach that should be avoided. First, it's tight coupling, as it requires Custom Element to have detailed knowledge about other Custom Element. Second, it wouldn't scale. What if there are ten different types of Custom Elements to which information should be passed? That would require ten different searches.

Another option is leaving orchestration to the parent. The parent would listen to events from one Custom Element and change properties on the other. This breaks the separation of responsibilities as the parent (in our case, the Decide service) is now responsible for implementing logic that belongs to someone else (in our case, the Checkout Service).

What is needed is a communication channel that will enable a publish-subscribe pattern. This will ensure proper decoupling. The classic implementation of such a channel is events based bus. The publisher raises events with bubbling enabled (by default, it's not), so subscribers can listen for those events on the window object. This is an established approach, but it's not the one I've decided to implement. An events-based bus is a little bit "too public" for me. In the case of multiple Custom Elements communicating, there are a lot of events on the window object, and I would prefer more organization. Luckily, modern browsers provide an alternative way to implement such a channel - the Broadcast Channel API. You can think about Broadcast Channel API as a simple message bus that provides the capability of creating named channels. The hidden power of Broadcast Channel API is that it allows communication between windows/tabs, iframes, web workers, and service workers.

Using Broadcast Channel API in Blazor once again requires JavaScript interop. I've decided to use this opportunity to build a component library that provides easy access to it. I'm not going to describe the process of creating a component library in this post, but if you are interested just let me know and I'm happy to write a separate post about it. If you want to use the Broadcast Channel API, it's available on NuGet.

After building and publishing the component library, I referenced it in the Checkout project and registered the service it provides.

...

var builder = WebAssemblyHostBuilder.CreateDefault(args);

builder.RootComponents.RegisterAsCustomElement<BuyButton>("checkout-buy");

builder.Services.AddBroadcastChannel();

...

In the checkout button component, I've injected the service. The channel can be created by calling CreateOrJoinAsync, and I'm doing that in OnAfterRenderAsync. I've also made the component implement IAsyncDisposable, where the channel is disposed to avoid JavaScript memory leaks. The last part was calling PostMessageAsync as part of OnButtonClick to send the message to the channel. This completes the publisher.

...

@implements IAsyncDisposable

...
@inject IBroadcastChannelService BroadcastChannelService

...

@code {
    ...
    private IBroadcastChannel? _broadcastChannel;

    ...

    protected override async Task OnAfterRenderAsync(bool firstRender)
    {
        if (firstRender)
        {
            _broadcastChannel = await BroadcastChannelService.CreateOrJoinAsync("checkout:item-added");
        }
    }

    private async Task OnButtonClick(MouseEventArgs e)
    {
        ...

        if (_broadcastChannel is not null)
        {
            await _broadcastChannel.PostMessageAsync(new CheckoutItem { Sku = Sku, Edition = Edition });
        }

        ...
    }

    ...

    public async ValueTask DisposeAsync()
    {
        if (_broadcastChannel is not null)
        {
            await _broadcastChannel.DisposeAsync();
        }
    }
}

The mini-cart component will be the subscriber. I've added there the same code for injecting the service, joining the channel, and disposing of it. The main difference here is that the component will subscribe to the channel Message event instead of sending anything. The BroadcastChannelMessageEventArgs contains the message which has been sent in the Data property as JsonDocument, which can be deserialized to the desired type. In the mini-cart component, I'm using the message to add items.

@using System.Text.Json;

@implements IAsyncDisposable

@inject IBroadcastChannelService BroadcastChannelService

@(_items.Count == 0  ? "Your cart is empty." : $"You've picked {_items.Count} tractors:")
@foreach (var item in _items)
{
    <img src="https://mi-fr.org/img/@(item.Sku)_@(item.Edition).svg" />
}

@code {
    private IList<CheckoutItem> _items = new List<CheckoutItem>();
    private IBroadcastChannel? _broadcastChannel;
    private JsonSerializerOptions _jsonSerializerOptions = new JsonSerializerOptions { PropertyNamingPolicy = JsonNamingPolicy.CamelCase };

    protected override async Task OnAfterRenderAsync(bool firstRender)
    {
        if (firstRender)
        {
            _broadcastChannel = await BroadcastChannelService.CreateOrJoinAsync("checkout:item-added");
            _broadcastChannel.Message += OnMessage;
        }
    }

    private void OnMessage(object? sender, BroadcastChannelMessageEventArgs e)
    {
        _items.Add(e.Data.Deserialize<CheckoutItem>(_jsonSerializerOptions));

        StateHasChanged();
    }

    public async ValueTask DisposeAsync()
    {
        if (_broadcastChannel is not null)
        {
            await _broadcastChannel.DisposeAsync();
        }
    }
}

The last thing I did in the Checkout service was exposing the mini-cart component.

...

var builder = WebAssemblyHostBuilder.CreateDefault(args);

builder.RootComponents.RegisterAsCustomElement<BuyButton>("checkout-buy");
builder.RootComponents.RegisterAsCustomElement<MiniCart>("checkout-minicart");

builder.Services.AddBroadcastChannel();

...

Now the mini-cart could be included in the HTML owned by the Decide service.

<html>
  ...
  <body class="decide_layout">
    ...
    <div class="decide_details">
      <checkout-buy sku="porsche"></checkout-buy>
    </div>
    ...
    <div class="decide_summary">
      <checkout-minicart></checkout-minicart>
    </div>
    ...
  </body>
</html>

Playing With the Complete Sample

The complete sample is available on GitHub. You can run it locally by spinning up all the services, but I've also included a GitHub Actions workflow that can deploy the whole solution to Azure (you just need to fork the repository and provide your own credentials).

Micro Frontends in Action With ASP.NET Core - Composition via Blazor WebAssembly Based Web Components

noreply@blogger.com (Tomasz Pęczek) — Wed, 17 Aug 2022 19:21:00 +0000

This is another post in my series on implementing the samples from Micro Frontends in Action in ASP.NET Core:

Server-Side Routing via YARP in Azure Container Apps
Composition via YARP Transforms and Server-Side Includes (SSI)
Composition via Blazor WebAssembly Based Web Components
Communication Patterns for Blazor WebAssembly Based Web Components
Universal Rendering With Blazor WebAssembly Based Web Components

This time I'm jumping from server-side composition to client-side composition. I've used the word jumping because I haven't fully covered server-side composition yet. There is one more approach to server-side composition which I intend to cover later, but as lately I was doing some Blazor WebAssembly work I was more eager to write this one.

Expanding The Project

As you may remember from the first post, the project consists of two services that are hosted in Azure Container Apps.

Both services are using server-side rendering for their frontends, and the Decide service is loading Inspire service frontend fragment via Ajax. It's time to bring a new service to the picture, the Checkout service.

The Checkout service is responsible for checkout flow. As this flow is more sophisticated than what Decide and Inspire services provide, the Checkout service requires client-side rendering to provide the experience of a single page application. This requirement creates a need for isolation and encapsulation of the Checkout service frontend fragment. There is a suite of technologies that can help solve that problem - Web Components.

Web Components

Web Components aim at enabling web developers to create reusable elements with well-encapsulated functionality. To achieve that, they bring together four different specifications:

Custom Elements, which allows defining your own tags (custom elements) with their business logic.
Shadow DOM, which enables scripting and styling without collisions with other elements.
HTML Templates, which allows writing markup templates.
ES Module, which defines a consistent way for JavaScript inclusion and reuse.

The Custom Elements is the most interesting one in this context. The way to create a Custom Element is to implement an ES6 class that extends HTMLElement and register it via window.customElements.define. The class can also implement a set of lifecycle methods (constructor, connectedCallback, disconnectedCallback, or attributeChangedCallback). This allows for initializing a SPA framework (Angular, React, Vue, etc.) and instructing it to use this as a root for rendering.

This is exactly what is needed for the Checkout service, where the SPA framework will be Blazor WebAssembly.

Creating a Blazor WebAssembly Based Custom Element

The way Blazor WebAssembly works fits nicely with Custom Elements. If you've ever taken a look at the Program.cs of a Blazor WebAssembly project, you might have noticed some calls to builder.RootComponents.Add. This is because the WebAssembly to which your project gets compiled is designed to perform rendering into elements. Thanks to that a Blazor WebAssembly application can be wrapped into a Custom Element, it just requires proper initialization. You will be happy to learn, that this work has already been done. As part of AspLabs Steve Sanderson has prepared a package and instructions on how to make Blazor components available as Custom Elements. Let's do it.

I've started with an empty Blazor WebAssembly application hosted in ASP.NET Core, to which I've added a component that will server as a button initiating the checkout flow (the final version of that component will also contain some confirmation toast, if you're interested you can find it here).

<button type="button" @onclick="OnButtonClick">buy for @(String.IsNullOrWhiteSpace(Sku) ? "???" : _prices[Sku])</button>
...

@code {
    // Dictionary of tractor prices
    private IDictionary<string, int> _prices = new Dictionary<string, int>
    {
        { "porsche", 66 },
        { "fendt", 54 },
        { "eicher", 58 }
    };

    ...
}

Next, I've added the Microsoft.AspNetCore.Components.CustomElements package to the project.

<Project Sdk="Microsoft.NET.Sdk.BlazorWebAssembly">
  ...
  <ItemGroup>
    ...
    <PackageReference Include="Microsoft.AspNetCore.Components.CustomElements" Version="0.1.0-alpha.*" />
  </ItemGroup>
</Project>

I've removed all .razor files besides the above component and _Imports.razor. After that, I modified the Program.cs by removing the builder.RootComponents.Add calls and adding builder.RootComponents.RegisterAsCustomElement to expose my component as a checkout-buy element.

...

var builder = WebAssemblyHostBuilder.CreateDefault(args);

builder.RootComponents.RegisterAsCustomElement<BuyButton>("checkout-buy");

await builder.Build().RunAsync();

This is it. After the build/publish the service will server the custom element through the _content/Microsoft.AspNetCore.Components.CustomElements/BlazorCustomElements.js script, so it's time to plug it into the page served by Decide service.

Using a Blazor WebAssembly Based Custom Element

Before the Decide service can utilize the custom element, it is necessary to set up the routing. As described in previous posts, all services are hidden behind a YARP-based proxy, which routes the incoming requests based on prefixes (to avoid conflicts). So far, both services were built in a way where the prefixes were an integral part of their implementation (they were included in actions and static content paths). With the Checkout service that would be hard to achieve, due to Blazor WebAssembly static assets.

There is a way to control the base path for Blazor WebAssembly static assets (through StaticWebAssetBasePath project property) but it doesn't affect the BlazorCustomElements.js path. So, instead of complicating the service implementation to handle the prefix, it seems a lot better to make the prefixes a proxy concern and remove them there. YARP has an out-of-the-box capability to do so through PathRemovePrefix transform. There is a ready-to-use extension method (.WithTransformPathRemovePrefix) which allows adding that transform to a specific route.

...

var routes = new[]
{
    ...
    (new RouteConfig
    {
        RouteId = Constants.CHECKOUT_ROUTE_ID,
        ClusterId = Constants.CHECKOUT_CLUSTER_ID,
        Match = new RouteMatch { Path = Constants.CHECKOUT_ROUTE_PREFIX + "/{**catch-all}" }
    }).WithTransformPathRemovePrefix(Constants.CHECKOUT_ROUTE_PREFIX),
    ...
};

var clusters = new[]
{
    ...
    new ClusterConfig()
    {
        ClusterId = Constants.CHECKOUT_CLUSTER_ID,
        Destinations = new Dictionary<string, DestinationConfig>(StringComparer.OrdinalIgnoreCase)
        {
            { Constants.CHECKOUT_SERVICE_URL, new DestinationConfig() { Address = builder.Configuration[Constants.CHECKOUT_SERVICE_URL] } }
        }
    }
};

builder.Services.AddReverseProxy()
    .LoadFromMemory(routes, clusters);

...

Now the Decide service views can be modified to include the custom element. The first step is to add the static assets.

<html>
  <head>
    ...
    <link href="/checkout/static/components.css" rel="stylesheet" />
  </head>
  <body class="decide_layout">
    ...
    <script src="/checkout/_content/Microsoft.AspNetCore.Components.CustomElements/BlazorCustomElements.js"></script>
    <script src="/checkout/_framework/blazor.webassembly.js"></script>
  </body>
</html>

Sadly, this will not be enough for Blazor to work. When Blazor WebAssembly starts, it requests additional boot resources. They must be loaded from the Checkout service as well, which means including the prefix in URIs. This can be achieved thanks to the JS initializers feature which has been added in .NET 6. The automatic start of Blazor WebAssembly can be disabled, and it can be started manually which allows providing a function to customize the URIs.

<html>
  ...
  <body class="decide_layout">
    ...
    <script src="/checkout/_framework/blazor.webassembly.js" autostart="false"></script>
    <script>
      Blazor.start({
        loadBootResource: function (type, name, defaultUri, integrity) {
          return `/checkout/_framework/${name}`;
        }
      });
    </script>
  </body>
</html>

Finally, the custom element can be used. It will be available through a tag matching the name provided as a parameter to builder.RootComponents.RegisterAsCustomElement.

<html>
  ...
  <body class="decide_layout">
    ...
    <div class="decide_details">
      <checkout-buy sku="porsche"></checkout-buy>
    </div>
    ...
  </body>
</html>

The Expanded Project

As with previous samples, I've created a GitHub Actions workflow that deploys the solution to Azure. After the deployment, if you navigate to the URL of the ca-app-proxy Container App, you will see a page with following layout.

You can click the button rendered by the custom element and see the toast notification.

This approach highlights one of the benefits of micro frontends, the freedom to choose the fronted stack. Thanks to the encapsulation of the Blazor WebAssembly app into a custom element, the Decide service can host it in its static HTML without understanding anything except how to load the scripts. If we would want to hide even that (something I haven't done here) we could create a script that encapsulates Blazor WebAssembly loading and initialization. That script could be a shared asset that loads Blazor WebAssembly from CDN (which besides the performance benefit would be also a way to go for a solution with multiple services using Blazor WebAssembly).

Exploring Communication of Rate Limits in ASP.NET Core With Rate Limit Headers

noreply@blogger.com (Tomasz Pęczek) — Thu, 28 Jul 2022 12:25:00 +0000

Rate limiting (sometimes also referred to as throttling) is a key mechanism when it comes to ensuring API responsiveness and availability. By enforcing usage quotas, it can protect an API from issues like:

Denial Of Service (DOS) attacks
Degraded performance due to traffic spikes
Monopolization by a single consumer

Despite its importance, the typical approach to rate limiting is far from perfect when it comes to communicating usage quotas by services and (as a result) respecting those quotas by clients. It shouldn't be a surprise that various services were experimenting with different approaches to solve this problem. The common pattern in the web world is that some of such experiments start to gain traction which results in standardization efforts. This is exactly what is currently happening around communicating services usage quotas with RateLimit Fields for HTTP Internet-Draft. As rate limiting will have built-in support with .NET 7, I thought it might be a good time to take a look at what this potential standard is bringing. But before that, let's recall how HTTP currently supports rate limiting (excluding custom extensions).

Current HTTP Support for Rate Limiting

When it comes to current support for rate limiting in HTTP, it's not much. If a service detects that a client has reached the quota, instead of a regular response it may respond with 429 (Too Many Request) or 503 (Service Unavailable). Additionally, the service may include a Retry-After header in the response to indicate how long the client should wait before making another request. That's it. It means that client can be only reactive. There is no way for a client to get the information about the quota to avoid hitting it.

In general, this works. That said, handling requests which are above the quota still consumes some resources on the service side. Clients would also prefer to be able to understand the quota and adjust their usage patterns instead of handling it as an exceptional situation. So as I said, it's not much.

Proposed Rate Limit Headers

The RateLimit Fields for HTTP Internet-Draft proposes four new headers which aim at enabling a service to communicate usage quotas and policies:

RateLimit-Limit to communicate the total quota within a time window.
RateLimit-Remaining to communicate the remaining quota within the current time window.
RateLimit-Reset to communicate the time (in seconds) remaining in the current time window.
RateLimit-Policy to communicate the overall quota policy.

The most interesting one is RateLimit-Policy. It is a list of quota policy items. A quota policy item consists of a quota limit and a single required parameter w which provides a time window in seconds. Custom parameters are allowed and should be treated as comments. Below you can see an example of RateLimit-Policy which informs that client is allowed to make 10 requests per second, 50 requests per minute, 1000 requests per hour, and 5000 per 24 hours.

RateLimit-Policy: 10;w=1, 50;w=60, 1000;w=3600, 5000;w=86400

The only headers which intend to be required are RateLimit-Limit and RateLimit-Reset (RateLimit-Remaining is strongly recommended). So, how an ASP.NET Core based service can server those headers.

Communicating Quotas When Using ASP.NET Core Middleware

As I've already mentioned, built-in support for rate limiting comes to .NET with .NET 7. It brings generic purpose primitives for writing rate limiters as well as a few ready-to-use implementations. It also brings a middleware for ASP.NET Core. The below example shows the definition of a fixed time window policy which allows 5 requests per 10 seconds. The OnRejected callback is also provided to return 429 (Too Many Request) status code and set the Retry-After header value based on provided metadata.

using System.Globalization;
using System.Threading.RateLimiting;
using Microsoft.AspNetCore.RateLimiting;

var builder = WebApplication.CreateBuilder(args);

var app = builder.Build();

app.UseHttpsRedirection();

app.UseRateLimiter(new RateLimiterOptions
{
    OnRejected = (context, cancellationToken) =>
    {
        if (context.Lease.TryGetMetadata(MetadataName.RetryAfter, out var retryAfter))
        {
            context.HttpContext.Response.Headers.RetryAfter = ((int)retryAfter.TotalSeconds).ToString(NumberFormatInfo.InvariantInfo);
        }

        context.HttpContext.Response.StatusCode = StatusCodes.Status429TooManyRequests;

        return new ValueTask();
    }
}.AddFixedWindowLimiter("fixed-window", new FixedWindowRateLimiterOptions(
    permitLimit: 5,
    queueProcessingOrder: QueueProcessingOrder.OldestFirst,
    queueLimit: 0,
    window: TimeSpan.FromSeconds(10),
    autoReplenishment: true
)));

app.MapGet("/", context => context.Response.WriteAsync("-- Demo.RateLimitHeaders.AspNetCore.RateLimitingMiddleware --"))
    .RequireRateLimiting("fixed-window");

app.Run();

The question is, if and how this can be extended to return the rate limit headers? The answer is, sadly, that there seems to be no way to provide the required ones right now. All the information about rate limit policies is well hidden from public access. It would be possible to provide RateLimit-Limit and RateLimit-Policy as they are a direct result of provided options. It is also possible to provide RateLimit-Remaining, but it requires rewriting a lot of the middleware ecosystem to get the required value. What seems completely impossible to get right now is RateLimit-Reset as timers are managed centrally deep in System.Threading.RateLimiting core without any access to their state. There is an option to provide your own timers, but it would mean rewriting the entire middleware stack and taking a lot of responsibility from System.Threading.RateLimiting. Let's hope that things will improve.

Communicating Quotas When Using AspNetCoreRateLimit Package

That built-in support for rate limiting is something that is just coming to .NET. So far the ASP.NET Core developers were using their own implementations or non-Microsoft packages for this purpose. Arguably, the most popular rate limiting solution for ASP.NET Core is AspNetCoreRateLimit. The example below provides similar functionality to the one from the built-in rate limiting example.

using AspNetCoreRateLimit;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddMemoryCache();

builder.Services.Configure<IpRateLimitOptions>(options =>
{
    options.EnableEndpointRateLimiting = true;
    options.StackBlockedRequests = false;
    options.HttpStatusCode = 429;
    options.GeneralRules = new List<RateLimitRule>
    {
        new RateLimitRule { Endpoint = "*", Period = "10s", Limit = 5 }
    };
});

builder.Services.AddInMemoryRateLimiting();

builder.Services.AddSingleton<IRateLimitConfiguration, RateLimitConfiguration>();

var app = builder.Build();

app.UseHttpsRedirection();

app.UseIpRateLimiting();

app.MapGet("/", context => context.Response.WriteAsync("-- Demo.RateLimitHeaders.AspNetCore.RateLimitPackage --"));

app.Run();

The AspNetCoreRateLimit has its own custom way of communicating quotas with HTTP headers. In the case of the above example, you might receive the following values in response.

X-Rate-Limit-Limit: 10s
X-Rate-Limit-Remaining: 4
X-Rate-Limit-Reset: 2022-07-24T11:30:47.2291052Z

As you can see, they provide potentially useful information but not in a way that RateLimit Fields for HTTP is going for. Luckily, AspNetCoreRateLimit is not as protective about its internal state and algorithms, the information needed can be accessed and served in a different way.

The information about the current state is kept in IRateLimitCounterStore. This is a dependency that could be accessed directly, but the method for generating needed identifiers lives in ProcessingStrategy so it will be better to create an implementation of it dedicated for purpose of just getting the counters state.

internal interface IRateLimitHeadersOnlyProcessingStrategy : IProcessingStrategy
{ }

internal class RateLimitHeadersOnlyProcessingStrategy : ProcessingStrategy, IRateLimitHeadersOnlyProcessingStrategy
{
    private readonly IRateLimitCounterStore _counterStore;
    private readonly IRateLimitConfiguration _config;

    public RateLimitHeadersOnlyProcessingStrategy(IRateLimitCounterStore counterStore, IRateLimitConfiguration config) : base(config)
    {
        _counterStore = counterStore;
        _config = config;
    }

    public override async Task ProcessRequestAsync(ClientRequestIdentity requestIdentity, RateLimitRule rule,
        ICounterKeyBuilder counterKeyBuilder, RateLimitOptions rateLimitOptions, CancellationToken cancellationToken = default)
    {
        string rateLimitCounterId = BuildCounterKey(requestIdentity, rule, counterKeyBuilder, rateLimitOptions);

        RateLimitCounter? rateLimitCounter = await _counterStore.GetAsync(rateLimitCounterId, cancellationToken);
        if (rateLimitCounter.HasValue)
        {
            return new RateLimitCounter
            {
                Timestamp = rateLimitCounter.Value.Timestamp,
                Count = rateLimitCounter.Value.Count
            };
        }
        else
        {
            return new RateLimitCounter
            {
                Timestamp = DateTime.UtcNow,
                Count = _config.RateIncrementer?.Invoke() ?? 1
            };
        }
    }
}

The second thing that is needed are rules which apply to specific endpoint and identity. Those can be retrieved from specific (either based on IP or client identifier) IRateLimitProcessor. The IRateLimitProcessor is also a tunnel to IProcessingStrategy, so it's nice we have a dedicated one. But what about the identity I've just mentioned? The algorithm to retrieve it lies in RateLimitMiddleware, so access to it will be needed. There are two options here. One is to inherit from RateLimitMiddleware and the other is to create an instance of one of its implementation and use it as a dependency. The first case would require hiding the base implementation of Invoke as it can't be overridden. I didn't like that, so I went with keeping an instance as a dependency approach. This led to the following code.

internal class IpRateLimitHeadersMiddleware
{
    private readonly RequestDelegate _next;
    private readonly RateLimitOptions _rateLimitOptions;
    private readonly IpRateLimitProcessor _ipRateLimitProcessor;
    private readonly IpRateLimitMiddleware _ipRateLimitMiddleware;

    public IpRateLimitHeadersMiddleware(RequestDelegate next,
        IRateLimitHeadersOnlyProcessingStrategy processingStrategy, IOptions<IpRateLimitOptions> options, IIpPolicyStore policyStore,
        IRateLimitConfiguration config, ILogger<IpRateLimitMiddleware> logger)
    {
        _next = next;
        _rateLimitOptions = options?.Value;
        _ipRateLimitProcessor = new IpRateLimitProcessor(options?.Value, policyStore, processingStrategy);
        _ipRateLimitMiddleware = new IpRateLimitMiddleware(next, processingStrategy, options, policyStore, config, logger);
    }

    public async Task Invoke(HttpContext context)
    {
        ClientRequestIdentity identity = await _ipRateLimitMiddleware.ResolveIdentityAsync(context);

        if (!_ipRateLimitProcessor.IsWhitelisted(identity))
        {
            var rateLimitRulesWithCounters = new Dictionary<RateLimitRule, RateLimitCounter>();

            foreach (var rateLimitRule in await _ipRateLimitProcessor.GetMatchingRulesAsync(identity, context.RequestAborted))
            {
                rateLimitRulesWithCounters.Add(
                    rateLimitRule,
                    await _ipRateLimitProcessor.ProcessRequestAsync(identity, rateLimitRule, context.RequestAborted)
                 );
            }
        }

        await _next.Invoke(context);

        return;
    }
}

The rateLimitRulesWithCounters contains all the rules applying to the endpoint in the context of the current request. This can be used to calculate the rate limit headers values.

internal class IpRateLimitHeadersMiddleware
{
    private class RateLimitHeadersState
    {
        public HttpContext Context { get; set; }

        public int Limit { get; set; }

        public int Remaining { get; set; }

        public int Reset { get; set; }

        public string Policy { get; set; } = String.Empty;

        public RateLimitHeadersState(HttpContext context)
        {
            Context = context;
        }
    }

    ...

    public async Task Invoke(HttpContext context)
    {
        ...
    }

    private RateLimitHeadersState PrepareRateLimitHeaders(HttpContext context, Dictionary<RateLimitRule, RateLimitCounter> rateLimitRulesWithCounters)
    {
        RateLimitHeadersState rateLimitHeadersState = new RateLimitHeadersState(context);

        var rateLimitHeadersRuleWithCounter = rateLimitRulesWithCounters.OrderByDescending(x => x.Key.PeriodTimespan).FirstOrDefault();
        var rateLimitHeadersRule = rateLimitHeadersRuleWithCounter.Key;
        var rateLimitHeadersCounter = rateLimitHeadersRuleWithCounter.Value;

        rateLimitHeadersState.Limit = (int)rateLimitHeadersRule.Limit;

        rateLimitHeadersState.Remaining = rateLimitHeadersState.Limit - (int)rateLimitHeadersCounter.Count;

        rateLimitHeadersState.Reset = (int)(
            (rateLimitHeadersCounter.Timestamp+ (rateLimitHeadersRule.PeriodTimespan ?? rateLimitHeadersRule.Period.ToTimeSpan())) - DateTime.UtcNow
            ).TotalSeconds;

        rateLimitHeadersState.Policy = String.Join(
            ", ",
            rateLimitRulesWithCounters.Keys.Select(rateLimitRule =>
                $"{(int)rateLimitRule.Limit};w={(int)(rateLimitRule.PeriodTimespan ?? rateLimitRule.Period.ToTimeSpan()
            ).TotalSeconds}")
        );

        return rateLimitHeadersState;
    }
}

The only thing that remains is setting the headers on the response.

internal class IpRateLimitHeadersMiddleware
{
    ...

    public async Task Invoke(HttpContext context)
    {
        ...

        if (!_ipRateLimitProcessor.IsWhitelisted(identity))
        {
            ...

            if (rateLimitRulesWithCounters.Any() && !_rateLimitOptions.DisableRateLimitHeaders)
            {
                context.Response.OnStarting(
                    SetRateLimitHeaders,
                    state: PrepareRateLimitHeaders(context, rateLimitRulesWithCounters)
                );
            }
        }

        ...
    }

    ...

    private Task SetRateLimitHeaders(object state)
    {
        var rateLimitHeadersState = (RateLimitHeadersState)state;

        rateLimitHeadersState.Context.Response.Headers["RateLimit-Limit"] = rateLimitHeadersState.Limit.ToString(CultureInfo.InvariantCulture);
        rateLimitHeadersState.Context.Response.Headers["RateLimit-Remaining"] = rateLimitHeadersState.Remaining.ToString(CultureInfo.InvariantCulture);
        rateLimitHeadersState.Context.Response.Headers["RateLimit-Reset"] = rateLimitHeadersState.Reset.ToString(CultureInfo.InvariantCulture);
        rateLimitHeadersState.Context.Response.Headers["RateLimit-Policy"] = rateLimitHeadersState.Policy;

        return Task.CompletedTask;
    }
}

After registering the RateLimitHeadersOnlyProcessingStrategy and IpRateLimitHeadersMiddleware (I've registered it after the IpRateLimitMiddleware) response will contain values similar to the following ones.

RateLimit-Limit: 5
RateLimit-Remaining: 4
RateLimit-Reset: 9
RateLimit-Policy: 5;w=10
X-Rate-Limit-Limit: 10s
X-Rate-Limit-Remaining: 4
X-Rate-Limit-Reset: 2022-07-25T20:57:32.0746592Z

The code works but certainly isn't perfect, so I've created an issue in hope that AspNetCoreRateLimit will get those headers built in.

Limiting the Number of Outbound Requests in HttpClient

The general rule around rate limit headers is that they should be treated as informative, so the client doesn't have to do anything specific with them. They are also described as generated at response time without any guarantee of consistency between requests. This makes perfect sense. In the simple examples above, multiple clients would be competing for the same quota so received headers values don't exactly tell how many requests a specific client can make within a given window. But, real-life scenarios are usually more specific and complex. It's very common for quotas to be per client or per IP address (this is why AspNetCoreRateLimit has concepts like request identity as a first-class citizen, the ASP.NET Core built-in middleware also enables sophisticated scenarios by using PartitionedRateLimiter at its core). In a such scenario, the client might want to use rate limit headers to avoid making requests which have a high likelihood of being throttled. Let's explore that, below is a simple code that can handle 429 (Too Many Request) responses and utilize the Retry-After header.

HttpClient client = new();
client.BaseAddress = new("http://localhost:5262");

while (true)
{
    Console.Write("{0:hh:mm:ss}: ", DateTime.UtcNow);

    int nextRequestDelay = 1;

    try
    {
        HttpResponseMessage response = await client.GetAsync("/");
        if (response.IsSuccessStatusCode)
        {
            Console.WriteLine(await response.Content.ReadAsStringAsync());
        }
        else
        {
            Console.Write($"{(int)response.StatusCode}: {await response.Content.ReadAsStringAsync()}");

            string? retryAfter = response.Headers.GetValues("Retry-After").FirstOrDefault();
            if (Int32.TryParse(retryAfter, out nextRequestDelay))
            {
                Console.Write($" | Retry-After: {nextRequestDelay}");
            }

            Console.WriteLine();
        }
    }
    catch (Exception ex)
    {
        Console.WriteLine(ex.Message);
    }

    await Task.Delay(TimeSpan.FromSeconds(nextRequestDelay));
}

Let's assume that the service is sending all rate limit headers and that they are dedicated to the client. We can rate limit the HttpClient by creating a DelegatingHandler which will read the RateLimit-Policy header value and instantiate a FixedWindowRateLimiter based on it. The FixedWindowRateLimiter will be used to rate limit the outbound requests - if a lease can't be acquired a locally created HttpResponseMessage will be returned.

internal class RateLimitPolicyHandler : DelegatingHandler
{
    private string? _rateLimitPolicy;
    private RateLimiter? _rateLimiter;

    private static readonly Regex RATE_LIMIT_POLICY_REGEX = new Regex(@"(\d+);w=(\d+)", RegexOptions.Compiled);

    public RateLimitPolicyHandler() : base(new HttpClientHandler())
    { }

    protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
    {
        if (_rateLimiter is not null)
        {
            using var rateLimitLease = await _rateLimiter.WaitAsync(1, cancellationToken);
            if (rateLimitLease.IsAcquired)
            {
                return await base.SendAsync(request, cancellationToken);
            }

            var rateLimitResponse = new HttpResponseMessage(HttpStatusCode.TooManyRequests);
            rateLimitResponse.Content = new StringContent($"Service rate limit policy ({_rateLimitPolicy}) exceeded!");

            if (rateLimitLease.TryGetMetadata(MetadataName.RetryAfter, out var retryAfter))
            {
                rateLimitResponse.Headers.Add("Retry-After", ((int)retryAfter.TotalSeconds).ToString(NumberFormatInfo.InvariantInfo));
            }

            return rateLimitResponse;
        }

        var response = await base.SendAsync(request, cancellationToken);

        if (response.Headers.Contains("RateLimit-Policy"))
        {
            _rateLimitPolicy = response.Headers.GetValues("RateLimit-Policy").FirstOrDefault();

            if (_rateLimitPolicy is not null)
            {
                Match rateLimitPolicyMatch = RATE_LIMIT_POLICY_REGEX.Match(_rateLimitPolicy);

                if (rateLimitPolicyMatch.Success)
                {
                    int limit = Int32.Parse(rateLimitPolicyMatch.Groups[1].Value);
                    int window = Int32.Parse(rateLimitPolicyMatch.Groups[2].Value);

                    _rateLimiter = new FixedWindowRateLimiter(new FixedWindowRateLimiterOptions(
                        limit,
                        QueueProcessingOrder.NewestFirst,
                        0,
                        TimeSpan.FromSeconds(window),
                        true
                    ));

                    string? rateLimitRemaining = response.Headers.GetValues("RateLimit-Remaining").FirstOrDefault();
                    if (Int32.TryParse(rateLimitRemaining, out int remaining))
                    {
                        using var rateLimitLease = await _rateLimiter.WaitAsync(limit - remaining, cancellationToken);
                    }
                }
            }
        }

        return response;
    }
}

The above code also uses the RateLimit-Remaining header value to acquire leases for requests which are no longer available in the initial window.

Now depending if the sample code is run with the RateLimitPolicyHandler in the HttpClient pipeline or not, the console output will be different as those 429 will be coming from a different place.

Opinions

The rate limit headers seem like an interesting addition for communicating services usage quotas. Properly used in the right situations might be a useful tool, it is just important not to treat them as guarantees.

Serving rate limit headers from ASP.NET Core right now has its challenges. If they will become a standard and gain popularity, I think this will change.

If you want to play with the samples, they are available on GitHub.

Micro Frontends in Action With ASP.NET Core - Composition via YARP Transforms and Server-Side Includes (SSI)

noreply@blogger.com (Tomasz Pęczek) — Tue, 05 Jul 2022 14:46:00 +0000

I'm continuing my series on implementing the samples from Micro Frontends in Action in ASP.NET Core:

Server-Side Routing via YARP in Azure Container Apps
Composition via YARP Transforms and Server-Side Includes (SSI)
Composition via Blazor WebAssembly Based Web Components
Communication Patterns for Blazor WebAssembly Based Web Components
Universal Rendering With Blazor WebAssembly Based Web Components

In the previous post, I described how I've deployed the two services which provide fragments of the frontend to the single Container Apps environment and then hidden them behind a YARP-based proxy. This technique (called server-side routing) solves some problems related to bad user experience (multiple domains), browser security (CORS), or search engines. That said, there are some other problems on the table.

The first common problem is performance. Server-side routing does improve performance over solution where fragments are loaded directly from different domains by removing multiple DNS lookups, SSL handshakes, etc. Still, separated AJAX calls can have a very negative impact on overall page load time, especially on slower connections.

The second common problem is layout shifts. When fragments are being loaded, it can often cause already visible page content to "jump". This is frustrating for the end-users.

A solution to both of these problems can be providing a complete page to the browser in a response to the first request.

Server-Side Composition

Server-side composition is a technique, where the page is being fully assembled (which means requesting all the required fragments) on the server. The composition can be done either by a central service (a proxy) or can be done in a decentralized manner, where every service requests fragments it requires to build its own UI. As the current solution already has a proxy in place, I've decided to start with a centralized approach. There are two possible mechanisms discussed in the book for this purpose: Server-Side Includes (SSI) and Edge-Side Includes (ESI).

Server-Side Includes (SSI)

Server-Side Includes is a quite old mechanism, it dates back to the NCSA HTTPd web server. It defines a set of directives (called commands or elements in some implementations) that can be placed in HTML and evaluated by the server while the page is being served. Currently, SSI is supported by Apache, Nginx, and IIS. The subset of supported directives varies between implementations, but the most useful and always available one is include, which the server replaces with a file or result of a request. All that a service needs to do, is put that directive as part of the returned markup.

<html>
  ...
  <body class="decide_layout">
    ...
    <aside class="decide_recos">
      <!--#include virtual="/inspire/fragment/recommendations/porsche" -->
    </aside>
  </body>
</html>

All the magic needs to happen at the proxy level, the only question is how?

Supporting SSI in YARP With Response Body Transform

Every well-respected reverse proxy provides more capabilities than just routing and YARP is no different. One of the capabilities provided by YARP, which goes beyond routing, is transforms. Transforms allow for modifying parts of the request or response as part of the flow. Currently, there are three categories of transforms:

Request
Response
Response Trailers

In every one of those categories, YARP provides a number of built-in transforms which allow for modifying path, query string, client certificates, and headers. There are no built-in transforms for request and response body, which probably makes sense as transforms including request and response body are slightly tricky and potentially dangerous. The first tricky part is that direct forwarding ignores any modifications to the response body which transforms could make, so the proxy service needs to be switched to a "full" reverse proxy experience.

app.Run();

var builder = WebApplication.CreateBuilder(args);

...

builder.Services.AddReverseProxy();

var app = builder.Build();

app.MapReverseProxy();

app.Run();

Moving away from direct forwarding means that there is no longer a way to define path prefixes directly. The out-of-the-box approach for configuring the reverse proxy requires is through configuration files. This would mean that I would have to change the way in which the deployment workflow provides the proxy with URLs to other services. That's something I really didn't want to do. Luckily, there is a possibility of implementing configuration providers to load the configuration programmatically from any source. The documentation even contains an example of an in-memory configuration provider which I've basically copy-pasted (looking at the YARP backlog it seems that the team has noticed its usefulness and it will be available out-of-the-box as well). This allowed me to keep the routes and clusters (this is how destinations are represented in YARP) configuration in the code.

...

var routes = new[]
{
    ...
    new RouteConfig
    {
        RouteId = Constants.DECIDE_ROUTE_ID,
        ClusterId = Constants.DECIDE_CLUSTER_ID,
        Match = new RouteMatch { Path = "/decide/{**catch-all}" }
    },
    ...
};

var clusters = new[]
{
    new ClusterConfig()
    {
        ClusterId = Constants.DECIDE_CLUSTER_ID,
        Destinations = new Dictionary<string, DestinationConfig>(StringComparer.OrdinalIgnoreCase)
        {
            { Constants.DECIDE_SERVICE_URL, new DestinationConfig() { Address = builder.Configuration[Constants.DECIDE_SERVICE_URL] } }
        }
    },
    ...
};

builder.Services.AddReverseProxy()
    .LoadFromMemory(routes, clusters);

...

With the configuration in place, it's time for the transform. There are two main ways for adding transforms. One is a callback and the other is ITransformProvider implementation. The ITransformProvider implementation gives more flexibility and isolation, so I've decided to go with it. As the implementation will be a registered dependency, it gives full access to dependency injection. It also gives validation capabilities for routes and clusters.

The simplest ("no-op") implementation of ITransformProvider can look like below.

internal class SsiTransformProvider : ITransformProvider
{
    public void ValidateRoute(TransformRouteValidationContext context)
    { }

    public void ValidateCluster(TransformClusterValidationContext context)
    { }

    public void Apply(TransformBuilderContext transformBuildContext)
    {
        transformBuildContext.AddResponseTransform(TransformResponse);
    }

    private ValueTask TransformResponse(ResponseTransformContext responseContext)
    {
        return default;
    }
}

In order to register that ITransformProvider implementation (and make TranformResponse part of the flow) it is enough to call AddTransforms.

...

builder.Services.AddReverseProxy()
    .LoadFromMemory(routes, clusters)
    .AddTransforms<SsiTransformProvider>();

...

This is where it is important to understand further specifics of transforms that are working with the request or response body. As a result of the Apply method from the above implementation, the TranformResponse will be executed for every flow going through YARP. This is too broad because if that transform deals with request or response body, it will come with a performance penalty. It will have to read (essentially buffer) the response from the destination. The moment the body has been read, it will also have to be written to the HttpContext of the YARP response, otherwise the response will be empty. This is happening because YARP doesn't buffer the response as part of the flow (due to performance reasons), instead it attempts to read the stream which in this case is at the end.

The performance penalty means that there is a need to limit the scope of impact by adding the transform only to certain routes. The routes which should be transformed need to be somehow marked. For that purpose I've decided to include additional information in the routes through metadata. Metadata is a dictionary available on RouteConfig. I've defined a specific key and value for which the Apply method will check before adding the transform. I've also added a statically available dictionary which can be used to set the metadata.

internal class SsiTransformProvider : ITransformProvider
{
    private const string SSI_METADATA_FLAG = "SSI";
    private const string SSI_METADATA_FLAG_ON = "ON";

    public static IReadOnlyDictionary<string, string> SsiEnabledMetadata { get; } = new Dictionary<string, string>()
    {
        { SSI_METADATA_FLAG, SSI_METADATA_FLAG_ON }
    };

    ...

    public void Apply(TransformBuilderContext transformBuildContext)
    {
        if (transformBuildContext.Route.Metadata is not null
            && transformBuildContext.Route.Metadata.ContainsKey(SSI_METADATA_FLAG)
            && transformBuildContext.Route.Metadata[SSI_METADATA_FLAG] == SSI_METADATA_FLAG_ON)
        {
            transformBuildContext.AddResponseTransform(TransformResponse);
        }
    }

    ...
}

Thanks to the in-memory configuration provider, including those metadata mean just setting one more property. While doing this I've also increased the granularity of routes to further limit the affected scope.

...

var routes = new[]
{
    ...
    new RouteConfig
    {
        RouteId = Constants.DECIDE_ROUTE_ID + "-static",
        ClusterId = Constants.DECIDE_CLUSTER_ID,
        Match = new RouteMatch { Path = "/decide/static/{**catch-all}" }
    },
    new RouteConfig
    {
        RouteId = Constants.DECIDE_ROUTE_ID,
        ClusterId = Constants.DECIDE_CLUSTER_ID,
        Match = new RouteMatch { Path = "/decide/{**catch-all}" },
        Metadata = SsiTransformProvider.SsiEnabledMetadata
    },
    ...
};

...

Now it's time for the actual transformation. To focus on the logic needed to support SSI include directive, let's get the response body reading and writing out of the way.

internal class SsiTransformProvider : ITransformProvider
{
    ...

    private ValueTask TransformResponse(ResponseTransformContext responseContext)
    {

        string proxyResponseContent = await responseContext.ProxyResponse.Content.ReadAsStringAsync();

        responseContext.SuppressResponseBody = true;

        ...

        byte[] proxyResponseContentBytes = Encoding.UTF8.GetBytes(proxyResponseContent);
        responseContext.HttpContext.Response.ContentLength = proxyResponseContentBytes.Length;
        await responseContext.HttpContext.Response.Body.WriteAsync(proxyResponseContentBytes);
    }
}

In order to get the include directive from the response body, I've decided to use (I don't believe I'm saying this) regex. The snippet below will grab all the directives. The group with index one of the resulting match will contain the directive name (all others than include are to be ignored) while the group with index two will contain parameters for further parsing (I'm doing this with another regex, but ultimately I care only for virtual parameter).

internal class SsiTransformProvider : ITransformProvider
{
    private static readonly Regex SSI_DIRECTIVE_REGEX = new Regex(
        @"<!--\#([a-z]+)\b([^>]+[^\/>])?-->",
        RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.IgnorePatternWhitespace
    );

    ...

    private ValueTask TransformResponse(ResponseTransformContext responseContext)
    {

        ...

        var directives = SSI_DIRECTIVE_REGEX.Matches(proxyResponseContent)

        ...
    }
}

To get the content to which the include directive is pointing, I will need to make an HTTP request to a specific service. The configuration provided for YARP already has an URL for that service. YARP also maintains a HttpClient instance dedicated to the cluster to which the service belongs. It would be nice to reuse it. In order to do that, first I needed to identify the endpoint for the path to which the virtual parameter is pointing.

Endpoint? virtualEndpoint = null;

var endpointDataSource = context.RequestServices.GetService<EndpointDataSource>();
if (endpointDataSource is not null)
{
    var virtualPath = new PathString(directive.Parameters[VIRTUAL_PARAMETER]);
    foreach (Endpoint possibleVirtualEndpoint in endpointDataSource.Endpoints)
    {
        var routeEndpoint = possibleVirtualEndpoint as RouteEndpoint;
        if (routeEndpoint is not null)
        {
            var routeTemplateMatcher = new TemplateMatcher(new RouteTemplate(routeEndpoint.RoutePattern), _emptyRouteValueDictionary);
            if (routeTemplateMatcher.TryMatch(virtualPath, _emptyRouteValueDictionary))
            {
                virtualEndpoint = possibleVirtualEndpoint;
                break;
            }
        }
    }
}

The endpoint also has a metadata collection. In this collection, YARP keeps the route model, which includes the cluster model.

ClusterModel? cluster = null;

foreach (var endpointMetadata in virtualEndpoint.Metadata)
{
    var proxyRoute = endpointMetadata as RouteModel;
    if (proxyRoute is not null)
    {
        cluster = proxyRoute.Cluster?.Model;
        break;
    }
}

The cluster model contains the configured destinations (with URLs) and that mentioned HttpClient instance. All that remains is to build the request URI, make the request, and read the content.

string virtualUri = cluster.Config.Destinations.FirstOrDefault().Value.Address + parameters["virtual"];

HttpResponseMessage response = await cluster.HttpClient.SendAsync(new HttpRequestMessage(HttpMethod.Get, virtualUri), CancellationToken.None);

string directiveOutputContent = await response.Content.ReadAsStringAsync();

Once the content has been read, the directive can be replaced in the body.

proxyResponseContent = proxyResponseContent.Substring(0, directives[directiveIndex].Index)
    + directiveOutputContent
    + proxyResponseContent.Substring(directives[directiveIndex].Index + directives[directiveIndex].Length);

This Is Just a Proof of Concept

Yes, this code (even the little more polished version available in the repository) is just a POC. It's missing constraints, error checking, support for multiple destinations in a cluster, support for other parameters of the include directive, and much more.

There should be also further performance considerations. The approach that the sample code takes (buffer the body, request content for include directives in parallel, and then build the final response body) is typical for SSI, but an approach that streams the body whenever possible could be considered. This is for example how ESI (which is a more modern mechanism) is sometimes implemented.

The only goal of this post (and related sample) is to show some YARP capabilities which can be used for achieving server-side composition at its level.

Micro Frontends in Action With ASP.NET Core - Server-Side Routing via YARP in Azure Container Apps

noreply@blogger.com (Tomasz Pęczek) — Tue, 28 Jun 2022 13:35:00 +0000

Recently, I've been reading Micro Frontends in Action and while doing it I've decided that I want to implement the samples in ASP.NET Core and host them on Azure. Then I had a thought, that maybe I can use this as an opportunity to play with some technologies I didn't have a chance to play with yet, like YARP or Azure Container Apps. Once I did that, the next thought was that maybe I should write down some of this and maybe someone will find it useful. So here we are, possibly starting a new series around micro frontends techniques:

Server-Side Routing via YARP in Azure Container Apps
Composition via YARP Transforms and Server-Side Includes (SSI)
Composition via Blazor WebAssembly Based Web Components
Communication Patterns for Blazor WebAssembly Based Web Components
Universal Rendering With Blazor WebAssembly Based Web Components

Micro Frontends in Action

One of the praises for Micro Frontends in Action says that it's an excellent starting point to understanding how to introduce micro frontends in your projects. I'm about one-third through it and I'm willing to agree with that statement. The book starts with very simple techniques like page transition via links, composition via iframe, and composition via Ajax. Then it moves to server-side composition, advanced techniques for client-side composition, communication patterns, and more (I don't know, I haven't gotten there yet). Up to this point, it has been interesting and engaging (this is my opinion and just to be clear - nobody is paying me to read the book and provide an opinion about it).

The book contains samples for every discussed technique and it was just too tempting not to implement them with technologies I like. That doesn't mean I will implement every single one of those samples. I will also certainly not describe every single one of those implementations. I'm doing it for fun.

I also recommend you read the book if you are interested in the subject - a blog post describing some specific implementation will not provide you with information about benefits, drawbacks, and when to use a certain technique.

The Landscape So Far

The technique I'm going to describe in this post is server-side routing. Prior to introducing that technique, the project consists of two services: Decide and Inspire. The Decide service is loading frontend fragments provided by Inspire service via Ajax request.

Under the hood, both services are simple ASP.NET Core MVC applications that serve views based on static HTML from original samples, where the URLs are generated based on configuration.

@using Microsoft.Extensions.Configuration
@inject IConfiguration Configuration
@{string decideServiceUrl = Configuration["DECIDE_SERVICE_URL"];}
<link href="@(Context.Request.Scheme + "://" + Context.Request.Host + "/static/fragment.css")" rel="stylesheet" />
<div class="inspire_fragment">
  <h2 class="inspire_headline">Recommendations</h2>
  <div class="inspire_recommendations">
    <a href="@(decideServiceUrl + "/product/fendt")"><img src="https://mi-fr.org/img/fendt.svg" /></a>
    <a href="@(decideServiceUrl + "/product/eicher")"><img src="https://mi-fr.org/img/eicher.svg" /></a>
  </div>
</div>

The Inspire service additionally defines a CORS policy to enable requesting the fragments from different domain via Ajax.

var builder = WebApplication.CreateBuilder(args);

string decideServiceCorsPolicyName = "decide-service-cors-policy";

builder.Services.AddCors(options =>
{
    options.AddPolicy(name: decideServiceCorsPolicyName, policy =>
    {
        policy.WithOrigins(builder.Configuration["DECIDE_SERVICE_URL"]);
        policy.AllowAnyHeader();
        policy.WithMethods("GET");
    });
});

...

app.UseRouting();

app.UseCors(decideServiceCorsPolicyName);

...

Both services are containerized and deployed as two Container Apps within a single Container Apps environment.

az containerapp create \
  -n ${DECIDE_CONTAINERAPP} \
  -i ${CONTAINER_REGISTRY}.azurecr.io/decide:latest \
  -g ${RESOURCE_GROUP} \
  --environment ${CONTAINERAPPS_ENVIRONMENT} \
  --ingress external \
  --target-port 3001 \
  --min-replicas 1 \
  --registry-server ${CONTAINER_REGISTRY}.azurecr.io

az containerapp create \
  -n ${INSPIRE_CONTAINERAPP} \
  -i ${CONTAINER_REGISTRY}.azurecr.io/inspire:latest \
  -g ${RESOURCE_GROUP} \
  --environment ${CONTAINERAPPS_ENVIRONMENT} \
  --ingress external \
  --target-port 3002 \
  --min-replicas 1 \
  --registry-server ${CONTAINER_REGISTRY}.azurecr.io

This results in the following Container Apps solution.

I've created a GitHub Actions workflow that performs all the necessary steps (creating Azure resources, building and pushing containers images, and deploying Container Apps) to set up the entire solution from scratch. There is one tricky step in that workflow. Both services must know each other URLs but those are available only after Container Apps are created. To solve this I'm first getting the ingress information for every app (with az containerapp ingress show) and then write them to environment variables (using the multiline strings approach).

jobs:
  ..
  deploy-to-container-apps:
  ..
  steps:
    ..
    - name: Get Services Ingress
      run: |
        echo 'DECIDE_CONTAINERAPP_INGRESS_JSON<<EOF' >> $GITHUB_ENV
        az containerapp ingress show -n ${DECIDE_CONTAINERAPP} -g ${RESOURCE_GROUP} >> $GITHUB_ENV
        echo 'EOF' >> $GITHUB_ENV
        echo 'INSPIRE_CONTAINERAPP_INGRESS_JSON<<EOF' >> $GITHUB_ENV
        az containerapp ingress show -n ${INSPIRE_CONTAINERAPP} -g ${RESOURCE_GROUP} >> $GITHUB_ENV
        echo 'EOF' >> $GITHUB_ENV
    ..

Next, I'm using the fromJson expression to get FQDN from ingress information and update the Container Apps.

jobs:
  ..
  deploy-to-container-apps:
  ..
  steps:
    ..
    - name: Configure Services URLs
      run: |
        az containerapp update -n ${DECIDE_CONTAINERAPP} -g ${RESOURCE_GROUP} --set-env-vars INSPIRE_SERVICE_URL=https://${{ fromJSON(env.INSPIRE_CONTAINERAPP_INGRESS_JSON).fqdn }}
        az containerapp update -n ${INSPIRE_CONTAINERAPP} -g ${RESOURCE_GROUP} --set-env-vars DECIDE_SERVICE_URL=https://${{ fromJSON(env.DECIDE_CONTAINERAPP_INGRESS_JSON).fqdn }}
    ..

The Challenge

There is a problem with this solution. There are two services and each of them is available to public requests under a different domain. This has several drawbacks:

Bad user experience (requests flying to different domains).
Internal infrastructure of the solution is exposed publicly.
Performance (two DNS lookups, two SSL handshakes, etc.).
Indexing by search engines.

To solve this a central service (a proxy), where all requests will arrive, is needed.

Introducing YARP as a Solution

For about a year, the ASP.NET Core stack have its own reverse proxy - YARP. It provides a lot of routing features like headers-based routing, session affinity, load balancing, or destination health checks. In this scenario I'm going to use direct forwarding to simply forward requests to specific services based on a path. I've also decided to set up the rules from code instead of using configuration (it seemed simpler as I still could provide service URLs through environment variables). For this purpose, I've created a MapForwarder extension method.

public static void MapForwarder(this IEndpointRouteBuilder endpoints, string pattern, string serviceUrl)
{
    var forwarder = endpoints.ServiceProvider.GetRequiredService<IHttpForwarder>();
    var requestConfig = new ForwarderRequestConfig { ActivityTimeout = TimeSpan.FromMilliseconds(500) };

    endpoints.Map(pattern, async httpContext =>
    {
        var error = await forwarder.SendAsync(httpContext, serviceUrl, HttpClient, requestConfig, HttpTransformer.Default);

        if (error != ForwarderError.None)
        {
            var errorFeature = httpContext.GetForwarderErrorFeature();
            var exception = errorFeature?.Exception;
        }
    });
}

I've followed the approach of defining service-specific path prefixes as well as specific prefixes for important pages.

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddHttpForwarder();

var app = builder.Build();

string decideServiceUrl = app.Configuration["DECIDE_SERVICE_URL"];
string inspireServiceUrl = app.Configuration["INSPIRE_SERVICE_URL"];

app.UseRouting();
app.UseEndpoints(endpoints =>
{
    // Per service prefixes
    endpoints.MapForwarder("/", decideServiceUrl);

    // Per service prefixes
    endpoints.MapForwarder("/decide/{**catch-all}", decideServiceUrl);
    endpoints.MapForwarder("/inspire/{**catch-all}", inspireServiceUrl);

    // Per page prefixes
    endpoints.MapForwarder("/product/{**catch-all}", decideServiceUrl);
    endpoints.MapForwarder("/recommendations/{**catch-all}", inspireServiceUrl);
});

app.Run();

As it will be now the proxy responsibility to know the addresses of both services, they no longer need to be able to point to each other. All the paths can now be relative, as long as they follow the established rules.

<link href="/inspire/static/fragment.css" rel="stylesheet" />
<div class="inspire_fragment">
  <h2 class="inspire_headline">Recommendations</h2>
  <div class="inspire_recommendations">
    <a href="/product/fendt"><img src="https://mi-fr.org/img/fendt.svg" /></a>
    <a href="/product/eicher"><img src="https://mi-fr.org/img/eicher.svg" /></a>
  </div>
</div>

There is also no need for CORS anymore, as from a browser perspective there are no cross-origin requests. So I've removed that code from the Inspire service.

Now it's time to hide the services from the public. For starters let's change their ingress to internal.

az containerapp create \
  -n ${DECIDE_CONTAINERAPP} \
  -i ${CONTAINER_REGISTRY}.azurecr.io/decide:latest \
  -g ${RESOURCE_GROUP} \
  --environment ${CONTAINERAPPS_ENVIRONMENT} \
  --ingress internal \
  --target-port 3001 \
  --min-replicas 1 \
  --registry-server ${CONTAINER_REGISTRY}.azurecr.io

az containerapp create \
  -n ${INSPIRE_CONTAINERAPP} \
  -i ${CONTAINER_REGISTRY}.azurecr.io/inspire:latest \
  -g ${RESOURCE_GROUP} \
  --environment ${CONTAINERAPPS_ENVIRONMENT} \
  --ingress internal \
  --target-port 3002 \
  --min-replicas 1 \
  --registry-server ${CONTAINER_REGISTRY}.azurecr.io

This way the services are now accessible only from within the Container Apps environment. The proxy can be deployed to the same Container Apps environment and expose the desired traffic outside.

az containerapp create \
  -n ${PROXY_CONTAINERAPP} \
  -i ${CONTAINER_REGISTRY}.azurecr.io/proxy:latest \
  -g ${RESOURCE_GROUP} \
  --environment ${CONTAINERAPPS_ENVIRONMENT} \
  --ingress external \
  --target-port 3000 \
  --min-replicas 1 \
  --registry-server ${CONTAINER_REGISTRY}.azurecr.io \
  --env-vars INSPIRE_SERVICE_URL=https://${INSPIRE_CONTAINERAPP_FQDN} DECIDE_SERVICE_URL=https://${DECIDE_CONTAINERAPP_FQDN}

After this change, the Container Apps solution looks like in the below diagram.

That's All Folks

At least for this post. You can find the final solution here and its GitHub Actions workflow here. The services have become a little bit strange in the process (they server static HTML in an unnecessarily complicated way) but they are not the focus here, the server-side routing technique is.

I don't know if or when I'm going to play with another technique, but if I will it's probably going to be something around server-side composition.

ASP.NET Core 6 and IAsyncEnumerable - Receiving Async Streamed JSON in Blazor WebAssembly

noreply@blogger.com (Tomasz Pęczek) — Mon, 06 Dec 2021 14:00:00 +0000

Back in July, I've shared my experiments around new JSON async streaming capabilities in ASP.NET Core 6. Last week I've received a question about utilizing these capabilities in the Blazor WebAssembly application. The person asking the question has adopted the DeserializeAsyncEnumerable based client code, but it didn't seem to work properly. All the results were always displayed at once. As I didn't have a Blazor WebAssembly sample as part of my streaming JSON objects demo, I've decided I'll add one and figure out the answer to the question along the way.

Blazor WebAssembly and IAsyncEnumerable

Before I focus on the problem of results not being received in an async stream manner, I think it is worth discussing the way of working with IAsyncEnumerable in Blazor WebAssembly. What's the challenge here? The first one is that await foreach can't be used in the page markup, only in the code block. So the markup must use a synchronous loop.

<table>
    <thead>
        <tr>
            <th>Date</th>
            <th>Temp. (C)</th>
            <th>Temp. (F)</th>
            <th>Summary</th>
        </tr>
    </thead>
    <tbody>
        @foreach (WeatherForecast weatherForecast in weatherForecasts)
        {
            <tr>
                <td>@weatherForecast.DateFormatted</td>
                <td>@weatherForecast.TemperatureC</td>
                <td>@weatherForecast.TemperatureF</td>
                <td>@weatherForecast.Summary</td>
            </tr>
        }
    </tbody>
</table>

That brings us to the second challenge. If the await foreach can be used only in the code block, how the streamed results can be rendered as they come? Here the solution comes in form of the StateHasChanged method. Calling this method will trigger a render. With this knowledge, we can adopt the DeserializeAsyncEnumerable based code from my previous post.

@code {

    private List<WeatherForecast> weatherForecasts = new List<WeatherForecast>();

    private async Task StreamWeatherForecastsJson()
    {
        weatherForecasts = new List<WeatherForecast>();

        StateHasChanged();

        using HttpResponseMessage response = await Http.GetAsync("api/WeatherForecasts/negotiate-stream", HttpCompletionOption.ResponseHeadersRead);

        response.EnsureSuccessStatusCode();

        using Stream responseStream = await response.Content.ReadAsStreamAsync();

        await foreach (WeatherForecast weatherForecast in JsonSerializer.DeserializeAsyncEnumerable<WeatherForecast>(
            responseStream,
            new JsonSerializerOptions
            {
                PropertyNameCaseInsensitive = true,
                DefaultBufferSize = 128
            }))
        {
            weatherForecasts.Add(weatherForecast);

            StateHasChanged();
        }
    }
}

Running that code put me in the exact same spot where the person asking the question was. All the results were rendered at once after the entire wait time. What to do, when you have no idea what might be wrong and where? Dump what you can to the console ;). No, I'm serious. Debugging through console.log is in fact quite useful in many situations and I'm not ashamed of using it here. I've decided that the diagnostic version will perform direct response stream reading.

@code {

    ...

    private async Task StreamWeatherForecastsJson()
    {
        Console.WriteLine($"-- {nameof(StreamWeatherForecastsJson)} --");
        Console.WriteLine($"[{DateTime.UtcNow:hh:mm:ss.fff}] Requesting weather forecasts . . .");

        using HttpResponseMessage response = await Http.GetAsync("api/WeatherForecasts/negotiate-stream", HttpCompletionOption.ResponseHeadersRead);

        response.EnsureSuccessStatusCode();

        Console.WriteLine($"[{DateTime.UtcNow:hh:mm:ss.fff}] Receving weather forecasts . . .");

        using Stream responseStream = await response.Content.ReadAsStreamAsync();

        Console.WriteLine($"[{DateTime.UtcNow:hh:mm:ss.fff}] Weather forecasts stream obtained . . .");

        while (true)
        {
            byte[] buffer = ArrayPool<byte>.Shared.Rent(128);
            int bytesRead = await responseStream.ReadAsync(buffer);

            Console.WriteLine($"[{DateTime.UtcNow:hh:mm:ss.fff}] ({bytesRead}/{buffer.Length}) {Encoding.UTF8.GetString(buffer[0..bytesRead])}");

            ArrayPool<byte>.Shared.Return(buffer);

            if (bytesRead == 0)
            {
                break;
            }
        }

        Console.WriteLine($"[{DateTime.UtcNow:hh:mm:ss.fff}] Weather forecasts has been received.");
        Console.WriteLine();
    }
}

Below you can see the output from browser developer tools.

-- StreamWeatherForecastsJson --
[08:04:01.183] Requesting weather forecasts . . .
[08:04:01.436] Receving weather forecasts . . .
[08:04:02.420] Weather forecasts stream obtained . . .
[08:04:02.426] (128/128) [{"dateFormatted":"06.12.2021","temperatureC":28,"temperatureF":82,"summary":"Hot"},{"dateFormatted":"07.12.2021","temperatureC"
[08:04:02.429] (128/128) :36,"temperatureF":96,"summary":"Scorching"},{"dateFormatted":"08.12.2021","temperatureC":-7,"temperatureF":20,"summary":"Mild"}
[08:04:02.431] (128/128) ,{"dateFormatted":"09.12.2021","temperatureC":-6,"temperatureF":22,"summary":"Hot"},{"dateFormatted":"10.12.2021","temperatureC"
[08:04:02.433] (128/128) :40,"temperatureF":103,"summary":"Cool"},{"dateFormatted":"11.12.2021","temperatureC":44,"temperatureF":111,"summary":"Swelterin
[08:04:02.434] (128/128) g"},{"dateFormatted":"12.12.2021","temperatureC":-3,"temperatureF":27,"summary":"Balmy"},{"dateFormatted":"13.12.2021","temperat
[08:04:02.435] (128/128) ureC":1,"temperatureF":33,"summary":"Sweltering"},{"dateFormatted":"14.12.2021","temperatureC":3,"temperatureF":37,"summary":"Ho
[08:04:02.437] (88/128) t"},{"dateFormatted":"15.12.2021","temperatureC":19,"temperatureF":66,"summary":"Mild"}]tureC":3,"temperatureF":37,"summary":"Ho
[08:04:02.438] (0/128) t"},{"dateFormatted":"15.12.2021","temperatureC":19,"temperatureF":66,"summary":"Mild"}]tureC":3,"temperatureF":37,"summary":"Ho
[08:04:02.439] Weather forecasts has been received.

So the call which is blocking the whole thing seems to be ReadAsStreamAsync, when it returns the entire response is already available. All I knew at this point was that Blazor WebAssembly is using a special HttpMessageHandler. I needed to dig deeper.

Digging Into BrowserHttpHandler

There is a number of things that have dedicated implementations for Blazor WebAssembly. The HttpClient stack is one of those things. Well, there is no access to native sockets in the browser, so the HTTP calls must be performed based on browser-provided APIs. The BrowserHttpHandler is implemented on top of Fetch API. Inspecting its code shows that it can provide response content in one of two forms.

The first one is BrowserHttpContent, which is based on arrayBuffer method. This means, that it will always read the response stream to its completion, before making the content available.

The second one is StreamContent wrapping WasmHttpReadStream, which is based on readable streams. This one allows for reading response as it comes.

How does BrowserHttpHandler decide which one to use? In order for WasmHttpReadStream to be used, two conditions must be met - the browser must support readable streams and the WebAssemblyEnableStreamingResponse option must be enabled on the request. Now we are getting somewhere. Further search for WebAssemblyEnableStreamingResponse will reveal a SetBrowserResponseStreamingEnabled extension method. Let's see what happens if it's used.

@code {

    ...

    private async Task StreamWeatherForecastsJson()
    {
        Console.WriteLine($"-- {nameof(StreamWeatherForecastsJson)} --");
        Console.WriteLine($"[{DateTime.UtcNow:hh:mm:ss.fff}] Requesting weather forecasts . . .");

        HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Get, "api/WeatherForecasts/negotiate-stream");
        request.SetBrowserResponseStreamingEnabled(true);

        using HttpResponseMessage response = await Http.SendAsync(request, HttpCompletionOption.ResponseHeadersRead);

        response.EnsureSuccessStatusCode();

        ...
    }
}

This gives the desired output.

-- StreamWeatherForecastsJson --
[08:53:14.722] Requesting weather forecasts . . .
[08:53:15.002] Receving weather forecasts . . .
[08:53:15.009] Weather forecasts stream obtained . . .
[08:53:15.018] (84/128) [{"dateFormatted":"06.12.2021","temperatureC":31,"temperatureF":87,"summary":"Cool"}
[08:53:15.057] (84/128) ,{"dateFormatted":"07.12.2021","temperatureC":18,"temperatureF":64,"summary":"Cool"}
[08:53:15.166] (86/128) ,{"dateFormatted":"08.12.2021","temperatureC":10,"temperatureF":49,"summary":"Chilly"}
[08:53:15.276] (84/128) ,{"dateFormatted":"09.12.2021","temperatureC":33,"temperatureF":91,"summary":"Mild"}
[08:53:15.386] (88/128) ,{"dateFormatted":"10.12.2021","temperatureC":-14,"temperatureF":7,"summary":"Freezing"}
[08:53:15.492] (84/128) ,{"dateFormatted":"11.12.2021","temperatureC":12,"temperatureF":53,"summary":"Warm"}
[08:53:15.600] (86/128) ,{"dateFormatted":"12.12.2021","temperatureC":6,"temperatureF":42,"summary":"Bracing"}
[08:53:15.710] (85/128) ,{"dateFormatted":"13.12.2021","temperatureC":48,"temperatureF":118,"summary":"Mild"}
[08:53:15.818] (89/128) ,{"dateFormatted":"14.12.2021","temperatureC":13,"temperatureF":55,"summary":"Scorching"}
[08:53:15.931] (88/128) ,{"dateFormatted":"15.12.2021","temperatureC":44,"temperatureF":111,"summary":"Chilly"}]
[08:53:15.943] (0/128) 
[08:53:15.946] Weather forecasts has been received.

Corrected Implementation

This means, that in order to have stream-based access to the response body (regardless if it's JSON or something else), one needs to explicitly enable it on the request. So the code which is able to receive async streamed JSON and properly deserialize it to IAsyncEnumerable should look like this.

@code {

    ...

    private async Task StreamWeatherForecastsJson()
    {
        weatherForecasts = new List<WeatherForecast>();

        StateHasChanged();

        HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Get, "api/WeatherForecasts/negotiate-stream");
        request.SetBrowserResponseStreamingEnabled(true);

        using HttpResponseMessage response = await Http.SendAsync(request, HttpCompletionOption.ResponseHeadersRead);

        response.EnsureSuccessStatusCode();

        using Stream responseStream = await response.Content.ReadAsStreamAsync();

        await foreach (WeatherForecast weatherForecast in JsonSerializer.DeserializeAsyncEnumerable<WeatherForecast>(
            responseStream,
            new JsonSerializerOptions
            {
                PropertyNameCaseInsensitive = true,
                DefaultBufferSize = 128
            }))
        {
            weatherForecasts.Add(weatherForecast);

            StateHasChanged();
        }
    }
}

This works exactly as expected - the results are being rendered as they are being returned from the backend (with precision resulting from the size of the buffer).

Lesson Learned

Despite the fact that there is no dedicated TFM for Blazor WebAssembly, and it uses the "runs everywhere" one (net5.0, net6.0, etc.) there are platform differences. You will notice the APIs which aren't supported very quickly because they will throw PlatformNotSupportedException. But there are also more sneaky ones, ones which behavior is different. Blazor WebAssembly needs to be approached with this thought in the back of your mind. This will allow you to properly handle situations when things are not working the way you've expected.

Leveraging Azure Cosmos DB Partial Document Update With JSON Patch in an ASP.NET Core Web API

noreply@blogger.com (Tomasz Pęczek) — Tue, 30 Nov 2021 13:24:00 +0000

A couple of weeks ago the Cosmos DB team has announced support for patching documents. This is quite a useful and long-awaited feature, as up to this point the only way to change the stored document was to completely replace it. This feature opens up new scenarios. For example, if you are providing a Web API on top of Cosmos DB, you can now directly implement support for the PATCH request method. I happen to have a small demo of such Web API, so I've decided to refresh it and play with leveraging this new capability.

Adding PATCH Requests Support to an ASP.NET Core Web API

An important part of handling a PATCH request is deciding on the request body format. The standard approach for that, in the case of JSON-based APIs, is JSON Patch. In fact, Cosmos DB is also using JSON Patch, so it should be easier to leverage it this way.

ASP.NET Core provides support for JSON Patch, but I've decided not to go with it. Why? It's designed around applying operations to an instance of an object and as a result, has internal assumptions about the supported list of operations. In the case of Cosmos DB, the supported operations are different, and that "applying" capability is not needed (it was a great way to implement PATCH when Cosmos DB provided on replace option). I've figured out it will be better to start fresh, with a lightweight model.

public class JsonPatchOperation
{
    [Required]
    public string Op { get; set; }

    [Required]
    public string Path { get; set; }

    public object Value { get; set; }
}

public class JsonPatch : List<JsonPatchOperation>
{ }

This class is quite generic and should allow for handling any request which body is compliant with JSON Patch structure. The first concretization I want to introduce is the list of available operations. Cosmos DB currently supports five operations: Add, Set, Remove, Replace, and Increment. In this demo (at least for now) I'm skipping the Increment because it would require a little bit different handling than others.

public enum JsonPatchOperationType
{
    Add,
    Set,
    Remove,
    Replace,,
    Invalid
}

As you can see, in the above enumeration I've also added the Invalid value. This will give me a way to represent the operations I don't intend to support through a specific value instead of e.g. throwing an exception.

public class JsonPatchOperation
{
    private string _op;
    private JsonPatchOperationType _operationType;

    [Required]
    public string Op
    {
        get { return _op; }

        set
        {
            JsonPatchOperationType operationType;
            if (!Enum.TryParse(value, ignoreCase: true, result: out operationType))
            {
                operationType = JsonPatchOperationType.Invalid;
            }

            _operationType = operationType;

            _op = value;
        }
    }

    public JsonPatchOperationType OperationType => _operationType;

    ...
}

Having not supported operations represented through a specific value also allows me to implement IValidatableObject which checks for them. This way, if the class is used as an action model, making a request with an unsupported operation will trigger a 400 Bad Request response.

public class JsonPatchOperation : IValidatableObject
{
    ...

    public IEnumerable<ValidationResult> Validate(ValidationContext validationContext)
    {
        if (OperationType == JsonPatchOperationType.Invalid)
        {
            yield return new ValidationResult($"Not supported operation: {Op}.", new[] { nameof(Op) });
        }
    }
}

Now all is needed is an action that will support the PATCH method.

[Route("api/[controller]")]
[ApiController]
public class CharactersController : Controller
{
    ...

    [HttpPatch("{id}")]
    public async Task<ActionResult<Character>> Patch(string id, JsonPatch update)
    {
        ...
    }

    ...
}

The next step will be proxying the deserialized JSON Patch to Cosmos DB .NET SDK.

Utilizing Cosmos DB Partial Updates

The Cosmos DB .NET SDK exposes partial document updates through the PatchItemAsync method on a container. This method expects a collection of PatchOperation instances. Instances representing specific operations can be created through static methods on PatchOperation which names correspond to operations names. So the conversion from JsonPatch to PatchOperation collection requires calling the appropriate method for every JsonPatchOperation. This is something a simple extensions method should handle.

public static class JsonPatchExtensions
{
    public static IReadOnlyList<PatchOperation> ToCosmosPatchOperations(this JsonPatch jsonPatchOperations)
    {

        List<PatchOperation> cosmosPatchOperations = new List<PatchOperation>(jsonPatchOperations.Count);
        foreach (JsonPatchOperation jsonPatchOperation in jsonPatchOperations)
        {
            switch (jsonPatchOperation.OperationType)
            {
                case JsonPatchOperationType.Add:
                    cosmosPatchOperations.Add(PatchOperation.Add(jsonPatchOperation.Path, jsonPatchOperation.Value));
                    break;
                case JsonPatchOperationType.Remove:
                    cosmosPatchOperations.Add(PatchOperation.Remove(jsonPatchOperation.Path));
                    break;
                case JsonPatchOperationType.Replace:
                    cosmosPatchOperations.Add(PatchOperation.Replace(jsonPatchOperation.Path, jsonPatchOperation.Value));
                    break;
                case JsonPatchOperationType.Set:
                    System.Int32 test = 25;
                    cosmosPatchOperations.Add(PatchOperation.Set(jsonPatchOperation.Path, jsonPatchOperation.Value));
                    break;
            }
        }

        return cosmosPatchOperations;
    }
}

Making the proper call in our action.

[Route("api/[controller]")]
[ApiController]
public class CharactersController : Controller
{
    ...

    [HttpPatch("{id}")]
    public async Task<ActionResult<Character>> Patch(string id, JsonPatch update)
    {
        ...

        ItemResponse<Character> characterItemResponse = await _starWarsCosmosClient.Characters.PatchItemAsync<Character>(
            id,
            PartitionKey.None,
            update.ToCosmosPatchOperations());

        return characterItemResponse.Resource;
    }

    ...
}

And we have some testable code. In order to test I've decided to attempt two operations: Set on /height and Add on /weight. This is represented by below JSON Patch body.

[
  {
    "op": "set",
    "path": "/height",
    "value": 195
  },
  {
    "op": "add",
    "path": "/weight",
    "value": 90
  }
]

I've hit F5, crated the request in Postman, and clicked Send. What I've received was a 500 Internal Server Error with a weird Newtonsoft.Json deserialization exception. A quick look into Cosmos DB Explorer revealed that the document now looks like this.

{
    ...
    "height": {
        "valueKind": 4
    },
    "weight": {
        "valueKind": 4
    },
    ...
}

This is not what I was expecting. What happened? The fact that PatchItemAsync worked without an exception suggested that this must be how Cosmos DB .NET SDK interpreted the JsonPatchOperation.Value. Through debugging I've quickly discovered that what JsonPatchOperation.Value actually holds is System.Text.Json.JsonElement. The Cosmos DB .NET SDK has no other way of dealing with that than serializing public properties - regardless of how smart the implementation is. This is because Cosmos DB .NET SDK (at least for now) is based on Newtonsoft.Json.

So, this is going to be a little bit harder. Conversion from JsonPatch to PatchOperation collection will require deserializing the value along the way. As this is still part of deserializing the request, I've figured out it will be best to put it into JsonPatchOperation.

public class JsonPatchOperation : IValidatableObject
{
    ...

    public T GetValue<T>()
    {
        return ((JsonElement)Value).Deserialize<T>();
    }
}

This will need to be called with the right type parameter, so a mapping between paths and types needs to be obtained. I've decided to make this information available through JsonPatch by making it generic and spicing with reflection.

public class JsonPatch<T> : List<JsonPatchOperation>
{
    private static readonly IDictionary<string, Type> _pathsTypes;

    static JsonPatch()
    {
        _pathsTypes = typeof(T).GetProperties().ToDictionary(p => $"/{Char.ToLowerInvariant(p.Name[0]) + p.Name[1..]}", p => p.PropertyType);
    }

    public Type GetTypeForPath(string path)
    {
        return _pathsTypes[path];
    }
}

A disclaimer is in order. This simple code will work only in this simple case. I'm looking only at top-level properties because my document only has top-level properties. But, in general, a path can be targeting a nested property e.g. /mather/familyName. This code will have to get more complicated to handle such a case.

To make the conversion from JsonPatch to PatchOperation work correctly, sadly, more reflection is needed as those generic calls have to be made with the right parameters at the runtime. The needed pieces can live together with the conversion extension method.

public static class JsonPatchExtensions
{
    private static MethodInfo _createAddPatchOperationMethodInfo = typeof(JsonPatchExtensions)
        .GetMethod(nameof(JsonPatchExtensions.CreateAddPatchOperation), BindingFlags.NonPublic | BindingFlags.Static);
    private static MethodInfo _createReplacePatchOperationMethodInfo = typeof(JsonPatchExtensions)
        .GetMethod(nameof(JsonPatchExtensions.CreateReplacePatchOperation), BindingFlags.NonPublic | BindingFlags.Static);
    private static MethodInfo _createSetPatchOperationMethodInfo = typeof(JsonPatchExtensions)
        .GetMethod(nameof(JsonPatchExtensions.CreateSetPatchOperation), BindingFlags.NonPublic | BindingFlags.Static);

    ...

    private static PatchOperation CreateAddPatchOperation<T>(JsonPatchOperation jsonPatchOperation)
    {
        return PatchOperation.Add(jsonPatchOperation.Path, jsonPatchOperation.GetValue<T>());
    }

    private static PatchOperation CreateReplacePatchOperation<T>(JsonPatchOperation jsonPatchOperation)
    {
        return PatchOperation.Replace(jsonPatchOperation.Path, jsonPatchOperation.GetValue<T>());
    }

    private static PatchOperation CreateSetPatchOperation<T>(JsonPatchOperation jsonPatchOperation)
    {
        return PatchOperation.Set(jsonPatchOperation.Path, jsonPatchOperation.GetValue<T>());
    }

    private static PatchOperation CreatePatchOperation<T>(
        MethodInfo createSpecificPatchOperationMethodInfo,
        JsonPatch<T> jsonPatchOperations,
        JsonPatchOperation jsonPatchOperation)
    {
        Type jsonPatchOperationValueType = jsonPatchOperations.GetTypeForPath(jsonPatchOperation.Path);

        MethodInfo createSpecificPatchOperationWithValueTypeMethodInfo =
            createSpecificPatchOperationMethodInfo.MakeGenericMethod(jsonPatchOperationValueType);

        return (PatchOperation)createSpecificPatchOperationWithValueTypeMethodInfo.Invoke(null, new object[] { jsonPatchOperation });
    }
}

The code above has been structured to isolate the reflection related part and cache the well-known things. Of course, this is a subject of personal preference, but I hope it's readable.

The last remaining thing is modifying the extension method.

public static class JsonPatchExtensions
{
    ...

    public static IReadOnlyList<PatchOperation> ToCosmosPatchOperations<T>(this JsonPatch jsonPatchOperations)
    {
        List<PatchOperation> cosmosPatchOperations = new List<PatchOperation>(jsonPatchOperations.Count);
        foreach (JsonPatchOperation jsonPatchOperation in jsonPatchOperations)
        {
            switch (jsonPatchOperation.OperationType)
            {
                case JsonPatchOperationType.Add:
                    cosmosPatchOperations.Add(CreatePatchOperation(_createAddPatchOperationMethodInfo, jsonPatchOperations, jsonPatchOperation));
                    break;
                case JsonPatchOperationType.Remove:
                    cosmosPatchOperations.Add(PatchOperation.Remove(jsonPatchOperation.Path));
                    break;
                case JsonPatchOperationType.Replace:
                    cosmosPatchOperations.Add(CreatePatchOperation(_createReplacePatchOperationMethodInfo, jsonPatchOperations, jsonPatchOperation));
                    break;
                case JsonPatchOperationType.Set:
                    cosmosPatchOperations.Add(CreatePatchOperation(_createSetPatchOperationMethodInfo, jsonPatchOperations, jsonPatchOperation));
                    break;
            }
        }

        return cosmosPatchOperations;
    }

    ...
}

Adjusting the action code, building the demo, running, going to Postman, sending a request, and it works!

What’s Missing?

This is demoware, so there is always something missing. I've already mentioned that parts of the code are handling only simple cases. But there are two additional subjects which require more attention.

One is validation. This code is not validating if paths provided for operations represent valid properties. It's also not validating if values can be converted to the right types and if the operations result in an invalid state of the entity. First, two things should be achievable at the JsonPatch level. The last one will require additional code in action.

The other subject is performance around that reflection code. It can be improved by caching the generic methods for specific types, but as the solution grows it might not be enough. It is worth thinking about another option here.

I might tackle those two subjects, but I don't know if and when. You can keep an eye on the demo project because if I do, the code will certainly end there.

Server-Sent Events and ASP.NET Core - Disconnecting a Client

noreply@blogger.com (Tomasz Pęczek) — Mon, 08 Nov 2021 20:28:00 +0000

One of the common requests for my Server-Sent Events library is the ability to disconnect clients from a server. My usual answer was that this is best to be implemented as a logical operation where the server sends an event with a specific type and clients to react to it by closing the connection. I was avoiding putting disconnect functionality into the library because it's not fully defined by the protocol.

Disconnecting a Client in Server-Sent Events

Disconnecting a client from a server is a little bit tricky when it comes to Server-Sent Events. The reason for that is automatic reconnect. If the server simply closes the connection, the client will wait a defined period of time and reconnect. The only way to prevent the client from reconnecting is to respond with a 204 No Content status code. So complete flow of disconnecting a client should look like on the diagram below.

There is just one challenge here - the server needs to be able to tell that it is the same client trying to reconnect.

Identifying Clients in Server-Sent Events

Server-Sent Events doesn't provide any specific way of identifying clients. No dedicated session mechanism. This is a place where choices and opinions are starting, a place where a library should try to stay generic. I've given a lot of thought to whether I want it to include any specific implementation with the library and I've decided to provide only the contract and a no-op implementation.

public interface IServerSentEventsClientIdProvider
{
    Guid AcquireClientId(HttpContext context);

    void ReleaseClientId(Guid clientId, HttpContext context);
}

This gives consumers full freedom (and responsibility) to implement this aspect in whichever way they prefer. For example, below is a simple cookie-based implementation. In the case of a production scenario, you probably want to think a little bit more about cookie options, if the value should be protected, etc. Also, remember that cookies bear legal requirements.

internal class CookieBasedServerSentEventsClientIdProvider : IServerSentEventsClientIdProvider
{
    private const string COOKIE_NAME = ".ServerSentEvents.Guid";

    public Guid AcquireClientId(HttpContext context)
    {
        Guid clientId;

        string cookieValue = context.Request.Cookies[COOKIE_NAME];
        if (String.IsNullOrWhiteSpace(cookieValue) || !Guid.TryParse(cookieValue, out clientId))
        {
            clientId = Guid.NewGuid();

            context.Response.Cookies.Append(COOKIE_NAME, clientId.ToString());
        }

        return clientId;
    }

    public void ReleaseClientId(Guid clientId, HttpContext context)
    {
        context.Response.Cookies.Delete(COOKIE_NAME);
    }
}

Tracking Disconnected Server-Sent Events Clients

Being able to identify a client is only the first step. The second required thing is tracking the disconnected clients, so when they attempt to reconnect the server can respond with 204 No Content.

public interface IServerSentEventsNoReconnectClientsIdsStore
{
    Task AddClientId(Guid clientId);

    Task<bool> ContainsClientId(Guid clientId);

    Task RemoveClientId(Guid clientId);
}

This part doesn't seem to be complicated until you consider scaling out. When there are multiple instances behind a load balancer, there is a possibility that reconnect attempt will reach a different instance than the one to which the client has been previously connected. This is why I've decided to include two implementations of the above store in the library. One is simply keeping the identifiers in memory, while the second is backed by distributed cache.

Putting Things Together

The high-level flow which library currently performs to handle Server-Sent Events requests looks like below.

public class ServerSentEventsMiddleware<TServerSentEventsService> ...
{
    ...

    public async Task Invoke(HttpContext context, IPolicyEvaluator policyEvaluator)
    {
        if (CheckAcceptHeader(context.Request.Headers))
        {
            if (!await AuthorizeAsync(context, policyEvaluator))
            {
                return;
            }

            ...

            await context.Response.AcceptAsync(_serverSentEventsOptions.OnPrepareAccept);

            ServerSentEventsClient client = new ServerSentEventsClient(clientId, context.User, context.Response, _clientDisconnectServicesAvailable);

            ...

            await ConnectClientAsync(context.Request, client);

            await context.RequestAborted.WaitAsync();

            await DisconnectClientAsync(context.Request, client);
        }
        else
        {
            await _next(context);
        }
    }

    ...
}

To enable the disconnect capability, this flow needs to be adjusted to acquire the client identifier as soon as possible and prevent connection (by responding with 204) if the identifier represents a client which shouldn't be allowed to reconnect.

public class ServerSentEventsMiddleware<TServerSentEventsService> ...
{
    ...

    public async Task Invoke(HttpContext context, IPolicyEvaluator policyEvaluator)
    {
        if (CheckAcceptHeader(context.Request.Headers))
        {
            if (!await AuthorizeAsync(context, policyEvaluator))
            {
                return;
            }

            Guid clientId = _serverSentEventsClientIdProvider.AcquireClientId(context);

            if (await PreventReconnectAsync(clientId, context))
            {
                return;
            }

            ...

            await context.Response.AcceptAsync(_serverSentEventsOptions.OnPrepareAccept);

            ...

            await DisconnectClientAsync(context.Request, client);
        }
        else
        {
            await _next(context);
        }
    }

    ...

    private async Task PreventReconnectAsync(Guid clientId, HttpContext context)
    {
        if (!await _serverSentEventsNoReconnectClientsIdsStore.ContainsClientIdAsync(clientId))
        {
            return false;
        }

        response.StatusCode = StatusCodes.Status204NoContent;

        _serverSentEventsClientIdProvider.ReleaseClientId(clientId, context);

        await _serverSentEventsNoReconnectClientsIdsStore.RemoveClientIdAsync(clientId);

        return true;
    }

    ...
}

You can also see that as part of reconnecting prevention I'm releasing the client identifier and removing it from the "no reconnect" identifiers store. The goal is to allow any underlying implementations to clear any data and make sure that a stale identifier doesn't "stick" with the client.

One more thing which requires adjustment are operations performed when a client is being disconnected. If the client is being disconnected by a server, its identifier should be added to the store. If the disconnect happens on the client-side, the client identifier should be released to avoid staleness.

public class ServerSentEventsMiddleware<TServerSentEventsService> ...
{
    ...

    private async Task DisconnectClientAsync(HttpRequest request, ServerSentEventsClient client)
    {
        ...

        if (client.PreventReconnect)
        {
            await _serverSentEventsNoReconnectClientsIdsStore.AddClientIdAsync(client.Id);
        }
        else
        {
            _serverSentEventsClientIdProvider.ReleaseClientId(client.Id, request.HttpContext);
        }

        ...
    }

    ...
}

Coming Soon to a NuGet Feed Near You

The code is ready and I'm going to push it out with the next release of Lib.AspNetCore.ServerSentEvents. So, if you have been waiting for disconnect capabilities in the library they soon will be there. That said, it still leaves some decisions and implementation to consumers. By sharing parts of my thought process and some implementation details I wanted to make clear why it is like this.