cfrenzel

Throwback: Prototype of AR Furniture Layout (2009)

2020-02-28T00:00:00-05:00

Was going through some old video and found rough demo of a prototype that I built back in 2009. I was experimenting with AR at the time but nothing really came of it; so I thought I’d throw it up here. The prototype uses a webcam under a sheet of glass to track markers on the bottom of blocks. Here I’m experimenting with the layout of some office furniture.

Throwback: Projector with Microsoft Kinect (2013)

2020-02-27T00:00:00-05:00

Found some more footage that I thought was pretty cool from back in 2013. This uses a microsoft kinect to track people at a party. I used Quartz Composer to build some visuals and projected it realtime on the side of a barn. Some of the visualizations are influenced by the audio, but I never got much going with body tracking. Might have to expand on this one in the future.

Learning From Open Source Part1: MediatR

2020-01-14T00:00:00-05:00

It can be intimidating and time consuming to explore a mature open source project. Often an attempt to find inspiration results in a surface level understanding of a few components with no real insight into the magic. In this series I hope to find projects of just the right size and complexity to “get in there” and learn something valuable. Let’s start by taking a look at MediatR.

I use MediatR in many projects because of its “simple to use” yet powerful implementation of the Mediator Pattern. A mediator sits between method callers and receivers creating a configurable layer of abstraction that determines how callers and receivers get wired up.

Let’s look at a quick example of a Controller calling some Application Layer code with MediatR.

public class CreateTicketCommand : IRequest<CreateTicketResponse>
{
    public string Description { get; set; }
    public string Department { get; set; }
    public string Severity { get; set; }
}

public class TicketController: Controller
{
   private readonly IMediator _mediator;

    public TicketController(IMediator mediator)
    {
        _mediator = mediator;
    }

    [HttpPost]
    public async Task<ActionResult> Create(CreateTicketModel model)
    {
        var command = new CreateTicketCommand()
        {
            Description = model.Description,
            Department = model.Department,
            Severity = model.Severity
        };

        var res = await _mediator.Send(command);
        return RedirectToAction("Show", new { id = res.TicketId});
    }
}

There are a few things to notice here.

The Command defines the type of it’s response IRequest<CreateTicketResponse>
The IMediator is injected into the Controller
There is a single Send method on the IMediator used to dispatch all command/request types
The Controller doesn’t know who is handling the command

Here’s a sample handler for CreateTicketCommand

public class CreateTicketHandler : IRequestHandler<CreateTicketCommand, CreateTicketResponse>
{
    private readonly ApplicationDbContext _db;

    public CreateTicketHandler(ApplicationDbContext db) =>  _db = db;

    public async Task<CreateTicketResponse> Handle(CreateTicketCommand command, CancellationToken cancellationToken)
    {
        Ticket ticket = new Ticket(command.Description, command.Department, command.Severity);
        _db.Tickets.Add(ticket);
        await _db.SaveChangesAsync();
        return new CreateTicketResponse() { TicektId = ticket.Id };
    }
}

Notice:

Handler doesn’t know who sent the command
Handler specifies the the message type that it handles and the response type IRequestHandler<CreateTicketCommand, CreateTicketResponse>
Handler has constructor parameters that must be injected by the caller

We configure MediatR in our app’s startup with a single line

  services.AddMediatR(typeof(Program));

Here’s what I’m thinking

I never registered any handlers explicitly; so I know that MediatR is scanning my assembly for handlers. This is pretty common and shouldn’t be much different than something like ASP.NET MVC finding your controllers, but I’d like to look under the hood a little.
The call to _mediator.Send(command) stands out as the interesting bit. Somehow this method can take any request type, find a concrete handler implementation for that request type, instantiate it, and call it. Let’s try to uncover the magic behind this!

Going to the Source

Let’s dig in. First download the source or browse online at https://github.com/jbogard/MediatR/tree/master/src/MediatR

> git clone https://github.com/jbogard/MediatR.git .

You’ll first notice a handful of simple interfaces and Mediator.cs with the core class. Since we’re curious about it’s Send method let’s take a look inside. The file is surprisingly small, and at first glance it looks like the Send method is actually doing some of the real work.

        public Mediator(ServiceFactory serviceFactory)
        {
            _serviceFactory = serviceFactory;
        }

        public Task<TResponse> Send<TResponse>(IRequest<TResponse> request, CancellationToken cancellationToken = default)
        {
            if (request == null)
            {
                throw new ArgumentNullException(nameof(request));
            }

            var requestType = request.GetType();

            var handler = (RequestHandlerWrapper<TResponse>)_requestHandlers.GetOrAdd(requestType,
                t => Activator.CreateInstance(typeof(RequestHandlerWrapperImpl<,>).MakeGenericType(requestType, typeof(TResponse))));

            return handler.Handle(request, cancellationToken, _serviceFactory);
        }

First things first, above I made a call using Send(command), but the send method here is generic Send<TResponse>(IRequest<TResponse> request). It turns out that the compiler can infer the type argument; so you can omit it. Our call above looks much prettier than Send<CreateTicketResponse>(command) (generics methods). Moving on…

I’m starting to get excited because the Send method is basically a fat one-liner, yet it appears to be instantiating a handler and calling it (which is all a mediator really does). Let’s dig a little deeper.

var handler = (RequestHandlerWrapper<TResponse>)_requestHandlers.GetOrAdd(requestType,
                t => Activator.CreateInstance(typeof(RequestHandlerWrapperImpl<,>).MakeGenericType(requestType, typeof(TResponse))));

The outer _requestHandlers.GetOrAdd is just checking if a handler already exists before creating a new one. The real magic seems to be:

Activator.CreateInstance(typeof(RequestHandlerWrapperImpl<,>).MakeGenericType(requestType, typeof(TResponse)))

We’re getting close to something. We’re taking a RequestHandlerWrapperImpl<,> calling .MakeGenericType(requestType, typeof(TResponse)) on it then instantiating the result, which is apparently assignable to RequestHandlerWrapper<TResponse>. Let’s plug in our command type above to see what it looks like.

typeof(RequestHandlerWrapperImpl<,>).MakeGenericType(typeof(CreateTicketCommand), typeof(CreateTicketResponse))

This gives us a RequestHandlerWrapperImpl<CreateTicketCommand,CreateTicketResponse> which is assignable to RequestHandlerWrapper<CreateTicketResponse>. For now let’s think of “RequestHandlerWrapper” as some kind of wrapper around our actual handler (CreateTicketHandler). We know that instantiating our actual handler will require Dependency Injection to resolve the dependencies (like ApplicationDbContext); so perhaps the wrapper is hiding these details.

But whats up with the cast from RequestHandlerWrapperImpl down to the less generic RequestHandlerWrapper? Believe it or not, this seemingly benign cast is part of the real magic. The bigger concept at play here is a technique for allowing non-generic code to call into generic code. It’s a little hard to see here because the return type is still generic, but notice how the TRequest goes away when casting RequestHandlerWrapperImpl<TRequest,TResponse> to RequestHandlerWrapper<TResponse>.

Lets take a look inside:

 
 internal abstract class RequestHandlerWrapper<TResponse> : RequestHandlerBase
    {
        public abstract Task<TResponse> Handle(IRequest<TResponse> request, CancellationToken cancellationToken,
            ServiceFactory serviceFactory);
    }
 
 internal class RequestHandlerWrapperImpl<TRequest, TResponse> : RequestHandlerWrapper<TResponse>
        where TRequest : IRequest<TResponse>
    {
         public override Task<TResponse> Handle(IRequest<TResponse> request, CancellationToken cancellationToken,
            ServiceFactory serviceFactory)
        {
            Task<TResponse> Handler() => GetHandler<IRequestHandler<TRequest, TResponse>>(serviceFactory).Handle((TRequest) request, cancellationToken);

            return serviceFactory
                .GetInstances<IPipelineBehavior<TRequest, TResponse>>()
                .Reverse()
                .Aggregate((RequestHandlerDelegate<TResponse>) Handler, (next, pipeline) => () => pipeline.Handle((TRequest)request, cancellationToken, next))();
        }

Cool. So RequestHandlerWrapperImpl<TRequest, TResponse> implements the abstract RequestHandlerWrapper<TResponse>. So we have the generic implementation extending the non-generic (with respect to TRequest). Ultimately, RequestHandlerWrapper<TResponse> is providing a single non-generic handler interface that we can use to make calls into all the generic implementations for each request type. Let’s see how the generic version accomplishes this:

 public override Task<TResponse> Handle(IRequest<TResponse> request, CancellationToken cancellationToken,
            ServiceFactory serviceFactory)
        {
            Task<TResponse> Handler() => GetHandler<IRequestHandler<TRequest, TResponse>>(serviceFactory).Handle((TRequest) request, cancellationToken);

            return serviceFactory
                .GetInstances<IPipelineBehavior<TRequest, TResponse>>()
                .Reverse()
                .Aggregate((RequestHandlerDelegate<TResponse>) Handler, (next, pipeline) => () => pipeline.Handle((TRequest)request, cancellationToken, next))();
        }

MediatR has some cool features around pipelines, but a basic mediator doesn’t need any of that. Let’s focus on how the RequestHandlerWrapperImpl<TRequest, TResponse> does it’s job of calling our CreateTicketHandler with this line:

Task<TResponse> Handler() => GetHandler<IRequestHandler<TRequest, TResponse>>(serviceFactory).Handle((TRequest) request, cancellationToken);

Let’s break this down

First we call GetHandler() on our base RequestHandlerBase class. This should return our actual handler (CreateTicketHandler). We’ll take a look inside a little later

GetHandler<IRequestHandler<TRequest, TResponse>>(serviceFactory)

Then we call the .Handle() method on the handler (CreateTicketHandler). We must cast the IRequest<TResponse> to TRequest to complete the bridge from the non-generic to generic.

.Handle((TRequest) request, cancellationToken)

With our command it would look like this

//request is an IRequest<CreateTicketResponse> from the non-generic
.Handle((CreateTicketCommand) request, cancellationToken)

Now instead of immediately running the Handle() method above, we’re using a lambda to create Task that we can pass around and call later

Task<TResponse> Handler() => 

At this point awaiting the Handler would wrap up our dispatch to CreateTicketHandler. Let’s back up one step and see how the instantiation of our handler with Dependency Injection went down in RequestHandlerBase.

 protected static THandler GetHandler<THandler>(ServiceFactory factory)
 {
     THandler handler;
     try
     {
        handler = factory.GetInstance<THandler>();
     }

Interesting, I really expected more code. Let’s take a look at ServiceFactory to see what’s up.

public delegate object ServiceFactory(Type serviceType);

public static class ServiceFactoryExtensions
{
    public static T GetInstance<T>(this ServiceFactory factory)
        => (T) factory(typeof(T));

    public static IEnumerable<T> GetInstances<T>(this ServiceFactory factory)
            => (IEnumerable<T>) factory(typeof(IEnumerable<T>));
}

Mind Blown! ServiceFactory is just a delegate. As a bonus the method GetInstance<THandler> is an extension method on the delegate. That’s right C# supports extension methods on delegates. Essentially, ServiceFactory is a single method wrapper around our Dependency Injection container. The only burden ServiceFactory puts on the underlying container is to have a method that takes in a Type parameter and returns an instance of that type. Using a couple of extension methods we overlay a nicer generic interface T GetInstance<T> and IEnumerable<T> GetInstances<T>. You don’t see this kind of expressiveness in strongly-typed languages every day.

MediatR's support for all the different dependency injection frameworks boils down to a simple delegate. Ironically, if you look into one of the Dependency Injection integrations like MediatR.Extensions.Microsoft.DependencyInjection, it has about as much code for registering all the bells and whistles for handlers as the core of MediatR itself.

Simple Domain Events with EFCore and MediatR

2020-01-03T00:00:00-05:00

This post relates to the Domain Driven Design (DDD) concept of Domain Events. These events originate in the Domain Model and are broadcast within a Bounded Context. These are not events used directly for integration. For the purpose of this implementation I want to frame things as EFCore entities publishing events that can be handled locally by one or more subscribers within a Unit Of Work. For more information about what Domain Events are/aren’t and what they can be used for, check out Domain Driven Design by Eric Evans and Implementing Domain Driven Design by Vaughn Vernon.

Beyond the initial difficulty of understanding what Domain Events are, lies figuring out a way to implement the things. How the heck can you cleanly publish an event from an Entity? How would you wire up listeners, and where in the architecture would the listeners live? Our entities are often in a core assembly that doesn’t have any dependencies. There is no concept of a UnitOfWork/Transaction at this level, and they sure as heck don’t have access to anything interesting like databases or an Application Layer where you might normally think about hydrating other entities and handling events.

This post describes a method to allow EFCore entities to publish Domain Events. I’ve seen this technique used a handful of times, but to make this implementation a little more interesting Domain Events will be published as MediatR notifications that can be handled in the Application Layer. In addition, this must be done without the entities taking on any external dependencies. Specifically the entities won’t have a dependency on MediatR.

Sound good? Let’s get started!

The Entity Side

The entity needs to call “Publish” on something. One of the simplest implementations from the entity’s perspective is just to have the entity inherit from a base class that contains the publish logic. In this implementation the entity won’t actually do the event dispatching, it will just hold a collection of events that a dispatcher will later examine.

First let’s define an interface for the Entity

using System;
using System.Collections.Concurrent;

namespace DomainEventsMediatR.Domain
{
    public interface IEntity
    {
        IProducerConsumerCollection<IDomainEvent> DomainEvents { get; }
    }
}

 public interface IDomainEvent { }

Now a base class implementation for our EFCore entities marking the DomainEvents as [NotMapped] to let EFCore know that they are not to be persisted to the db.

We also add a helper for entities to initialize there own Id's. It can be very useful and efficient for entities to have their Id's generated locally "on or before instantiation" rather than "on save" or in the database. This allows transient entities to reference eachother by Id, to store Id's in Domain Events, and generally to use Id's in all kinds of eventual consistency scenarios. You can pass the entity itself in the domain event, but remember that it hasn't been persisted yet; so you can't trust the transient Id assigned by EFcore (a new Id will be assigned by the database when persisted).

using System;
using System.Collections.Concurrent;
using System.ComponentModel.DataAnnotations.Schema;

namespace DomainEventsMediatR.Domain
{ 
    public abstract class Entity : IEntity
    {     
        [NotMapped]
        private readonly ConcurrentQueue<IDomainEvent> _domainEvents = new ConcurrentQueue<IDomainEvent>();

        [NotMapped]
        public IProducerConsumerCollection<IDomainEvent> DomainEvents => _domainEvents;

        protected void PublishEvent(IDomainEvent @event)
        {
            _domainEvents.Enqueue(@event);
        }

        protected Guid NewIdGuid()
        {
            return MassTransit.NewId.NextGuid();
        }
    }
}

Now a Domain Event: BacklogItemCommitted and an entity: BacklogItem that publishes the event when it is commited to a Sprint

namespace DomainEventsMediatR.Domain
{
    public class BacklogItemCommitted : IDomainEvent
    {
        public Guid BacklogItemId { get; }
        public Guid SprintId { get; set; }
        public DateTime CreatedAtUtc { get; }

        private BacklogItemCommitted() { }

        public BacklogItemCommitted(BacklogItem b, Sprint s)
        {
            this.BacklogItemId = b.Id;
            this.CreatedAtUtc = b.CreatedAtUtc;
            this.SprintId = s.Id;
        }    
    }
}


using System;
using System.ComponentModel.DataAnnotations;

namespace DomainEventsMediatR.Domain
{
    public class BacklogItem : Entity
    {
        public Guid Id { get; private set; }

        [MaxLength(255)]
        public string Description { get; private set; }
        public virtual Sprint Sprint { get; private set; }
        public DateTime CreatedAtUtc { get; private set; } = DateTime.UtcNow;

        private BacklogItem() { }

        public BacklogItem(string desc)
        {
            this.Id = NewIdGuid();
            this.Description = desc;
        }
    
        public void CommitTo(Sprint s)
        {
            this.Sprint = s;
            this.PublishEvent(new BacklogItemCommitted(this, s));
        }
    }
}

The real magic with this technique is how the Domain Events are dispatched. Currently they’re just sitting in the Entity. We’ll use some hooks in our DbContext to dispatch them, but first let’s define an interface for the dispatcher

using System.Threading.Tasks;

namespace DomainEventsMediatR.Domain
{
    public interface IDomainEventDispatcher
    {
        Task Dispatch(IDomainEvent devent);
    }
}

Now we can configure the dispatcher to be injected into our DbContext constructor

 public class ApplicationDbContext : DbContext
 {
    private readonly IDomainEventDispatcher _dispatcher;

    public ApplicationDbContext(DbContextOptions<ApplicationDbContext> options,
        IDomainEventDispatcher dispatcher)
        : base(options)
    {
        _dispatcher = dispatcher;
    }

We can hook into EFCore and dispatch Domain Events before entities are persisted by overriding SaveChanges

public override int SaveChanges()
{
    _preSaveChanges().GetAwaiter().GetResult();
    var res = base.SaveChanges();
    return res;
}

public override async Task<int> SaveChangesAsync(CancellationToken cancellationToken = default(CancellationToken))
{
    await _preSaveChanges();
    var res = await base.SaveChangesAsync(cancellationToken);
    return res;
}

private async Task _preSaveChanges()
{
    await _dispatchDomainEvents();
}

private async Task _dispatchDomainEvents()
{
    var domainEventEntities = ChangeTracker.Entries<IEntity>()
        .Select(po => po.Entity)
        .Where(po => po.DomainEvents.Any())
        .ToArray();

    foreach (var entity in domainEventEntities)
    {
        IDomainEvent dev;
        while (entity.DomainEvents.TryTake(out dev))
            await _dispatcher.Dispatch(dev);
    }
}

The Dispatcher

We need an implementation of IDomainEventDispatcher that will publish the Domain Event as a MediatR notification. We’ll implement this in our Application Layer. We do have to deal with the small issue of our Domain Event not being a valid MediatR INotification. We’ll overcome this by creating a generic INotification to wrap our Domain Event.

Create a custom generic INotification.

using System;
using MediatR;
using DomainEventsMediatR.Domain;

namespace DomainEventsMediatR.Application
{
    public class DomainEventNotification<TDomainEvent> : INotification where TDomainEvent : IDomainEvent
    {
        public TDomainEvent DomainEvent { get; }

        public DomainEventNotification(TDomainEvent domainEvent)
        {
            DomainEvent = domainEvent;
        }
    }
}

Create a Dispatcher that wraps Domain Events in MediatR notificatoins and publishes them

using System;
using System.Threading.Tasks;
using Microsoft.Extensions.Logging;
using MediatR;
using DomainEventsMediatR.Domain;

namespace DomainEventsMediatR.Application
{
    public class MediatrDomainEventDispatcher : IDomainEventDispatcher
    {
        private readonly IMediator _mediator;
        private readonly ILogger<MediatrDomainEventDispatcher> _log;
        public MediatrDomainEventDispatcher(IMediator mediator, ILogger<MediatrDomainEventDispatcher> log)
        {
            _mediator = mediator;
            _log = log;
        }

        public async Task Dispatch(IDomainEvent devent)
        {

            var domainEventNotification = _createDomainEventNotification(devent);
            _log.LogDebug("Dispatching Domain Event as MediatR notification.  EventType: {eventType}", devent.GetType());
            await _mediator.Publish(domainEventNotification);
        }
       
        private INotification _createDomainEventNotification(IDomainEvent domainEvent)
        {
            var genericDispatcherType = typeof(DomainEventNotification<>).MakeGenericType(domainEvent.GetType());
            return (INotification)Activator.CreateInstance(genericDispatcherType, domainEvent);

        }
    }
}

Create a handler for the BacklogItemCommitted Domain Event

using System;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.Logging;
using MediatR;
using DomainEventsMediatR.Domain;
using DomainEventsMediatR.Persistence;

namespace DomainEventsMediatR.Application
{
    public class OnBacklogItemCommitted
    {
        public class Handler : INotificationHandler<DomainEventNotification<BacklogItemCommitted>>
        {
            private readonly ApplicationDbContext _db;
            private readonly ILogger<Handler> _log;
        
            public Handler(ApplicationDbContext db,  ILogger<Handler> log)
            {
                _db = db;
                _log = log;
            }

            public Task Handle(DomainEventNotification<BacklogItemCommitted> notification, CancellationToken cancellationToken)
            {
                var domainEvent = notification.DomainEvent;
                try
                {
                    _log.LogDebug("Handling Domain Event. BacklogItemId: {itemId}  Type: {type}", domainEvent.BacklogItemId, notification.GetType());
                    //from here you could 
                    // - create/modify entities within the same transaction as the backlogItem commit
                    // - trigger the publishing of an integration event on a servicebus (don't write it directly though, you need an outbox scoped to this transaction)
                                      
                    //Remember NOT to call SaveChanges on dbcontext if making db changes when handling DomainEvents
                    return Task.CompletedTask;
                }
                catch (Exception exc)
                {
                    _log.LogError(exc, "Error handling domain event {domainEvent}", domainEvent.GetType());
                    throw;
                }
            }
        }

    }
}

Now we just need to configure dependency injection in our application and we’re done

     services.AddTransient<IDomainEventDispatcher, MediatrDomainEventDispatcher>();
     services.AddMediatR(typeof(MediatrDomainEventDispatcher).GetTypeInfo().Assembly);

You can find the full source code for this post at https://github.com/cfrenzel/DomainEventsWithMediatR

Quickly Create Your Own .NET Code Templates and Use Them From Anywhere

2019-12-20T00:00:00-05:00

Whether you need to throw together a quick console app or scaffold an enterprise solution, it can be a real time suck just creating, naming and referencing projects. Setting up boilerplate logging, dependency injection, data access, messaging, gulp and other tools can send you hunting through previous work to copy and paste code. Let’s put an end to all that once and for all with less than an hour of work using dotnet new templating! The advantages of this approach include:

- Use a tool that's already on any development machine       
- No new templating language to learn
- Use runnable Solution/Project/Files as templates
- Bundle many templates into a single distributable package
- Access templates from any machine with a single command

Let's get started:

Our goal is to be able to easily create and distribute custom templates to any machine; so let’s first take a look at what templates already exist on our machine:

> dotnet new -l

Templates                                         Short Name               Language          Tags
----------------------------------------------------------------------------------------------------------------------------------
Console Application                               console                  [C#], F#, VB      Common/Console
Class library                                     classlib                 [C#], F#, VB      Common/Library

You should see a list of templates longer but similar to above. Our custom templates will show up in this list when we’re done. To create a new project from the template named console in the list above we can type:

>  dotnet new console -n SampleFromTemplate

This will create a new folder with a console app named SampleFromTemplate. It’s ready to go with nuget packages restored and the namespaces set to SampleFromTemplate.

SampleFromTemplate
    └─── SampleFromTemplate.csproj
    └─── Program.cs
    └─── /obj

To begin creating custom templates with dotnet new simply create a normal project or solution (or just one or more files) that represents your boilerplate code for a given scenario. That’s almost all there is to it. Adding a configuration file to setup some metadata and behavior will result in a resuable template. The template folder structure for a simple console app project will look something like this:

└───mycustomtemplate
    └─── Templating.ConsoleApp.csproj
    └─── Program.cs
    └─── /.template_config
        └───  template.json

Start with any existing project and from the project root folder

>  mkdir .template_config

You have control over whether the generated output of your template is placed in a new folder or just dumped in the output location. If you want everything inside a folder then include the folder at the top level of the template beside the .template_config folder. Otherwise you can leave it up to the user to specify on the command line using the -o option.

If you want to create empty folders inside your template such as /src /test /doc /build /migrations. For now you will need to place a file named -.- inside the folder otherwise the empty folder will be ignored in the output

Add a template.json to the .template_config folder

{
  "$schema": "http://json.schemastore.org/template",
  "author": "Camron Frenzel",
  "classifications": [ "cfrenzel", "core", "console" ],
  "tags": {
    "language": "C#"
  },
  "identity": "demo.console",
  "name": "demo.console_2.2",
  "shortName": "dm-console-2.2",
  "sourceName": "Templating",
  "sources": [
    {
      "modifiers": [
        { "exclude": [ ".vs/**", ".template_config/**" ] }
      ]
    }
  ],
}

identity a unique name for the template
name for display purposes
shortName what users will type to specify your template
sources -> exclude: This is a little trick to keep some unwanted files out of the template

sourceName the name in the source tree to replace with the user specified name (using -n or --name).

sourceName is important!. dotnet new will replace all the folders/files/namespaces/etc.. containing this name with whatever the user passes in on the command line. For example: If I’m using a convention such as

  └─── Templating.sln
  └─── /src
      └─── /Templating.ConsoleApp
          └─── Templating.ConsoleApp.csproj
      └─── /Templating.Domain
          └─── Templating.Domain.csproj
      └─── /Templating.Application
          └─── Templating.Application.csproj

Then passing in -n Demo will produce:

  >  dotnet new demo.console_2.2 -n Demo

  └─── Demo.sln
  └─── /src
      └─── /Demo.ConsoleApp
          └─── Demo.ConsoleApp.csproj
      └─── /Demo.Domain
          └─── Demo.Domain.csproj
      └─── /Demo.Application
          └─── Demo.Application.csproj
    
   namespaces: Templating.ConsoleApp -> Demo.ConsoleApp

At this point you should be comfortable with these concepts
- a template is a normal solution/project/file
- add a .template_config folder with a template.config file in it to configure a template
- the user will pass in a –name MyApp to the template that will replace the configured sourceName text in all folders/solutions/projects/namespaces

Tip! To have nuget restore automatically - add this to your template.config

"symbols": {
    "skipRestore": {
      "type": "parameter",
      "datatype": "bool",
      "description": "If specified, skips the automatic restore of the project on create.",
      "defaultValue": "false"
    }
  },
  "postActions": [
    {
      "condition": "(!skipRestore)",
      "description": "Restore NuGet packages required by this project.",
      "manualInstructions": [
        { "text": "Run 'dotnet restore'" }
      ],
      "actionId": "210D431B-A78B-4D2F-B762-4ED3E3EA9025",
      "continueOnError": true
    }
  ]

Issue! If your template creates one or more projects, often you would like the generated projects to be automatically added to an existing solution. This is supported, but I haven't had any luck with it. The essence of the problem seems to be a bug rendering the output project path/name.

  "primaryOutputs": [
    { "path": "SolutionName.ConsoleApp/SolutionName.ConsoleApp.csproj" }
  ],

  "postActions": [
    {
      "description": "Add project to solution",
      "manualInstructions": [],
      "primaryOutputIndexes": "0",
      "actionId": "D396686C-DE0E-4DE6-906D-291CD29FC5DE",
      "continueOnError": true
    }
  ]

https://github.com/dotnet/templating/issues/1489

If you haven’t created your own template at this point you can follow along by downloading a console app template with logging/DI/configuration here

Installing a template

We could install our template locally from the template root folder.

dotnet new -i .

List the installed templates and you should see your template listed

dotnet new -l

You can use it by passing in it’s shortName and provide a name

    >  dotnet new {shortname} -n DemoApp

To remove your template

dotnet new -u

You should see your template along with an Uninstall command:. This command will come in handy as things can get confusing when managing multiple versions of your templates and installing them from different sources.

 dotnet new -u C:\temptemplate\temptemplate

Not bad, but the workflow leaves a lot to be desired. It would be a pain to manage even a modest number of templates using this method.

Packaging templates

The dotnet new templating tool supports installing templates from nuget packages locally or in remote repositories. Multiple templates can be included in a single package, which allows adding and removing collections of templates from the internet with a single command.

Packaging templates took some tinkering for me; so let’s get straight to what works by creating a special project that will help us get all of our templates into a single package. The structure of our multi-template solution will look like this:

 └─── /my-dotnet-templates   
      └─── my-dotnet-templates.sln
      └─── /templates
           └─── Directory.Build.props //metadata for package
           └─── templates.csproj
           └─── /ConsoleApp //template 1
                └─── Templating.sln
                └─── /.template_config
                     └───template.json
                └─── /Templating.ConsoleApp
                     └─── Templating.ConsoleApp.csproj
           └─── /WebApp //template 2
                └─── Templating.sln
                └─── /.template_config
                     └───template.json
                └─── /Templating.WebbApp
                     └─── Templating.WebApp.csproj

The idea is that you have a solution with a /templates folder and a special project: template.csproj that will aid in building the multi-template package. Within the /templates folder you will have a folder for each template. The folder for each template should contain everything you need to develop and test the template. You won’t be able to run the template from our special templates.csproj so it’s nice to have a seperate solution for running/editing each template.

You can start by creating templates.csproj as a console app. Open the .csproj file and edit it to look like this:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <PackageType>Template</PackageType>
    <TargetFramework>netcoreapp2.2</TargetFramework>
    <PackageId>cfrenzel-dotnet-new-templates-2.2</PackageId>
    <Title>cfrenzel dotnet-new-templates</Title>
    <IncludeContentInPack>true</IncludeContentInPack>
    <IncludeBuildOutput>false</IncludeBuildOutput>
    <ContentTargetFolders>content</ContentTargetFolders>
  </PropertyGroup>

  <ItemGroup>
    <Content Include="ConsoleApp\**" Exclude="ConsoleApp\SolutionName.sln;ConsoleApp\**\bin\**;ConsoleApp\**\obj\**;ConsoleApp\**\.vs\**" />
    <Content Include="EFCore.MigrationProjects\**" Exclude="EFCore.MigrationProjects\**\bin\**;EFCore.MigrationProjects\**\obj\**;EFCore.MigrationProjects\**\.vs\**" />
    <Content Include="Solution\**" Exclude="Solution\**\bin\**;Solution\**\obj\**;Solution\**\.vs\**" />
    <Compile Remove="**\*" />
  </ItemGroup>

</Project>

<PackageType>Template</PackageType> - set special package type for project
For each template in our package we are adding a <Content> tag that specifies which files to include and exclude
- <Content Include="ConsoleApp\**" - include everything from our /ConsoleApp folder
- Exclude="ConsoleApp\Templating.sln;ConsoleApp\**\bin\**;ConsoleApp\**\obj\**;ConsoleApp\**\.vs\**" /> - exclude the solution file and bin/obj folders
<Compile Remove="**\*" /> - we’re not interested in the output of the compiled project
We can specify metadata for the package in Directory.Build.props

<Project>
  <PropertyGroup>
    <Authors>Camron Frenzel</Authors>
    <RepositoryUrl>https://github.com/cfrenzel/dotnet-new-templates-2.2.git</RepositoryUrl>
    <PackageProjectUrl>https://github.com/cfrenzel/dotnet-new-templates-2.2</PackageProjectUrl>
    <Description>dotnet new templates for core 2.2</Description>
    <PackageTags>template dotnet console migration web</PackageTags>
    <PackageLicense></PackageLicense>
    <Version>1.0.0</Version>   
  </PropertyGroup>
</Project>

Now we can create our nuget package using templates.csproj

dotnet pack .\templates\templates.csproj -o .\artifacts\ --no-build

And install all of our templates locally from our .nupkg file

dotnet new -i .\artifacts\cfrenzel-dotnet-new-templates-2.2.1.0.0

Find the Uninstall command: to remove

dotnet new -u

Publish package for access online

You can host your template .nupkg for free using my MyGet or make it official/perminent using NuGet.org. This can be as simple as typing

dotnet nuget push artifacts\**\*.nupkg -s "https://www.myget.org/F/{youraccount}/api/v2/package" -k {yourkey}

Then installing your templates from anywhere using

dotnet new --install {yourpackagename}  --nuget-source https://www.myget.org/F/{youraccount}/api/v3/index.json

If you publish it to nuget.org you don't even have to specify the package url! dotnet new --install {yourpackagename}

-Since we’re pretty much one configuration file away from free continuous build and deployment, let’s setup AppVeyor to build and publish our template package every time we commit a change to the source code.

Save you code to Github or wherever
Add an appveyor.yml to build and publish template package to MyGet

version: '1.{build}'
image: Visual Studio 2019
environment:
  MyGetApiKey:
    secure: 56nW3KcP4naYX9mlsVEIKLj5xPdfmpt6lMALR6wQmorRQOaoUOtlwMZ2V0BtGTAM
  NugetApiKey:
    secure: /54XAunyBETRa1Fp/qSrwvebSnTAcHDO2OVZ+exMtQtOtrBzHKvp4RC1AB8RD2PQ
pull_requests:
  do_not_increment_build_number: true
branches:
  only:
  - master
nuget:
  disable_publish_on_pr: true
test: off
build_script:
  - dotnet restore 
  - dotnet pack .\templates\templates.csproj -o .\artifacts\ --no-build
deploy_script:
  - ps: dotnet nuget push artifacts\**\*.nupkg -s "https://www.myget.org/F/cfrenzel-ci/api/v2/package" -k $env:MyGetApiKey
 

After you link your AppVeyor project to your source repo, your template .nupkg will be updated in MyGet/Nuget every time you commit to master.

Final Thoughts

Though templating with dotnet new has some more powerful features including:

conditional logic
custom parameters
post actions
multi-language
see docs - template.json

I really appreciate the simplicity of the tool. You simply use what you’re already doing to make doing what you’re already doing faster. No need to learn a new complex template language.

full source code for this post.

Publishing .NET Core NuGet Packages with Nuke and AppVeyor

2019-12-17T00:00:00-05:00

This article builds on concepts discussed by Andrew Lock, Jimmy Bogard and Georg Dangl. Here we’re going use Nuke to make build, packaging and publishing even nicer!!!

I’ve been eking out build solutions using various Powershell based tools for years. They serve their purpose, but I always dread getting familiar with the scripts again when I need to make a change. I recently used Nuke on a project and for the first time I feel like I didn’t waste any time fighting with it.

Nuke creates a CSharp Console App within your solution containing a simple Build.cs file that can handle a variety of common build/deployment tasks out of the box. The real joy is that you can now author and debug platform independent build scripts in C# within your favorite IDE!

Let’s jump in.

Using Nuke to Build

Install Nuke

> dotnet tool install Nuke.GlobalTool --global

Add Nuke to your solution - let the wizard get you started

> nuke :setup

Warning: For me the Nuke build project defaulted to .NET Core 3.0. This isn't necessarily a problem, but it's worth noting. This was true when buidling an app on .NET Core 2.2; so it's a bit odd for my build environment to require .Net Core 3.0
TODO:/// Figure out Nuke's logic for framework selection and see if it's configurable

You’ll notice a new project in your solution named _build. Take note of a few files

Build.cs - a fluent “make style” build Class in C#
- Defines targets and their dependencies
Two scripts used to run builds. These scripts will install dotnet if it doesn’t exist and then call your build application. Choose one based on your build environment.
- build.ps1 - a powershell script used to execute builds (platform independent - must have powershell installed)
- build.sh - a shell script version (linux/osx/etc..)

Now compile your code

>  .\build.ps1 Compile

Success! I’ll admit that compiling a project isn’t that impressive, but we’re now scripting in C#. Let’s take it a step further and make a NuGet package.

Add a Pack step to our build script

Target Pack => _ => _
      .DependsOn(Compile)
      .Executes(() =>
      {
          DotNetPack(s => s
              .SetProject(Solution.GetProject("Nuke.Sample"))
              .SetConfiguration(Configuration)
              .EnableNoBuild()
              .EnableNoRestore()
              .SetDescription("Sample package produced by NUKE")
              .SetPackageTags("nuke demonstration c# library")
              .SetNoDependencies(true)
              .SetOutputDirectory(ArtifactsDirectory / "nuget"));

      });

We want our NuGet package to specify an author, repository, homepage, etc… We could do this programatically from Nuke

 Target Pack => _ => _
      .DependsOn(Compile)
      .Executes(() =>
      {
          DotNetPack(s => s
               ***
              .SetAuthors("Your Name")
              .SetPackageProjectUrl("https://github.com/yourrepo/NukeSample")
               ***
      });

But it’s simpler to add a Directory.Build.props to your solution folder

 <Project>
  <PropertyGroup>
    <Authors>Your Name</Authors>
    <RepositoryUrl>https://github.com/yourrepo/NukeSample</RepositoryUrl>
    <PackageProjectUrl>https://github.com/yourrepo/NukeSample</PackageProjectUrl>
    <PackageLicense>https://github.com/yourrepo/NukeSample/blob/master/LICENSE</PackageLicense>
  </PropertyGroup>
</Project>

Now call our new Pack target

>  .\build.ps1 Pack

Now we’ve got our nuget package: artifacts\nuget\Nuke.Sample.1.0.0.nupkg. If we unzip the .nupkg file we can take a look inside at our Nuke.Sample.nuspec file.

<?xml version="1.0" encoding="utf-8"?>
<package xmlns="http://schemas.microsoft.com/packaging/2012/06/nuspec.xsd">
  <metadata>
    <id>Nuke.Sample</id>
    <version>1.0.0</version>
    <authors>Your Name</authors>
    <owners>Your Name</owners>
    <requireLicenseAcceptance>false</requireLicenseAcceptance>
    <projectUrl>https://github.com/yourrepo/NukeSample</projectUrl>
    <description>Sample package produced by NUKE</description>
    <tags>nuke demonstration c# library</tags>
    <repository url="https://github.com/yourrepo/NukeSample" />
    <dependencies>
      <group targetFramework=".NETStandard2.0" />
    </dependencies>
  </metadata>
</package>

Success! Not bad for a few minutes of our time. Before we move on let’s touch on versioning. If you have an approach that you love, it shouldn’t be hard to work it into our current workflow with Nuke. Here we’ll consider a manual option and using the popular GitVersion tool.

Manual Versioning

Let’s add add a couple of lines to our Directory.Build.props.

 <Project>
  <PropertyGroup>
    ---
    <VersionPrefix>0.1.1</VersionPrefix>
    <VersionSuffix>alpha</VersionSuffix>
  </PropertyGroup>
</Project>

Now let’s call our Pack target again

>  .\build.ps1 Pack

Our package name reflects the new version: artifacts\nuget\Nuke.Sample.0.1.1-alpha.nupkg. If we unzip and look inside Nuke.Sample.nuspec we can see the updated version.

<?xml version="1.0" encoding="utf-8"?>
<package xmlns="http://schemas.microsoft.com/packaging/2012/06/nuspec.xsd">
  <metadata>
    <id>Nuke.Sample</id>
    <version>0.1.1-alpha</version>

Versioning with GitVersion tool

Nuke has great integration with the GitVersion tool. You’ll need to read the docs to fully understand how GitVersion determines the current version name for a branch, but to use - simply:

Add these 2 properties to your Build.cs class

  [GitRepository] readonly GitRepository GitRepository;
  [GitVersion] readonly GitVersion GitVersion;

Add the .SetVersion(GitVersion.NuGetVersionV2) to your Pack Target

 DotNetPack(s => s
           ---
          .SetVersion(GitVersion.NuGetVersionV2)
          .SetNoDependencies(true)
          .SetOutputDirectory(ArtifactsDirectory / "nuget"));

Now GitVersion will work it’s magic to determine the current version name!

Publishing to a NuGet Repository with Nuke

Now that we have our source compiling and our package versioned and waiting in our artifacts folder, lets use Nuke to push it to a repository where it can be used by others.

In order to make this as flexible as possible, we’ll pass the nuget repository’s url and auth_key as parameters to the Nuke build script. Inside the script Nuke makes it easy for us to

Require that it’s a Release build
Require that the url and auth_key have been set
Get values from commandline / environment using c# fields
Add 2 Fields to to your Build file with the [Parameter] attribute

    [Parameter] string NugetApiUrl = "https://api.nuget.org/v3/index.json"; //default
    [Parameter] string NugetApiKey;

Add a Push Target to your Build file

 Target Push => _ => _
       .DependsOn(Pack)
       .Requires(() => NugetApiUrl)
       .Requires(() => NugetApiKey)
       .Requires(() => Configuration.Equals(Configuration.Release))
       .Executes(() =>
       {
           GlobFiles(NugetDirectory, "*.nupkg")
               .NotEmpty()
               .Where(x => !x.EndsWith("symbols.nupkg"))
               .ForEach(x =>
               {
                   DotNetNuGetPush(s => s
                       .SetTargetPath(x)
                       .SetSource(NugetApiUrl)
                       .SetApiKey(NugetApiKey)
                   );
               });
       });

Push to a NuGet repository

> ./build.ps1 Push --NugetApiUrl "https://api.nuget.org/v3/index.json" --NugetApiKey "yoursecretkey"

Using AppVeyor for Continuous Integration and Deployment

AppVeyor is a CI/CD tool with good support for windows/dotnet (and linux). For open source projects you can setup a free account to build and deploy every time you publish changes to source control. Here we’re going to use GitHub, but you could configure something similar with Azure DevOps

We’re going to build a popular workflow as described by Andrew Lock and Jimmy Bogard. It uses two seperate nuget repositories to publish under different conditions:

Build all commits on master branch and publish to MyGet.org
- Useful for reviewing and testing packages before releasing to the world
- Nightly/experimental builds
If a commit is tagged we want to build and publish to NuGet.org
- Allows us to use git tags to control versioning and our intent to publish to the world
Build-only for pull request
- We don’t want to publish a nuget package on pull requests, but we will confirm that the pull request builds

We can accomplish all of this with a simple appveyor.yml file.

Add appveyor.yml to our root folder

version: '{build}'
image: Ubuntu
environment:
MyGetApiKey:
    secure: 56nW3KcP4naYX9mlsVEIKLj5xPdfmpt6lMALR6wQmorRQOaoUOtlwMZ2V0BtGTAM
NugetApiKey:
    secure: /54XAunyBETRa1Fp/qSrwvebSnTAcHDO2OVZ+exMtQtOtrBzHKvp4RC1AB8RD2PQ
pull_requests:
do_not_increment_build_number: true
branches:
only:
- master
nuget:
disable_publish_on_pr: true
build_script:
- ps: ./build.ps1
test: off
deploy_script:
- ps: ./build.ps1 Pack
- ps: ./build.ps1 Push --NugetApiUrl "https://www.myget.org/F/cfrenzel-ci/api/v2/package" --NugetApiKey $env:MyGetApiKey
- ps: | 
    if ($env:APPVEYOR_REPO_TAG  -eq "true"){
        ./build.ps1 Push --NugetApiUrl "https://api.nuget.org/v3/index.json" --NugetApiKey $env:NugetApiKey
    }

There are a couple of important bits here

build_script:
    - ps: ./build.ps1

This tells appveyor to call our Nuke build script during the build phase

 deploy_script:
    - ps: ./build.ps1 Pack
    - ps: ./build.ps1 Push --NugetApiUrl "https://www.myget.org/F/cfrenzel-ci/api/v2/package" --NugetApiKey $env:MyGetApiKey
    - ps: | 
        if ($env:APPVEYOR_REPO_TAG  -eq "true"){
            ./build.ps1 Push --NugetApiUrl "https://api.nuget.org/v3/index.json" --NugetApiKey $env:NugetApiKey
        }

This tells appveyor to run a series of powershell commands during the Deploy phase.

We call Pack to create the nuget package.
Then we Push it to MyGet.org using secure environment variables that we declared earlier
Then we check an appveyor environement variable APPVEYOR_REPO_TAG to see if the branch has a tag
If it does we Push to NuGet.Org

For a full working example with multiple nuget packages in a single solution checkout out my repo:

https://github.com/cfrenzel/Eventfully/blob/master/build/Build.cs https://github.com/cfrenzel/Eventfully/blob/master/appveyor.yml https://github.com/cfrenzel/Eventfully