

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <title>Evan Dontje</title>
    
    <meta name="author" content="Evan Dontje">

    <!-- Le HTML5 shim, for IE6-8 support of HTML elements -->
    <!--[if lt IE 9]>
      <script src="http://html5shim.googlecode.com/svn/trunk/html5.js"></script>
    <![endif]-->

    <!-- Le styles -->
    <link href='http://fonts.googleapis.com/css?family=Vollkorn' rel='stylesheet' type='text/css'>
    <link href="/assets/themes/twitter/bootstrap/css/bootstrap.min.css" rel="stylesheet">
    <link href="/assets/themes/twitter/bootstrap/css/bootstrap.responsive.css" rel="stylesheet">
    <link href="/assets/themes/twitter/css/pygments.css" rel="stylesheet" type="text/css" media="all">
    <link href="/assets/themes/twitter/css/style.css?body=1" rel="stylesheet" type="text/css" media="all">

    <!-- Le fav and touch icons -->
    <link rel="shortcut icon" href="/images/favicon.ico">
    <link rel="apple-touch-icon" href="/images/apple-touch-icon-iphone.png">
    <link rel="apple-touch-icon" sizes="72x72" href="/images/apple-touch-icon-ipad.png">
    <link rel="apple-touch-icon" sizes="114x114" href="/images/apple-touch-icon-iphone4.png">
  </head>

  <body>

    <div class="head">
      <div class="container">
      <div class="row">
        <div class="left-header">
          <h1><a class="brand" href="/">Evan Dontje</a></h1>
        </div>
        <div class="right-header">
          <ul class="nav">
            <li class="buffer">{</li>
            
              <li><a href="/">Home </a></li>
            
            
            
            


  
    
      
      	
      	<li><a href="/about_me/">About Me</a></li>
      	
      
    
  
    
      
      	
      	<li><a href="/all_posts/">All Posts</a></li>
      	
      
    
  
    
      
    
  
    
  
    
      
    
  
    
  
    
  
    
  
    
  



            <li class="buffer">}</li>
          </ul>
        </div>
        </div>
      </div>
    </div>

    <div class="container">

      <div class="content">
          <div class="post-content">
            




   <h2 class="title"><a href="/2018/05/14/full-framework-wsfederation-to-owin-conversion/">Full Framework WSFederation to OWIN Conversion</a></h2>
   <p class="author">
    <span class="date">14 May 2018</span>
  </p>
  <div class="content">
    <p>If you have been using WSFederation in a .net web application for more than a year or two, chances are that it is configured using the <code class="highlighter-rouge">Microsoft.IdentityModel.Web</code> or <code class="highlighter-rouge">System.IdentityModel.Services</code> libraries. Two HTTP modules are added to the application, <code class="highlighter-rouge">WSFederationAuthenticationModule</code> and <code class="highlighter-rouge">SessionAuthenticationModule</code>, to handle the WSFederation protocol and configuration was done by inheriting those classes or configuring on application start via the web.config. However, in newer versions of asp.net using “middleware” is preferred by using <a href="https://docs.microsoft.com/en-us/aspnet/aspnet/overview/owin-and-katana/an-overview-of-project-katana#the-open-web-interface-for-net-owin">OWIN</a> in both full framework applications and .net core. The purpose of OWIN is to abstract the underlying web server from the web application. HttpModules are tightly coupled to <code class="highlighter-rouge">System.Web</code> and therefore the IIS webserver. Using OWIN does require some configuration and setup changes which I will detail in this post.</p>

<h3 id="basic-owin-setup">Basic OWIN setup</h3>
<p>First, if you don’t already have OWIN configured for your application install the <code class="highlighter-rouge">Microsoft.Owin</code> and <code class="highlighter-rouge">Microsoft.Owin.Host.SystemWeb</code> nuget packages. Then add a startup class like the one below to your application:</p>

<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">using</span> <span class="nn">System</span><span class="p">;</span>
<span class="k">using</span> <span class="nn">System.Threading.Tasks</span><span class="p">;</span>
<span class="k">using</span> <span class="nn">Microsoft.Owin</span><span class="p">;</span>
<span class="k">using</span> <span class="nn">Owin</span><span class="p">;</span>

<span class="na">[assembly: OwinStartup(typeof(OwinApp.Startup))]</span>
<span class="k">namespace</span> <span class="nn">OwinApp</span>
<span class="p">{</span>
    <span class="k">public</span> <span class="k">class</span> <span class="nc">Startup</span>
    <span class="p">{</span>
        <span class="k">public</span> <span class="k">void</span> <span class="nf">Configuration</span><span class="p">(</span><span class="n">IAppBuilder</span> <span class="n">app</span><span class="p">)</span>
        <span class="p">{</span>
        <span class="p">}</span>
    <span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>

<h3 id="convert-sessionauthenticationmodule-into-owin-configuration">Convert SessionAuthenticationModule into OWIN configuration</h3>
<p>Once OWIN is installed, we can begin configuring WSFederation. Previously a <code class="highlighter-rouge">SessionAuthenticationModule</code> would have been customized to set up properties for the cookies that will store session information:</p>

<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">public</span> <span class="k">class</span> <span class="nc">CustomSessionAuthenticationModule</span> <span class="p">:</span> <span class="n">SessionAuthenticationModule</span>
<span class="p">{</span>
  <span class="k">protected</span> <span class="k">override</span> <span class="k">void</span> <span class="nf">InitializePropertiesFromConfiguration</span><span class="p">()</span>
  <span class="p">{</span>
      <span class="n">CookieHandler</span><span class="p">.</span><span class="n">RequireSsl</span> <span class="p">=</span> <span class="k">true</span><span class="p">;</span>
      <span class="n">CookieHandler</span><span class="p">.</span><span class="n">Name</span> <span class="p">=</span> <span class="s">"FederatedAuthCookie"</span><span class="p">;</span>
  <span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>

<p>and configured as a HTTP module in the web.config:</p>

<div class="language-xml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nt">&lt;modules&gt;</span>
  <span class="nt">&lt;add</span> <span class="na">name=</span><span class="s">"SessionAuthenticationModule"</span> <span class="na">type=</span><span class="s">"MyApp.CustomSessionAuthenticationModule, MyApp"</span> <span class="na">preCondition=</span><span class="s">"managedHandler"</span> <span class="nt">/&gt;</span>
<span class="nt">&lt;/modules&gt;</span>
</code></pre></div></div>

<p>In the OWIN pipeline, we’ll configure the cookie using <code class="highlighter-rouge">CookieAuthentication</code> classes and helper methods.</p>

<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">public</span> <span class="k">class</span> <span class="nc">Startup</span>
<span class="p">{</span>
    <span class="k">public</span> <span class="k">void</span> <span class="nf">Configuration</span><span class="p">(</span><span class="n">IAppBuilder</span> <span class="n">app</span><span class="p">)</span>
    <span class="p">{</span>
        <span class="n">app</span><span class="p">.</span><span class="nf">UseCookieAuthentication</span><span class="p">(</span> <span class="k">new</span> <span class="n">CookieAuthenticationOptions</span>
        <span class="p">{</span>
          <span class="c1">// converted from the CookieHandler.Name = "FederatedAuthCookie"; line in SessionAuthenticationModule</span>
          <span class="n">CookieName</span> <span class="p">=</span> <span class="s">"FederatedAuthCookie"</span><span class="p">,</span>
          <span class="c1">// converted from the CookieHandler.RequireSsl = true; line in SessionAuthenticationModule</span>
          <span class="n">CookieSecure</span> <span class="p">=</span> <span class="n">CookieSecureOption</span><span class="p">.</span><span class="n">Always</span>
        <span class="p">}</span> <span class="p">);</span>
    <span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>

<h3 id="convert-wsfederationauthenticationmodule-into-owin-configuration">Convert WSFederationAuthenticationModule into OWIN configuration</h3>
<p>Next, we’ll convert our custom <code class="highlighter-rouge">WSFederationAuthenticationModule</code> to use the <code class="highlighter-rouge">WsFederationAuthenticationMiddleware</code> from the OWIN pipeline.</p>

<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">public</span> <span class="k">class</span> <span class="nc">CustomWsFederationAuthenticationModule</span> <span class="p">:</span> <span class="n">WSFederationAuthenticationModule</span>
<span class="p">{</span>
  <span class="k">protected</span> <span class="k">override</span> <span class="k">void</span> <span class="nf">InitializeModule</span><span class="p">(</span> <span class="n">HttpApplication</span> <span class="n">context</span> <span class="p">)</span>
  <span class="p">{</span>
      <span class="k">base</span><span class="p">.</span><span class="nf">InitializeModule</span><span class="p">(</span> <span class="n">context</span> <span class="p">);</span>

      <span class="n">RedirectingToIdentityProvider</span> <span class="p">+=</span> <span class="n">OnRedirectingToIdentityProvider</span><span class="p">;</span>
  <span class="p">}</span>

  <span class="k">protected</span> <span class="k">override</span> <span class="k">void</span> <span class="nf">InitializePropertiesFromConfiguration</span><span class="p">()</span>
  <span class="p">{</span>
      <span class="n">Issuer</span> <span class="p">=</span> <span class="n">InstanceWideSettings</span><span class="p">.</span><span class="n">BaseStsUrl</span><span class="p">;</span>
  <span class="p">}</span>

  <span class="k">private</span> <span class="k">void</span> <span class="nf">OnRedirectingToIdentityProvider</span><span class="p">(</span> <span class="kt">object</span> <span class="n">sender</span><span class="p">,</span> <span class="n">RedirectingToIdentityProviderEventArgs</span> <span class="n">args</span> <span class="p">)</span>
  <span class="p">{</span>
      <span class="c1">// setting the realm in the OnRedirecting event allows it to be dynamic for multi-tenant applications</span>
      <span class="n">args</span><span class="p">.</span><span class="n">SignInRequestMessage</span><span class="p">.</span><span class="n">Realm</span> <span class="p">=</span> <span class="n">Settings</span><span class="p">.</span><span class="n">BaseUrl</span><span class="p">;</span>
  <span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>The code above will be removed and replaced with the <code class="highlighter-rouge">UseWSFederationAuthentication</code> helper below</p>

<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">public</span> <span class="k">void</span> <span class="nf">Configuration</span><span class="p">(</span><span class="n">IAppBuilder</span> <span class="n">app</span><span class="p">)</span>
<span class="p">{</span>
   <span class="p">...</span>
   <span class="n">app</span><span class="p">.</span><span class="nf">UseWsFederationAuthentication</span><span class="p">(</span> <span class="k">new</span> <span class="n">WsFederationAuthenticationOptions</span>
   <span class="p">{</span>
     <span class="c1">// Pulls in STS Url and other metadata (like signing certificates)</span>
     <span class="n">MetadataAddress</span> <span class="p">=</span> <span class="n">Settings</span><span class="p">.</span><span class="n">StsMetadataUrl</span><span class="p">,</span>
     <span class="n">Notifications</span> <span class="p">=</span> <span class="k">new</span> <span class="n">WsFederationAuthenticationNotifications</span>
     <span class="p">{</span>
         <span class="c1">// replaces the OnRedirectingToIdentityProvider event</span>
         <span class="n">RedirectToIdentityProvider</span> <span class="p">=</span> <span class="n">notification</span> <span class="p">=&gt;</span>
         <span class="p">{</span>
           <span class="n">notification</span><span class="p">.</span><span class="n">ProtocolMessage</span><span class="p">.</span><span class="n">Wtrealm</span> <span class="p">=</span> <span class="n">Settings</span><span class="p">.</span><span class="n">PresentationUrlRoot</span><span class="p">;</span>
         <span class="p">}</span>
     <span class="p">};</span>
     <span class="c1">// Name this authentication type (for WIF)</span>
     <span class="n">AuthenticationType</span> <span class="p">=</span> <span class="n">WsFederationAuthenticationDefaults</span><span class="p">.</span><span class="n">AuthenticationType</span><span class="p">,</span>
     <span class="c1">// Tells the pipeline to use a cookie authenication we configured above to store the WIF session</span>
     <span class="n">SignInAsAuthenticationType</span> <span class="p">=</span> <span class="n">CookieAuthenticationDefaults</span><span class="p">.</span><span class="n">AuthenticationType</span>
   <span class="p">}</span> <span class="p">);</span>
<span class="p">}</span>
</code></pre></div></div>

<h3 id="move-globalasaxcs-wsfederation-configuration-into-owin-configuration">Move Global.asax.cs WSFederation configuration into OWIN configuration</h3>
<p>Now that we’ve converted the two WSFederation HttpModules we can finish configuring the OWIN pipeline by converting either the WSFederation configuration in the web.config or that was configured on application start. In my case, I preferred to set up WSFederation in code using the <code class="highlighter-rouge">FederationConfigurationCreated</code> event like the code below:</p>

<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">FederatedAuthentication</span><span class="p">.</span><span class="n">FederationConfigurationCreated</span> <span class="p">+=</span> <span class="p">(</span> <span class="n">sender</span><span class="p">,</span> <span class="n">args</span> <span class="p">)</span> <span class="p">=&gt;</span>
<span class="p">{</span>
  <span class="n">args</span><span class="p">.</span><span class="n">FederationConfiguration</span><span class="p">.</span><span class="n">IdentityConfiguration</span><span class="p">.</span><span class="n">AudienceRestriction</span><span class="p">.</span><span class="n">AudienceMode</span> <span class="p">=</span> <span class="n">ystem</span><span class="p">.</span><span class="n">IdentityModel</span><span class="p">.</span><span class="n">Selectors</span><span class="p">.</span><span class="n">AudienceUriMode</span><span class="p">.</span><span class="n">Always</span><span class="p">;</span>

  <span class="c1">// this method loads the list of relying parties for a multi-tenant application.</span>
  <span class="n">List</span><span class="p">&lt;</span><span class="kt">string</span><span class="p">&gt;</span> <span class="n">relyingParties</span> <span class="p">=</span> <span class="nf">GetRelyingParties</span><span class="p">();</span>
  <span class="n">relyingParties</span><span class="p">.</span><span class="nf">ForEach</span><span class="p">(</span> <span class="n">rp</span> <span class="p">=&gt;</span> <span class="n">args</span><span class="p">.</span><span class="n">FederationConfiguration</span><span class="p">.</span><span class="n">IdentityConfiguration</span><span class="p">.</span><span class="n">AudienceRestriction</span><span class="p">.</span><span class="n">AllowedAudienceUris</span><span class="p">.</span><span class="nf">Add</span><span class="p">(</span> <span class="k">new</span> <span class="nf">Uri</span><span class="p">(</span> <span class="n">rp</span>  <span class="p">)</span> <span class="p">);</span>

  <span class="c1">// This code loads the metadata url, parses it and and updates the configuration with details from it like the signing certificates</span>
  <span class="n">args</span><span class="p">.</span><span class="n">FederationConfiguration</span><span class="p">.</span><span class="n">IdentityConfiguration</span><span class="p">.</span><span class="n">IssuerNameRegistry</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">CustomMetadataParser</span><span class="p">(</span> <span class="n">Settings</span><span class="p">.</span><span class="n">StsMetadataUrl</span> <span class="p">);</span>
<span class="p">};</span>
</code></pre></div></div>

<p>The items configured above can be added to the <code class="highlighter-rouge">UseWsFederationAuthentication</code> configuration:</p>

<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">public</span> <span class="k">void</span> <span class="nf">Configuration</span><span class="p">(</span><span class="n">IAppBuilder</span> <span class="n">app</span><span class="p">)</span>
<span class="p">{</span>
   <span class="p">...</span>
   <span class="n">app</span><span class="p">.</span><span class="nf">UseWsFederationAuthentication</span><span class="p">(</span> <span class="k">new</span> <span class="n">WsFederationAuthenticationOptions</span>
   <span class="p">{</span>
     <span class="p">...</span>
     <span class="n">TokenValidationParameters</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">TokenValidationParameters</span><span class="p">()</span>
     <span class="p">{</span>
         <span class="c1">// this replaces the IdentityConfiguration.AudienceRestriction setup</span>
         <span class="n">ValidAudiences</span> <span class="p">=</span> <span class="nf">GetRelyingParties</span><span class="p">(),</span>
         <span class="n">ValidateAudience</span> <span class="p">=</span> <span class="k">true</span>
     <span class="p">},</span>
     <span class="c1">// Pulls in STS Url and other metadata (like signing certificates) so we don't have to do custom metadata parsing</span>
     <span class="n">MetadataAddress</span> <span class="p">=</span> <span class="n">Settings</span><span class="p">.</span><span class="n">StsMetadataUrl</span><span class="p">,</span>
     <span class="p">...</span>
   <span class="p">}</span> <span class="p">);</span>
<span class="p">}</span>
</code></pre></div></div>

<p>Additionally, in the Global.asax.cs file if you wanted to have access to WSFederation events you could declare special methods on your <code class="highlighter-rouge">HttpApplication</code> class and those would be invoked while the WSFederation protocol was executing. Two examples that I’ve used are shown below:</p>

<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">void</span> <span class="nf">WSFederationAuthenticationModule_SessionSecurityTokenCreated</span><span class="p">(</span> <span class="kt">object</span> <span class="n">sender</span><span class="p">,</span> <span class="n">SessionSecurityTokenCreatedEventArgs</span> <span class="n">e</span> <span class="p">)</span>
<span class="p">{</span>
   <span class="c1">// extend the expiration of the session cookie to make it last 1 year</span>
   <span class="n">TimeSpan</span> <span class="n">expiration</span> <span class="p">=</span> <span class="n">TimeSpan</span><span class="p">.</span><span class="nf">FromYears</span><span class="p">(</span> <span class="m">1</span> <span class="p">);</span>
   <span class="n">e</span><span class="p">.</span><span class="n">SessionToken</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">SessionSecurityToken</span><span class="p">(</span> <span class="n">e</span><span class="p">.</span><span class="n">SessionToken</span><span class="p">.</span><span class="n">ClaimsPrincipal</span><span class="p">,</span> <span class="n">e</span><span class="p">.</span><span class="n">SessionToken</span><span class="p">.</span><span class="n">Context</span><span class="p">,</span> <span class="n">now</span><span class="p">,</span> <span class="n">now</span><span class="p">.</span><span class="nf">Add</span><span class="p">(</span> <span class="n">expiration</span> <span class="p">)</span> <span class="p">)</span> <span class="p">{</span> <span class="n">IsPersistent</span> <span class="p">=</span> <span class="k">true</span> <span class="p">};</span>

   <span class="n">e</span><span class="p">.</span><span class="n">WriteSessionCookie</span> <span class="p">=</span> <span class="k">true</span><span class="p">;</span>
<span class="p">}</span>

<span class="k">void</span> <span class="nf">WSFederationAuthenticationModule_RedirectingToIdentityProvider</span><span class="p">(</span> <span class="kt">object</span> <span class="n">sender</span><span class="p">,</span> <span class="n">RedirectingToIdentityProviderEventArgs</span> <span class="n">e</span> <span class="p">)</span>
<span class="p">{</span>
   <span class="c1">// add client id parameter to outgoing wsfederation request</span>
   <span class="n">e</span><span class="p">.</span><span class="n">SignInRequestMessage</span><span class="p">.</span><span class="n">Parameters</span><span class="p">.</span><span class="nf">Add</span><span class="p">(</span> <span class="s">"client_id"</span><span class="p">,</span> <span class="n">Settings</span><span class="p">.</span><span class="n">ClientId</span> <span class="p">);</span>
<span class="p">}</span>
</code></pre></div></div>

<p>Again, these items can be replicated in the <code class="highlighter-rouge">UseWsFederationAuthentication</code> configuration:</p>

<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">public</span> <span class="k">void</span> <span class="nf">Configuration</span><span class="p">(</span><span class="n">IAppBuilder</span> <span class="n">app</span><span class="p">)</span>
<span class="p">{</span>
   <span class="p">...</span>
   <span class="n">app</span><span class="p">.</span><span class="nf">UseWsFederationAuthentication</span><span class="p">(</span> <span class="k">new</span> <span class="n">WsFederationAuthenticationOptions</span>
   <span class="p">{</span>
     <span class="p">...</span>
     <span class="n">Notifications</span> <span class="p">=</span> <span class="k">new</span> <span class="n">WsFederationAuthenticationNotifications</span>
     <span class="p">{</span>
         <span class="c1">// replaces the WSFederationAuthenticationModule_RedirectingToIdentityProvider method</span>
         <span class="n">RedirectToIdentityProvider</span> <span class="p">=</span> <span class="n">notification</span> <span class="p">=&gt;</span>
         <span class="p">{</span>
           <span class="n">notification</span><span class="p">.</span><span class="n">ProtocolMessage</span><span class="p">.</span><span class="n">Parameters</span><span class="p">.</span><span class="nf">Add</span><span class="p">(</span> <span class="s">"client_id"</span><span class="p">,</span> <span class="n">Settings</span><span class="p">.</span><span class="n">ClientId</span> <span class="p">);</span>
         <span class="p">},</span>
         <span class="c1">// replaces the WSFederationAuthenticationModule_SessionSecurityTokenCreated method</span>
         <span class="n">SecurityTokenValidated</span> <span class="p">=</span> <span class="n">notification</span> <span class="p">=&gt;</span>
         <span class="p">{</span>
           <span class="kt">var</span> <span class="n">newAuthenticationProperties</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">AuthenticationProperties</span><span class="p">(</span> <span class="n">authenticationTicket</span><span class="p">.</span><span class="n">Properties</span><span class="p">.</span><span class="n">Dictionary</span> <span class="p">);</span>

           <span class="n">DateTime</span> <span class="n">now</span> <span class="p">=</span> <span class="n">DateTime</span><span class="p">.</span><span class="n">UtcNow</span><span class="p">;</span>
           <span class="n">TimeSpan</span> <span class="n">expiration</span> <span class="p">=</span> <span class="n">TimeSpan</span><span class="p">.</span><span class="nf">FromYears</span><span class="p">(</span> <span class="m">1</span> <span class="p">);</span>

           <span class="n">newAuthenticationProperties</span><span class="p">.</span><span class="n">IssuedUtc</span> <span class="p">=</span> <span class="n">now</span><span class="p">;</span>
           <span class="n">newAuthenticationProperties</span><span class="p">.</span><span class="n">ExpiresUtc</span> <span class="p">=</span> <span class="n">now</span><span class="p">.</span><span class="nf">Add</span><span class="p">(</span> <span class="n">expiration</span> <span class="p">);</span>
           <span class="n">authenticationProperties</span><span class="p">.</span><span class="n">IsPersistent</span> <span class="p">=</span> <span class="k">true</span><span class="p">;</span>

           <span class="k">return</span> <span class="k">new</span> <span class="nf">AuthenticationTicket</span><span class="p">(</span> <span class="n">claimsIdentity</span><span class="p">,</span> <span class="n">authenticationProperties</span> <span class="p">);</span>
         <span class="p">}</span>
     <span class="p">};</span>
     <span class="p">...</span>
   <span class="p">}</span> <span class="p">);</span>
<span class="p">}</span>
</code></pre></div></div>

<h3 id="wrap-up">Wrap up</h3>
<p>At this point, all old WSFederation code is replaced and WSFederation actions are handled using the OWIN pipeline. One thing to note - we are not able to re-use existing sessions so existing user sessions will be invalidated by this code change. Once the user logs in again at the STS they’ll be issued a new cookie that will work with the OWIN pipeline cookie authentication code.</p>

  </div>
  <p class="post-separator"><p>
  <p class="post-separator-padding"><p>


   <h2 class="title"><a href="/2018/05/04/setting-up-code-analysis-in-vs-2017-projects/">Setting up Code Analysis in Visual Studio 2017 projects</a></h2>
   <p class="author">
    <span class="date">04 May 2018</span>
  </p>
  <div class="content">
    <p>In older versions of Visual Studio, FXCop was the standard for static analysis in .NET. In Visual Studio 2017, along with the release of the roslyn compiler, the landscape is different. Static analysis is no longer something you install on the machine and configure in the project, it’s delivered via nuget packages. In this post, I’ll lay out how I suggest setting up static analysis for C# projects. The setup should work equally well for Full Framework and .NET Core.</p>

<h3 id="which-analyzers-to-use">Which analyzers to use</h3>

<p>The first question to answer is which code analyzers to use for your project. Before we just had FXCop but now you can google for “roslyn analyzers” and find a plethora of options. I typically stick to the <a href="https://github.com/dotnet/roslyn-analyzers">stock Microsoft options</a> but there are plenty of third-party options too like <a href="https://github.com/DotNetAnalyzers/StyleCopAnalyzers">Style Cop</a>.</p>

<p>In this example, I’ll use the <a href="https://www.nuget.org/packages/Microsoft.CodeAnalysis.FxCopAnalyzers/">Microsoft.CodeAnalysis.FxCopAnalyzers</a>, which is a meta package of four other analyzer packages:</p>
<ul>
  <li><a href="https://www.nuget.org/packages/Microsoft.CodeQuality.Analyzers">Microsoft.CodeQuality.Analyzers</a>
    <ul>
      <li>The bulk of the “classic” FXCop errors are here. For instance, implementing <code class="highlighter-rouge">IDisposable</code> properly and passing URIs instead of strings are both checked in this package.</li>
    </ul>
  </li>
  <li><a href="https://www.nuget.org/packages/Microsoft.NetCore.Analyzers/">Microsoft.NetCore.Analyzers</a>
    <ul>
      <li>.NET core specific warnings/errors appear here, but many are more generic like requiring <code class="highlighter-rouge">ICultureInfo</code> to be passed to methods that can accept it.</li>
    </ul>
  </li>
  <li><a href="https://www.nuget.org/packages/Microsoft.NetFramework.Analyzers/">Microsoft.NetFramework.Analyzers</a>
    <ul>
      <li>Full framework .net warnings are checked here like handling <code class="highlighter-rouge">ISerializable</code> correctly.</li>
    </ul>
  </li>
  <li><a href="https://www.nuget.org/packages/Text.Analyzers/">Text.Analyzers</a>
    <ul>
      <li>This package provides some basic spell checking (disabled by default).</li>
    </ul>
  </li>
</ul>

<h3 id="install-and-configure-analyzers">Install and configure analyzers</h3>

<p>To get started, install the <code class="highlighter-rouge">Microsoft.CodeAnalysis.FxCopAnalyzers</code> nuget package into all projects in your solution. This by itself will generate warnings and do the code analysis using the default ruleset. I want to take it one step further to use shared rulesets for all projects in the solution.</p>

<h3 id="add-shared-rulesets">Add shared rulesets</h3>

<p>To utilize shared rulesets, edit the <code class="highlighter-rouge">csproj</code> files to reference a shared analyzer ruleset. This file will be used to configure which rules are enabled/disabled in your solution. I prefer to have two ruleset files, one for production source code and one for tests so that I can be flexible on the rules I use to analyze test code.</p>

<ol>
  <li>
    <p>Edit the projects by adding the following lines to the csproj. They can go anywhere but I typically add them underneath the TargetFramework/RootNamespace property group. I use a tools folder at the root of my git repository but you can put the ruleset file anywhere.</p>

    <div class="language-xml highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nt">&lt;PropertyGroup&gt;</span>
   <span class="nt">&lt;CodeAnalysisRuleSet&gt;</span>..\..\tools\Source.ruleset<span class="nt">&lt;/CodeAnalysisRuleSet&gt;</span>
 <span class="nt">&lt;/PropertyGroup&gt;</span>
</code></pre></div>    </div>
  </li>
  <li>
    <p>Add a ruleset file in the location specified above. The exact contents of the ruleset file will vary but if you use FxCop analyzers a good place to start is with this default file - <a href="https://gist.github.com/dontjee/4a151dea7bc1169f9dd051da70bec35e">https://gist.github.com/dontjee/4a151dea7bc1169f9dd051da70bec35e</a>. It enables some of the most important rules as warnings.</p>
  </li>
  <li>
    <p>Repeat the process above for test projects, or any other projects you want to use different rulesets, using a different ruleset file.</p>

    <div class="language-xml highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nt">&lt;PropertyGroup&gt;</span>
   <span class="nt">&lt;CodeAnalysisRuleSet&gt;</span>..\..\tools\Tests.ruleset<span class="nt">&lt;/CodeAnalysisRuleSet&gt;</span>
 <span class="nt">&lt;/PropertyGroup&gt;</span>
</code></pre></div>    </div>
  </li>
</ol>

<h3 id="build-and-fixignore-warnings">Build and fix/ignore warnings</h3>

<p>Now that we’ve configured the ruleset files, the next step is to do a <code class="highlighter-rouge">rebuild</code> of the solution and fix or ignore any warnings that pop up. One rule that I often disable is <code class="highlighter-rouge">CA2007 Do not directly await a Task without calling ConfigureAwait</code>. This makes sense when building libraries to be consumed in other projects but when building applications, this rule isn’t necessary. To disable the rule follow the steps below</p>

<ol>
  <li>Find the rule in the analyzer list under Dependencies-&gt;Analyzers-&gt;[The-Analyzer-Name]. In the case of <code class="highlighter-rouge">CA2007</code>, the analyzer name is <code class="highlighter-rouge">Microsoft.CodeQuality.Analyzers</code>.
 <img src="/images/2018/5/code_analyzers_location.png" alt="Dual_write-example" /></li>
  <li>
    <p>Under the code analyzer, find the rule you want to disable and right-click on it and set the <code class="highlighter-rouge">Rule Set Severity</code> to <code class="highlighter-rouge">None</code>.
 <img src="/images/2018/5/code_analyzer_rule_right_click.png" alt="Code-Analyzer-Rule-Right-Click" /></p>

    <p>This adds the following block to the corresponding ruleset file to disable the rule:</p>

    <div class="language-xml highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nt">&lt;Rules</span> <span class="na">AnalyzerId=</span><span class="s">"Microsoft.CodeQuality.Analyzers"</span> <span class="na">RuleNamespace=</span><span class="s">"Microsoft.CodeQuality.Analyzers"</span><span class="nt">&gt;</span>
   <span class="nt">&lt;Rule</span> <span class="na">Id=</span><span class="s">"CA2007"</span> <span class="na">Action=</span><span class="s">"None"</span> <span class="nt">/&gt;</span>
 <span class="nt">&lt;/Rules&gt;</span>
</code></pre></div>    </div>
  </li>
</ol>

<p>Repeat the process above for all rules you wish to disable or fix the warnings that show up. Once that’s done, the code analysis setup is complete for your solution.</p>

<h3 id="extra-credit---set-up-builds-to-fail-on-analyzer-warnings">Extra Credit - set up builds to fail on analyzer warnings</h3>

<p>Now that you have a clean build with no warnings I suggest configuring the continuous build (I hope you have one!) to report warnings as errors so that the build will fail if any new code analysis violations show up. To do this, add the following MSBuild property to the compile step of your build - <code class="highlighter-rouge">/p:TreatWarningsAsErrors="true"</code>.</p>

  </div>
  <p class="post-separator"><p>
  <p class="post-separator-padding"><p>


   <h2 class="title"><a href="/2017/12/20/avoid-dual-writes-with-sql-server-and-change-tracking-part-2/">Avoid dual writes with sql server and change tracking - Part 2</a></h2>
   <p class="author">
    <span class="date">20 December 2017</span>
  </p>
  <div class="content">
    <p>In my last post I suggested using the <a href="/2017/12/13/avoid-dual-writes-with-sql-server-and-change-tracking-part-1/">database stream writer pattern</a> to avoid writing to multiple data stores outside of a transaction from your application. This post will detail the implementation. For my example the application is a  c# application writing to SQL Server and tracking changes using the Change Tracking feature. The data model for this example is similar to Youtube. It contains users, channels and media. A user has one or more channels which in turn have one or more pieces of media associated with them. The full schema is contained in <a href="https://gist.github.com/dontjee/aebaabd7737f113382a6f0384015232c">this gist</a>.</p>

<p>At a high level, there are 3 pieces of the architecture to consider.</p>
<ol>
  <li>The primary application which will write to the SQL database. It will not deal with writes to the downstream data stores and will largely be ignored by this post.</li>
  <li>The SQL Server database which will be configured to track changes to all necessary tables.</li>
  <li>The application to monitor the change tracking stream from the database and push updates to their datastore (cache, search, etc).</li>
</ol>

<h4 id="database-implementation">Database Implementation</h4>

<p>First <a href="https://docs.microsoft.com/en-us/sql/relational-databases/track-changes/about-change-tracking-sql-server">change tracking</a> must be enabled on the SQL Server database and tables. The SQL below enables change tracking on all three tables configured to retain changes for seven days and enables automatic cleanup of expired changes.</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">ALTER</span> <span class="k">DATABASE</span> <span class="n">BlogPostExample</span>
<span class="k">SET</span> <span class="n">CHANGE_TRACKING</span> <span class="o">=</span> <span class="k">ON</span>  
  <span class="p">(</span><span class="n">CHANGE_RETENTION</span> <span class="o">=</span> <span class="mi">7</span> <span class="n">DAYS</span><span class="p">,</span> <span class="n">AUTO_CLEANUP</span> <span class="o">=</span> <span class="k">ON</span><span class="p">)</span>  

<span class="k">ALTER</span> <span class="k">TABLE</span> <span class="n">dbo</span><span class="p">.</span><span class="n">UserAccount</span>
<span class="n">ENABLE</span> <span class="n">CHANGE_TRACKING</span>  
<span class="k">WITH</span> <span class="p">(</span><span class="n">TRACK_COLUMNS_UPDATED</span> <span class="o">=</span> <span class="k">ON</span><span class="p">)</span>

<span class="k">ALTER</span> <span class="k">TABLE</span> <span class="n">dbo</span><span class="p">.</span><span class="n">Channel</span>
<span class="n">ENABLE</span> <span class="n">CHANGE_TRACKING</span>  
<span class="k">WITH</span> <span class="p">(</span><span class="n">TRACK_COLUMNS_UPDATED</span> <span class="o">=</span> <span class="k">ON</span><span class="p">)</span>

<span class="k">ALTER</span> <span class="k">TABLE</span> <span class="n">dbo</span><span class="p">.</span><span class="n">Media</span>
<span class="n">ENABLE</span> <span class="n">CHANGE_TRACKING</span>  
<span class="k">WITH</span> <span class="p">(</span><span class="n">TRACK_COLUMNS_UPDATED</span> <span class="o">=</span> <span class="k">ON</span><span class="p">)</span>
</code></pre></div></div>

<p>One final table is required to track the current position in the change tracking stream our monitor has consumed. The following SQL will create the table and initialize it with the minimum change tracking version currently in the database:</p>

<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">CREATE</span> <span class="k">TABLE</span> <span class="p">[</span><span class="n">dbo</span><span class="p">].[</span><span class="n">CacheChangeTrackingHistory</span><span class="p">]</span> <span class="p">(</span>
   <span class="p">[</span><span class="n">CacheChangeTrackingHistoryId</span><span class="p">]</span>               <span class="n">INT</span>   <span class="k">IDENTITY</span> <span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span> <span class="k">NOT</span> <span class="k">NULL</span><span class="p">,</span>
   <span class="p">[</span><span class="n">TableName</span><span class="p">]</span>                             <span class="n">NVARCHAR</span> <span class="p">(</span><span class="mi">512</span><span class="p">)</span>   <span class="k">NOT</span> <span class="k">NULL</span><span class="p">,</span>
   <span class="p">[</span><span class="n">LastSynchronizationVersion</span><span class="p">]</span>            <span class="n">BIGINT</span>   <span class="k">NOT</span> <span class="k">NULL</span><span class="p">,</span>
<span class="p">);</span>
<span class="k">ALTER</span> <span class="k">TABLE</span> <span class="p">[</span><span class="n">dbo</span><span class="p">].[</span><span class="n">CacheChangeTrackingHistory</span><span class="p">]</span>
   <span class="k">ADD</span> <span class="k">CONSTRAINT</span> <span class="p">[</span><span class="n">PK_CacheChangeTrackingHistory</span><span class="p">]</span> <span class="k">PRIMARY</span> <span class="k">KEY</span> <span class="n">CLUSTERED</span> <span class="p">([</span><span class="n">CacheChangeTrackingHistoryId</span><span class="p">]</span> <span class="k">ASC</span><span class="p">);</span>

<span class="c1">-- Add default values for last sync version</span>
<span class="k">INSERT</span> <span class="k">INTO</span> <span class="n">dbo</span><span class="p">.</span><span class="n">CacheChangeTrackingHistory</span><span class="p">(</span> <span class="n">TableName</span><span class="p">,</span> <span class="n">LastSynchronizationVersion</span> <span class="p">)</span>
<span class="k">VALUES</span> <span class="p">(</span><span class="s1">'dbo.UserAccount'</span><span class="p">,</span> <span class="n">CHANGE_TRACKING_MIN_VALID_VERSION</span><span class="p">(</span><span class="n">Object_ID</span><span class="p">(</span><span class="s1">'dbo.UserAccount'</span><span class="p">)))</span>
<span class="k">INSERT</span> <span class="k">INTO</span> <span class="n">dbo</span><span class="p">.</span><span class="n">CacheChangeTrackingHistory</span><span class="p">(</span> <span class="n">TableName</span><span class="p">,</span> <span class="n">LastSynchronizationVersion</span> <span class="p">)</span>
<span class="k">VALUES</span> <span class="p">(</span><span class="s1">'dbo.Channel'</span><span class="p">,</span> <span class="n">CHANGE_TRACKING_MIN_VALID_VERSION</span><span class="p">(</span><span class="n">Object_ID</span><span class="p">(</span><span class="s1">'dbo.Channel'</span><span class="p">)))</span>
<span class="k">INSERT</span> <span class="k">INTO</span> <span class="n">dbo</span><span class="p">.</span><span class="n">CacheChangeTrackingHistory</span><span class="p">(</span> <span class="n">TableName</span><span class="p">,</span> <span class="n">LastSynchronizationVersion</span> <span class="p">)</span>
<span class="k">VALUES</span> <span class="p">(</span><span class="s1">'dbo.Media'</span><span class="p">,</span> <span class="n">CHANGE_TRACKING_MIN_VALID_VERSION</span><span class="p">(</span><span class="n">Object_ID</span><span class="p">(</span><span class="s1">'dbo.Media'</span><span class="p">)))</span>
</code></pre></div></div>

<h4 id="change-monitor-implementation">Change Monitor Implementation</h4>

<p>With the database properly configured we can start on the application that will consume the change tracking stream from SQL Server. The changes can be accessed by the <code class="highlighter-rouge">CHANGETABLE( CHANGES &lt;TABLE_NAME&gt; )</code> function. I will focus on <code class="highlighter-rouge">UserAccount</code> changes but the code will apply equally to the <code class="highlighter-rouge">Channel</code> and <code class="highlighter-rouge">Media</code> tables. When our monitor application starts, a loop is started to process change tracking updates and push them to downstream data stores. In this case the only downstream data store is the Cache represented by the ICache interface. If we had multiple downstream systems to update, the application would start one monitoring loop with a distinct change tracking history table for each system.</p>

<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">static</span> <span class="kd">class</span> <span class="nc">ChangeTracker</span>
<span class="o">{</span>
    <span class="n">internal</span> <span class="kd">static</span> <span class="n">async</span> <span class="n">Task</span> <span class="nf">StartChangeTrackingMonitorLoopAsync</span><span class="o">(</span> <span class="n">CancellationToken</span> <span class="n">token</span><span class="o">,</span> <span class="n">ICache</span> <span class="n">userAccountCache</span> <span class="o">)</span>
    <span class="o">{</span>
       <span class="k">while</span> <span class="o">(</span> <span class="kc">true</span> <span class="o">)</span>
       <span class="o">{</span>
          <span class="k">if</span> <span class="o">(</span> <span class="n">token</span><span class="o">.</span><span class="na">IsCancellationRequested</span> <span class="o">)</span>
          <span class="o">{</span>
             <span class="k">break</span><span class="o">;</span>
          <span class="o">}</span>

          <span class="n">using</span> <span class="o">(</span> <span class="n">ChangeTrackingBatch</span><span class="o">&lt;</span><span class="n">UserAccountChangeModel</span><span class="o">&gt;</span> <span class="n">userAccountChangesBatch</span> <span class="o">=</span> <span class="n">GetLatestUserChanges</span><span class="o">()</span> <span class="o">)</span>
          <span class="o">{</span>
             <span class="n">UserAccountChangeModel</span><span class="o">[]</span> <span class="n">userAccountChanges</span> <span class="o">=</span> <span class="o">(</span> <span class="n">await</span> <span class="n">userAccountChangesBatch</span><span class="o">.</span><span class="na">GetItemsAsync</span><span class="o">()</span> <span class="o">).</span><span class="na">ToArray</span><span class="o">();</span>
              <span class="n">foreach</span><span class="o">(</span> <span class="n">var</span> <span class="n">userAccount</span> <span class="n">in</span> <span class="n">userAccountChanges</span> <span class="o">)</span>
              <span class="o">{</span>
                <span class="n">userAccountCache</span><span class="o">.</span><span class="na">UpdateObject</span><span class="o">(</span> <span class="s">"user_account"</span><span class="o">,</span> <span class="n">userAccount</span><span class="o">.</span><span class="na">OperationType</span><span class="o">,</span> <span class="n">userAccount</span><span class="o">.</span><span class="na">UserAccountId</span><span class="o">,</span> <span class="n">userAccount</span> <span class="o">);</span>
              <span class="o">}</span>
              <span class="n">userAccountChangesBatch</span><span class="o">.</span><span class="na">Commit</span><span class="o">();</span>
          <span class="o">}</span>

          <span class="n">await</span> <span class="n">Task</span><span class="o">.</span><span class="na">Delay</span><span class="o">(</span> <span class="mi">1000</span> <span class="o">);</span>
       <span class="o">}</span>
    <span class="o">}</span>

    <span class="kd">private</span> <span class="n">Task</span><span class="o">&lt;</span><span class="n">IEnumerable</span><span class="o">&lt;</span><span class="n">UserAccountChangeModel</span><span class="o">&gt;&gt;</span> <span class="nf">GetLatestUserChangesAsync</span><span class="o">()</span>
    <span class="o">{</span>
         <span class="n">string</span> <span class="n">cmd</span> <span class="o">=</span> <span class="s">"
DECLARE @last_synchronization_version BIGINT = (SELECT LastSynchronizationVersion FROM dbo.CacheChangeTrackingHistory WHERE TableName = 'dbo.UserAccount')

DECLARE @current_synchronization_version BIGINT = CHANGE_TRACKING_CURRENT_VERSION();
SELECT ct.UserAccountId, ua.Email, ua.DisplayName, ua.CreateDate
		, CASE WHEN ct.SYS_CHANGE_OPERATION = 'I' THEN 'Insert' WHEN ct.SYS_CHANGE_OPERATION = 'U' THEN 'Update' ELSE 'Delete' END AS OperationType
FROM dbo.UserAccount AS ua
	RIGHT OUTER JOIN CHANGETABLE(CHANGES dbo.UserAccount, @last_synchronization_version) AS ct ON ua.UserAccountId = ct.UserAccountId

UPDATE dbo.CacheChangeTrackingHistory
SET LastSynchronizationVersion = @current_synchronization_version
WHERE TableName = 'dbo.UserAccount'
"</span><span class="o">;</span>
         <span class="k">return</span> <span class="k">new</span> <span class="n">ChangeTrackingBatch</span><span class="o">&lt;</span><span class="n">UserAccountChangeModel</span><span class="o">&gt;(</span> <span class="n">_connectionString</span><span class="o">,</span> <span class="n">cmd</span> <span class="o">);</span>
    <span class="o">}</span>
<span class="o">}</span>
</code></pre></div></div>

<div class="language-csharp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">internal</span> <span class="k">class</span> <span class="nc">ChangeTrackingBatch</span><span class="p">&lt;</span><span class="n">T</span><span class="p">&gt;</span> <span class="p">:</span> <span class="n">IDisposable</span>
<span class="p">{</span>
  <span class="k">private</span> <span class="k">readonly</span> <span class="kt">string</span> <span class="n">_command</span><span class="p">;</span>
  <span class="k">private</span> <span class="n">SqlTransaction</span> <span class="n">_transaction</span><span class="p">;</span>
  <span class="k">private</span> <span class="n">IEnumerable</span><span class="p">&lt;</span><span class="n">T</span><span class="p">&gt;</span> <span class="n">_items</span><span class="p">;</span>
  <span class="k">private</span> <span class="n">SqlConnection</span> <span class="n">_connection</span><span class="p">;</span>
  <span class="k">private</span> <span class="k">readonly</span> <span class="kt">object</span> <span class="n">_param</span><span class="p">;</span>

  <span class="k">public</span> <span class="nf">ChangeTrackingBatch</span><span class="p">(</span> <span class="kt">string</span> <span class="n">connectionString</span><span class="p">,</span> <span class="kt">string</span> <span class="n">command</span><span class="p">,</span> <span class="kt">object</span> <span class="n">param</span> <span class="p">=</span> <span class="k">null</span> <span class="p">)</span>
  <span class="p">{</span>
     <span class="n">_connection</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">SqlConnection</span><span class="p">(</span> <span class="n">connectionString</span> <span class="p">);</span>
     <span class="n">_command</span> <span class="p">=</span> <span class="n">command</span><span class="p">;</span>
     <span class="n">_param</span> <span class="p">=</span> <span class="n">param</span><span class="p">;</span>
  <span class="p">}</span>

  <span class="k">public</span> <span class="k">async</span> <span class="n">Task</span><span class="p">&lt;</span><span class="n">IEnumerable</span><span class="p">&lt;</span><span class="n">T</span><span class="p">&gt;&gt;</span> <span class="nf">GetItemsAsync</span><span class="p">(</span> <span class="p">)</span>
  <span class="p">{</span>
     <span class="k">if</span> <span class="p">(</span> <span class="n">_items</span> <span class="p">!=</span> <span class="k">null</span> <span class="p">)</span>
     <span class="p">{</span>
        <span class="k">return</span> <span class="n">_items</span><span class="p">;</span>
     <span class="p">}</span>

     <span class="n">_connection</span><span class="p">.</span><span class="nf">Open</span><span class="p">();</span>
     <span class="n">_transaction</span> <span class="p">=</span> <span class="n">_connection</span><span class="p">.</span><span class="nf">BeginTransaction</span><span class="p">(</span> <span class="n">System</span><span class="p">.</span><span class="n">Data</span><span class="p">.</span><span class="n">IsolationLevel</span><span class="p">.</span><span class="n">Snapshot</span> <span class="p">);</span>
     <span class="n">_items</span> <span class="p">=</span> <span class="k">await</span> <span class="n">_connection</span><span class="p">.</span><span class="n">QueryAsync</span><span class="p">&lt;</span><span class="n">T</span><span class="p">&gt;(</span> <span class="n">_command</span><span class="p">,</span> <span class="n">_param</span><span class="p">,</span> <span class="n">_transaction</span> <span class="p">);</span>
     <span class="k">return</span> <span class="n">_items</span><span class="p">;</span>
  <span class="p">}</span>

  <span class="k">public</span> <span class="k">void</span> <span class="nf">Commit</span><span class="p">()</span>
  <span class="p">{</span>
     <span class="n">_transaction</span><span class="p">?.</span><span class="nf">Commit</span><span class="p">();</span>
     <span class="n">_connection</span><span class="p">?.</span><span class="nf">Close</span><span class="p">();</span>
  <span class="p">}</span>

  <span class="k">public</span> <span class="k">void</span> <span class="nf">Dispose</span><span class="p">()</span>
  <span class="p">{</span>
     <span class="nf">Dispose</span><span class="p">(</span> <span class="k">true</span> <span class="p">);</span>
     <span class="n">GC</span><span class="p">.</span><span class="nf">SuppressFinalize</span><span class="p">(</span> <span class="k">this</span> <span class="p">);</span>
  <span class="p">}</span>

  <span class="k">protected</span> <span class="k">virtual</span> <span class="k">void</span> <span class="nf">Dispose</span><span class="p">(</span> <span class="kt">bool</span> <span class="n">disposing</span> <span class="p">)</span>
  <span class="p">{</span>
     <span class="k">if</span> <span class="p">(</span> <span class="n">disposing</span> <span class="p">)</span>
     <span class="p">{</span>
        <span class="n">_transaction</span><span class="p">?.</span><span class="nf">Dispose</span><span class="p">();</span>

        <span class="n">_connection</span><span class="p">?.</span><span class="nf">Dispose</span><span class="p">();</span>
     <span class="p">}</span>
  <span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>

<p>One additional thing to note is the use of <a href="https://docs.microsoft.com/en-us/dotnet/framework/data/adonet/sql/snapshot-isolation-in-sql-server">SnapshotIsolationMode</a> for the database transaction. This makes sure we’re working with a consistent view of the database to prevent any collisions with the change tracking cleanup process.</p>

<h4 id="wrap-up">Wrap Up</h4>
<p>At this point the solution is complete. Any updates to the <code class="highlighter-rouge">UserAccount</code> table will be tracked and pushed into the cache by the change tracker class. If the update to the cache fails, the transaction will be rolled back. The monitor application will then retry applying changes in order until the change is pushed into the cache successfully or the change is cleaned up by change tracking retention settings.</p>

<p>This solution is tied to the scalability of SQL Server, so for a write heavy application a different architecture may be necessary. For example, SQL Server could be replaced by a log stream like <a href="https://kafka.apache.org">Kafka</a>. However, for moderate scale applications this architecture will be more than adequate to handle load. We’ve solved the resiliency and race condition issues from the dual write scenario by ensuring that any successful database write will be pushed to downstream systems, in order. We’ve also improved the overall architecture of the primary application by removing the writes to secondary data stores. Plus we haven’t introduced any new data stores to learn and manage. For reference, <a href="https://github.com/dontjee/WriteThroughDatabaseToCache">the full application source code is available on GitHub</a>.</p>

  </div>
  <p class="post-separator"><p>
  <p class="post-separator-padding"><p>


   <h2 class="title"><a href="/2017/12/13/avoid-dual-writes-with-sql-server-and-change-tracking-part-1/">Avoid dual writes with sql server and change tracking - Part 1</a></h2>
   <p class="author">
    <span class="date">13 December 2017</span>
  </p>
  <div class="content">
    <h4 id="problem-consistent-updates-to-multiple-data-stores">Problem: Consistent updates to multiple data stores</h4>
<p>A common problem in web applications is the need to persist updates to multiple data stores (sql, cache, search, etc). Rarely does an application deal only with one data store. How do we get one update from the application into all data stores? The most common approach is <strong>dual writes</strong> where the application simply writes to each data store in parallel or serial order. This is compelling because it’s easy to implement and works well with low traffic, low error scenarios.</p>

<p><img src="/images/2017/12/dual-write-example.png" alt="Dual_write-example" /></p>

<p>However, there are many tricky errors that can arise. The most common being a failure of one of the writes. One data store has the new data and one has stale data. Another problem is race conditions among the different data store updates like in the diagram below. In this example, the value will be ‘2’ in the SQL database and ‘3’ in the Redis cache. No errors were thrown in this case making it even harder to track down.</p>

<p><img src="/images/2017/12/dual-write-race-condition.png" alt="Dual_write-example" /></p>

<p>How do we deal with these problems? One approach is to build a process to monitor the databases, look for drift between the two, then correct the one that’s out of line. However, this is difficult because you have to infer which database is correct. It’s also slow to analyze the whole database so the difference between the two databases may be “in the wild” for some time.</p>

<p>A better solution is the <strong><a href="https://engineering.linkedin.com/distributed-systems/log-what-every-software-engineer-should-know-about-real-time-datas-unifying">unified log pattern</a></strong>. At a high level, the unified log pattern is implemented by persisting all writes to one stream data store like <a href="https://kafka.apache.org">Kafka</a> or <a href="https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-what-is-event-hubs">Azure Event Hubs</a>. The log is consumed by one or more applications that persist the updates to dependent data stores as shown below.</p>

<p><img src="/images/2017/12/unified-log-example.png" alt="Unified-Log-Example" /></p>

<p>This avoids many of the problems from the dual write scenario but it is difficult to introduce into existing systems that currently write to more traditional data stores like SQL. Additionally, if your team is used to working with traditional databases, a log data store can be a difficult mindset shift.</p>

<p>What else can we do? A better solution is to only write to the SQL database. Then consume the changelog of the database and update any dependent data stores. I call this approach <strong>database stream writer</strong>. This idea comes from a series of blog posts <a href="https://www.oreilly.com/learning/making-sense-of-stream-processing/page/1/events-and-stream-processing">Martin Kleppmann did on Events and Stream Processing</a>. With this system you can continue writing to the existing SQL database but your primary application no longer has to deal with updating dependent data stores.</p>

<p><img src="/images/2017/12/database-leader-example.png" alt="Database-leader-example" /></p>

<h4 id="how-do-we-implement-the-database-stream-writer-pattern">How do we implement the database stream writer pattern?</h4>

<p>This pattern is implemented by turning Change Data Capture on for your database. Change Data Capture exposes the changes made to a database to a third party application based on its commit log. Many database vendors support this feature, for example, postgresql exposes a method called <a href="https://www.postgresql.org/docs/9.6/static/logicaldecoding-explanation.html">logical decoding</a> that can be used to parse updates from the write ahead log. SQL Server calls this feature <a href="https://docs.microsoft.com/en-us/sql/relational-databases/track-changes/about-change-data-capture-sql-server">change data capture</a> where updates to tables are stored in system tables inside the same database and exposed via table functions. SQL Server also has a lighter weight feature called <a href="https://docs.microsoft.com/en-us/sql/relational-databases/track-changes/about-change-tracking-sql-server">change tracking</a> that does not track individual column updates but simply tracks when a table row is modified. For many applications just knowing that a row changed is enough information to make necessary updates to dependent data stores (like invalidating a cache).</p>

<p>With a change data capture system in place, the last step is to write an application to consume the change data capture stream and write the data to downstream systems. <a href="/2017/12/13/avoid-dual-writes-with-sql-server-and-change-tracking-part-2/">In my next post</a>, I will detail how to implement a system like this using SQL Server’s Change Tracking and a C# application for consuming the change stream.</p>

  </div>
  <p class="post-separator"><p>
  <p class="post-separator-padding"><p>


   <h2 class="title"><a href="/2017/08/15/solutions-for-async-await-in-mvc-action-filters/">Solutions For Async/Await In MVC Action Filters</a></h2>
   <p class="author">
    <span class="date">15 August 2017</span>
  </p>
  <div class="content">
    
<p>Async/await has been available in .net for years but until the release of asp.net core there was no way to create a MVC <code class="highlighter-rouge">ActionFilter</code> that uses async/await properly. Since async was not supported by the framework there was no truly safe way to call async code from an <code class="highlighter-rouge">ActionFilter</code>. This has changed in asp.net core but if you are using ASP.Net 5 or below you’re stuck.</p>

<p>Recently, I found a workaround to using an async <code class="highlighter-rouge">HttpModule</code> to load whatever data the <code class="highlighter-rouge">ActionFilter</code> will need. You could also do all the work of the <code class="highlighter-rouge">ActionFilter</code> in the <code class="highlighter-rouge">HttpModule</code> but I prefer to keep the filter because it ties more closely into the rest of the MVC pipeline. My example will demonstrate moving async code out of an <code class="highlighter-rouge">AuthorizationFilter</code> but the pattern will work with any <code class="highlighter-rouge">ActionFilter</code>.</p>

<h4 id="actionfilter-to-fix">ActionFilter to fix</h4>
<p>This is an example authorization filter that does async work as part of the authorization of the request. Because attributes do not have async methods to override we’re stuck calling <code class="highlighter-rouge">.Wait()</code> and <code class="highlighter-rouge">.Result</code> to synchronously execute the task. This code is <a href="https://blog.stephencleary.com/2012/07/dont-block-on-async-code.html">ripe for deadlocks</a>.</p>

<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>public class WebAuthorizationFilter : AuthorizeAttribute
{
  public override void OnAuthorization( AuthorizationContext filterContext )
  {
     if ( AllowAnonymous( filterContext ) )
     {
        return;
     }

     Task&lt;bool&gt; isAuthorizedTask = DoAsyncAuthorizationWork( filterContext.HttpContext );
     isAuthorizedTask.Wait();

     bool isAuthorized = isAuthorizedTask.Result;
     if ( !isAuthorized)
     {
        filterContext.Result = new UnauthorizedResult();
     }
  }
}
</code></pre></div></div>

<p>There are 2 classes necessary to add the module:</p>

<h4 id="new-httpmodule-to-handle-async-code">New HttpModule to handle async code</h4>
<p>Any async code goes in this class. Call necessary methods then add state to <code class="highlighter-rouge">HttpContext.Items</code>.</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>public class WebAuthorizationAsyncModule : IHttpModule
{
    public void Init( HttpApplication context )
    {
       var authWrapper = new EventHandlerTaskAsyncHelper( AuthorizeRequestAsync );

       // Execute module early in pipeline during request authorization
       // To execute the module after the MVC route has been bound, use `context.AddOnPostAcquireRequestStateAsync` instead
       context.AddOnAuthorizeRequestAsync( authWrapper.BeginEventHandler, authWrapper.EndEventHandler );
    }

    private static async Task AuthorizeRequestAsync( object sender, EventArgs e )
    {
       HttpApplication httpApplication = (HttpApplication) sender;
       HttpContext context = httpApplication.Context;

       bool isAuthorized = await DoAsyncAuthorizationWork( context );

       // Store the result in HttpContext.Items for later access
       context.Items.Add( "IsAuthorized", isAuthorized );
    }
}
</code></pre></div></div>

<h4 id="module-registration-startup-class">Module Registration Startup Class</h4>
<p>This class registers the <code class="highlighter-rouge">HttpModule</code> created above with asp.net. You can also register in the web.config but I prefer to keep this kind of configuration in code.</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>public class PreApplicationStartCode
{
  public static void Start()
  {
    DynamicModuleUtility.RegisterModule( typeof( WebAuthorizationAsyncModule ) );
  }
}
</code></pre></div></div>

<p>The second code change required is to add the <code class="highlighter-rouge">PreApplicationStartCode</code> class to the startup classes registered with asp.net. To do this use the <code class="highlighter-rouge">PreApplicationStartMethod</code> attribute on your <code class="highlighter-rouge">HttpApplication</code> class in <code class="highlighter-rouge">Global.asax.cs</code>.</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>[assembly: PreApplicationStartMethod( typeof( Some.Code.PreApplicationStartCode ), "Start" )]
namespace Some.Code
 {
    public class WebApplication : HttpApplication
    {
      ...
</code></pre></div></div>

<h4 id="action-filter-changes">Action Filter changes</h4>
<p>This is the same authorization filter from above changed to read the authorization result from <code class="highlighter-rouge">Httpcontext.Items</code> instead of doing the work directly.</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>public class WebAuthorizationFilter : AuthorizeAttribute
{
  public override void OnAuthorization( AuthorizationContext filterContext )
  {
     if ( AllowAnonymous( filterContext ) )
     {
        return;
     }

     bool isAuthorized =  (bool) filterContext.HttpContext.Items["isAuthorized"];
     if ( !isAuthorized)
     {
        filterContext.Result = new UnauthorizedResult();
     }
  }
}
</code></pre></div></div>

<h4 id="next-steps">Next Steps</h4>
<p>This example was deliberately simple and as such it executes for every request. To only execute the module for specific URLs you can inspect the <code class="highlighter-rouge">HttpContext.Request.Url</code> property. Or if you delay execution of the module until after the ‘Acquire State’ pipeline step in asp.net (<code class="highlighter-rouge">context.AddOnPostAcquireRequestStateAsync</code> in Module.Init) and access the MVC route values using <code class="highlighter-rouge">HttpContext.Request.RequestContext.RouteData.Values</code> you can only execute the module code for specific Controllers/Actions in MVC.</p>

  </div>
  <p class="post-separator"><p>
  <p class="post-separator-padding"><p>


<hr />
<div>
  <a href="/all_posts">See All Posts</a>
</div>

          </div>
      </div>

      <footer>
        <div class="container">
          <div class="row">
            <div class="span2">
              <p>Evan Dontje</p>
              <p>evan.dontje@gmail.com</p>
            </div>
            <div class="span2">
              <p><a href="https://stackoverflow.com/cv/evandontje">stackoverflow.com/cv/evandontje</a></p>
              <p><a href="http://github.com/dontjee">github.com/dontjee</a></p>
              <!--<p><a href="http://twitter.com/dontjee">twitter.com/dontjee</a></p>-->
            </div>
          </div>
        </div>
      </footer>

    </div> <!-- /container -->

    


  <script type="text/javascript">
  var _gaq = _gaq || [];
  _gaq.push(['_setAccount', 'UA-4415874-1']);
  _gaq.push(['_trackPageview']);

  (function() {
    var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
    ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
    var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
  })();
</script>



  </body>
</html>

