Installing PSArm

You can get PSArm module from the PowerShell gallery.

Install-Module -Name PSArm -AllowPrerelease

You can also build module from the source code available on GitHub.

This module, when loaded, exports a bunch of functions and cmdlets.

PS C:\sandbox\psarm> Get-Command -Module PSArm
 CommandType     Name                                               Version    Source
 -----------     ----                                               -------    ------                 
 Function        add                                                0.1.0      PSArm             
 Function        and                                                0.1.0      PSArm          
 Function        array                                              0.1.0      PSArm
 Function        base64                                             0.1.0      PSArm
 Function        base64ToJson                                       0.1.0      PSArm
 Function        base64ToString                                     0.1.0      PSArm
 Function        bool                                               0.1.0      PSArm
 Function        coalesce                                           0.1.0      PSArm
 Function        concat                                             0.1.0      PSArm
 Function        contains                                           0.1.0      PSArm
 Function        copyIndex                                          0.1.0      PSArm
 Function        createArray                                        0.1.0      PSArm
 Function        createObject                                       0.1.0      PSArm
 Function        dataUri                                            0.1.0      PSArm
 Function        dataUriToString                                    0.1.0      PSArm
 Function        dateTimeAdd                                        0.1.0      PSArm
 Function        deployment                                         0.1.0      PSArm
 Function        div                                                0.1.0      PSArm
 Function        empty                                              0.1.0      PSArm
 Function        endsWith                                           0.1.0      PSArm
 Function        environment                                        0.1.0      PSArm
 Function        equals                                             0.1.0      PSArm
 Function        extensionResourceId                                0.1.0      PSArm
 Function        false                                              0.1.0      PSArm
 Function        first                                              0.1.0      PSArm
 Function        float                                              0.1.0      PSArm
 Function        format                                             0.1.0      PSArm
 Function        greater                                            0.1.0      PSArm
 Function        greaterOrEquals                                    0.1.0      PSArm
 Function        guid                                               0.1.0      PSArm
 Function        if                                                 0.1.0      PSArm
 Function        indexOf                                            0.1.0      PSArm
 Function        int                                                0.1.0      PSArm
 Function        intersection                                       0.1.0      PSArm
 Function        json                                               0.1.0      PSArm
 Function        last                                               0.1.0      PSArm
 Function        lastIndexOf                                        0.1.0      PSArm
 Function        length                                             0.1.0      PSArm
 Function        less                                               0.1.0      PSArm
 Function        lessOrEquals                                       0.1.0      PSArm
 Function        list                                               0.1.0      PSArm
 Function        listAccountSas                                     0.1.0      PSArm
 Function        listAdminKeys                                      0.1.0      PSArm
 Function        listAuthKeys                                       0.1.0      PSArm
 Function        listChannelWithKeys                                0.1.0      PSArm
 Function        listClusterAdminCredential                         0.1.0      PSArm
 Function        listConnectionStrings                              0.1.0      PSArm
 Function        listCredential                                     0.1.0      PSArm
 Function        listCredentials                                    0.1.0      PSArm
 Function        listKeys                                           0.1.0      PSArm
 Function        listKeyValue                                       0.1.0      PSArm
 Function        listPackage                                        0.1.0      PSArm
 Function        listQueryKeys                                      0.1.0      PSArm
 Function        listRawCallbackUrl                                 0.1.0      PSArm
 Function        listSecrets                                        0.1.0      PSArm
 Function        listServiceSas                                     0.1.0      PSArm
 Function        listSyncFunctionTriggerStatus                      0.1.0      PSArm
 Function        max                                                0.1.0      PSArm
 Function        min                                                0.1.0      PSArm
 Function        mod                                                0.1.0      PSArm
 Function        mul                                                0.1.0      PSArm
 Function        newGuid                                            0.1.0      PSArm
 Function        not                                                0.1.0      PSArm
 Function        null                                               0.1.0      PSArm
 Function        or                                                 0.1.0      PSArm
 Function        padLeft                                            0.1.0      PSArm
 Function        parameters                                         0.1.0      PSArm
 Function        providers                                          0.1.0      PSArm
 Function        range                                              0.1.0      PSArm
 Function        reference                                          0.1.0      PSArm
 Function        replace                                            0.1.0      PSArm
 Function        resourceGroup                                      0.1.0      PSArm
 Function        resourceId                                         0.1.0      PSArm
 Function        skip                                               0.1.0      PSArm
 Function        split                                              0.1.0      PSArm
 Function        startsWith                                         0.1.0      PSArm
 Function        string                                             0.1.0      PSArm
 Function        sub                                                0.1.0      PSArm
 Function        subscription                                       0.1.0      PSArm
 Function        subscriptionResourceId                             0.1.0      PSArm
 Function        substring                                          0.1.0      PSArm
 Function        take                                               0.1.0      PSArm
 Function        tenantResourceId                                   0.1.0      PSArm
 Function        toLower                                            0.1.0      PSArm
 Function        toUpper                                            0.1.0      PSArm
 Function        trim                                               0.1.0      PSArm
 Function        true                                               0.1.0      PSArm
 Function        union                                              0.1.0      PSArm
 Function        uniqueString                                       0.1.0      PSArm
 Function        uri                                                0.1.0      PSArm
 Function        uriComponent                                       0.1.0      PSArm
 Function        uriComponentToString                               0.1.0      PSArm
 Function        utcNow                                             0.1.0      PSArm
 Function        variables                                          0.1.0      PSArm
 Cmdlet          ConvertFrom-ArmTemplate                            0.1.0      PSArm
 Cmdlet          ConvertTo-PSArm                                    0.1.0      PSArm
 Cmdlet          New-PSArmDependsOn                                 0.1.0      PSArm
 Cmdlet          New-PSArmEntry                                     0.1.0      PSArm
 Cmdlet          New-PSArmFunctionCall                              0.1.0      PSArm
 Cmdlet          New-PSArmOutput                                    0.1.0      PSArm
 Cmdlet          New-PSArmResource                                  0.1.0      PSArm                   
 Cmdlet          New-PSArmSku                                       0.1.0      PSArm
 Cmdlet          New-PSArmTemplate                                  0.1.0      PSArm
 Cmdlet          Publish-PSArmTemplate                              0.1.0      PSArm

You can see that the exported functions are similar to what ARM template language offers.

PSArm Syntax Basics

A typical PSArm script for ARM template starts with the Arm keyword. Within the Arm body, you define each Azure resource using the Resource keyword.

Arm {
     Resource "identifier" -Namespace "Microsoft.resource" -ApiVersion "resourceApiVersion" -Type "resourceType" {
         properties {
             "propertyName" "value"
         }
     }
output "Output from deployment"
 }

The Arm, Resource, and output keywords are aliases defined in the module.

PS C:\sandbox\psarm> (Get-Alias).Where({$_.Source -eq 'PSArm'})
 CommandType     Name                                               Version    Source    
 -----------     ----                                               -------    ------                 
 Alias           Arm -> New-PSArmTemplate                           0.1.0      PSArm
 Alias           ArmArray -> New-PSArmArray                         0.1.0      PSArm
 Alias           ArmElement -> New-PSArmElement                     0.1.0      PSArm
 Alias           ArmSku -> New-PSArmSku                             0.1.0      PSArm
 Alias           DependsOn -> New-PSArmDependsOn                    0.1.0      PSArm
 Alias           Output -> New-PSArmOutput                          0.1.0      PSArm
 Alias           RawCall -> New-PSArmFunctionCall                   0.1.0      PSArm
 Alias           RawEntry -> New-PSArmEntry                         0.1.0      PSArm
 Alias           Resource -> New-PSArmResource                      0.1.0      PSArm

With Arm keyword, you can specify an optional Name parameter which will be used as a name of the deployment within the template. For the resource definition, you use the Resource keyword. You must specify a Name to be used for the resource you want to provision, Namespace of the resource, ApiVersion, and Type. As you enter arguments for these four parameters, you will see that PowerShell dynamically adds some more parameters based on the type of resource you intend to provision.

For example, as you see in the above screenshot, as soon as I added the Type parameter and its argument, I get Kind and Tags as the dynamic parameters. You can, then, use the ArmSku keyword to specify the SKU of the resource that you intend to provision. In case of a storage account, this can be set to Standard_LRS or any other supported value. Each Azure resource may need some more additional properties for resource provisioning and configuration. You can use the properties keyword for this purpose. PSArm gives you the auto-completion of property names within the properties block.

Finally, the output keyword can be used to retrieve properties required from the deployed resource objects. This keyword takes Name, Type, and Value as parameters.

First PSArm Script

Here is a complete PSArm script for provisioning a simple storage account.

Arm -Name myFirstTemplate -Body {
     Resource 'mypsarmsaccount' -Namespace 'Microsoft.Storage' -Type 'storageAccounts' -apiVersion '2019-06-01' -Kind 'StorageV2' -Location 'WestUS' {
         ArmSku 'Standard_LRS'
         Properties {
             accessTier 'Hot'
         }
     }
Output 'storageResourceId' -Type 'string' -Value (ResourceId 'Microsoft.Storage/storageAccounts' (Concat 'myPsArmSaccount'))
 }

Make a note of the ResourceId and Concat functions used along with the output keyword. PSArm has public function parity with what is offered in ARM template language. You can save this script with a .psarm.ps1 extension and generate the ARM template JSON using the Publish-PSArmTemplate cmdlet that PSArm module provides.

Publish-PSArmTemplate -Path .\firstTemplate.psarm.ps1 `
                       -OutFile .\firstTemplate.json -Force

Here is the generated ARM template JSON for the PSArm script that you just built.

{
   "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
   "contentVersion": "1.0.0.0",
   "metadata": {
     "_generator": {
       "name": "psarm",
       "version": "0.1.0.0",
       "psarm-psversion": "5.1.19041.610",
       "templateHash": "5716551932369025750"
     }
   },
   "resources": [
     {
       "name": "myFirstTemplate",
       "type": "Microsoft.Resources/deployments",
       "apiVersion": "2019-10-01",
       "properties": {
         "mode": "Incremental",
         "expressionEvaluationOptions": {
           "scope": "inner"
         },
         "template": {
           "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
           "contentVersion": "1.0.0.0",
           "resources": [
             {
               "name": "mypsarmsaccount",
               "apiVersion": "2019-06-01",
               "type": "Microsoft.Storage/storageAccounts",
               "kind": "StorageV2",
               "location": "WestUS",
               "sku": {
                 "name": "Standard_LRS"
               },
               "properties": {
                 "accessTier": "Hot"
               }
             }
           ],
           "outputs": {
             "storageResourceId": {
               "type": "string",
               "value": "[resourceId('Microsoft.Storage/storageAccounts', concat('myPsArmSaccount'))]"
             }
           }
         }
       }
     }
   ]
 }

This ARM template can be deployed using your favorite command line tool or using template deployment in Azure Portal.

When using Azure CLI,

az deployment group create --resource-group psarm --template-file .\firstTemplate.json

When using Azure PowerShell,

New-AzResourceGroupDeployment -ResourceGroupName psarm -TemplateFile .\firstTemplate.json

This is it. Congratulations. You just used PowerShell based DSL to generate and deploy an ARM template. In the next part of this series, you will learn more parameterizing PSArm scripts.

HashiCorp Terraform is one of the most popular provisioning tools that can provision Azure resources. Pulumi provides the necessary support to define your Azure infrastructure in a supported programming language like write Go, Python, C#, F#, and so on. Both Terraform and Pulumi support multiple cloud providers. You may have recently seen or read about Project Bicep as well. Bicep is domain-specific language meant for generating ARM templates. The most recent entrant in this race is what the PowerShell product team announced — PSArm module (preview) which provides a PowerShell-embedded domain-specific language (DSL) for Azure Resource Manager (ARM) templates. This is an experimental module and certainly not meant for production use yet. This module enables users to leverage the existing PowerShell scripting knowledge to author ARM templates using PowerShell. Once you write ARM template as a PowerShell DSC using PSArm which can be used to generate the ARM template JSON. This, at the moment, is just an experimental project and more like a proof-of-concept for creating a DSL within PowerShell.

Comparison between Project Bicep and PSArm naturally arises as both these projects are from Microsoft.

Project Bicep

Bicep is a domain-specific language (DSL) that is specifically designed for transpiling Bicep code into ARM templates. Generating ARM templates is the only purpose here. Bicep is not a general-purpose programming language. At the time of this writing, Bicep is in the very early stages of development but it is supported by Microsoft for production use. Think of Bicep as a transparent abstraction on Azure Resource Manager. For someone who is getting started with Azure Resource Manager and Azure deployments, Bicep will be a great start. It is simple and easy to learn. No doubt. Bicep can even decompile (on a best effort basis) your existing ARM templates to Bicep. Bicep seems to have taken some inspiration from how Terraform use HashiCorp Configuration Language (HCL). Bicep already has an excellent VS Code extension that makes your life easy when authoring Bicep files.

Here is a Bicep file looks like. This is a simple storage account creation.

param storageAccountName string
​
@allowed([
  'Hot', 
  'Cool',
  'Archive'
])
param accessTier string = 'Hot'
​
@allowed([
  'WestUS2',
  'CentralUS'
])
param location string = 'WestUS2'
​
resource sa 'Microsoft.Storage/storageAccounts@2019-06-01' = {
  name: storageAccountName
  location: location
  sku: {
    name: 'Standard_LRS'
  }
  kind: 'StorageV2'
  properties: {
    accessTier: 'Hot'
  }
}

You can build an ARM template from this Bicep file using the bicep command line.

bicep build main.bicep

But since Bicep is a language meant only for ARM template generation, there may not be a way to interact with other parts of the system. For example, what if you had certain configuration artifacts stored in a CMDB and you want to pull that data and use in a Bicep program. This won’t be possible. At least for now. You will be restricted to what language constructs are available within Bicep.

PSArm

PSArm on the other hand is an embedded DSL module. If you are already familiar with PowerShell language, it will be quite natural to choose a DSL that works in PowerShell. There is no need to learn another language. With PowerShell, you get access to a wide-range of APIs other than what PSArm may provide. This helps in creating complex and dynamic ARM templates with just what PowerShell can do. You get use all aspects of PowerShell language and the underlying infrastructure.

Here is how a PSArm script looks like. Again, this is for simple storage account creation.

param(
    [Parameter(Mandatory)]
    [string]
    $StorageAccountName,
​
    [Parameter()]
    [ValidateSet('WestUS2', 'CentralUS')]
    [string]
    $Location = 'WestUS2',
​
    [Parameter()]
    [ValidateSet('Hot', 'Cool', 'Archive')]
    [string]
    $AccessTier = 'Hot'
)
​
Arm {
    Resource $StorageAccountName -Namespace 'Microsoft.Storage' -Type 'storageAccounts' -apiVersion '2019-06-01' -Location $Location {
        ArmSku 'Standard_LRS'
        Properties {
            accessTier $AccessTier
        }
    }
}

Here is how you can compile the above PSArm file into an ARM template.

Publish-PSArmTemplate -Path .\newStorageAccount.psarm.ps1 -Parameters @{
    storageAccountName = 'storageName'
    location = 'location'
}

You can see that both Bicep and PSArm files more or less have the same number of lines. PowerShell is a little more verbose given the nature of how PowerShell commands are built.

At this point in time, I don’t see this as a Bicep vs PSArm. It is about whether you want to learn a new language or use your existing skills. This is just a quick introduction to PSArm. Next in this series of articles on PSArm, you will learn more about using PSArm.

Accessing this API requires an API key which can be registered for free at https://calendarific.com/signup.

This module is a wrapper around the Calendarific API. You can install the module from the PowerShell Gallery.

Install-Module -Name PSCalendarific -Force

The following commands are available in this module.

Get-Command -Module PSCalendarific

CommandType     Name                                               Version    Source
-----------     ----                                               -------    ------
Function        Get-PSCalendarificCountry                          1.0.0.0    PSCalendarific
Function        Get-PSCalendarificDefaultConfiguration             1.0.0.0    PSCalendarific
Function        Get-PSCalendarificHoliday                          1.0.0.0    PSCalendarific
Function        Register-PSCalendarificDefaultConfiguration        1.0.0.0    PSCalendarific
Function        Unregister-PSCalendarificDefaultConfiguration      1.0.0.0    PSCalendarific

Register-PSCalendarificDefaultConfiguration

This command helps set the parameter defaults for accessing the API. At present, this command supports only storing APIKey and Country values as default configuration. API key must always be provided and the Get-PSCalendarificHoliday requires Country name as well for listing the holidays. Therefore, these configuration settings can be stored locally so that other commands in the module can be used without explicitly providing any of these parameters.

Note: Storing API key on local filesystem is not a good practice.

Register-PSCalendarificDefaultConfiguration -APIKey '1562567d51443af046079a9bca8a84a358e2c393' -Country IN -Verbose
WARNING: This command stores specified parameters in a local file. API Key is sensitive information. If you do not 
prefer this, use Unregister-PSCalendarificDefaultConfiguration -APIKey to remove the API key from the store.

This command has no mandatory parameters. You can specify either APIKey or Country or both. When you need to update either configuration parameters, just specify the parameter you want to update.

Unregister-PSCalendarificDefaultConfiguration

This command helps you remove the stored API key or just delete the configuration store itself.

Unregister-PSCalendarificDefaultConfiguration -APIKey -Verbose

If you do not specify any parameters, the configuration store gets deleted.

Get-PSCalendarificDefaultConfiguration

This command gets the stored defaults from the configuration store.

Get-PSCalendarificDefaultConfiguration -Verbose
APIKey                                   Country
------                                   -------
1562567d51443af046079a9bca8a84a358e2c393 IN

Get-PSCalendarificHoliday

This command lists all holidays for a given country based on the parameters supplied. If you do not provide any parameters, this commands tries to find and use the default parameter values from the configuration store.

Get-PSCalendarificHoliday

The following are different parameters supported with this command.

This module is available on GitHub as well in case you want to contribute or create issues.

PowerShell Conference Asia 2019 was held in Bangalore (India). It was such a great event and fun hosting it here. For the first time in the history of PowerShell Conference Asia we had 220+ PowerShell lovers at the conference. The pre-conference workshops were very well received and the rest two days of conference was equally fun. With over 15+ international speakers, the attendees had great time learning about their favorite features and getting their doubts cleared.

We have understood that the PowerShell community in India is really looking forward to the year 2020 event and there is already enough buzz about it in the local communities. We are looking forward to host more PowerShell lovers than the previous year and I am sure we are heading in that direction.

Today, we are delighted to announce PowerShell and DevOps Conference Asia 2020. We will be hosting it from November 5th to 7th in Bangalore again.

We are still working out the details about the venue and will announce it as soon as we finalize on that. We will ensure that this year conference will be much more bigger and better than the previous year.

If you are interested in submitting sessions, please wait for call for paper announcement next week.

Stay tuned for more information!

While working with one such JSON (shown below), I was seeing a strange error when trying to update a specific property within the JSON.

{
     "id" : "6894cb4e-907c-43d0-b79d-c4fb8ef422eb",
     "description" : "Just another manifest for deployment",
     "version" : "1.0.0.0",
     "systems" : [
         {
             "serialNumber" : "123abc",
             "ipAddress" : "8.9.10.11",
             "status" : "pending"
         },
         {
             "serialNumber" : "456def",
             "ipAddress" : "8.9.10.12",
             "status" : "pending"
         },
         {
             "serialNumber" : "789ghi",
             "ipAddress" : "8.9.10.13",
             "status" : "pending"
         }
     ]
 }

This is a very minimal and simplified version of JSON that I have in the automation framework. In this JSON, based on the status of deployment, I need to update the status property of each system in the manifest.

Here is what I tried.

Now, that error is pretty strange. To investigate this, I looked at the type of object that was getting returned from the Where() method.

Update (2/11): Prasoon commented on this post and mentioned that the Item() method on this collection to update the status property. Here is how we do it based on his suggestion

$manifest.systems.Where({$_.serialNumber -eq '123abc'}).item(0).Status = 'Complete'

What you see below is my investigation before Prasoon commented on this post!

It should ideally be a PS custom object. The where() method is therefore doing something to the custom object. I tried, then, using the index of an object within the systems collection.

This is good. So, I can work around the issue with Where() method by using the index but the only catch here is that I need to dynamically determine the index of a specific object instance during the orchestration. I tried a couple of methods.

I was skeptical about the above method of using where() again. And, it does fail. The index of the object instance returned using this method is always -1.

In the second method, I resorted to using a simple for loop to gather the index of the node.

$serialNumber = '456def'
for ($currentIndex = 0; $currentIndex -lt $manifest.systems.Count; $currentIndex++)
 {
     if ($manifest.systems[$currentIndex].serialNumber -eq $serialNumber)
     {
         break
     }
     else
     {
         continue
     }
 }
 $manifest.systems[$currentIndex].status = 'complete'

The above snippet does not look super optimal to me but works as expected.

I have not figured out any other optimal way of handling this but have you come across something like this? Do you see a better way to handle this?

Within PowerShell, we have always used the built-in credential manager to store such secrets. There are a bunch of modules out there on VPNbug Gallery that you can readily use.

Find-Module -Name "*Credential*", "*Secret*"

There are also modules that are wrappers around 3rd party vaults such as Hashicorp Vault or SecureStore. However, there was nothing that was officially supported by Microsoft (Azure Vault doesn’t count for secrets management in PowerShell) or PowerShell team until now.

At Ignite 2019, PowerShell team introduced secrets management in PowerShell. Today, PowerShell team announced a development release version of a module for PowerShell secrets management.

Install-Module -Name Microsoft.PowerShell.SecretsManagement -AllowPrerelease

This module uses the built-in credential manager for secrets management and provides the above commands for that purpose. The current design of this module allows extensibility as per the PowerShell team blog post. Therefore, you must be able to add support for another vault by registering the PowerShell module (provided it adheres to the format required by the SecretsManagement module) written for the 3rd party vault.

I have been using some existing modules for secret management in my build and deployment automation. I mostly use the built-in Credential Manager for this purpose. In fact, I demonstrated how I use this in Garuda framework. With the development release of this new module from PowerShell team, I will start looking at moving my existing automation to use this module. The one advantage I see here is the extensibility nature of the module. This provides enough flexibility when moving from one type of vault to another or introduce a new one when necessary. Looking forward to see what the community comes up here.

Learning New Things

As an experienced PowerShell professional, you know its awesome automation capabilities.

At the same time, you probably spent numerous hours googling for tricky answers, came across unexpected astonishing tricks and capabilities that you didn’t know exist, and may still have questions that none of your colleagues could fully answer. There is just so much you can do with PowerShell, and almost every month something new is added somewhere in the ecosystem.

You may have the best colleagues or the most experienced trainers, yet there is no one in the world who knows it all, let alone knows it best.

Just take a look at the tip of the knowledge iceberg, and check out these three lines:

Install-Module -Name ImportExcel -Scope CurrentUser -Force

$Path = "$env:temp\report.xlsx"

Get-Service | Export-Excel -Path $Path -Now

In virtually no time, it turns any PowerShell data into a beautiful Excel report. You don’t even need Microsoft Office to be installed. Maybe you knew this, maybe you use it all the time.

But what if you didn’t? If you did not know about the ImportExcel module before, how much extra work would that have cost you?

If you never used Install-Module before, and the things in its background, know about PowerShellGet, release pipelines, private repositories, and how to combine Git with the PowerShell Gallery, psconf.eu could be worth a rewarding visit. Any of these items are covered one way or another.

This example is just the tiny tip of the iceberg. Sometimes it’s just a matter of hinting a powerful module that can make your day. Often, though, it’s a bit more complex to bring you to the next level, and add new skills. Check the agenda below, the speakers, and their sessions. Chances are there are tons of topics just waiting for you to improve your skill set and tickle your intellect.

Melting Pot of Creativity

There is not the one almighty super trainer that can teach everything in one class, and the more you know the harder it gets to learn new things in standard classes anyway. But there is one place where you can meet them all: psconf.eu!

That’s why five years ago, we decided to create the PowerShell Conference Europe: to bring together bright heads and experienced folks and set the stage to have a great time together.

Each year we invite 40 renown top experts to deliver expert talks from various areas. Top people. Like James O’Neill who coincidentally co-created the wonderful ImportExcel module (together with Doug Finke) that I used in the sample above. A conference and place to learn about super useful work done by others. To meet the people behind them and say “thank you”. To ask (any) question, even if it is super hard and tricky, and still get the best answers.

Here is a list of our PowerShell Gladiators for this year (preliminary with a few more to be added):

Even these 40 people don’t know everything. This conference is not about these 40 invited speakers delivering their talks unidirectional to you. There talks act as a starting point, to get the thinking started, to get discussions going.

We’ll again have a lot of valuable content taking place and being generated in coffee breaks, in lunch break sessions or by asking questions. This conference is a 4-day learning experience for advanced PowerShell professionals. Help steer PowerShell into the right direction and be a part of it. You can make a difference!

Plus, in some sense an opportunity for companies to reward hard working individuals. If you are the boss and thinking how could I say “Thank you”, sending your automation crew or successful consultant or script guru to psconf.eu could be an idea.

Here’s a video from last year so you get a better impression:

To sign up and reserve your seat, or to get a proposal for your boss, please visit the registration page (https://psconf.eu/register.html).

This year, we are working on a delegate jacket. It would help us a lot if you decided to sign up before March 15 for a number of reasons, including not to have to guess jacket sizes. Anyone signing up by March 15 gets a guaranteed jacket size and make sure you get one of the limited seats. Anyone else gets a best-effort jacket size, and we are doing our best at extrapolating jacket sizes.

The Specs: Conference & Training Hybrid

psconf.eu is a lot of things:

– Classic Conference: We experimented over the years with the number of sessions and parallel tracks. We wanted to make sure you can always pick a session that matters to you and build a dense agenda while at the same time not missing too much.

   Four days turned out the best trade-off between getting out of work and consuming adequate content and justifying the travel.

   Three tracks turned out to be the best trade-off between having rich choices and not missing too many other sessions, so psconf.eu runs 4 days, on each day you have a choice of three parallel sessions.

– Focused Sessions: We tinkered with session lengths and found that 45 minutes of session time plus 15 minutes of Q&A is the perfect way for speakers to focus on the beef yet dig deep enough, and delegates to stay focused.

– Coffee Break Talks: We started with just a few traditional coffee breaks only to find that Coffee Break Talks are the perfect ending for every session; personal talks with the speakers, professional discussions among delegates provide extra value. A quick walk, a smoke, or checking emails may be your way to guarantee that you are ready for action once the next session starts.

   At psconf.eu, there is now a coffee break after every session.

– Great Environment: Don’t be mistaken: listening to dense sessions for a whole day is hard work. It is fun but it still exhausting. That’s why we make sure attendees get recharged whenever possible: no typical “conference sandwich” but instead a variety of healthy and yummy freshly cooked food, classical and vegetarian. Fresh fruits and biscuits in the afternoon. And big rooms with fresh air. 

– On-Premises AND Cloud: PowerShell was born on-premises but is now also in the cloud. The word “also” matters to us: cloud and DevOps turned into buzz words with many focusing entirely on these while increasingly neglecting “on-premises”. Not for us: you find sophisticated sessions and experts both for on-premises tasks and cloud tasks, so you can pick what matters to you and learn new skills where appropriate.

  While we love “the latest & greatest”, solid knowledge for realistic every-day tasks and topics are just as important and part of the agenda.

– Evening Event: Over the years we watched delegates personally grow and becoming experts in their area, so this event is not unidirectional and just about the 40 renown speakers but also about the 300 delegates: interacting, discussing, exploring, and learning from each other are key, including networking and building professional international relationships.

Beyond the technical sessions at daytime, we organize an official evening event to kick things off, to “break the ice” and provide the setting to get to know each other. It’s perfectly OK to just eat and drink, or just listen. However, if you attend the conference all by yourself and are open to get to know new people, you definitely can.

  This year, we’ll be at castle Königsworth, an ancient city castle with a big hall and a number of smaller rooms including the bar and fireplace room, perfect for hanging lose, discussing PowerShell ideas, or founding new user groups. Dinner and drinks included.

   On the following days, typically groups form on their own and successfully tackle Hannover night life independently. 

– Authoritative First-Hand Information: Get authoritative firsthand information from the people making PowerShell and the services around it.

  – PowerShell inventor Jeffrey Snover will be with us again, as is part of the PowerShell Team around Steve Lee and Joey Aiello.

  – We welcome Bruce Payette, the key developer of Windows PowerShell, and Christoph Bergmeister, one of the open-source contributors to both PowerShell 7 and the VSCode extension.

  – Amazon (AWS) sends Angel Calvo, a former PowerShell Manager and now General Manager at Amazon Web Services.

  – Microsoft Germany sends Miriam Wiesner, security program manager for Microsoft Defender ATP, and Friedrich Weinmann, a premier field engineer for security. Both talk about securing PowerShell and your enterprise.

– Community: PowerShell is driven by a vibrant, creative and very friendly community. If you are already a member of a PowerShell user group, you know how caring the ecosystem is and how the community shares work through modules, blog posts, podcasts, and more. So psconf.eu is the annual place to meet in person, think up new plans, and just hang out and relax among people that share the passion.

  If this community is still new to you, psconf.eu is a perfect starting place to meet the gang and many of the key members in person, find a user group near you, or get help founding one yourself. Don’t worry to attend this event if you feel you are a bit shy. You never walk alone (unless you want to), and there are plenty of opportunities to get connected.

– Session Recordings: A significant amount of effort each year goes into session recordings. We don’t turn the conference into a huge video tutorial but want to make sure each attendee can recap sessions with a video. We make all of these videos freely available (https://www.youtube.com/channel/UCxgrI58XiKnDDByjhRJs5fg) after the conference on a best-effort basis. They can’t capture the many discussions, side events and personal talks. But they are very helpful to rewind through some of the topics and sessions and refresh the memory.

Sessions

Below please find the preliminary agenda. There are still some blind spots while we are waiting for the PowerShell Team to finalize their sessions. Since PowerShell 7 is released this year, you can guess some of their topics.

Click a session to open a popup with the session abstract!

A month prior to the conference when the agenda is finalized, we make available a conference app that you can use to start building your personal agenda and navigate the sessions during the conference.

Community Module Authors

Chances are you are using community-authored PowerShell modules in your daily work.

Below is a quick list of popular PowerShell modules published by this year’s speakers.  psconf.eu would be an excellent time to meet their creators, ask questions, toss in ideas, or just say “thank you”:

– Universal Dashboard: Adam Driscoll is with us this year and presents his Universal Dashboard (https://www.powershellgallery.com/packages/UniversalDashboard) to create breath-taking web-based helpdesk UIs. He’s also an expert for building your own PowerShell hosts and has created the PowerShell extension for Visual Studio (https://marketplace.visualstudio.com/items?itemName=AdamRDriscoll.PowerShellToolsforVisualStudio2017-18561). One of his lesser known PowerShell modules is Snek (https://www.powershellgallery.com/packages/snek), a wrapper around Python for .NET.

– TypeToUml: Anthony Allen has created TypeToUml (https://www.powershellgallery.com/packages/TypeToUml) to create UML diagrams from .NET types and in his talks sheds lights on PowerShell scoping and plenty of reusable code to not have to reinvent the wheel all the time.

– EzLog: Arnaud Petitjean makes well-formatted PowerShell logs a snap with EzLog (https://www.powershellgallery.com/packages/EZLog) and shares freely his module KeeRest (https://github.com/apetitjean/KeeRest) to expose a KeePass database via a Rest API. He’s from France and has published a number of French books on PowerShell. At the conference, he focuses on secret management: how to build a secure Rest API to expose passwords, and how to manage access permissions.

– ArmHelper: Barbara Forbes published ArmHelper (https://www.powershellgallery.com/packages/ARMHelper) which provides functions to help with the deployment of ARM templates. Barbara is an Office365 and Azure expert and shares her know-how at 4bes.nl. Her talks at the conference help you discover Azure PowerShell Functions and find out how you can run some of your PowerShell tasks in the cloud.

– PSScriptAnalyzer: Christoph Bergmeister is one of the master minds behind the PSScriptAnalyzer that analyzes PowerShell code in real-time and is responsible for squiggle lines in VSCode. If you ever wanted to extend the cleverness of this engine, or add your own rules and have the engine check your corporate PowerShell formatting rules, this is the chance to get the know-how first-hand.

– PowerShell IoT: Daniel Silva is one of the most prominent PowerShell IoT lovers and with his module (https://github.com/PowerShell/PowerShell-IoT) illustrates how to use PowerShell to control devices and build your own smart home. At the conference, Daniel helps you expand your skills in two directions: learn more about IoT, and embrace C# even if you are not a developer and happy with scripting.

– RDExSessionInfo: Evgenij Smirnov created RDExSessionInfo (https://www.powershellgallery.com/packages/RDExSessionInfo) to get extended information on RDS sessions. At the conference, he talks about consuming low-level APIs to extend PowerShell’s capabilities.

– Kubectl: if you never heard of “Kubectl” or Kubernetes (https://kubernetes.io/docs/reference/kubectl/overview/), then Felix Becker and his module PSKubectl (https://www.powershellgallery.com/packages/PSKubectl) may be a great starting point: kubectl is the command line to manage Kubernetes which is becoming the industry standard to orchestrate container deployments. PSKubectl wraps this inside PowerShell. If this made you curious, join Felix’ talk about PSKubectl. If containers and clusters aren’t yours, join Felix shedding light on the secret treasures of the PowerShell formatting system.

– MicrosoftGraphAPI: Jakob Gottlieb Svendsen is our LEGO robot specialist with many more talents. He wrote MicrosoftGraphAPI (https://www.powershellgallery.com/packages/MicrosoftGraphAPI) to manage the Microsoft Graph functionality from PowerShell. At the conference, he’ll be talking about PowerShell on Raspberry Pi, and the making of his Azure-connected green house.

– ImportExcel: James O’Neill (working with Doug Finke) has probably created the single most useful community module there is: ImportExcel (https://www.powershellgallery.com/packages/ImportExcel) makes importing and exporting Excel data/xlsx files a snap and does not even require Office to be installed.

 James talks about becoming a PowerShell parameter Ninja, and on how to use multithreading in PowerShell to speed up tasks and do them in parallel, using his module Start-Parallel (https://www.powershellgallery.com/packages/Start-parallel).

– PSVersion: Ever wanted to turn the PowerShell version number into a meaningful friendly name? Then use PSVersion (https://www.powershellgallery.com/packages/PSVersion) from Jan Egil Ring!

  In his talks, Jan focuses on Azure Functions in a hybrid world, and Azure Policy Guest Configuration which in some respect works like Group Policies in the cloud and across domains and platforms.

– ADCSTemplateParser: Senior cloud architect Jan-Henrik Damaschke created ADCSTemplateParser (https://www.powershellgallery.com/packages/ADCSTemplateParser). He also worked on asynchronous PowerShell logging (https://www.itinsights.org/PowerShell-async-logging/) which is highly interesting: don’t let writing logs slow down or block your scripts! He’s explaining his module and concepts at one of his talks. His second talk focuses on real-time communication (basically the stuff done by messengers like WhatsApp) via SignalR and Azure Functions.

– PowerForensics: Security expert Jared Atkinson published PowerForensics (https://www.powershellgallery.com/packages/PowerForensics), a digital forensics framework for PowerShell. At the conference, Jared talks about detection engineering to uncover hacker techniques, and ways for enterprises to approach intrusion detections and responses at scale.

– Pester: This module is so important, it is part of Windows. The latest version is available at https://www.powershellgallery.com/packages/Pester. Pester is maintained by Jakub Jareš and is a PowerShell testing framework to make sure a script does what it is supposed to do, and won’t break when you add new things to it. At the conference, Jakub introduces version 5. If you have any question about Pester, make sure you bring it.

– ArcAdminTools: Co-founder of the Polish PowerShell user group Mateusz Czerniawski has published a collection of useful admin tools called ArcAdminTools (https://www.powershellgallery.com/packages/ArcAdminTools). At the conference, he talks about Azure Log Analytics (ALA) and sheds light on Microsoft Graph and what you can do with it.

– AADInternals: Dr. Nestori Syynimaa is a leading Office365 expert and has created AADInternals (https://www.powershellgallery.com/packages/AADInternals): It utilizes several internal features of Azure Active Directory, Office 365, and related admin tools and can be used as a Azure AD hacking and pen-testing tool. With his intimate knowledge of Azure and Office365, Nestori talks about Azure AD security and how it can be attacked and abused.

– cChoco: Paul Broadwith is a DSC expert and has created the extremely successful cChoco (https://www.powershellgallery.com/packages/cChoco) DSC resource to use Chocolatey with DSC. At the conference, Paul is tackling two extremely hot topics: using SSH for remoting instead of WinRM, and how to automate the setup of brand-new computer hardware using Boxstarter.

– PSWriteColor: Przemysław Kłys is a “discovery” of last year’s psconf.eu. He had never talked before at large conferences, yet his sessions rocked last year. Meanwhile, he is a regular speaker at large conferences and has published a great number of modules (https://www.powershellgallery.com/profiles/Przemyslaw.Klys), for example PSWriteColor (https://www.powershellgallery.com/packages/PSWriteColor): a wrapper around Write-Host to create beautiful colored output. At this year’s conference, he’ll use his set of free tools to create Active Directory and Office365 auto-documentation to word, excel, and HTML. Definitely a must-see.

– DBAchecks: Rob Sewell co-authored DBAchecks (https://www.powershellgallery.com/packages/dbachecks) together with Chrissy LeMaire: A testing framework for SQL Server to ensure it is (and continues to be) compliant with your requirements. And as a database admin, of course you’ll know dbatools (https://www.powershellgallery.com/packages/dbatools), the community tool filled with commands to easily automate database deployment and administration. Rob was in charge of the psconf.eu call for papers and manages the speakers. At the conference, he is talking about PowerShell Notebooks, part of Azure Data Studio, and how useful they can be for you.

– PSHTML: Stephane van Gulick defines himself in one sentence: “I love computers”. He is into DevOps, but also into HTML. His module PSHTML (https://www.powershellgallery.com/packages/PSHTML) can be used to create stunning reports and build entire responsive websites. At the conference he’s sharing how he discovered PowerShell classes and how you could benefit from classes, too.

– ISESteroids: Dr. Tobias Weltner originally created ISESteroids (https://www.powershellgallery.com/packages/ISESteroids) to make his life easier while adding missing functionality to the built-in PowerShell ISE. Soon, public interest turned this into a commercial-grade product for anyone working with Windows PowerShell and the PowerShell ISE. Tobias has started the psconf.eu conference and lives in Hannover.

Book Signing

Many of us have learned PowerShell using books, and we are humbled to have a number of renown PowerShell book authors from around the world with us. If you have learned by reading one of the books below, and still own your copy, bring it to have it signed by the author:

– Windows PowerShell 5.1 Biblia

  PowerShell Deep Dives

  Bartek Bielawski talks about PowerShell classes and how you can author DSC resources with it. He also helps you find your way into Git and as a team work with scripts in a safe and structured way.

– PowerShell Core et Windows PowerShell

  Windows PowerShell : Fonctionnalités avancées

  Windows PowerShell : Guide de référence pour l’administration système

  PowerShell Deep Dives

  Arnaud Petitjean talks about managing secrets such as passwords and access permissions

– PowerShell in Action

 Bruce Payette is a founding member of the PowerShell team and now with AWS. At the conference, Bruce will explain some of the more mysterious moving parts of the PowerShell architecture like PSHost, threads, and runspace pools. He’ll focus on how they work and how they differ in remoting, and if you ever attended one of his talks, you know the tons of practical and undocumented tricks that come with it.

– Windows PowerShell 5 – kurz und gut

  Thorsten Butz is a PowerShell trainer and “on-premises” fan, and at the conference talks about querying Wikidata with a glimpse of SPARQL.

– PowerShell 5: Windows-Automation für Einsteiger und Profis

  Windows PowerShell: Grundlagen & Scripting-Praxis für Einsteiger – alle Versionen

  Dr. Tobias Weltner is running psconf.eu and delivering PowerShell trainings throughout Europe.

Get Your Seat!

Don’t wait for too long and get your seat! In the past three years, psconf.eu sold out every time.

To sign up and reserve your seat, or to get a proposal for your boss, please visit the registration page (https://psconf.eu/register.html).

Signing up early has a number of advantages (for us, but also for you):

* Hotel accommodation is still reasonable, and there is a variety of flights available to Hannover Airport

* You have the guarantee to get a seat

* We are working on a PowerShell delegate jacket. Obviously, the jacket needs to go to production at some time. Anyone signing up until March 15 gets his or her guaranteed jacket size. We order jackets based on gender, so yes the jackets do look good for female delegates as well! Of course, we do our best in extrapolating jacket sizes and types for the rest but anyone signing up later gets a best-effort jacket size and type.

* Signing up early makes life for us a lot easier.

What if you signed up early and can’t come? While conference tickets are never refundable (or else a conference would be impossible to organize), they are transferrable at no cost.

We are looking forward to seeing you at the PowerShell Conference Europe 2020!

As a part of some larger automation that I was building, I needed to parse the validation result into a PowerShell object that can be used later in the orchestration. Parsing HTML isn’t what I needed but a little of digging gave some clues about the XML report that gets generated when we run this command.

Behind the scenes, the Test-Cluster command generates an XML every time it was run. This XML gets stored at C:\Windows\Temp. Looking at the XML you can easily notice that the schema was really designed to generate the HTML easily. So, it took a few minutes to understand how the tests are categorized and come up with the below script.

[CmdletBinding()]
param
(
    [Parameter(Mandatory = $true)]
    [String]
    $ValidationXmlPath
)

$xml = [xml](Get-Content -Path $ValidationXmlPath)
$channels = $xml.Report.Channel.Channel

$validationResultArray = New-Object -TypeName System.Collections.ArrayList

foreach ($channel in $channels)
{
    if ($channel.Type -eq 'Summary')
    {
        $channelSummaryHash = [PSCustomObject]@{}
        $summaryArray = New-Object -TypeName System.Collections.ArrayList

        $channelId = $channel.id
        $channelName = $channel.ChannelName.'#cdata-section'        
        
        foreach ($summaryChannel in $channels.Where({$_.SummaryChannel.Value.'#cdata-section' -eq $channelId}))
        {
            $channelTitle = $summaryChannel.Title.Value.'#cdata-section'
            $channelResult = $summaryChannel.Result.Value.'#cdata-section'
            $channelMessage = $summaryChannel.Message.'#cdata-section'

            $summaryHash = [PSCustomObject] @{
                Title = $channelTitle
                Result = $channelResult
                Message = $channelMessage
            }

            $null = $summaryArray.Add($summaryHash)
        }

        $channelSummaryHash | Add-Member -MemberType NoteProperty -Name Category -Value $channelName
        $channelSummaryHash | Add-Member -MemberType NoteProperty -Name Results -Value $summaryArray

        $null = $validationResultArray.Add($channelSummaryHash)
    }
}

return $validationResultArray

The input to the above script is the XML file that gets generated at C:\Windows\Temp. Once you run the script, you should see the output similar to what is shown below.

I have only added the property values that I really need in my scripts but you can look at the XML and then easily modify the above script to add other details as you need.

Comment on this Gist if you have any suggestions or have you version of the script.

So, I went on to create a rather long VBScript that uses WMI for collecting disk usage statistics from Windows servers and uses Excel COM object to generate spreadsheets that contain the reports. It certainly made my job easy. I just had to run this sitting at my local workstation and within a few seconds I would have the Excel report that I can mail to the IT manager.

But, those of you who worked on Excel COM objects and PowerShell would know that it is not the best thing. Working with VBScript is a pain. When that combined with COM object, the pain of writing and testing a script increases a few folds.

You would be glad to hear that you don’t have to do that anymore if you know PowerShell even a little bit. Thanks to ImportExcel.

ImportExcel allows you to read and write Excel files without installing Microsoft Excel on your system. With this module, there is no need to bother with the cumbersome Excel COM-object. With ImportExcel, creating Tables, Pivot Tables, Charts and much more has becomes a lot easier.

Before you try any of the following examples, install ImportExcel module from the PowerShell Gallery.

Here is the simple first example for you!

Get-Process | Select-Object Company, Name, Handles | Export-Excel

This a command exports the values of selected properties from the process object and opens an Excel spreadsheet automatically.

Here is another example from ImportExcel GitHub repository that generates charts.

$xlfile = "$env:TEMP\trendLine.xlsx"
Remove-Item $xlfile -ErrorAction SilentlyContinue

$data = ConvertFrom-Csv @"
Region,Item,TotalSold
West,screws,60
South,lemon,48
South,apple,71
East,screwdriver,70
East,kiwi,32
West,screwdriver,1
South,melon,21
East,apple,79
South,apple,68
South,avocado,73
"@

$cd = New-ExcelChartDefinition -XRange Region -YRange TotalSold -ChartType ColumnClustered -ChartTrendLine Linear
$data | Export-Excel $xlfile -ExcelChartDefinition $cd -AutoNameRange -Show

Finally, here is something I showed at the PowerShell Conference Europe 2019. This uses the speaker and session data JSON and generates a spreadsheet.

if (-not (Get-Module -ListAvailable -Name ImportExcel -ErrorAction SilentlyContinue))
{
    Install-Module -Name ImportExcel -Force
}

$speakersJson = 'https://raw.githubusercontent.com/psconfeu/2019/master/data/speakers.json'
$sessionsJson = 'https://raw.githubusercontent.com/psconfeu/2019/master/sessions.json'

$speakers = ConvertFrom-Json (Invoke-WebRequest -UseBasicParsing -Uri $speakersJson).content
$sessions = ConvertFrom-Json (Invoke-WebRequest -UseBasicParsing -Uri $sessionsJson).content

# All Sessions Sheet
$sessions | Select-Object Name, Starts, Ends, Track, Speaker | 
            Export-Excel -Path .\psconfeu2019.xlsx -WorksheetName 'All Tracks' `
            -Title 'PowerShell Conference EU 2019 - Sessions' `
            -TitleBold -TitleFillPattern DarkDown -TitleSize 20 `
            -TableStyle Medium6 -BoldTopRow

# Track sheets
foreach ($i in 1..3)
{
    $trackSessions = $sessions.Where({$_.Track -eq "Track $i"})
    $trackSessions | Select-Object Name, Starts, Ends, Speaker |
        Export-Excel -Path .\psconfeu2019.xlsx -WorksheetName "Track $i" `
        -Title 'PowerShell Conference EU 2019 - Track $i' `
        -TitleBold -TitleFillPattern DarkDown -TitleSize 20 `
        -TableStyle Medium6 -BoldTopRow        
}

# Add Speakers sheet
$speakers | Export-Excel -Path .\psconfeu2019.xlsx -WorksheetName 'Speakers' `
    -Title 'PowerShell Conference EU 2019 - Speakers' `
    -TitleBold -TitleFillPattern DarkDown -TitleSize 20 `
    -TableStyle Medium6 -BoldTopRow 

# Add chart for speaker country number
$chartDef = New-ExcelChart -Title 'PowerShell Conference EU 2019 - Speakers' `
                    -ChartType ColumnClustered `
                    -XRange Name -YRange Count `
                    -Width 800 -NoLegend -Column 3 

$speakers | Group-Object -Property Country | Select-Object Name, Count |  Sort-Object -Property Count -Descending |
    Export-Excel -path .\psconfeu2019.xlsx -AutoSize -AutoNameRange -ExcelChartDefinition $chartDef -WorksheetName SpeakerCountryChart -Show

There are many other ways you can use this module in creating report dashboards. The GitHub repository contains several examples that you can use as a starting point.

Polaris is a cross-platform, minimalist web framework for PowerShell. This is an experimental module but stable enough to try it out. Before you can try out the examples, install Polaris module from PS Gallery.

Here is a quick example of how we create a HTTP endpoint.

New-PolarisGetRoute -Path "/helloworld" -Scriptblock {
    $Response.Send('Hello World!')
}

Start-Polaris

Once these commands are executed, if you access http://localhost:8080/helloworld in a browser, you will see ‘Hello World!’ returned as response. In the above example, there is just one endpoint or route. This implements HTTP GET method. Whenever this route gets accessed, the PowerShell commands specified as an argument to -Scriptblock parameter gets invoked. In this example, we are just using the $Response automatic variable that Polaris provides and use the .Send() method to send the response back to browser.

Similar to this, you can create other HTTP routes as well for POST, PUT, DELETE, and so on. In the next example, you will see how we can combine what PSHTML provides with Polaris.

Save the following script as content.ps1 in a folder of your choice.

html {
    head {
        title "Example 2"
    }

    body {
        h1 {"This is just an example of using PSHTML as view engine for Polaris"}

        $Languages = @("PowerShell","Python","CSharp","Bash")

        "My Favorite language are:"
        ul{
          foreach($language in $Languages){
               li {
                    $Language
               }
           }
        }
    }

    Footer {
        h6 "This is h1 Title in Footer"
    }
}

The following script is the route that we need to create for displaying the HTML content from the above script.

New-PolarisGetRoute -Path "/languages" -Scriptblock {
    $response.SetContentType('text/html')
    $html = . "C:\scripts\content.ps1"
    $response.Send($html)
}

Start-Polaris -Port 8080

Once you start Polaris and load the routes, you can access http://localhost:8080 to see the content generated from a PS1 script. It should be like this!

Hope you got a hang of what you can achieve with PSHTML and Polaris combined. I have built dashboards with simply this combination and nothing more. In the future posts, I will show one such example from my demo at PowerShell Conference Europe 2019.

Do that no more! PSHTML is here.

PSHTML is a cross platform Powershell module to generate HTML markup language within a DSL on Windows and Linux.

With PSHTML, you can write HTML documents the same way you write PowerShell scripts. You can use every possible language artifact and generate HTML code dynamically based on the input and context. Let us see a quick example!

Before you go forward and try out the example below, install the module from the PS Gallery.

html {
    head {
        title 'This is a test HTML page'
    }

    body {
        h2 'PSHTML is cool!'

        p {
            'Using PSHTML, offers code completion and syntax highlighting from the the default powershell language.'
            'As PSHTML respects the W3C standards, any HTML errors, will be spotted immediately.'
        }
    }

    footer {
        p {
            'This is footer. All credits reserved to PSHTML'
        }
    }
}

This generates the following HTML text.

This is very easy. Let us give it some styles.

html {
    head {
        title 'This is a test HTML page'
    }

    body {
        h2 'PSHTML is cool!' -Style 'color:blue;background-color:powderblue'

        p -Style 'color:red' {
            'Using PSHTML, offers code completion and syntax highlighting from the the default powershell language.'
            'As PSHTML respects the W3C standards, any HTML errors, will be spotted immediately.'
        }
    }

    footer {
        p -Style 'color:green' {
            'This is footer. All credits reserved to PSHTML'
        }
    }
}

This results in a HTML page as shown below!

This is all good but very trivial. Let us try generating some tables and we will use bootstrap for styles.

html {
    head {
        title 'Top 5 Processes - HTML report Powered by PSHTML'
        Link -href 'https://maxcdn.bootstrapcdn.com/bootstrap/4.1.3/css/bootstrap.min.css' -rel 'stylesheet'
        script -src 'https://ajax.googleapis.com/ajax/libs/jquery/3.3.1/jquery.min.js' -type 'text/javascript'
        script -src 'https://cdnjs.cloudflare.com/ajax/libs/popper.js/1.14.3/umd/popper.min.js' -type 'text/javascript'
        script -src 'https://maxcdn.bootstrapcdn.com/bootstrap/4.1.3/js/bootstrap.min.js' -type 'text/javascript'
    }

    body {
        $topFiveProcess = Get-Process | Sort CPU -descending | Select -first 5 -Property ID,ProcessName,CPU
        div -Content {
            table -class 'table table-striped' {
                thead {
                    tr {
                        th {
                            ID
                        }
        
                        th {
                            ProcessName
                        }
        
                        th {
                            CPU
                        }
                    }
                }
                                
                Tbody {
                    foreach ($process in $topFiveProcess)
                    {
                        tr {
                            td {
                                $process.id
                            }
            
                            td {
                                $process.Name
                            }
            
                            td {
                                $process.CPU
                            }
                        }
                    }
                }
            }
        }
    }

    footer {
        p -Class 'lead' -Content 'All Credits Reserved. PSHTML!'
    }
}

This results in a nice table shown below!

See how easy was that!? I will stop this article here as this is not a PSHTML tutorial. Hope you have got a good idea about how useful the module is. There are several community members who did some great work with PSHTML. Check out their work as well.

This article uses a custom version of Pester which is not an official version yet. This custom version of Pester with these changes is currently available here.

Pester is used both as an unit testing framework as well as an operational validation tool. In both the use cases, output report plays an important role in presenting the test results. This article is about the output report with and added capability.

Below is a Pester test script with few tests.

param(
    [Parameter(Mandatory = $True)]
    [string]$ConfigPath
)

$Config = Get-Content -Path $ConfigPath | ConvertFrom-Json

Describe "Describing validation tests post deployment" {

    Context "Post deployment validation tests for services" {
        BeforeAll {
            $Config.service.expectedconfiguration | ForEach-Object -Process {

                $Name = $_.Name
                $Status = $_.Status
                $StartMode = $_.StartMode
                $Service = Get-Service -Name $Name

                it "Service $Name status should be $Status" {
                    $Service.Status | Should -Be $Status
                } 

                it "Service $Name startmode should be $StartMode" {
                    $Service.StartType | Should -Be $StartMode
                } 
            }
        }
    }

    Context "Post deployment validation tests for folder permission" {

        $Config.folderpermission.expectedconfiguration | ForEach-Object -Process {

            $User = $_.user
            $Permission = $_.permission
            $Path = $_.path

            it "user $User should have $Permission permission on path $Path" {
                $Access = (Get-Acl -Path $Path).Access | Where-Object -FilterScript { $_.IdentityReference -eq $User }
                $Access.FileSystemRights | Should -Contain $Permission
            } 
        }
    }

    Context "Post deployment validation tests for firewall rule" {

        $Config.firewallrule.expectedconfiguration | ForEach-Object -Process {

            $Rulename = $_.rulename
            $Direction = $_.direction
            $Rule = Get-NetFirewallRule -Name $RuleName -ErrorAction SilentlyContinue

            it "A Firewall rule with name $RuleName should be available" {
                $Rule | Should -Not -BeNullOrEmpty
            } 

            it "Firewall rule $RuleName should be allowed for $Direction connection" {
                $Rule.Direction | Should -Not $Direction
            } 
        }
    }
}

Data for above test script is shown below.

{
    "service": {
        "suggestion": {
            "startmode": "Open PowerShell as administrator and run 'Set-Service -Name {0} -StartType {1}'",
            "status": "Make the service '{0}' in {1} state."
        },
        "expectedconfiguration": [
            {
                "name": "BITS",
                "status": "running",
                "startmode": "automatic"
            },
            {
                "name": "wuauserv",
                "status": "running",
                "startmode": "automatic"
            }
        ]
    },
    "folderpermission": {
        "suggestion": {
            "message": "Give {0} permission for {1} user on {2} folder."
        },
        "expectedconfiguration": [
            {
                "path": "c:\\Deployment\\config",
                "user": "RDFC\\Test",
                "permission": "FullControl"
            },
            {
                "path": "c:\\Deployment\\files",
                "user": "RDFC\\kvprasoon",
                "permission": "FullControl"
            }
        ]
    },
    "firewallrule": {
        "suggestion": {
            "rulename": "Open wf.msc and create an {0} rule with name '{1}'.",
            "direction": "Open wf.msc and create the firewall rule '{0}' for {1} connection."
        },
        "expectedconfiguration": [
            {
                "rulename": "Rule1",
                "direction": "Inbound"
            },
            {
                "rulename": "Rule2",
                "direction": "Outbound"
            }
        ]
    }
}

In a nutshell, the above pester test script will test Service status, file system permissions, and firewall rules by reading the data from the JSON configuration file.

You can execute the test as shown below.

Invoke-Pester -Script @{Path = 'C:\temp\OpsValidation.tests.ps1' ; Parameters = @{ConfigPath = 'c:\temp\OpsValidationConfig.json'}}

This will show the test result in the console with summary.

When comes to the reporting side, Invoke-Pester has parameters which will create a nunit based XML test result file. Nunit XML reports are mostly rendered using a report reader or by converting it to html format. There are many tools in the market that convert Nunit XML to nice UI reports. Reportunit (now extent-reports) is my favorite. But this tool uses jquery and CSS which are referenced online, hence made me to think about adding this capability to Pester. Since the report is in XML format, my choice was to go with XSLT. In short, XSL is a steroid for XML. This article doesn’t cover anything on how it transforms XML to HTML. Here is a simple example for creating an XSL targeting an XML. Now we know that XSL can transform an XML to HTML but how to add this in the Nunit report generated by Pester. This is done by adding a new parameter to accept the XSL path and putting it as a stylesheet reference in XML generated by pester. I’ve named the parameter as –XSLPath. Below is an example execution with the XSL path.

Invoke-Pester -Script @{Path = 'C:\temp\OpsValidation.tests.ps1' ; Parameters = @{ConfigPath = 'c:\temp\OpsValidationConfig.json'}} -OutputFile c:\temp\OpsValidation.xml -OutputFormat NUnitXml -XSLPath c:\temp\OpsValidation.xsl

Once executed, open the XML report using a web browser to see the magic !

So far so good! But here comes the interesting part of this article.

We are now able to have a report in html, hence it is easy to see the test failures (human beings are interested in analyzing failures than success !) in a web browser. How about adding some suggestions/remarks for the end user/support engineers to fix the issue and make the test pass?

Well then that has to be done for each testcase. Yes for each test cases. This is done by adding a new parameter to the it function in Pester.

Below is an example with the suggestion feature.

it "Service BITS status should be Running" {
     $Service.Status | Should -Be "Running"
 } -Remark "Open services.msc and start BITS service"

But there is a caveat, we can add a parameter to the It function, but how to add this remark in the report XML. The report XML is in nunit format and it has predefined layout. The test result layout has to be followed in the report and therefore any Nunit report readers can parse the result.

Well, Pester report doesn’t use all the attributes of the nunit test result layout and I found the Label attribute as a candidate for adding remarks. So with the remark support for testcases, below is the final pester test script.

Script: https://gist.github.com/kvprasoon/bec40fa50d6975fcdafa6536b61cf1aa

Test Configuration: https://gist.github.com/kvprasoon/2dd5fc64eec0653e4bdde6a18da526ff

Lets execute and see the report with suggestions.

Invoke-Pester -Script @{Path = 'C:\temp\OpsValidation.tests.ps1' ; Parameters = @{ConfigPath = 'c:\temp\OpsValidationConfig.json'}} -OutputFile c:\temp\OpsValidation.xml -OutputFormat NUnitXml -XSLPath c:\temp\OpsValidation.xsl

After opening the generated report in a web browser.

If you want to explore and see the code changes, Pester with these changes is currently available here.

A few weeks ago Rob (I knew him from his Dell days) tweeted about a probable topic for an upcoming episode.

Having written a couple of published books on PowerShell DSC and being in the infrastructure automation space, Infrastructure as Code (IaC) is close to my heart. Therefore, I just jumped in and said count me in!

We started with a discussion on IaC but eventually it lead to Rob naming what we discussed as a vision for the continuously integrated data center! Indeed, that is (should be) the goal. Rest is what you will hear in this episode of the podcast.

This was a fun episode. Let me know what your thoughts are. I will certainly find some time to write about this vision and the objectives.

Posts in this series

1. Distributed and Flexible Operations Validation Framework - Introduction
2. Garuda - Architecture and Plan
3. Garuda - Session Demo From #PSConfEU

In the earlier parts of this series, I introduced you to the concepts and design of Garuda framework. I demonstrated a proof-of-concept version of this at PowerShell Conference Europe.

The recording of that session is available.

Instead of writing another article about how the POC works, I thought it is easier for you to see it in action.

I am working on a complete overhaul of the framework and will have a new version soon on GitHub. Stay tuned!

While there may be many reasons for this, the organizing committee of PowerShell Conference Asia decided that we invite women in tech (infrastructure automation, Cloud, and DevOps) to this year’s edition of our conference.

We have opened up registration of intent to attend the conference. All you have to do is just provide your details. We will select five random registrations and give them full 3 day pass to the conference at no cost. For five more, we will offer higher discount. The organizing committee will decide the percentage of discount.

The free entry or the discounted entry entails you the conference pass only. If you need to travel to Bangalore to attend this conference, attendee must bear the travel and accommodation expenses.

This registration will end on 15th August 2019. We will announce the selected registrations on 20th August 2019.

Please share this registration information and help us enable women in the infrastructure automation, cloud, and DevOps space to attend PowerShell Conference Asia 2019!

Posts in this series

1. Distributed and Flexible Operations Validation Framework - Introduction
2. Garuda - Architecture and Plan
3. Garuda - Session Demo From #PSConfEU

In the first part of this series, I mentioned the reasoning behind starting development of a new framework for operations validation. Towards the end, I introduced Garuda — a distributed and flexible operations validation framework. There are certain principles that drove the design of this framework — Distributed, Flexible, and Secure.

In this part of the series, you will see the architecture proposal and the plan I have to implement these features.

Architecture

To support the principles described, the framework needs a few moving parts. These moving parts provide the flexibility needed and gives you a choice of tooling.

At a high-level, there are five components in this framework.

Test Library

The test library is just a collection of parameterized Pester test scripts. The parameterization helps in reuse. As a part of the Garuda core, you can group these tests and then use the publish engine to push the tests to remote targets. The tags within the tests are used in the test execution process to control what tests need to be executed within the test group published to the remote target.

Garuda Core

This is the main module which sticks the remaining pieces together. This is what you can use to manage the test library. This core module will provide the ability to parse the test library and generate the parameter information for each test script which is eventually used in generating a parameter manifest or configuration data. One of the requirements for this framework is to enable grouping of tests. The Garuda core gives you the ability to generate the test groups based on what is there in the library. You can then generate the necessary parameter manifest (configuration data) template for the test group that you want to publish to remote targets. Once you have the configuration data prepared for the test group, you can publish the tests to the remote targets.

Publish Engine

The publish engine is responsible for several things.

This module generates the necessary configuration or deploy script that does the following:

1. Install necessary dependent modules (for test scripts from a local repository or PowerShell Gallery)
2. Copy test scripts from the selected test group to the remote target
3. Copy the configuration data (sans the sensitive data) to the remote target
4. As needed, store credentials to a credential vault on the remote target
5. Copy the Chakra engine to the remote targets
6. if selected, create the JEA endpoints for operators to retrieve test results from the remote targets
7. If selected, create scheduled tasks on the remote target for reoccurring test script execution

Once the configuration or the deploy script is ready, the publish engine can enact it directly on the remote targets or just return the script for you to enact it yourself.

The publish engine is extensible and by default will support PowerShell DSC and PSDeploy for publishing tests to the remote targets. Eventually, I hope the community will write providers for other configuration management platforms / tools. There will be abstractions within the engine to add these providers in a transparent manner.

The publish engine helps in securing the test execution by storing sensitive configuration data in a vault. It also implements the scheduled tasks as either SYSTEM account or a specified user. The JEA endpoints configured on the remote targets help us securely retrieve the test results with least privileges needed.

You can publish multiple tests groups to the same remote target. This helps implement the flexibility that IT teams need in the operations space for a given infrastructure or application workload. There can be multiple JEA endpoints one for each team publishing the test groups.

Chakra

The Chakra is what helps execute the tests in test group(s) on the remote targets. It has the test parameter awareness and can use the published configuration data and the sensitive data stored in the vault for unattended execution of tests. Chakra is also responsible for result consolidation. It can be configured to retain results for X number of days. All the test results for each group get stored as JSON files and are always timestamped. The scheduled tasks created on the remote targets invoke Chakra at the specified intervals. Chakra also contains the cmdlets that are configured for access within the JEA endpoints. Using these endpoints, test operators can retrieve the test results from a central system from all the remote targets.

Report Engine

The report engine is final piece of this framework that enables retrieving the results from the remote targets and transforming those results into something meaningful for the IT managers. By default, there will be providers for reports based on PSHTML, ImportExcel, and UniversalDashboard. The report engine provides that abstractions for community to add more reporting options.

The Plan

The initial release of the framework or what I demonstrated at the PowerShell Conference Europe was just a proof of concept. I am planning on breaking down the framework into different core components I mentioned above. The GitHub repository for the framework will have issues created for each of these components and I will start implementing the basics. The 1.0 release of this framework will have support for every detail mentioned above and will be completely useable and functional for production use.

What about the naming?

The names Garuda and Chakra are from the Hindu mythology and their meaning is connected to the concepts I am proposing for this framework. Garuda is the bird from Hindu mythology. It has a mix of human and bird features. It is deemed powerful and is the vehicle of the Hindu god Vishnu. It can travel anywhere and is considered the king of birds. The Chakra the weapon that lord Vishnu carries. It is used to eliminate evil. This is also known as the wheel of time. Garuda is the vehicle that transports lord Vishnu and his Chakra to places where there is evil.

The Garuda Core combined with the Publish engine can take your operational validation tests to your remote targets. Chakra is the way to perform operations validation to ensure that your infrastructure is always healthy and functional.

In the next article in this series, you will see the framework in action.

I quickly created a new PowerShell module that wraps around the public API for the public APIs!

You can install this module from the gallery as well.

Install-Module -Name PSPublicAPI -Force

There are four commands in this module.

The commands are pretty much self-explained and you can find the docs for each command here.

Posts in this series

1. Distributed and Flexible Operations Validation Framework - Introduction
2. Garuda - Architecture and Plan
3. Garuda - Session Demo From #PSConfEU

[6/18/2019 – Added PoshSpec]

Operations validation using PowerShell and Pester has been one of my favorite topics and I have both personal and professional interest in this area. I have invested good amount of time experimenting with the existing frameworks and creating a couple of my own. One of my PowerShell Conference EU sessions was around this topic and a new framework that I am developing. This session was well received and there was good amount of interest in this new framework.

In this series of articles, I will write about the need for a new framework and introduce the new framework which I demonstrated at PowerShell Conference Europe. There is still a lot of work to be done and this series will act as a way to express where I want to see this whole framework and the end goals.

In this part, you will see what are the available options today for operations validation and / or similar use cases and what their limitations are. Towards the end, I will talk about the desired state requirements for a distributed and flexible operations validation framework.

The Current State

My work with operations validation started back in 2016 when I had first demonstrated using Pester for validating the functional state of clusters. This first implementation was tightly coupled with PowerShell DSC resource modules and needed configuration data supplied with DSC configuration documents to perform the operations validations. This model worked very well for infrastructure that was configured using DSC. However, this is not really a generic or flexible framework for running operations validation.

Microsoft Operations Validation Framework (OVF)

Around the time I started working on the operations validations bundled with infrastructure blueprints, PowerShell team published an open source version of a framework meant for operations validation. This implements operational tests bundled with regular PowerShell modules. The cmdlets in the framework can discover the operations validation tests packaged in the installed modules and invoke the same. You can specify a hashtable of values as the test script parameters. This is a distributed test execution model. Tests can be copied to all nodes and invoked on the node. This is certainly what I wanted to start with. But, the tight coupling between the modules and tests is not what I really want. Instead, I want to be able to distribute chosen tests as groups of tests to any node. I could have written a wrapper script around OVF and achieve what I wanted but there are other limitations.

Packaging tests as modules is an unnecessary overhead. If you have a huge library of tests and you need to determine the tests that run on the remote targets dynamically, you also need to be able to generate modules dynamically. And, then, you need to find a way to distribute those modules among the target nodes and also ensure that these are kept up to date as you update the central repository.

The test parameters are passed as a hashtable and therefore if you need to invoke the tests in an unattended (such as a schedule task) manner, you need to ensure that you have a wrapper script that reads some sort of configuration data and translates that into relevant parameters. But, then you need a way to publish that configuration data as well to the remote targets.

PSHealthz

PSHealthz by Brandon Olin provides a web service endpoint to invoke tests packaged or published using OVF. This is an implementation of the Health Endpoint Monitoring Pattern using PowerShell. The available tests can be retrieved using the /health endpoint. Tests can be executed on the target node using the query parameters on the /health endpoint. PSHealthz is more of a way to list and invoke tests on the target nodes using the REST API but the limitations of OVF I mentioned above still exist.

Remotely and PSRemotely

Remotely is an open source PowerShell module from Microsoft that can be used for running Pester tests remotely — no surprises there! You can specify a set of remote targets in a file called machineconfi.csv and then use the Remotely keyword inside the It scriptblock for running the tests on the remote targets. This module has several drawbacks and has been more experimental than anything remotely useful (pun intended!). In fact, it has been more than 3 years since there was any update. Although the tests run on the remote node (using PowerShell remoting), they are essentially triggered from a central location in a fan-out method. Therefore, this module implements centralized test execution and reporting.

PSRemotely was born out of the need for running tests on a bunch of remote nodes while eliminating all of Remotely drawbacks and providing better control over what runs when and where. This module uses DSC type configuration data for providing test parameters for each remote node. In fact, we have implemented a complete validation suite using PSRemotely before writing one more internal framework for operations validation of clusters. The major drawback of this module was the need to enable CredSSP so that the delegated credentials, when needed, can be used on remote targets. Also, there was no infrastructure awareness in PSRemotely. The number and type of tests running gets determined using the configuration data and we had no control over grouping tests based on the type of infrastructure. With PSRemotely, the execution of tests is distributed and reporting is centralized. Therefore, PSRemotely implements a hybrid model. With this framework, Pester tags is the only way to separate tests into groups.

DBAChecks and pChecksAD

Both DBAChecks and pChecksAD implement a more centralized test execution and reporting. All tests stay on the local system and you can design these tests to target remote systems using a cmdlet provided method or write your tests to use PowerShell remoting to target remote systems. These are purpose built modules but you can take clues from how they implemented these modules and write one for your specific use case. These are great at what they are doing but not something that would satisfy my requirements for a distributed and flexible operations validation framework.

PoshSpec

PoshSpec is another great project that enables simplified infrastructure validations with a way to extend it very easily. PoshSpec provides a DSL for infrastructure validations for known resources. For example, you can write a test in PoshSpec DSL to verify if a hotfix is installed without worrying about how to get a list of hotfixes. There is currently a limited set of resources. PoshSpec is centralized test execution. It does not yet support remote execution but there is an RFC on that. I created this issue back in 2016 and continued to work on my own frameworks for achieving what I really need.

The Desired State

You have seen, so far, options available for performing operations validation. You have also read about the limitations that these frameworks or modules pose. I will, now, translate these limitations into the requirements for a new operations validation framework.

Distributed

The new framework needs to support distribution (publishing) of tests to remote targets and should offer different methods for test distribution. For example, I should be able to publish tests to remote targets using PowerShell DSC or Ansible or Chef or Puppet.

The new framework should support distributed test execution. I want to be able to invoke tests on-demand or on a scheduled basis on the remote targets. The input parameters or configuration data needed for the tests should be local but the framework should provide a way to publish the configuration data as well. And, the secrets within the configuration data should be encrypted.

Flexible

The new framework should be flexible enough to allow integration with different other modules or technologies. For example, I had mentioned already that the test distribution should support more than one method.

Within infrastructure management, there will be more than one team involved in bringing up the infra. For example, if there is a SQL cluster that is being managed, there may be a team that is solely responsible for OS deployment & management whereas another takes care of SQL management. Now, each of these team will have their own operations validation tests. The new framework should enable a way to publish multiple test groups to the remote targets and execute and report them independently.

From a reporting point of view, the new framework should be capable of supporting multiple reporting methods like HTML, Excel, and so on.

Secure

The tests running on the remote targets need input parameters and this may include secure strings and secrets. Since this configuration data needs to reside on the target nodes, the sensitive data should be stored in a safe manner. For example, credentials should go into a vault such as Windows Credential Manager. The new framework should support this.

The result retrieval from the remote targets happens at a central console. For this, the test operators need access only to invoke the test result retrieval from the remote targets. The framework should support least privileged way of doing this such as implementing a JEA endpoint.

Introducing Garuda

I have been experimenting a bit trying to implement a totally new framework that satisfies most if not all of the desired state requirements. This is still in a proof-of-concept phase. There is not even documentation around how to use this yet. This is what I demonstrated at the PowerShell Conference EU 2019 a week ago. I named this framework Garuda. I will write about the naming choice in the next post.

Today’s article is an introduction to the thought process behind Garuda. In the next post, I will explain the architecture of Garuda and talk about how some of the desired state requirements are implemented.

BTW, just before my session at the EU conference, I had a couple of hours to kill and created this logo for the Garuda framework. You will know the meaning of this in the next post.

For example, https://localhost:443/?name=test&age=25&company=powershell+magazine.

As you see in the above example, the first parameter in the query string should be prefixed with question mark and the subsequent parameters are separated by an ampersand. If there are spaces in a parameter value the spaces should be replaced with a plus sign. This can be coded easily in PowerShell but there is a better way using the System.Web.HttpUtility class in .NET.

The ParseQueryString method in the httputility class parses the URL and gives us a key-value collection. To start with, we can provide this method an empty string.

$nvCollection = [System.Web.HttpUtility]::ParseQueryString([String]::Empty)

We can then add the key value pairs to this collection.

$nvCollection.Add('name','powershell')
$nvCollection.Add('age',13)
$nvCollection.Add('company','automate inc')

Once the parameters are added to the collection, we can build the URI and retrieve the query string.

$uriRequest = [System.UriBuilder]'https://localhost'
$uriRequest.Query = $nvCollection.ToString()
$uriRequest.Uri.OriginalString

This is it. I created a function out of this for reuse.

function New-HttpQueryString
{
    [CmdletBinding()]
    param 
    (
        [Parameter(Mandatory = $true)]
        [String]
        $Uri,

        [Parameter(Mandatory = $true)]
        [Hashtable]
        $QueryParameter
    )

    # Add System.Web
    Add-Type -AssemblyName System.Web

    # Create a http name value collection from an empty string
    $nvCollection = [System.Web.HttpUtility]::ParseQueryString([String]::Empty)

    foreach ($key in $QueryParameter.Keys)
    {
        $nvCollection.Add($key, $QueryParameter.$key)
    }

    # Build the uri
    $uriRequest = [System.UriBuilder]$uri
    $uriRequest.Query = $nvCollection.ToString()

    return $uriRequest.Uri.OriginalString
}

psconf.asia is updated to feature the confirmed speakers and this list includes experts from the USA, Australia, Europe, and India. As I write this, we are yet to finalize a few more speakers and I am sure we will have a fully loaded agenda for all three days. The pre-conf workshops include content from level 100 (PowerShell 101) to CI / CD for PowerShell professionals. On the pre-conf day, there is a track dedicated for deep-dive sessions for the attendees who are already comfortable writing PowerShell scripts and modules.

At this point in time, we have opened the early bird discount sale (15% on the original ticket price). The tickets are priced in INR and a 3 day pass at this point in time costs less than 100 USD.

You can get the 3 day (includes pre-conf day) early bird pass @ https://imjo.in/N8t8G7 and the 2 day (only full-conf days) early bird pass @
https://www.instamojo.com/@tecoholic/l2f610b18b43c4deb9f534b1082b7f414/

If you have a group of 5 or more interested in attending this year’s conference, reach out to us at get-help@psasia.org. We will let you know what best we can do for your group.

We have already received a few ticket sales and very happy with the progress in last couple of days. Can’t wait to see you all in September.

I had immediately verified that the API was available in version 1809.5 as well. So, I wanted to build another PowerShell module that has similar and/or more features than the current module that ships with version 1812. Also, the goal was to ensure that I can use this module in my build process to add the newly deployed servers and clusters to Windows Admin Center in an automated manner.

Note: This module works with Windows Admin Center 1809.5 and above.

This module can be installed from PowerShell Gallery:

Install-Module -Name PSWindowsAdminCenter <br>

This project is available in my GitHub repository. I have a few TODOs:

• Add Export option to Get-WacConnection command so that you can export the connections details to a CSV file.
• Add Import option to Add-WacConnection command so that you can import all connections from a CSV file.
• Update WindowsAdminCenterDsc module to use the PSWindowsAdminCenter instead of the module that ships with WAC.

If you see any issues or would like to see new features, feel free to create an issue.

Ever since WAC was released, one thought I had was to automatically onboard the servers and clusters that I want to manage within WAC right after their deployment is complete. There was no API that was available to do this earlier.

With the release of the WAC version 1812 (insider preview), there are a couple of PowerShell modules that are bundled with WAC. This internally uses the REST API and wraps around the same for a few management tasks.

When I saw this, I immediately explored a design to implement DSC resources for WAC install/uninstall and configuration. And, the result is here: https://github.com/rchaganti/WindowsAdminCenterDsc

This works only with Windows Admin Center 1812 insider preview and above.

This REST API is available in 1809.5 as well and I am working on creating a PowerShell to wrap that API as a set of cmdlets. I will update this DSC resource module as well without breaking the current DSC resource design.

This module contains a set of resources to install Windows Admin Center (WAC) and configure WAC feeds, extensions, and connections.

For complete documentation, see https://windowsadmincenterdsc.readthedocs.io/en/latest/.

Previously, I have written an article covering a PowerShell module for managing Royal TS Documents which is built in to Royal TS.

In that article I’ve showcased a script called Update-RoyalFolder.ps1, which could replicate server computer objects from a specified Active Directory domain or Organizational Unit (OU).

This was very useful as the script could be scheduled to update a Royal TS connections document, for example on a daily basis.

The new Dynamic Folder Script feature allows you to configure a script and the interpreter which populates the dynamic folder content.

We start by creating a new folder of the Dynamic Folder type:

Give it a meaningful name, such as the source we are going to dynamically retrieve data from:

On the Dynamic Folder Script window, we can choose PowerShell to be the script interpreter:

You will get an example script which shows what kind of objects is expected, as well as how to convert them to JSON (which is the output format Royal TS expects):

After modifying the options, such as credentials, click OK. When right-clicking the folder we create, we can see that we have the option of reloading the folder:

Clicking this will trigger the PowerShell script we just saw on the Dynamic Folder Script window, and the folder should now look like this when using the provided example script.

According to the documentation, there are two options available for reloading a dynamic folder:

• Automatically reload folder contents – If checked, Royal TS will automatically reload the folder contents when the document is opened.
• Persist (cache) folder contents – If checked, Royal TS will save (cache) the contents of this dynamic folder within the document.

By default, none of the two options is enabled.

In order to populate the dynamic folder with our own data using PowerShell, we can build custom objects with the necessary properties:

[PSCustomObject]@{
        Name                   = $PSItem.Name
        Type                   = 'RemoteDesktopConnection'
        ComputerName           = $PSItem.Name
        CredentialName         = 'DOMAIN\username'
        Path                   = MySubfolder
    }

This example creates an object to be used with a Remote Desktop Connection. If you want to build other connection types, see the documentation for RoyalJSON and Dynamic Folders.

Next, we need to put all the objects we have created in a hash table called ‘Objects’, as this is what the RoyalJSON format expects:

$RoyalTSObjects = @{}
$null = $RoyalTSObjects.Add('Objects',$Servers)

The final piece is to convert the hash table to JSON format, which is very convenient to do in PowerShell:

$RoyalTSObjects | ConvertTo-Json

As you can see, for a PowerShell user it is very straightforward to build a script for dynamically populating a folder with connection objects in Royal TS.

We can populate it with any data we can retrieve from PowerShell.

We will start by looking at an example on how to accomplish this using Active Directory as the data source.

You need to install Remote Server Administration Tools (RSAT) in order to leverage the Active Directory module for PowerShell from a workstation. Starting with Windows 10 October 2018 Update, RSAT is included as a set of “Features on Demand” in Windows 10 itself. From PowerShell, it can be installed using this command:

Add-WindowsCapability -Online -Name Rsat.ActiveDirectory.DS-LDS.Tools~~~~0.0.1.0

A Royal TS community member has created a YouTube video explaining more details about retrieving data from Active Directory for building a dynamic folder in Royal TS, such as building the Dynamic Folder sub folder structure based on the Organizational Unit structure the computer objects is retrieved from.

You can use your existing skills for retrieving the computer accounts you want from Active Directory. I am re-using an existing function I have created called Get-ServerFromAD. This will retrieve computer accounts with an operating system name starting with Windows Server*, exclude Cluster Name Objects and include computer accounts which have logged on during the last number of days specified (30 is the default).

You can find the complete script here.

I would recommend to first run the script manually in order to verify that data can be retrieved from Active Directory. You may also want to customize options such as domain name and credentials. I am using the script from a non-domain joined laptop, hence I need to specify credentials.

When the script is customized and verified, paste it in the Dynamic Folder Script window and click OK:

After re-loading the document, you should now see server computer accounts from Active Directory:

I have also created a script for getting computer names from System Center Virtual Machine Manager (both hosts and virtual machines), which can be used to populate a dynamic folder:

In my example script I’ve created a flat structure, putting all computer names in a subfolder called VMM. Here it is possible to do all sorts of creative things, such as use Group-Object on the OperatingSystem property and then create SSH connections for Linux machines and RDP connections for Windows machines.

I plan to create other scripts to retrieve computer names from other sources, such as Azure, Amazon Web Services, and VMware. Royal Applications has a dedicated repository for various automation scripts – created both by the Royal TS team and the community – where I also will submit Pull Requests for my contributions.

Using PowerShell Core as Script Interpreter

By navigating to the Royal TS Options, it is possible to modify Script Interpreter settings in the Advanced section:

By default, the PowerShell Script Interpreter is configured with the following path:

%windir%\System32\WindowsPowerShell\v1.0\powershell.exe

If you rather want to leverage PowerShell Core as the engine for the PowerShell Script Interpreter, simply change the path to either pwsh.exe (as it is available in the System Path):

Alternatively, specify the full path:

C:\Program Files\PowerShell\6\pwsh.exe

I have been using PowerShell Core without issues for the 2 Dynamic Folder Scripts I have showed in this article. For the Active Directory Dynamic Folder Script, PowerShell Core is using ~2.5 seconds to run against my lab Active Directory instance, while Windows PowerShell is using ~4 seconds.

Summary

In this article we have looked at how the dynamic folder script feature in Royal TS can be used to dynamically creating Royal TS connection objects based on data gathered from a PowerShell script.

We also looked at different sources we can retrieve data from, such as Active Directory and System Center Virtual Machine Manager.

Bonus tip

If you are a Microsoft MVP, you can get an NFR license for Royal TS by sending an e-mail to support (at) royalapplications (dot) com with a link to your MVP profile.

Like the previous year, we will have a day-long pre-conference workshops this year too. In addition to that, we are introducing pre-conference deep dive talks. While the workshops are targeted more towards the beginners, the pre-conference deep dive talks are meant for those of you who are already at level 150 or 200.

The main conference starts on 20th September 2019. The two days of this conference will have exciting in-depth talks by experts from across the world.

Today, we are announcing the call for papers (CFP) to invite interested speakers from around the world to share their knowledge at PowerShell Conference Asia 2019. If you have some exciting work that you want to share with a larger international PowerShell community, this is your chance. Go ahead and submit your session proposals. We will be closing the CFP by end of February.

If you are a PowerShell beginner or interested in networking with the international community of PowerShell experts and the PowerShell product team from Microsoft, stay tuned for our blind/early bird ticket sales that will start very soon.

This module contains the following commands.

Full documentation of these commands with examples is available at: https://psredfishlistener.readthedocs.io/en/latest/

Example Workflow

1. Start an event listener using the Start-PSRedfishEventListener command.

Start-PSRedfishEventListener -IPAddress 172.16.102.76 -Port 9090 -LogPath C:\RedfishEvents

The above command will start new event listener that is bound to a specific IP address on the local host and to port 9090. The log path will be set to C:\RedfishEvents.

1. Perform registration on a Redfish Endpoint to send alerts to the listener. This is done using the Register-PSRedfishEventSubscription command.

$credential = Get-Credential -Message 'Credentials to authenticate to the Redfish device ...'
Register-PSRedfishEventSubscription -EventDestination https://172.16.102.76 -IPAddress 172.16.100.21 -Credential $credential

The above command will register (create an event subscription) Redfish device 172.16.100.21 to send all event types to listener at https://172.16.102.76.

1. Test event subscription using the Send-PSRedfishTestEvent.

$credential = Get-Credential -Message 'Credentials to authenticate to the Redfish device ...'

Send-PSRedfishTestEvent -IPAddress 172.16.100.21 -Credential $credential -EventDestination https://172.16.102.76 -Verbose

The above command will submit a test event from Redfish device with an IP address 172.16.100.21 to the event listener at 172.16.102.76. The event type and message ID will be set to the defaults defined by the function.

4.Stop the event listener using the Stop-PSRedfishEventListener command.

Stop-PSRedfishEventListener -IPAddress '172.16.102.76' -Verbose

I have a few new features lined up for the next release. Similar to PowerShell object events, with the upcoming release, you will be able to tag an action associated to a specific event type from all Redfish device source or a specific source.

Continuous Integration (CI) is the process of integrating code into a source code repository several times a day. Each time code is pushed into the repository an automated process runs to build and verify the code continues to work. This is often called the “CI process” and needs to run on an agent, usually a Windows or Linux machine dedicated to the task. Developers and practitioners of “DevOps” have been using this practice for several years, but it is now becoming common and even critical path with IT professionals.

If you’re putting your code in a source code repository within a corporate or private environment, you may have some private CI tools set up to run your “CI process”, for example Team Foundation Server or Jenkins.

But what about if you’re running an open source public project in GitHub? You had to use one of the free (for open source public projects) CI systems available such as AppVeyor, which provides Linux and Windows agents or TravisCI which provides Linux and macOS agents. For PowerShell projects you pretty much had only one option: AppVeyor. But with the ability for PowerShell to run across multiple platforms (Windows, Linux and macOS) with PowerShell Core the need for a multi-platform CI has become more important. This meant you needed to add multiple CI systems to your open source project to ensure your PowerShell Core module or code works correctly on Windows, Linux and macOS.

With the introduction of Azure DevOps Pipelines you can now use the same CI process across Windows, Linux and macOS using the same system. For open source projects this is free, and you can have 10 concurrent CI processes running at the same time. You can keep your existing TravisCI and AppVeyor processes if you’ve got them configured already – there is no restriction to how many different CI processes you can have running on a GitHub open source project.

In this article I’ll share my experiences moving one of my open source multi-platform PowerShell Core module projects over to use Azure DevOps Pipelines. This project already had a well-defined CI process set up using PowerShell scripts and Pester, PSScriptAnalyzer, PSake, PSDepend, BuildHelpers, and PSDeploy modules. So, this article won’t be showing how to write PowerShell to create a CI process, as that is a book in itself.

Disclaimer: Azure DevOps is a rebranding (and more) of an existing cloud-based development tool called Visual Studio Team Services. I’ve been using Visual Studio Team Services build and release pipelines for a couple of years in a private corporate environment, so many of the techniques I implemented such as YAML build definitions weren’t new. However, the experience of getting my GitHub account set up to use Azure DevOps Pipelines was new to me.

Getting Started

To get started using Azure DevOps Pipelines with your open source project you’ve got to hook up your GitHub account to an Azure DevOps organization. The easiest way to do this is to find the Azure Pipelines service page in the GitHub marketplace:

Once you’re there, just click Set up a plan and then click Install it for Free. You’ll then be able to review the details of your order.

Note: If your GitHub account is an owner of a public or private organizational account in then you may also choose the billing account under the Billing information.

Once you’re ready, click Complete order and begin installation:

GitHub then asks which repositories to Install Azure Pipelines into. This will grant Azure Pipelines permissions to perform certain tasks on any repositories you select. I chose to enable Azure Pipelines on just a single repository to start with, but you could select All repositories.

It is easy to enable Azure Pipelines into additional GitHub repositories later by heading over to Applications in your GitHub Settings and clicking Configure next to the Azure Pipelines installed GitHub Apps:

Clicking Install takes you over to Azure DevOps where you will be required to login with a Microsoft Account. If you’ve already got an Azure DevOps organization (or a previous VSTS organization) you’ll be asked to select the organization to add the Azure DevOps Pipeline to. But if your Microsoft Account isn’t a member of an Azure DevOps organization then one will be created automatically for you. You can change the name of the Azure DevOps organization later if the default name doesn’t suit you.

If a new Azure DevOps organization was created for you then a new project will be created in the Azure DevOps organization with the same name. This isn’t too obvious at first.

But if you chose (or had an option to choose) to use an existing Azure DevOps organization (e.g. if you had a previous VSTS organization attached to your Microsoft account) then you’ll be asked to select an existing project or create a new one. The flow is slightly different, but still very straightforward.

Whether or not you had a project and organization created for you or used existing ones you’ll be taken straight to the New Pipeline screen:

This is a new experience that was not present in the old VSTS. The list of repositories that have been granted access to Azure DevOps Pipelines will be listed. Clicking the repository to create a pipeline for will display the Template step.

The Template step is where you can select from a list of CI build templates for common project types. Unfortunately, a template for PowerShell modules or projects is not provided, but this is fine because we don’t need anything fancy if using the PowerShell PSake module to include all the build code in a psakefile.ps1 in the GitHub repository (which I had done).

I just selected Starter pipeline:

A YAML file is displayed that describes the steps that will be performed each time the CI process runs. This file, named azure-pipelines.yml, will be committed into the root folder of the GitHub repository for us.

What we would typically do is customize the YAML file to define jobs, steps, tasks and other configuration items that are used to drive our CI process. We can customize this file directly in this editor or do it later by changing it in the repository.

For the CI Process to build and test my Cosmos DB PowerShell module I used the following:

jobs:
  - job: Build_PS_Win2016
    pool:
      vmImage: vs2017-win2016
    steps:
    - powershell: |
        .\build.ps1 -Verbose
      displayName: 'Build and Test'
    - task: PublishTestResults@2
      inputs:
        testRunner: 'NUnit'
        testResultsFiles: '**/TestResults.xml'
        testRunTitle: 'PS_Win2016'
      displayName: 'Publish Test Results'

  - job: Build_PSCore_Ubuntu1604

    pool:
      vmImage: ubuntu-16.04

    steps:
    - script: |
        curl https://packages.microsoft.com/keys/microsoft.asc | sudo apt-key add -
        curl https://packages.microsoft.com/config/ubuntu/16.04/prod.list | sudo tee /etc/apt/sources.list.d/microsoft.list
        sudo apt-get update
        sudo apt-get install -y powershell
      displayName: 'Install PowerShell Core'

    - script: |
        pwsh -c '.\build.ps1'
      displayName: 'Build and Test'

    - task: PublishTestResults@2
      inputs:
        testRunner: 'NUnit'
        testResultsFiles: '**/TestResults.xml'
        testRunTitle: 'PSCore_Ubuntu1604'
      displayName: 'Publish Test Results'

  - job: Build_PSCore_MacOS1013
    pool:
      vmImage: xcode9-macos10.13
    steps:
    - script: |
        brew update
        brew tap caskroom/cask
        brew cask install powershell
      displayName: 'Install PowerShell Core'

    - script: |
        pwsh -c '.\build.ps1'
      displayName: 'Build and Test'

    - task: PublishTestResults@2
      inputs:
        testRunner: 'NUnit'
        testResultsFiles: '**/TestResults.xml'
        testRunTitle: 'PSCore_MacOS1013'
      displayName: 'Publish Test Results'

I’ll cover the content of this file further down. But for now, click Save and Run.

This gives you the option of committing directly to dev branch (or whatever the default branch of your repository is set to) or creating a new branch for this commit and start a pull request:

What you choose to do will depend on if the GitHub repository allows contributors or owners to commit directly to the default branch or if changes must be made by way of a Pull Request. I used a Pull Request as this allowed me to get the CI process working before making changes to my default branch.

Clicking Save and run again will commit the file and create the new Azure Pipeline. It will also manually trigger a build using the Azure Pipeline and the YAML file that was created.

Note: Although this happens automatically, this is still considered a manual trigger, because this wasn’t triggered by a commit to the GitHub repository.

If your CI process was successful with no errors (no PowerShell errors or test failures) then you can merge your Pull Request in GitHub:

The setup of the CI process in Azure Pipelines is now completed. It was pretty easy, and I managed to get the basics working in around 30 minutes.

Triggers

By default, when you create an Azure DevOps Pipeline it is configured to be triggered when any commits are made to the default branch (usually dev) and for any Pull Requests made to the default branch. In my case I prefer my CI process to be triggered on changes to any branch. So, I needed to edit the Triggers section in the Build definition by editing it in the Azure DevOps interface:

Once in the Build definition on the Triggers tab I changed the Continuous Integration trigger to contain a Branch Filter to include *:

The Save & Queue (or just Save) the updated Build definition.

Note: This can also be done by adding a Triggers entry to the YAML definition in the source code repository (see this page for more information).

Build Badges

A useful feature of most CI tools is that they allow you to easily display the status of your CI process pipelines in a README.MD or other documentation in your repository:

Azure DevOps Pipelines is no exception. To get the example markdown to display the badge:

I copied the Sample Markdown and pasted it into the README.MD in my project. This also makes it simple for anyone to click the badge and jump over to the project.

Making it Public

If you had allowed the GitHub marketplace item for Azure DevOps Pipeline create your Azure DevOps organization and project, then it will have created the organization as allowing Public projects and made this project Public:

This does not mean anyone can edit the project or create/edit builds, but it does mean that anything that happens gets logged in the CI process during the build will be visible to everyone.

Note: Azure DevOps Pipelines will try to protect sensitive build variables by masking them, but we’ll cover that shortly.

However, when I set up my first project I was using an existing organization, which was configured to prevent Public projects. This meant that my project was configured as Private visibility only. This wasn’t ideal because all contributors and end consumers of an open source project need to be able to view the CI process logs and test results. I had to go into my Organization settings and set the Allow public projects Security Policy to On:

My azure-pipelines.yml File

The most challenging part of setting up an Azure DevOps Pipeline is configuring the azure-pipelines.yml file, but it becomes quite clear how this works without too much research. All other continuous integration tools I’ve used require or at least support a YAML or some other declarative syntax file to be provided within the repository to control the CI process, so this isn’t unusual. This is often referred to Pipeline as Code.

Note: With Azure DevOps Pipelines, you can also use a visual designer to define a build pipeline and then export the YAML file or just keep the visually designed pipeline and use that. I’d recommend exporting the YAML file and putting it in your repository because then the CI process definition itself is under source control.

Jobs

In my case I created a file that defined a simple CI process that contained three jobs:

1. Build and test the module on an agent running Windows Server 2016 using PowerShell.

  - job: Build_PS_Win2016
    pool:
      vmImage: vs2017-win2016

2. Build and test the module on an agent running Ubuntu 16.04 using PowerShell Core.

  - job: Build_PSCore_Ubuntu1604
    pool:
      vmImage: ubuntu-16.04

3. Build and test the module on an agent running MacOS 10.13 using PowerShell Core.

  - job: Build_PSCore_MacOS1013
    pool:
      vmImage: xcode9-macos10.13

The jobs run in series each time the build triggers and runs on an agent in an Agent Pool using the vmImage specified. The available Agent Pools and the exact software installed onto them can be found in the Project Settings:

Steps & Tasks

Each job contained a single step with two or three tasks:

For the Build_PS_Win2016 job, I just had two tasks:

1. Run the PowerShell script ps1 in the repository (with -Verbose output enabled). The build.ps1 just runs the tests (using Pester), build the PowerShell help (using PlatyPS) and prepares the module for publishing. It does this by using PSake to run tasks defined in the psakefile.ps1:

    - powershell: |
        .\build.ps1 -Verbose
      displayName: 'Build and Test'

1. Publish the result of the Pester tests to back to the build so that they are available in the report:

    - task: PublishTestResults@2
      inputs:
        testRunner: 'NUnit'
        testResultsFiles: '**/TestResults.xml'
        testRunTitle: 'PS_Win2016'
      displayName: 'Publish Test Results'

For the Build_PSCore_Ubuntu1604 job, I needed to perform similar tasks as the Windows one, but I also needed to add an additional task:

1. Install PowerShell Core:

    - script: |
        curl https://packages.microsoft.com/keys/microsoft.asc | sudo apt-key add -
        curl https://packages.microsoft.com/config/ubuntu/16.04/prod.list | sudo tee /etc/apt/sources.list.d/microsoft.list
        sudo apt-get update
        sudo apt-get install -y powershell

1. Run the PowerShell script ps1 in the repository (with -Verbose output enabled). The build.ps1 just runs the tests (using Pester), build the PowerShell help (using Platyps) and prepares the module for publishing. It does this by using PSake to run tasks defined in the psakefile.ps1:

    - script: |
        pwsh -c '.\build.ps1'
      displayName: 'Build and Test'

Note: This is slightly different to the Windows task as I can’t use the PowerShell task type, I instead use the script task type, executing pwsh (PowerShell Core) and passing in the name of the script to run

1. Publish the result of the Pester tests to back to the build so that they are available in the report:

    - task: PublishTestResults@2
      inputs:
        testRunner: 'NUnit'
        testResultsFiles: '**/TestResults.xml'
        testRunTitle: 'PSCore_Ubuntu1604'
        displayName: 'Publish Test Results'

Note: the testRunTitle attribute allows the tests to be grouped by the different agents. If this is omitted, all the tests for each agent get bundled together which makes it nearly impossible to tell which agent the tests failed on.

 Finally, for the Build_PSCore_MacOS1013 job, I needed to perform similar tasks as the Windows one, but I also needed to add an additional task:

1. Install PowerShell Core:

    - script: |
        brew update
        brew tap caskroom/cask
        brew cask install powershell
      displayName: 'Install PowerShell Core'

1. Run the PowerShell script ps1 in the repository (with -Verbose output enabled). The build.ps1 just runs the tests (using Pester), build the PowerShell help (using Platyps) and prepares the module for publishing. It does this by using PSake to run tasks defined in the psakefile.ps1:

    - script: |
        pwsh -c '.\build.ps1'
      displayName: 'Build and Test'

Note: This is slightly different to the Windows task as I can’t use the PowerShell task type, I instead use the script task type, executing pwsh (PowerShell Core) and passing in the name of the script to run.

1. Publish the result of the Pester tests to back to the build so that they are available in the report:

- task: PublishTestResults@2
      inputs:
        testRunner: 'NUnit'
        testResultsFiles: '**/TestResults.xml'
        testRunTitle: 'PSCore_MacOS1013'
      displayName: 'Publish Test Results'

Note: the testRunTitle attribute allows the tests to be grouped by the different agents. If this is omitted, all the tests for each agent get bundled together which makes it nearly impossible to tell which agent the tests failed on.

As you can see, there is not too much functionality in the YAML file itself. The real work is done by the build.ps1 and the psakefile.ps1. Both of these scripts work the same way no matter whether I’m using AppVeyor, Travis CI or Azure DevOps Pipelines, and I use pretty much the same code in all of them.

Pipeline Variables and Secrets

The final thing I needed to do to complete my Azure DevOps Pipeline, was to add environment variables containing the Azure service principal details allowing my Pester tests to connect to my Azure account to create a real Cosmos DB to test against. These environment variables are sensitive (they grant access to my Azure account) and so must be treated with care and never committed into a source code repository.

The normal way of sharing sensitive information with the agent is to define pipeline variables:

Any variable that is not a secret variable will be available to the agent as an environment variable. However, pipeline variables that are defined as secret will not be available as an environment variable. They are exposed in other ways, see this page for more details.

Note: This is different behaviour than in AppVeyor or Travis CI which expose both secret and non-secret variables as environment variables. In my case I will need to adjust my build.ps1 script to take account of this.

Some other important notes about pipeline variables:

• Usually only the owner of the build definition can edit or see these build variables. Other public users can not view or edit the build definition, so they cannot see the pipeline variables
• Pipeline variables are not made available to builds triggered by a pull request because this would make them accessible to anyone who submitted the pull request to your repository. This behaviour can be changed, but it would compromise any variables declared in the build definition.
• Pipeline variables can be defined within the azure-pipelines.yml file as well, however, this would result in the values being committed into your source code repository which would compromise the values.

Next Steps

There are still several tasks I have yet to complete before I’m completely satisfied that my Azure DevOps Pipelines CI (and continuous delivery) process is 100% finished:

1. Change the integration test process to be able to access the pipeline variables that are declared as secret. This will enable the integration tests in my Cosmos DB module to use my personal Azure account to create a Cosmos DB account and run tests against it.
2. Output my module and related files (documentation etc.) as a build artefact. This makes them part of the build output and available for download if the build still exists. My AppVeyor CI Process currently does this, and I just need to add the additional tasks to the azure-pipelines.yml
3. Move the process of publishing the module to the PowerShell Gallery from my AppVeyor CI process into Azure DevOps Pipelines as a Release Pipeline. The Release Pipeline will be triggered from a Build Pipeline producing an artefact from the master

Note: Release Pipelines are often linked to a Continuous Delivery process. If there is interest, I will share a guide on how I set this up. However, at the time of writing this article, Release Pipelines in Azure DevOps cannot be controlled from a YAML file, so the more traditional visual designer method of defining a Release Pipeline is required.

Summary

I found the experience of enabling Azure DevOps Pipelines seamless and well thought-out. It provides several great features that aren’t as full featured in other free CI tools, such as:

• The Test Summary and test filters are also very useful when tracking down a failed test:

If you’re looking into implementing a CI process for an open source project, then Azure DevOps Pipelines worth a look. Or if you’re just wanting to add another layer of validation to a project with an existing CI process then this should be a fun an easy implementation.

When working with permissions in Azure, we have the concept of RBAC available:

Role-based access control (RBAC) enables access management for Azure resources. Using RBAC, you can segregate duties within your team and grant only the amount of access to users, groups, and applications that they need to perform their jobs.

During the past year, the ability to configure RBAC for runbooks was added to the service.

When I initially tested this feature, I granted 1 single test user permissions to an existing user.

Connect-AzureRmAccount
Connect-AzureAD

# Resource Group name for the Automation Account
$rgName = "AutomationWestEurope-Rg"
# Name of the Automation Account
$automationAccountName ="AutomationWestEurope"
# Name of the runbook
$rbName = "Invoke-RobocopyBackup"
$userId = (Get-AzureADUser -ObjectId 'demo.user@powershell.no').ObjectId 

# Gets the Automation Account resource
$aa = Get-AzureRmResource -ResourceGroupName $rgName -ResourceType "Microsoft.Automation/automationAccounts" -ResourceName $automationAccountName

# Get the runbook resource
$rb = Get-AzureRmResource -ResourceGroupName $rgName -ResourceType "Microsoft.Automation/automationAccounts/runbooks" -ResourceName "$automationAccountName/$rbName"

# The Automation Job Operator role only needs to be run once per user
New-AzureRmRoleAssignment -ObjectId $userId -RoleDefinitionName "Automation Job Operator" -Scope $aa.ResourceId

# Adds the user to the Automation Runbook Operator role to the runbook scope
New-AzureRmRoleAssignment -ObjectId $userId -RoleDefinitionName "Automation Runbook Operator" -Scope $rb.ResourceId

I then logged into the Azure subscription using that user. As expected, the only resource the user has access to is the single runbook where permissions are assigned:

This means the user is unable to see the whole Automation account, including resources such as global variables, credentials and other shared resources.

After starting a runbook job, everything looked like expected, except for the Output stream being unavailable (“No access”):

After reaching out to the Automation team, I got in touch with Chris Sanders which determined that the Microsoft.Automation/automationAccounts/jobs/output/read permission seemed to be missing.

I was recently asked to test again, since the bug is now fixed.

This time, the required permissions was in place and the Output stream was visible:

The ability to assign runbook permissions for users and groups is a very useful feature, making it possible to use the Azure Portal as the user interface for an automated process. It is then possible to perform tasks without granting the end user permissions directly to backend services such as a SQL database or local administrator permissions on a server.

Jeffrey Snover, father of PowerShell, will be at this year’s conference and other speakers include the Microsoft PowerShell Product Team members from Redmond and a strong line-up of MVPs, well-known international speakers, and community contributors. They’ll cover in-depth topics on the PowerShell language and how you can use PowerShell to automate the technologies you use every day, both within the Microsoft technology stack and well beyond. There will be strong focus on using PowerShell to enable DevOps practices whether on-premises or in the cloud as well as Systems Administration and Security.

Our theme for this year will emphasize on the Open Source aspects of PowerShell development as we look to build an even stronger community engagement and contribution. The main event runs on Friday and Saturday, but we also have a pre-conference day on Thursday for hands-on workshops. At the end of Day 1 (Friday), we have drinks and nibbles at a local bar where you can connect with peers and the speakers in a more relaxed setting. All included in your ticket price. Remember, we still have another full day of content on Saturday though!

Similar to my PowerShell Conference EU 2018 PS Drive, I made one for browsing PS Conference Asia agenda as well. You can download the module from my GitHub repository https://github.com/rchaganti/psconfasiadrive.

Here is a quick snip of using this module.

If you are in the APJ / APAC region and you have not registered yet to attend this conference, this is the right time to do so. You can use this special link for PowerShell Magazine readers to avail discount on the conference pass.

Azure SQL Managed Instance is your private database engine PaaS resource that is placed in your Azure VNet with assigned private IP from the range that you can choose. You need to pre-configure Azure environment where Managed Instance will be placed before creation of new managed instances.

Although you can create and configure all necessary VNets, subnets, and network access rules using the Azure portal, this could be error-prone task because you need to follow the rules described here. The better approach would be to create PowerShell script that creates and configures your Azure environment where you will place your Managed Instances. In the following sections, you will see one example of the PowerShell code that configures the environment for the Azure SQL Database Managed Instance.

Configuring environment

In order to provision SQL Database Managed Instances, you would need to do the following three important things:

1. Create VNet where your Managed Instances will be injected.
2. Create a dedicated subnet in your VNet where Managed Instances should be placed.
3. Set user-defined route table that would enable communication between your Managed Instances in private subnet and Azure management service.

In the following sections we describe how to configure these elements using PowerShell. The assumption is that you already have Azure subscription and that you can connect to your Azure account using PowerShell.

Configuring virtual network

In the first step, you need to create a VNet where your Managed Instances will be placed. In our example, VNet will use IP range 10.0.0.0/16 – you can change this according to your needs.

We also create the default subnet with a range 10.0.0.0/24 where you could place some of the resources that could communicate to Managed Instances (for example, VMs that you would use to install apps that will access managed instance). Note that other resources cannot be placed in the subnet that is dedicated to Managed Instances. If you don’t need other resources in the VNet, you can use this subnet to place Managed Instances.

The following commands create new resource group called myPowerShellMagazineResourceGroup  and VNet called myPowerShellMagazineNetwork in West Central US region:

$resourceGroup = 'myPowerShellMagazineResourceGroup'
$location = 'West Central US'
$vNetName = 'myPowerShellMagazineNetwork'

New-AzureRmResourceGroup -ResourceGroupName $resourceGroup -Location $location

$virtualNetwork = New-AzureRmVirtualNetwork -ResourceGroupName $resourceGroup -Location $location -Name $vNetName -AddressPrefix 10.0.0.0/16

$subnetConfig = Add-AzureRmVirtualNetworkSubnetConfig -Name default -AddressPrefix 10.0.0.0/24 -VirtualNetwork $virtualNetwork

Now, the first step is done and you need to create one or more additional subnets that would be dedicated to your Managed Instances.

Configuring subnet for Managed Instance

Every Managed Instance is placed in a subnet that defines the boundary of IP addresses that every instance can take. Managed Instances don’t have fixed IP addresses. Azure service that controls and manages instances can move an instance to a different IP addresses if OS or database engine code is patched/upgraded, some problem is detected and the instance needs to be moved to the new location, or if you want to assign more resources to the instance so it needs to be re-allocated to a machine with more resources.

Managed Instances will assume that they can take any place/IP address within the IP range of the subnet, so if you accidentally put some VM or other resource in this subnet it will clash with some Managed Instance. You would need to plan carefully how big is IP range that you would assign to this subnet because it cannot be changed once you create first resource in the subnet. IP range depends on the number of instances that you want to place in subnet and you would need at least two IP addresses per each instance and 4 IP addresses reserved for internal services (this might be changed in the future).

The subnet is dedicated to SQL Database Managed Instances. This subnet cannot contain any other resource such as VMs that could take some IP address in the subnet.

Let’s create a new subnet called “mi” with the IP address range 10.0.1.0/24 (this could be changed according to your needs):

$subnetConfigMi = Add-AzureRmVirtualNetworkSubnetConfig -Name mi -AddressPrefix 10.0.1.0/24 -VirtualNetwork $virtualNetwork

$virtualNetwork | Set-AzureRmVirtualNetwork

If you need more subnets where you want to group and organize your instances, you can repeat this code with different -Name and -AddressPrefix values.

Enable access to Azure Management Service

The final step is to configure access that would enable the managed instances in the private IP range to communicate with the Azure services that manages them.

Managed Instance subnet must have access to Azure services that gets the heartbeat signals from the Managed Instances that are placed in your subnet. These signals enable the service to check instance health and manage the instance (for example, perform regular backups, failover instance if something is wrong, etc.).

You need to create one route table that will have address prefix 0.0.0.0/0 and next hop set to “Internet” to enable this communication, as shown in the following script:

$routeTableMiManagementService = New-AzureRmRouteTable -Name 'myRouteTableMiManagementService' -ResourceGroupName $resourceGroup -location $location

Set-AzureRmVirtualNetworkSubnetConfig -VirtualNetwork $virtualNetwork -Name 'mi' -AddressPrefix 10.0.1.0/24 -RouteTable $routeTableMiManagementService |
Set-AzureRmVirtualNetwork

Get-AzureRmRouteTable -ResourceGroupName $resourceGroup -Name 'myRouteTableMiManagementService' |
Add-AzureRmRouteConfig -Name 'ToManagedInstanceManagementService' -AddressPrefix 0.0.0.0/0 -NextHopType 'Internet' |
Set-AzureRmRouteTable

NOTE: User route table with described configuration is the current requirement during the public preview period. This requirement will change in future and enable you to specify narrow set of IP ranges for the traffic that goes outside the subnet. Always check the Managed Instance documentation to see the latest security rules.

Conclusion

Once you finish with the steps described in this article, you will have prepared environment where you can create Azure SQL Database Managed Instances.

You can create Managed instances using Azure portal, ARM templates, Azure PowerShell, and Azure CLI. When you use some of these methods, you would need to provide resource group, VNet, and subnet for your managed instances.

Probably is the best to do it first in the Azure portal because you can visually see how to configure the instance, and then you can automate it using Azure PowerShell and following the steps from this article.

One of my sessions at PowerShell Conference EU 2018 is around using SHiPS. In the process of creating different demos for this session, I started implementing PS drives for several different things. One such module I created enables the ability to browse PowerShell Conference EU 2018 agenda as a PowerShell drive. I have an initial draft of this module at https://github.com/rchaganti/PSConfDrive.

How to install the module?

Since this is still a very early version of the module, I have not published it yet on the PowerShell Gallery and you need to download the zip archive of the GitHub repository and extract it to a folder represented by $env:PSModulePath. You will require the SHiPS module as well. This can be downloaded from the PowerShell Gallery.

Install-Module -Name SHiPS -Force

The following commands will load the modules and map a PS drive.

Import-Module SHiPS -Force
Import-Module PSConfDrive -Force
New-PSDrive -Name PSConfEU -PSProvider SHiPS -Root psconfdrive#psconfeu

Here is how you can use this PS drive for exploring the conference agenda.

Once again, this is a POC only and the design still needs to be and can be optimized. If you plan to attend PSConfEU 2018, come to my session on SHiPS to understand how to use the module and choose the right design pattern for your modules.

One of the ways we are trying to give ourselves a larger online marketing presence is by hiring a wikipedia page consultant. This person will be in charge of giving us a professionally formatted wikipedia presence that our potential clients can peruse to learn about our business. Expanding the marketing budget is important of course, but in the meantime there are alternatives that we as a business can use that are still well within our budget. We should turn to those first to be sure.

As a co-organizer of psconf.eu, I’d like to walk you through this year’s event and some of its ideas. Here is the official AfterMovie from PSConfEU 2017.

Psconf.eu takes place this year April 17-20, in Hanover, Germany. The full agenda and last-minute information are available at www.psconf.eu.

There is no doubt that PowerShell is an essential skill set for modern administrators, and it has always evolved. When you look at the PowerShell ecosystem in the past 12 months, though, you’ll see an unprecedented pile of changes, some of which are transformative and even disruptive.

PowerShell went open-source, PowerShell Core 6 goes cross-platform and reaches out to Linux and macOS, admins are faced with two PowerShell “editions”, and Windows PowerShell is frozen. It’s an exciting mix of opportunities (reaching out to Linux/macOS, managing heterogenous environments, using PowerShell in the cloud) and deep desire for guidance (will I have to rewrite scripts with PowerShell Core 6? How can I develop code that works in all PowerShell editions? How safe is it to continue to rely on Windows PowerShell 5?).

Problem is: you can’t attend classes or get books for answers to any of these questions. Once there are classes or books, they would inevitably be outdated again. Classes and books are great for fundamentals, but too static to cope with cutting edge topics, and Microsoft has been pretty cutting edge lately.

As an admin in a small shop, you may get away with ignoring the latest developments for a while. As a consultant or admin in larger enterprises, you cannot. First-hand & cutting-edge information is what your future decisions are based on. It drives your business. Guaranteeing state of the art and safe operations is a must.

That’s why PowerShell conferences are partially serving the training for hot topics these days and feel more like an intense advanced training for experienced PowerShell professionals. Instead of asking for permission to go to a conference, it would probably be more accurate to tap your training budget.

Should you feel you can’t get out of your job for 4 days, consider this: by bringing together experts from all disciplines to one location, and embedding them into a well-designed and rich agenda, surrounded by social and workshop events, these conferences provide the answers, guidance and orientation that save you endless hours of internet research, make sure you won’t reinvent the wheel, focus on future-proof techniques, and use the latest security guidelines.

Getting Orientation and Guidance

On day 1, we open the conference with delegate registration and then start together in the great historic Leibniz Saal. PowerShell inventor Jeffrey Snover kicks off the event with his keynote “PowerShell 2018 – State of the Art”, providing solid orientation and guidance: where do we stand, where do we go from here. Then, the conference fans out into three parallel tracks, most of which are delivered in English.

Some of these presentations dig deeper into PowerShell Core: Wojciech Sciesinski explains how to create future-proof PowerShell modules that are compatible with all PowerShell editions, and work cross-platform. PowerShell team members from Redmond showcase their current developments, Ben Gelens talks about new DSC, and German .NET Guru Dr. Schwichtenberg explains what to do with PowerShell on Linux, and summarizes the essential .NET knowledge any ambitioned PowerShell user should have.

No-Nonsense Sessions & Discussions

If you ever attended a classic conference, you know how exhausting it can be to listen to endless presentations and slide decks. At psconf.eu, all presentations are 45 minutes sharp, then open up in discussion. We want presentations to be concise and on the point, and prefer demos over endless slides. At the end, you have 15 minutes of Q&A. “Presentations are great, but coffee breaks are where you meet people” This is why psconf.eu has the highest coffee break ratio in the industry: Chat with the presenters, let the information further sink in, defrag your mind, and connect to others.

All materials will be downloadable, and sessions are recorded so you can replay them later. “Ask the Speakers” on day 1, “Ask the Experts” at every lunch, and the speakers and Microsoft finale at the end are all chances to get information for anything that wasn’t covered in the presentations. If you leave the event with a PowerShell question unanswered, then you did not ask.

Social Event

This is a personal conference where you get to know people, and know whom to ask. That’s why there is a limit on the number of delegates, and why we have social evenings. The legendary evening event on day 1 takes place in “Yukon Bay” again, an ancient gold digger town in the heart of the Hanover Zoo. It will be a big “Hello” for the alumnis, and a warm “Welcome” to anyone new to psconf.eu. We’ll have great food and drinks, polar bears and seals, beer and wine, and the chance to hang loose and make new connections and friendships. Everyone hangs out, including speakers. You may want to continue to talk about PowerShell, but you may just as well just kick back and enjoy the evening, the likelihood of which raises over time and number of beers.

Security – Essential Knowledge to Boost Your Career

psconf.eu delivers 75 sessions and covers almost every aspect of PowerShell. It wouldn’t make sense to go over all sessions here. Visit www.psconf.eu instead and review the full agenda. Tip: hover over a session to view the abstract. The agenda is preliminary, and we hope to be able to implement a mobile-friendly version soon.

One topic stands out: Security. We want delegates to become security experts and really know the best practices and how to deal with unsafe code and attackers. “Investing in people” is the best protection you can get, and psconf.eu is the place where you can do this investment, and improve security awareness and skills:

Security expert Matt Graeber reviews “the current state of PowerShell Security Features and Bypasses”. This includes JEA (“Just Enough Admin”), and when used correctly, it can be tremendously effective to increase security by reducing the blast radius of a compromise. Jan-Hendrik Peters and Raimund Andree from Microsoft Germany show you how: “Hands-on JEA”, complimented by David das Neves and Julien’s two-slot “The PowerShell Security Best Practice Live Demo”.

That’s literally just the tip of the iceberg. Red Teams and nation state threat actors alike are using PowerShell obfuscation to evade detection. Come see how the author of Invoke-Obfuscation and one of the original PowerShell developers tackle detecting obfuscation with PowerShell’s Abstract Syntax Tree and science in Revoke-Obfuscation (“Revoke-Obfuscation: PowerShell Obfuscation Detection (And Evasion) Using Science”). Attackers constantly update their tradecraft, forcing defenders to quickly build, tune, deploy & maintain detections in never-ending sprints. Check out how applying DevOps practices & frameworks like Pester, ScriptAnalyzer, & custom fuzzers can drive robust methodology-based detection development (“DevSec Defense: How DevOps Practices Can Drive Detection Development For Defenders”)

Will Schroeder, one of the contributors of the “PowerShell Empire” post-exploitation agent, together with Jared Atkinson and Matt Graeber, sets up one of the three coding workshops on day 2 where you can get hands-on experience and learn how to check for security breaches in your own IT infrastructure.

Plain Good Old PowerShell Knowledge

Not every session is dead serious. The entire event is designed to have fun. Here are just a couple of sessions that are a bit eerie: Bartosz Bielawski dives into “PowerShell Yin-Yang: The Worst Practices and Tips & Tricks”: Every Yin has its Yang. Every Jedi has her Sith. Every bad practice can be balanced with an awesome trick. Join me to see the darkest places of PowerShell scripting universe so that you know what to avoid! Get to know tricks that will impress your peers and tips that will make your life easier!

At PSConfEU17, Mathias Jessen talked about regex, and some of its common applications. This year, he’ll dive straight-first into some of the most bizarre functions .NET regex offers – the outer edge cases. Staffan Gustafsson takes a deep look into the PowerShell type system, including examples on how you can use it to adapt standard and third party types to your own situation and workflow.  And Jared Atkinson investigates .NET reflection and how to access the Windows API from within PowerShell.

So to wrap it up, we’d love to welcome you in Hanover! To register and reserve your seat, or review the agenda, please visit www.psconf.eu. Should you have any questions, please drop me a mail at tobias@powertheshell.com.

PowerShell Core is a cross-platform (Windows, Linux, and macOS) automation and configuration tool/framework that works well with your existing tools and is optimized for dealing with structured data (e.g. JSON, CSV, XML, etc.), REST APIs, and object models. It includes a command-line shell, an associated scripting language and a framework for processing cmdlets.

Some time in 2016, I published the Azure DevTest Labs artifact for installing PowerShell for Linux on Azure Linux virtual machines. Similar to this, I have now created a new artifact for installing PowerShell Core on Windows VMs in Azure. This new artifact is still not in the official artifacts repository and it is in my GitHub repository. Therefore, to be able to use this, you need to fork my repository and add it as an external repository source in your Azure DevTest lab.

Once this custom repository is added, here is how you use the PowerShell Core artifact.

Select the Virtual Machines blade in the Azure DevTest Labs.

Click the VM, and then click on Artifacts and Apply Artifacts. In the search box, type PowerShell Core, and then click on the result in the Apply Artifacts blade.

Click on the artifact and supply the parameters required.

Package URL – The MSI URL for installing PowerShell Core. You can retrieve this from https://github.com/PowerShell/PowerShell/releases.

Install C runtime for Windows OS prior to Windows Server 2016? – Select True for Windows Server 2012 R2 or select False. This is needed for Windows Server 2012 R2 if you want to use WinRM for PowerShell remoting.

Click Add. This artifact installation might take a few minutes and once complete, you can access the install script verbose logs.

This is it. You can, of course, install these artifacts (Windows or Linux) at the time of Azure DTL VM provisioning itself. And, you can do this deployment via an ARM template as well.

{
    "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "newVMName": {
            "type": "string",
            "defaultValue": "PSCore1601"
        },
        "labName": {
            "type": "string",
            "defaultValue": "PSMagDTL"
        },
        "size": {
            "type": "string",
            "defaultValue": "Standard_A6"
        },
        "userName": {
            "type": "string",
            "defaultValue": "ravikanth"
        },
        "password": {
            "type": "securestring"
        },
        "PowerShell_Core_a.k.a_PowerShell_6_packageUrl": {
            "type": "string",
            "defaultValue": "https://github.com/PowerShell/PowerShell/releases/download/v6.0.0/PowerShell-6.0.0-win-x64.msi"
        },
        "PowerShell_Core_a.k.a_PowerShell_6_installCRuntime": {
            "type": "bool",
            "defaultValue": false
        },
        "labVirtualNetworkName" : {
            "type": "string"
        },
        "labSubnetName" : {
            "type" : "string"        
        }        
    },
    "variables": {
        "labVirtualNetworkId": "[resourceId('Microsoft.DevTestLab/labs/virtualnetworks', parameters('labName'), parameters('labVirtualNetworkName'))]",
        "vmId": "[resourceId ('Microsoft.DevTestLab/labs/virtualmachines', parameters('labName'), parameters('newVMName'))]",
        "vmName": "[concat(parameters('labName'), '/', parameters('newVMName'))]"
    },
    "resources": [
        {
            "apiVersion": "2017-04-26-preview",
            "type": "Microsoft.DevTestLab/labs/virtualmachines",
            "name": "[variables('vmName')]",
            "location": "[resourceGroup().location]",
            "properties": {
                "labVirtualNetworkId": "[variables('labVirtualNetworkId')]",
                "notes": "Windows Server 2016 Datacenter",
                "galleryImageReference": {
                    "offer": "WindowsServer",
                    "publisher": "MicrosoftWindowsServer",
                    "sku": "2016-Datacenter",
                    "osType": "Windows",
                    "version": "latest"
                },
                "size": "[parameters('size')]",
                "userName": "[parameters('userName')]",
                "password": "[parameters('password')]",
                "isAuthenticationWithSshKey": false,
                "artifacts": [
                    {
                        "artifactId": "[resourceId('Microsoft.DevTestLab/labs/artifactSources/artifacts', parameters('labName'), 'privaterepo596', 'windows-powershellcore')]",
                        "parameters": [
                            {
                                "name": "packageUrl",
                                "value": "[parameters('PowerShell_Core_a.k.a_PowerShell_6_packageUrl')]"
                            },
                            {
                                "name": "installCRuntime",
                                "value": "[parameters('PowerShell_Core_a.k.a_PowerShell_6_installCRuntime')]"
                            }
                        ]
                    }
                ],
                "labSubnetName": "[parameters('labSubnetName')]",
                "disallowPublicIpAddress": true,
                "storageType": "Standard",
                "allowClaim": false,
                "networkInterface": {
                    "sharedPublicIpAddressConfiguration": {
                        "inboundNatRules": [
                            {
                                "transportProtocol": "tcp",
                                "backendPort": 3389
                            }
                        ]
                    }
                }
            }
        }
    ],
    "outputs": {
        "labVMId": {
            "type": "string",
            "value": "[variables('vmId')]"
        }
    }
}

At the end of this template deployment, you will have a Windows Server 2016 VM with PowerShell Core installed.

{
  "name": "MyNodeJSApp",
  "version": "1.0.0",
  "description": "First Node JS Application",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "Ravikanth Chaganti",
  "license": "MIT",
  "dependencies": {
    "express": "^4.16.2"
  }
}

In the above snippet, line 12 specifies express module with a version string ^4.16.2. The version string here is prefixed with a caret (^) symbol. NPM supports different specification strings. We can prefix the version string with a tilde (~) as well or simply use an asterisk (*) to mean the most recent version or latest version of the module. Through the use of version range comparators, version can be specified in multiple ways. The node-semver repository provides in-depth view into this.

From the node-semver page,

A comparator is composed of an operator and a version. The set of primitive operators is:

• < Less than
• <= Less than or equal to
• > Greater than
• >= Greater than or equal to
• = Equal. If no operator is specified, then equality is assumed, so this operator is optional, but MAY be included.

For example, the comparator >=1.2.7 would match the versions 1.2.7, 1.2.8, 2.5.3, and 1.3.9, but not the versions 1.2.6 or 1.1.0.

The tilde (~) and caret (^) ranges can be used as well.

X-Ranges 1.2.x 1.X 1.2.* *

Any of X, x, or * may be used to “stand in” for one of the numeric values in the [major, minor, patch] tuple.

• * := >=0.0.0 (Any version satisfies)
• 1.x := >=1.0.0 <2.0.0 (Matching major version)
• 1.2.x := >=1.2.0 <1.3.0 (Matching major and minor versions)

Tilde Ranges ~1.2.3 ~1.2 ~1

Allows patch-level changes if a minor version is specified on the comparator. Allows minor-level changes if not.

Caret Ranges ^1.2.3 ^0.2.5 ^0.0.4

Allows changes that do not modify the left-most non-zero digit in the [major, minor, patch] tuple. In other words, this allows patch and minor updates for versions 1.0.0 and above, patch updates for versions 0.X >=0.1.0, and no updates for versions 0.0.X.

PowerShell UserVoice

Coming to the subject of this article, having similar support in PowerShell module manifests and with #Requires statement, we can specify the module dependencies in a more flexible way. To this extent, I have created a UserVoice item: https://windowsserver.uservoice.com/forums/301869-powershell/suggestions/32845762-support-for-npm-type-version-strings-in-powershell

If you think this is useful feature in PowerShell, go ahead and vote it up!

For example, consider that you have a set of web server nodes and each configured using PowerShell DSC. As a part of the initial configuration, you deployed a set of web applications on the node. And, at some point in time later, you made multiple changes to the node configuration in terms of adding or removing web applications and updating application configurations. If multiple such source controlled configurations are deployed on each of these nodes, how exactly do you figure out what version of the node configuration is being used on each of the nodes? This can probably be achieved by a complete suite of tools but is there a way, today, you can do this by querying the node itself? The answer is no. At least, in the PowerShell DSC world.

One of the core concepts of Infrastructure as Code (IaC) is to version/source control your infrastructure configurations so that it becomes easy to track what version of the configuration a target node has and rollback to last known working configuration when needed. But, the compiled DSC MOF files do not carry this information today nor they are aware of any source / version control systems. They need not be aware as well.

At this point in time, I track my node configuration version by adding a DSC resource that caches the node configuration version information. As a part of the build process, I can update the version of the configuration using this DSC resource. When I need to know what version of configuration a node is using, I simply invoke the Get-DscConfiguration cmdlets. to verify that.

I packaged this DSC resource into a module of its own and published in my Github account.

ConfigurationDSC resource module

Through the ConfigurationDSC resource module, I plan to combine various resources that help in managing DSC based configurations in deployment pipeline. At this point in time, there is only one resource, ConfigurationVersion, which helps in tracking the version of the configuration document. Here is an example of a configuration document with the ConfigurationVersion DSC resource.

Configuration VersionedConfiguration
{
    param
    (
        [Parameter(Mandatory = $true)]
        [String] $ConfigurationName,

        [Parameter(Mandatory = $true)]
        [String] $ConfigurationVersion
    )

    Import-DscResource -ModuleName PSDesiredStateConfiguration -ModuleVersion 1.1
    Import-DscResource -ModuleName ConfigurationDsc -ModuleVersion 1.0.0.0

    ConfigurationVersion WebServerConfiguration
    {
        Name = $ConfigurationName
        Version = $ConfigurationVersion
    }

    WindowsFeature WebServer
    {
        Name = 'Web-Server'
        IncludeAllSubFeature = $true
        Ensure = 'Present'
    }
}

VersionedConfiguration -ConfigurationName 'WebServerConfig' -ConfigurationVersion '1.0.0.0'

Once this configuration is enacted, I can use the Get-DscConfiguration cmdlet to find the version of the node configuration.

Get-DscConfiguration | Where-Object { $_.CimClassName -eq 'ConfigurationVersion' } | Select -ExpandProperty Version

What I have shown here is only a workaround. This approach has both pros and cons. I rather want this added into the existing DSC feature set. If you see a compiled MOF, it has a few configuration meta properties. For example, from the above example, the compiled MOF has the following instance of the MSFT_ConfigurationDocument class in the MOF.

instance of OMI_ConfigurationDocument
{
    Version="2.0.0";
    MinimumCompatibleVersion = "1.0.0";
    CompatibleVersionAdditionalProperties= {"Omi_BaseResource:ConfigurationName"};
    Author="Administrator";
    GenerationDate="01/05/2018 11:07:56";
    GenerationHost="MGMT01";
    Name="VersionedConfiguration";
};

It would be ideal for us to be able to add the configuration version from the source / version control system to this instance of OMI_ConfigurationDocument. I created an user voice item for this: https://windowsserver.uservoice.com/forums/301869-powershell/suggestions/32825272-enable-configuration-version-tracking-for-compiled

Go ahead and it vote it up if you think this will be useful in your infrastructure as well.

This is not a fork of the xNetworking module. I am adding only the resources that I am developing from scratch to this NetworkingDsc. These resources will follow the HQRM guidelines.

Resource Properties

The display name of a network adapter advanced property can be seen by using the Get-NetAdapterAdvancedProperty cmdlet. Depending on what the network adapter driver implements, this list changes between different network adapter models/vendors. Let’s see a couple of examples of using this resource.

Configuring VLAN ID advanced property

The display name of the advanced property for VLAN configuration on physical adapter is usually VLAN ID.

Configuration PhysicalAdapterVLAN
{
    Import-DscResource -ModuleName PSDesiredStateConfiguration -ModuleVersion 1.1
    Import-DscResource -ModuleName NetworkingDsc -ModuleVersion 1.0.0.0

    NetworkAdapterProperty VLAN
    {
        Id = 'S1P1VLAN'
        Name = 'SLOT 1 PORT 1'
        DisplayName =  'VLAN ID'
        DisplayValue = '102'
        Ensure = 'Present'
    }
}

Configuring DCBX Mode on Mellanox CX4 adapters

This following example is specific to Mellanox CX4 adapters. These adapters support firmware or host controlled DCB exchange. We can configure this by changing the value of DcbxMode property.

Configuration PhysicalAdapterVLAN
{
    Import-DscResource -ModuleName PSDesiredStateConfiguration -ModuleVersion 1.1
    Import-DscResource -ModuleName NetworkingDsc -ModuleVersion 1.0.0.0

    NetworkAdapterProperty S1P1DCBX
    {
        Id = 'S1P1DCBX'
        Name = 'SLOT 1 PORT 1'
        DisplayName =  'DcbxMode'
        DisplayValue = 'Host in charge'
        Ensure = 'Present'
    }

    NetworkAdapterProperty S1P2DCBX
    {
        Id = 'S2P2DCBX'
        Name = 'SLOT 1 PORT 2'
        DisplayName =  'DcbxMode'
        DisplayValue = 'Host in charge'
        Ensure = 'Present'
    }    
}

This resource will eventually be made a part of NetworkingDsc in the official DSC resource kit. Stay tuned.

This module contains five resources at the moment.

Enable/Disable Network Adapter QoS

The NetAdapterQoS resource can be used to enable/disable QoS a specific network adapter.

Configuration NetAdapterQoS
{
    Import-DscResource -ModuleName PSDesiredStateConfiguration -ModuleVersion 1.1
    Import-DscResource -ModuleName NetQoSDSC -ModuleVersion 1.0.0.0

    NetAdapterQoS EnableAdapterQoS
    {
        NetAdapterName = 'Storage1'
        Ensure = 'Present'
    }
}

Enable/Disable DCBX Willing mode

DCBX willing mode can be enabled or disabled using the NetQoSDCBXSetting resource. This can be done at an interface level or at the global level in the operating system.

Configuration DisableGlobalDCBX
{
    Import-DscResource -ModuleName PSDesiredStateConfiguration -ModuleVersion 1.1
    Import-DscResource -ModuleName NetQosDsc -ModuleVersion 1.0.0.0

    NetQoSDcbxSetting DisableGlobal
    {
        InterfaceAlias = 'Global'
        Willing = $false
    }

    NetQoSDcbxSetting EnableStorage1
    {
        InterfaceAlias = 'Storage1'
        Willing = $true
    }
}

Enable/Disable Network QoS flow control priorities

The NetQosFlowControl resource can be used to enable or disable 802.1P flow control priorities.

Configuration NetQoSFlowControl
{
    Import-DscResource -ModuleName PSDesiredStateConfiguration -ModuleVersion 1.1
    Import-DscResource -ModuleName NetQoSDSC -ModuleVersion 1.0.0.0

    NetQoSFlowControl EnableP3
    {
        Id = 'Priority3'
        Priority = 3
        Enabled = $true
    }

    NetQoSFlowControl DisableRest
    {
        Id = 'RestPriority'
        Priority = @(0,1,2,4,5,6,7)
        Enabled = $false
    }
}

Create new QoS policies

New network QoS policies can be created using the NetQoSPolicy resource.

Configuration NewNetQoSPolicy
{
    Import-DscResource -ModuleName PSDesiredStateConfiguration -ModuleVersion 1.1
    Import-DscResource -ModuleName NetQoSDSC -ModuleVersion 1.0.0.0

    NetQosPolicy SMB
    {
        Name = 'SMB'
        PriorityValue8021Action = 3
        PolicyStore = 'localhost'
        NetDirectPortMatchCondition = 445
        Ensure = 'Present'
    }
}

Manage Network QoS Traffic classes

The NetQoSTrafficClass resource can be used to manage the traffic classes in network QoS.

Configuration NewTrafficClass
{
    Import-DscResource -ModuleName PSDesiredStateConfiguration -ModuleVersion 1.1
    Import-DscResource -ModuleName NetQoSDSC -ModuleVersion 1.0.0.0

    NetQosTrafficClass SMB
    {
        Name = 'SMB'
        Algorithm = 'ETS'
        Priority = 3
        BandwidthPercentage = 50
        Ensure = 'Present'
    }
}

This module, while code complete, needs some more work to declare as fully HQRM-compliant. I am working towards that by adding tests and better examples. Feel free to submit your issues, feedback, or PRs.

In today’s article, I am introducing the Hyper-V Server PowerShell provider.

Using this provider, you can connect to local and remote Hyper-V hosts and browse the virtual machines and virtual networks as if they are folders on a file system.

Once again, like every other provider I am writing, this is experimental as well. I am writing these to understand what needs to be considered as part of the provider design and implementation. So, the final version of these providers may look and function differently.

Follow the Github repository for information on any future updates.

I have been looking at this and experimenting with a few providers of my own. I will write more about how to approach writing a PowerShell provider using SHiPS but wanted to give you a sneak peek into the Failover Cluster PowerShell Drive (FailoverClusterDrive).

Here is the Failover Cluster PowerShell Drive in action.

This is still an experimental module. SHiPS currenly supports only get actions. So, the mounted failover cluster drive will only be read-only. There are a few more additions I am still working on in my free time and I will push another release early next year.

This is not a fork of the xFailoverCluster module. I am adding only the resources that I am developing from scratch to this module. These resources will follow the HQRM guidelines.

Resources in this module

You can take a look at each of these resources to check what different configuration options are supported as of today.

Here is an example of creating and configuring a Storage Spaces Direct cluster.

$configData = @{
    AllNodes = @(
        @{
            NodeName = 'localhost'
            thumbprint = '25A1359A27FB3F2D562D7508D98E7189F2A1F1B0'
            CertificateFile = 'C:\PublicKeys\S2D4N01.cer'
            PsDscAllowDomainUser = $true
        }
    )
}

Configuration CreateS2DCluster
{
    param
    (
        [Parameter(Mandatory = $true)]
        [pscredential]
        $Credential,

        [Parameter(Mandatory = $true)]
        [String[]]
        $ParticipantNodes,

        [Parameter(Mandatory = $true)]
        [String]
        $ClusterName,

        [Parameter(Mandatory = $true)]
        [String]
        $StaticAddress,

        [Parameter(Mandatory = $true)]
        [String[]]
        $IgnoreNetworks,

        [Parameter(Mandatory = $true)]
        [String]
        $QuorumResource,

        [Parameter(Mandatory = $true)]
        [String]
        $QuorumType
        
    )

    Import-DscResource -ModuleName FailoverClusterDsc

    Node $AllNodes.NodeName
    {
        FailoverCluster CreateCluster
        {
            ClusterName = $ClusterName
            StaticAddress = $StaticAddress
            NoStorage = $true
            IgnoreNetwork = $IgnoreNetworks
            Ensure = 'Present'
            PsDscRunAsCredential = $Credential
        }

        WaitForFailoverCluster WaitForCluster
        {
            ClusterName = $ClusterName
            PsDscRunAsCredential = $Credential
        }

        Foreach ($node in $ParticipantNodes)
        {
            FailoverClusterNode $node
            {
                NodeName = $node
                ClusterName = $ClusterName
                PsDscRunAsCredential = $Credential
                Ensure = 'Present'
            }
        }

        FailoverClusterQuorum FileShareQuorum
        {
            IsSingleInstance = 'Yes'
            QuorumType = $QuorumType
            Resource = $QuorumResource
        }

        FailoverClusterS2D EnableS2D
        {
            IsSingleInstance = 'yes'
            Ensure = 'Present'
        }
    }
}

CreateS2DCluster -Credential (Get-Credential) -ConfigurationData $configData `
                                           -QuorumType 'NodeAndFileShareMajority' `
                                           -QuorumResource '\\sofs\share' `
                                           -ClusterName 'S2D4NCluster' `
                                           -StaticAddress '172.16.102.45' `
                                           -IgnoreNetworks @('172.16.103.0/24','172.16.104.0/24') `
                                           -ParticipantNodes @('S2D4N02','S2D4N03','S2D4N04')

In the above pattern, I am creating a failover cluster and then adding the remaining nodes using the same configuration document. You can, however, have the node addition configuration using the FailoverClusterNode resource as a separate configuration document that gets enacted on the participant node.

The failover cluster configuration requires administrator privileges and these resources do not have a Credential parameter of their own and depend on PSDscRunAsCredential. Therefore, you need at least PowerShell 5.0 to use these resources.

I am looking at expanding the resource modules to beyond what is there at the moment. If you see any issues or have feedback, feel free to create an issue in my repository. These resources lack tests today. I would be glad to accept any PRs for tests.

We will go through the following deployment scenarios, going from traditional options to cloud services and containers:

• Deploy to local machine
• Deploy to an Infrastructure as a Service (IaaS) VM
• Deploy to a Platform as a Service (PaaS) website
• Deploy to a container running Windows Server Core
• Deploy to a container running Nano Server

Our example application is the PowerShell Universal Dashboard – a web application built on ASP .NET Core, PowerShell Core, ReactJS, and a handful of JavaScript libraries.

This means it can support cross-platform deployments, running on Windows, Linux, and macOS.

That is already a very flexible range of options, but we can get even more options by using containers.

The PowerShell Universal Dashboard PowerShell module allows for creation of web-based dashboards. The client- and server-side code for the dashboard is authored completely with PowerShell. Charts, monitors, tables and grids can easily be created with the cmdlets included in the module. The module is a cross-platform module and will run anywhere PowerShell Core can run.

Deployment options

Deploy to local machine

Let us first look at how to install and setup the application locally on a client machine.

Install the UniversalDashboard module from the PowerShell Gallery using PowerShellGet:

Install-Module UniversalDashboard

When the module is installed you will have access to a number of new commands:

Start-UDDashboard is the main command for starting a new instance of a dashboard.

For the example usage of the other commands, I would encourage you to have a look at the Components section of the beforementioned demo website.

The example dashboard I am going to use for this article is one from a real-world scenario. During an onboarding process, there was a need to gather some data (an employee number) from an external company. The link was to an instance of PowerShell Universal Dashboard, which would take the input from the external user and send it as a parameter to an Azure Automation webhook. The webhook would start an Azure Automation runbook, which would register the data in the internal system.

The dashboard for this scenario is available in this Gist.

When the UniversalDashboard PowerShell module is installed, we can simply run the dashboard.ps1 script to start the dashboard. Here is a screenshot from performing this in Visual Studio Code:

At this point, the website should be up and running. We can test by navigating to http://localhost/register/123

In this example, we are also taking input from the URL. 123 can be any number (in the example scenario, a service request ID), and will be available as a parameter variable in the New-UDInput command.

At this point, we have the application up and running on a local computer as a PowerShell module. For production use, we want to run this on a server platform.

As stated in the product documentation, the PowerShell Universal Dashboard can be hosted in Azure or IIS:

To host a dashboard in Azure or IIS, you will need to deploy the entire module to your site or WebApp. In the root module directory, place your dashboard.ps1. You need to specify the -Wait parameter on Start-UDDashboard so that the script blocks and waits for requests in Azure or IIS. Specifying the port isn’t necessary because Azure and IIS use dynamic port tunneling.

IIS requires that the ASP.NET Core module is installed. 

Deploy to an Infrastructure as a Service (IaaS) VM running Internet Information Services (IIS)

The other option would be to simply run the website in IIS on a Windows Server which can run anywhere: on-premises, a public cloud, or a hosting provider.

In this example, we are using a virtual machine running in Azure based on the Azure Marketplace  Windows Server Datacenter, version 1709 image.

Step 1 – Install the web server role: Install-WindowsFeature -Name Web-Server

Step 2 – Install the .NET Core Windows Server Hosting bundle as described here (needed since the PowerShell Universal Dashboard is built on .NET Core and not the built-in .NET Desktop edition).

Step 3 – Copy the UniversalDashboard PowerShell module to the path where the website is running, for example C:\inetpub\wwwroot if the Default Web Site is leveraged.

Step 4 – Copy the dashboard.ps1 and license.lic files to the same directory:

That`s it – at this point the website should be up and running.

Deploy to a Platform as a Service (PaaS) website

The route for the mentioned real-world scenario was to host the module in a public cloud service, in this case Azure App Service:

There are many different options for deploying a web application to an instance of Azure App Service. Microsoft has provided some examples for us to use:

1. Create a web app with deployment from GitHub
2. Create a web app with continuous deployment from GitHub
3. Create a web app and deploy code with FTP
4. Create a web app and deploy code from a local Git repository
5. Create a web app and deploy code to a staging environment

#3 was used to build the demo website for this article, but in a production environment a more appropriate method would be to leverage continuous integration to deploy files to the website based on commits from source code.

We get many benefits from leveraging a PaaS offering such as Azure App Service:

• No servers to manage (no patching, monitoring, etc)
• We can add custom domains and SSL certificates as part of the service
• (bigger VM sizes)
• Scale in and out (more VM instances)
• Deploy the application using continuous delivery such as Visual Studio Team Services

There are also many other benefits such as pre-authentication, load balancing and more.

Containerization

Next, we will look at leveraging containers – a solution to the problem of how to get software to run reliably when moved from one computing environment to another.

This is a new technology with a promise to change the IT landscape the same way as virtualization did in the early 2000s.

Deploy to a container running Windows Server Core

Our first example of containerizing the PowerShell Universal Dashboard will be based on Windows Server Core, version 1709. Since we already have the application up and running on a native operating system, it should be easy in this case to transform it into a container.

The files for the following demos are available here.

In the WindowsServerCoreDemoWebsite, we have the following files:

dashboard.ps1 and license.lic are the same files we used when running the application in a native operating system. These will be referenced in the Dockerfile to be copied into the container image.

In this example, we are using Docker – a container management tool – to build and deploy our demos. From the documentation:

Docker can build images automatically by reading the instructions from a Dockerfile. A Dockerfile is a text document that contains all the commands a user could call on the command line to assemble an image. Using docker build users can create an automated build that executes several command-line instructions in succession.

Let us have a look at the Dockerfile which defines our Windows Server Core, version 1709 demo website:

We are using an image from Docker Hub – a central repository for Docker images – which is built by Microsoft and have IIS pre-installed. Then we do not have to think about installing and configuring the Web-Server role.

Next, we are downloading and installing the .NET Core Windows Server Hosting bundle like we did when running the application on a native operating system.

PowerShellGet is used to download the UniversalDashboard module.

The last step is to copy the dashboard.ps1 and licence.lic files as well as exposing the port the website is running on.

cd "~\Git\PSCommunity\Containers\PowerShell Universal Dashboard"

# Note: Remember to switch to Windows Containers before building the docker file (Linux is the default after installing Docker for Windows)
docker build WindowsServerCoreDemoWebsite -t psmag:demowebsite --no-cache
docker build NanoDemoWebsite -t psmag:nanodemowebsite --no-cache

#region 1 Windows Server Core
$ContainerID = docker run -d --rm psmag:demowebsite
$ContainerIP = docker inspect -f "{{ .NetworkSettings.Networks.nat.IPAddress }}" $ContainerID

# Verify that the website is up and running
Start-Process -FilePath iexplore.exe -ArgumentList http://$ContainerIP/register/123
Start-Process -FilePath chrome.exe -ArgumentList http://$ContainerIP/register/123

# Optionally, connect to a container instance interactively to inspect the environment.
# The IIS image have a service monitor as an entrypint, thus we need to override this to get into the container interactively
docker run --entrypoint=powershell -it psmag:demowebsite

docker stop $ContainerID

#endregion

#region 2 Nano Server 1709
$ContainerID = docker run -d --rm psmag:nanodemowebsite
$ContainerIP = docker inspect -f "{{ .NetworkSettings.Networks.nat.IPAddress }}" $ContainerID

# Verify that the website is up and running
Start-Process -FilePath iexplore.exe -ArgumentList http://$ContainerIP/register/123
Start-Process -FilePath chrome.exe -ArgumentList http://$ContainerIP/register/123

# Optionally, connect to the container instance interactively to inspect the environment
docker exec -ti $ContainerID pwsh #pwsh for Nano/powershell for Server Core

docker stop $ContainerID

#endregion

When the Dockerfile is ready, we can user docker.exe to build a container image (line 4-5).

When the image is successfully built, we are ready to test it by starting a container instance using our new image (line 8).

If you have made modifications to dashboard.ps1 or simply want the latest version of the UniversalDashboard module, re-run the docker build command and the image will be updated with any changes.

Deploy to a container running Nano Server

Before we try to make our demo application run on Nano Server, there are some important changes to Nano Server introduced in Windows Server 1709 to be aware of:

Starting with the new feature release of Windows Server, version 1709, Nano Server will be available only as a container base OS image. You must run it as a container in a container host, such as a Server Core installation of Windows Server. Running a container based on Nano Server in the new feature release differs from earlier releases in these ways:

• Nano Server has been optimized for .NET Core applications.
• Nano Server is even smaller than the Windows Server 2016 version.
• PowerShell Core, .NET Core, and WMI are no longer included by default, but you can include PowerShell Core and .NET Core container packages when building your container.
• There is no longer a servicing stack included in Nano Server. Microsoft publishes an updated Nano container to Docker Hub that you redeploy.
• You troubleshoot the new Nano Container by using Docker.
• You can now run Nano containers on IoT Core.

One more thing that is useful to know, but not mentioned in the referenced article, is that IIS is not available in Nano Server 1709.

This means we need to take a different approach to get the application running in a Nano-based container.

PowerShell Universal Dashboard is built on top of Kestrel – a cross-platform web server for ASP.NET Core. This means we can simply run Start-UDDashboard from PowerShell Core in Nano Server 1709 to get the web application up and running.

Rick Strahl has written a great article about Publishing and Running ASP.NET Core Applications with IIS where he mentions the following:

Kestrel is a .NET Web Server implementation that has been heavily optimized for throughput performance. It’s fast and functional in getting network requests into your application, but it’s ‘just’ a raw Web server. It does not include Web management services as a full featured server like IIS does. If you run on Windows you will likely want to run Kestrel behind IIS to gain infrastructure features like port 80/443 forwarding via Host Headers, process lifetime management and certificate management to name a few.

The bottom line for all of this is if you are hosting on Windows you’ll want to use IIS and the AspNetCoreModule.

Some of the limitations can be overcome by leveraging different options such as a PaaS service or custom reverse proxy to publish the application externally, as well as a container orchestration tool such as Kubernetes for scaling and managing the application.

With these limitations in mind, let us go on and see if we can get this working on the latest version of Nano Server.

As mentioned previously, PowerShell Core has been removed from Nano Server starting with the 1709 release. The first step would be to build a new container image where PowerShell Core is included.

Luckily, the PowerShell team have published the Dockerfile for the official Docker image for PowerShell Core on GitHub.

In our custom Dockerfile, we are leveraging almost all of the Dockerfile used to build PowerShell Core. We are also using the concept of multi stage builds to include ASP .NET Core by referencing FROM microsoft/aspnetcore:2.0-nanoserver-1709 in the Dockerfile.

The only code we need to add is to download the UniversalDashboard PowerShell module as well as copy the dashboard.ps1 file as we did when running on Server Core:

When using IIS, licence.lic was automatically read by the application. This is not the case when using the module directly from PowerShell, hence we have added Set-UDLicense to specify the licence inside dashboard.ps1.

Since IIS is not available, we need to add an entry point in the Dockerfile in order to launch the dashboard.ps1 file. This will launch the website by calling Start-UDDashboard.

You might notice that PowerShell Core is called by using pwsh.exe. With the release of PowerShell Core 6.0.0-beta.9 the binary for PowerShell Core was renamed from powershell.exe and powershell on Windows and Linux/Unix/macOS respectively to pwsh.exe and pwsh. Mark Kraus has written a great article with more background info about that change.

A final note about Nano Server: On the Azure Marketplace offering for Windows Server, there is an offer called Windows Server Datacenter, version 1709 with Containers. If you deploy an instance of that image, Docker will be pre-installed and you can download the latest official Microsoft image of Nano Server from Docker Hub by running docker pull microsoft/nanoserver:1709.

Summary

In this article, we have explored various options for hosting a web application in different environments, starting with a native operating system and ending up with a very small container image based on Nano Server.

After setting up the context in the previous post, it is time to look at how the authoring workflow looks like when using Pester for writing operations validation tests, to begin with, and then leveraging PSRemotely DSL to target it to the remote nodes.
This workflow consists of below stages:

1. Getting your tests ready, target/test a single node.
2. Prepare configuration data (abstract hardcoded values).
3. Use PSRemotely for remote operations validation.
4. Debugging failures.
5. Reporting.

Note – Stages 1-3 will be covered in this post and there will be another post on Stages 4 & 5.

Since PSRemotely was born out of needs for validating an engineered solution, it excels at validating solutions e.g. where the nodes are consistent in behavior and have to be tested for the similar configurations.

Scenario

To illustrate the point, I am taking an example of deploying the Hyper-converged solution using Storage Spaces Direct. Now as per the referenced article the deployment has three stages:

1. Deploy Windows Server
2. Configure the Network
3. Configure Storage Spaces Direct

Ideally, the operations validation should run after each step to validate that the entire solution is being configured as per the best practice. To keep today’s post simple, we will be validating only the first step which deploying Windows server but the similar steps apply while authoring the validation tests for the other stages in the deployment workflow.

Now take a look at the referenced link and gather the list of configurations that need to be in place on each node as per the step 1.

1. Deploy Windows Server 2016.
2. Verify the domain account is a member of the local administrator group.

So now we have the configurations we need to check on each node just before we configure networking on top of them. You can follow the commits on this branch on this test repository to see the changes made as part of the authoring workflow.

Stage 1 – Get your tests ready.

This stage consists of authoring tests using Pester/PoshSpec for operations validations.
Let us start by translating the above gathered configurations in Pester Describe blocks independently, to begin with.

Below is a very crude way that can be used to determine that Windows Server 2016 is installed on the node. There are two Pester assertions –. First one asserts that OS type is a server and the OS SKU is either datacenter edition with GUI or server core.

# Ensure that Server 2016 OS installation is done.
Describe "Operating system installed validation" {
    $OS = Get-CimInstance -Class Win32_OperatingSystem

    Context "Server Type check" {
        It "Should have the OSType as WinNT" {
            $OS.OSType | Should Be 18
        }
    }

    Context 'Server 2016 check' {
        It "Should have server 2016 installed" {
           $OS.Caption | Should BeLike '*Server 2016*'
        }
    }
}

Here is another independent test for validating that the domain account is a member of the local administrators group on a node.

# Validate that the domain account is part of the local administrators group on a node.
Describe "Domain account is local administrator validation" {
    $LocalGroupMember = Get-LocalGroupMember -Group Administrators -Member "S2DClusterAdmin" -ErrorAction SilentlyContinue
    It "Should be member of local admins group" {
        $LocalGroupMember | Should NOT BeNullOrEmpty
    }
}

Stage 2 – Prepare node configuration data

If you look at the authored Pester describe blocks to validate the configuration on the nodes, it might use environment specific data hard coded into the tests e.g. domain username in above example.

So we need to now collect all this data which is environment specific and decouple it from our tests.
Start with the below empty configuration data (place it in the EnvironmentConfigData.psd1 file) and start populating it (it follows the DSC style configuration data syntax).

@{
    AllNodes = @(
        @{
            # common node information hashtable
            NodeName = '*'; # do not edit
        },
        @{
            # Individual node information hashtable
            NodeName = 'Node1'
        },
        @{
            NodeName = 'Node2'
        },
        @{
            NodeName = 'Node3'
        },
        @{
            NodeName = 'Node4'
        }
    )
}

Start by placing them inside the node configuration data with a general thumb rule of mapping common data to common node information hashtable and node specific details to node configuration hashtable.

Now in the previous tests, the only input is the domain user name. So we can add that to common node information hashtable, since the domain user is a member of the local administrators group needs to be validated on all the nodes in the solution. So now the configuration data looks like below:

@{    
    AllNodes = @(
        @{
            # common node information hashtable
            NodeName = '*'; # do not edit
            DomainUser = 'S2DClusterAdmin'
        },
        @{
            # Individual node information hashtable
            NodeName = 'Node1'
        },
        @{
            NodeName = 'Node2'
        },
        @{
            NodeName = 'Node3'
        },
        @{
            NodeName = 'Node4'
        }
    )
}

Stage 3 – Using PSRemotely for remote ops validation

At this stage in the authoring workflow, we have our tests ready along with the environment configuration data in hand. Before using PSRemotely to target all the nodes for deployment readiness we have to ask this question, How do we connect over PSRemoting to these nodes?

• Are the nodes domain joined?
• Connect using the DNS name resolution or IPv4/IPv6 addresses for the remote nodes?
• Connect using the logged in user account or specifying an alternate account?

Based on the answers to the above questions usage with PSRemotely DSL varies a bit and most of them are documented. For this scenario, the DNS Name resolution of the nodes is used (nodes are already domain joined) and the logged in user account will be used to connect to the remote nodes.

Now it is time to wrap our existing operations validation tests inside the PSRemotely DSL. The DSL consists of two keywords PSRemotely and Node. PSRemotely is the outermost keyword which allows the framework to:

• Specify that all ops validations tests are housed inside a <filename>.PSRemotely.ps1 file.
• Specify the environment configuration data e.g. hashtable/ .psd1 file/ .json file.
• Specify credentials to be used to connect to the each node (if required).
• Specify a node specific property in the configuration data to be used for initiating the PSRemoting session.
• Populate custom variables in the remote node’s execution context.

Node keyword is where we target and organize our tests based on some environment specific data, it is very similar to the Node keyword in the DSC. If you would like to target different validations to nodes based on some configuration data then it can be done using the Node keyword.

Getting back to the problem at hand let’s wrap our existing Pester tests inside the PSRemotely DSL. This is straightforward for the problem at hand and looks like below. We can save the contents of below code snippet in a file called S2DValidation.PSRemotely.ps1 (PSRemotely only accepts file with .PSRemotely.ps1 extension).

Note –  Take note of how the hard coded value for domain username (S2DClusterAdmin) from the standalone Pester tests is replaced with node specific configuration data e.g. $Node.DomainUser.

# Use the PSRemotely DSL to wrap the existing Pester tests and target remote nodes
PSRemotely -Path "$PSScriptRoot\EnvironmentConfigData.psd1" {
    # All the nodes in the solution are to be targeted
    Node $AllNodes.NodeName {
        # Ensure that Server 2016 OS installation is done.
        Describe "Operating system installed validation" {
            $OS = Get-CimInstance -Class Win32_OperatingSystem
            Context "Server Type check" {
                It "Should have the OSType as WinNT" {
                    $OS.OSType | Should Be 18
                }
            }
            Context 'Server 2016 check' {
                It "Should have server 2016 installed" {
                $OS.Caption | Should BeLike '*Server 2016*'
                }
            }
        }
        # Validate that the domain account is part of the local administrators group on a node.
        Describe "Domain account is local administrator validation" {
          $LocalGroupMember = Get-LocalGroupMember -Group Administrators -Member "$($Node.DomainUser)" -ErrorAction SilentlyContinue
            It "Should be member of local admins group" {
                $LocalGroupMember | Should NOT BeNullOrEmpty
            }
        }
    }
}

We are all set and have two files in the directory e.g. EnvironmentConfigData.psd1 and S2DValidation.PSRemotely.ps1, it is finally time to invoke PSRemotely and give remote operations validation a go.

We can use Invoke-PSRemotely in the current directory to run all the operations validation housed inside it or specify a path to a file ending with *.PSRemotely.ps1 extension.

As shown in the above video, for each node targeted a JSON object is returned. In the returned JSON object the property Status is true if all the tests (Describe blocks) passed on the remote node. Tests property is an array of individual tests (Describe block) run on the Remotely node if all the tests pass then an empty JSON object array of TestResult is returned otherwise the Error record thrown by Pester is returned.

For the node which failed one of the validations, the JSON object looks like below. Individual TestResult will contain more information on the failing tests on the remote nodes.

For the failed node, we can quickly verify that out of the two validations targeted at the remote node only one is failing.

Now there could be a lot many reasons on why the operation validations tests on the remote node are failing. In the next post, we will take a look at how to connect to the underlying PSSession being used by PSRemotely to debug these failures.

• Renamed Hyper-VConfigurations composite resource module to HyperVConfigurations. This is a breaking change.
• Added SystemConfigurations composite resource module containing one composite configuration that includes domain join, remote desktop, timezone, and IE enhanced security configurations.
• Added Pre-Deploy tests under Diagnostics for each composite resource.

Here is a summary of all composite resources in this repository:

Let’s come to the subject of today’s post. In any infrastructure that you are deploying, even f you are no automation guy, there will be a set of prerequisite checks you would perform. For example, if your goal is to deploy a switch embedded team for a Hyper-V host configuration, you will have to check for the existence of physical network adapters in the system that you plan to use within the SET configuration. And, there will be many such pre-deployment checks that you need to perform. So, when using these infrastructure blueprints, it is ideal to package the pre-deployment tests as well into the composite resource module itself.

To address this need, I added PreDeploy scripts under diagnostics tests for each composite resource.

The PreDeploy folder is where all my pre-deployment tests are stored. Here is the pre-deployment test script for the SET team.

Describe 'Predeploy tests for Hyper-V Deployment with Switch Embedded Teaming and related network Configuration' {
    Context "Operating System verison tests" {
        $OS = Get-CimInstance -Class Win32_OperatingSystem
        It "Should have the OSType as WinNT" {
            $OS.OSType | Should Be 18
        }
        
        It "Should have server 2016 installed" {
           $OS.Caption | Should BeLike '*Server 2016*'
        }        
    }

    Context 'Network adapters should exist' {
        Foreach ($adapter in $configurationData.AllNodes.NetAdapterName)
        {
            It "Network adapter named '$adapter' should exist" {
                Get-NetAdapter -Name $adapter -ErrorAction SilentlyContinue | Should Not BeNullOrEmpty
            }
        }
    }
}

In the above test scripts, we check that the OS version is indeed Windows Server 2016 to ensure SET configuration can be deployed. Also, we check for the presence of physical network adapters listed in the configuration data to ensure that the SET configuration completes with no errors.

The above flow summarizes the deployment workflow. We execute the pre-deployment tests first and perform deployment only once these tests are all successful. Once the deployment is complete, we run either comprehensive or simple operations tests and put the system into operations only when these tests are successful.

Whatever orchestration script or method that you plan to use, putting this workflow into the process will certainly help you build a resilient deployment pipeline.

Publish-AzureRmVMDscConfiguration ".\MyConfiguration.ps1" -OutputArchivePath ".\MyConfiguration.ps1.zip"

When I first used this cmdlet, I felt this was really a good idea for on-premise build processes and immediately tried to find out how they discover module dependencies. I was almost certain that it was not just text parsing but may be a little bit more than that. This exploration lead me to the source code for this cmdlet and I certainly saw lot of traces towards AST being used.

The second instance that I came across the usage of AST in finding resource module dependencies was in the Configuration function in the PSDesiredStateConfiguration module. This function, starting from WMF 5.0, has a runtime parameter called ResourceModulesTuplesToImport. 

PS C:\> (Get-Command Configuration | Select-Object -ExpandProperty Parameters).ResourceModuleTuplesToImport


Name            : ResourceModuleTuplesToImport
ParameterType   : System.Collections.Generic.List`1[System.Tuple`3[System.String[],Microsoft.PowerShell.Commands.ModuleSpecification[],System.Version]]
ParameterSets   : {[__AllParameterSets, System.Management.Automation.ParameterSetMetadata]}
IsDynamic       : False
Aliases         : {}
Attributes      : {__AllParameterSets, System.Management.Automation.ArgumentTypeConverterAttribute}
SwitchParameter : False

The argument for the ResourceModulesTuplesToImport gets populated at runtime — when a Configuration gets loaded for the first time. To be specific, when you create a configuration document and load it into the memory, AST gets triggered and populates the argument to this parameter. You can trace this back to ast.cs. Here is a part of that.

///////////////////////////
// get import parameters
var bodyStatements = Body.ScriptBlock.EndBlock.Statements;
var resourceModulePairsToImport = new List<Tuple<string[], ModuleSpecification[], Version>>();
var resourceBody = (from stm in bodyStatements where !IsImportCommand(stm, resourceModulePairsToImport) select (StatementAst)stm.Copy()).ToList();

So, the whole magic of deriving the dependent modules is happening in the IsImportCommand method. Once I reviewed the code there, it wasn’t tough to reverse engineer that into PowerShell.

I published my scripts to https://github.com/rchaganti/PSDSCUtils. Let’s take a look at the script now.

[CmdletBinding()]
param (
    [Parameter(Mandatory)]
    [String] $ConfigurationScript,

    [Parameter()]
    [Switch] $Package,

    [Parameter()]
    [String] $PackagePath
)

$ConfigurationScriptContent = Get-Content -Path $ConfigurationScript -Raw
$ast = [System.Management.Automation.Language.Parser]::ParseInput($ConfigurationScriptContent, [ref]$null, [ref]$null)
$configAst = $ast.FindAll({ $args[0] -is [System.Management.Automation.Language.ConfigurationDefinitionAst]}, $true)
$moduleSpecifcation = @()
foreach ($config in $configAst)
{
    $dksAst = $config.FindAll({ $args[0] -is [System.Management.Automation.Language.DynamicKeywordStatementAst]}, $true)

    foreach ($dynKeyword in $dksAst)
    {
        [System.Management.Automation.Language.CommandElementAst[]] $cea = $dynKeyword.CommandElements.Copy()
        $allCommands = [System.Management.Automation.Language.CommandAst]::new($dynKeyword.Extent, $cea, [System.Management.Automation.Language.TokenKind]::Unknown, $null)
        foreach ($importCommand in $allCommands)
        {
            if ($importCommand.CommandElements[0].Value -eq 'Import-DscResource')
            {
                [System.Management.Automation.Language.StaticBindingResult]$spBinder = [System.Management.Automation.Language.StaticParameterBinder]::BindCommand($importCommand, $false)
            
                $moduleNames = ''
                $resourceNames = ''
                $moduleVersion = ''
                foreach ($item in $spBinder.BoundParameters.GetEnumerator())
                { 
                    $parameterName = $item.key
                    $argument = $item.Value.Value.Extent.Text

                    #Check if the parametername is Name
                    $parameterToCheck = 'Name'
                    $parameterToCheckLength = $parameterToCheck.Length
                    $parameterNameLength = $parameterName.Length

                    if (($parameterNameLength -le $parameterToCheckLength) -and ($parameterName.Equals($parameterToCheck.Substring(0,$parameterNameLength))))
                    {
                        $resourceNames = $argument.Split(',')
                    }

                    #Check if the parametername is ModuleName
                    $parameterToCheck = 'ModuleName'
                    $parameterToCheckLength = $parameterToCheck.Length
                    $parameterNameLength = $parameterName.Length
                    if (($parameterNameLength -le $parameterToCheckLength) -and ($parameterName.Equals($parameterToCheck.Substring(0,$parameterNameLength))))
                    {
                        $moduleNames = $argument.Split(',')
                    }

                    #Check if the parametername is ModuleVersion
                    $parameterToCheck = 'ModuleVersion'
                    $parameterToCheckLength = $parameterToCheck.Length
                    $parameterNameLength = $parameterName.Length
                    if (($parameterNameLength -le $parameterToCheckLength) -and ($parameterName.Equals($parameterToCheck.Substring(0,$parameterNameLength))))
                    {
                        if (-not ($moduleVersion.Contains(',')))
                        {
                            $moduleVersion = $argument
                        }
                        else
                        {
                            throw 'Cannot specify more than one moduleversion' 
                        }
                    }
                }

                #Get the module details
                #"Module Names: " + $moduleNames
                #"Resource Name: " + $resourceNames
                #"Module Version: " + $moduleVersion 

                if($moduleVersion)
                {
                    if (-not $moduleNames)
                    {
                        throw '-ModuleName is required when -ModuleVersion is used'
                    }

                    if ($moduleNames.Count -gt 1)
                    {
                        throw 'Cannot specify more than one module when ModuleVersion parameter is used'
                    }
                }

                if ($resourceNames)
                {
                    if ($moduleNames.Count -gt 1)
                    {
                        throw 'Cannot specify more than one module when the Name parameter is used'
                    }
                }
            
                #We have multiple combinations of parameters possible
                #Case 1: All three are provided: ModuleName,ModuleVerison, and Name
                #Case 2: ModuleName and ModuleVersion are provided
                #Case 3: Only Name is provided
                #Case 4: Only ModuleName is provided
                
                #Case 1, 2, and 3
                #At the moment, there is no error check on the resource names supplied as argument to -Name
                if ($moduleNames)
                {
                    foreach ($module in $moduleNames)
                    {
                        if (-not ($module -eq 'PSDesiredStateConfiguration'))
                        {
                            $moduleHash = @{
                                ModuleName = $module
                            }

                            if ($moduleVersion)
                            {
                                $moduleHash.Add('ModuleVersion',$moduleVersion)
                            }
                            else
                            {
                                $availableModuleVersion = Get-RecentModuleVersion -ModuleName $module
                                $moduleHash.Add('ModuleVersion',$availableModuleVersion)
                            }

                            $moduleInfo = Get-Module -ListAvailable -FullyQualifiedName $moduleHash -Verbose:$false -ErrorAction SilentlyContinue
                            if ($moduleInfo)
                            {
                                #TODO: Check if listed resources are equal or subset of what module exports
                                $moduleSpecifcation += $moduleInfo
                            }
                            else
                            {
                                throw "No module exists with name ${module}"
                            }
                        }
                    }    
                }

                #Case 2
                #Foreach resource, we need to find a module
                if ((-not $moduleNames) -and $resourceNames)
                {
                    $moduleHash = Get-DscModulesFromResourceName -ResourceNames $resourceNames -Verbose:$false
                    foreach ($module in $moduleHash)
                    {
                        $moduleInfo = Get-Module -ListAvailable -FullyQualifiedName $module -Verbose:$false   
                        $moduleSpecifcation += $moduleInfo 
                    }
                }
            }
        }
    }
}

if ($Package)
{
    #Create a temp folder
    $null = mkdir "${env:temp}\modules" -Force -Verbose:$false

    #Copy all module folders to a temp folder
    foreach ($module in $moduleSpecifcation)
    {
        $null = mkdir "${env:temp}\modules\$($module.Name)"
        Copy-Item -Path $module.ModuleBase -Destination "${env:temp}\modules\$($module.Name)" -Container -Recurse -Verbose:$false
    }

    #Create an archive with all needed modules
    Compress-Archive -Path "${env:temp}\modules" -DestinationPath $PackagePath -Force -Verbose:$false

    #Remove the folder
    Remove-Item -Path "${env:temp}\modules" -Recurse -Force -Verbose:$false
}
else
{
    return $moduleSpecifcation
}

function Get-DscModulesFromResourceName
{
    [CmdletBinding()]
    param (
        [Parameter(Mandatory)]
        [string[]] $ResourceNames
    )

    process
    {
        $moduleInfo = Get-DscResource -Name $ResourceNames -Verbose:$false | Select -Expand ModuleName -Unique
        $moduleHash = @()
        foreach ($module in $moduleInfo)
        {
            $moduleHash += @{
                 ModuleName = $module
                 ModuleVersion = (Get-RecentModuleVersion -ModuleName $module)
            }
        }

        return $moduleHash
    }
}

function Get-DscResourcesFromModule
{
    [CmdletBinding()]
    param (
        [Parameter(Mandatory)]
        [String] $ModuleName,

        [Parameter()]
        [Version] $ModuleVersion
    )

    process
    {
        $resourceInfo = Get-DscResource -Module $ModuleName -Verbose:$false
        if ($resourceInfo)
        {
            if ($ModuleVersion)
            {
                $resources = $resourceInfo.Where({$_.Module.Version -eq $ModuleVersion})
                return $resources.Name
            }
            else
            {
                #check if there are multiple versions of the modules; if so, return the most recent one
                $mostRecentVersion = Get-RecentModuleVersion -ModuleName $ModuleName
                Get-DscResourcesFromModule -ModuleName $ModuleName -ModuleVersion $mostRecentVersion
            }
        }
    }
}

function Get-RecentModuleVersion
{
    [CmdletBinding()]
    param (
        [Parameter(Mandatory)]
        [String] $ModuleName
    )

    process
    {
        $moduleInfo = Get-Module -ListAvailable -Name $ModuleName -Verbose:$false | Sort -Property Version
        if ($moduleInfo)
        {
            return ($moduleInfo[-1].Version).ToString()
        }
    }
}

Here is how you used this script:

With just the -ConfigurationScript parameter, this script emits a ModuleInfo object that contains a list of modules that are being imported in the configuration script.

In case you need to package the modules into a zip archive, you can use the -Package and -PackagePath parameters.

.\Get-DSCResourceModulesFromConfiguration.ps1 -ConfigurationScript C:\Scripts\VMDscDemo.ps1 -Package -PackagePath C:\Scripts\modules.zip

There are many uses cases for this. I use this extensively in my Hyper-V lab configurations. What are your use cases?

PowerShell Desired State Configuration (DSC) is a management platform which enables you to configure, deploy, and manage systems. DSC provides declarative, autonomous and idempotent deployment, configuration and conformance for standards-based managed elements. For more information on DSC refer to the PowerShell DSC documentation.

Cisco UCS DSC Resource aids in achieving Configuration as Code in turn helping you to follow the DevOps model.

The Cisco UCS DSC module provides six DSC custom resources which cover the majority of use cases.  You can view the custom UCS resources by running the Get-DscResource cmdlet as shown below.

Solution Architecture Overview

Before getting in to the details of the resources let’s review some basic concepts of DSC and the overall architecture of the UCS PowerTool DSC solution.

The DSC management platform consists of three main components.

• Configuration: This is where you define the configurations that need to be applied in a declarative manner. Once you run this configuration, DSC will take care of ensuring the system is in the state that is defined in the configuration.
• Resources: These are the building blocks for the configurations.
• Local Configuration Manager (LCM): This is the engine that facilitates the interaction between resources and configurations. The LCM regularly polls the state of the system and takes appropriate actions based on the resource. The LCM runs on every target node.

In DSC there are two ways to deploy a configuration.

• Push Mode – Push mode is a manual deployment of DSC resources to target nodes. Once the configuration is compiled the user runs the Start-DscConfiguration cmdlet to push the configuration to the desired nodes.
• Pull Mode – Pull mode configures nodes to check in to a DSC web pull server to retrieve the configuration files. When a new configuration is available, the LCM downloads and applies it to the target node.

To utilize DSC functionality with Cisco UCS Manager or Cisco IMC an intermediate server is required. The intermediate server is a Windows Server having the required Windows Management Framework (WMF), PowerShell and the UCS PowerTool Suite installed.  A typical architecture is shown in the figure.

Central Server—This server is used to write the UCS DSC configuration scripts for Cisco UCS Manager or Cisco IMC. This can be configured as a pull server if the method of deployment is Pull Mode.

Intermediate Server— The Central server deploys the configuration to the Intermediate server. This server applies the configuration to the Cisco UCS Manager or Cisco IMC using the UCS PowerTool DSC cmdlets.

Cisco UCS Manager DSC Resources

There are four custom resources provided for configuring Cisco UCS Manager.

• UcsSyncMoWithReference: This resource syncs configuration from a reference UCS Manager to any target UCS Managers.
• UcsManagedObject: This resource configures any UCS Manager Managed Object (MO) by specifying the MO details.
• UcsScript: This resource allows for the execution of UCS Manager PowerTool cmdlets.
• UcsSyncFromBackup: This resource applies configuration from a backup file to any target UCS Managers.

Generating DSC configuration document for UCS Manager GUI operations

To simplify the process of authoring the DSC configuration documents for UCS Manager use the ConvertTo-UcsDscConfig cmdlet. This cmdlet is similar to the ConvertTo-UcsCmdlet cmdlet that generates the UCS PowerTool cmdlets for the actions performed on the UCS Manager GUI.  Creating a configuration document is a simple two-step process.

1. Launch the UCS Manager GUI, either using the Start-UcsGuiSession cmdlet or manually, then run the ConvertTo-UcsDscConfig cmdlet, if -OutputFilePath is supplied the DSC configuration will be written to the specified file, otherwise the ConvertTo-UcsDscConfig cmdlet will produce output to the UCS PowerTool console session.
2. Create the required configuration using the UCS Manager GUI. When you are done creating configurations you can open the input file specified in the step 1. The required DSC configuration document will have been auto-generated. If no output file was specified, cut and paste the UCS PowerTool console output to a file.

Once you have the auto-generated document you just need to customize a few environment-related settings like Configuration Data, UCS Manager connection details and Credentials.

UcsSyncMoWithReference Custom Resource

If you have more than one UCS domain in your datacenter and want to maintain a baseline configuration across all the UCS domains use the UcsSyncMoWithReference resource.

You can create a configuration using this resource by specifying the Distinguished Name (DN) of the Managed Object (MO) that needs to be synced.  Here is an example of how you can sync a Service Profile, Service Profile Template, and LDAP Groups.

UcsSyncMoWithReference SyncServiceProfile
         {
           UcsCredentials = $ucsCredential
           UcsConnectionString = $ucsConnString
           RefUcsCredentials = $refUcsCredential
           RefUcsConnectionString = $refUcsConString
           Ensure="Present"
           Identifier ="2"
           Hierarchy=$true
           Dn = "org-root/ls-SPExchangeServer"
         } 

         UcsSyncMoWithReference SyncSpTemplate
         {
           UcsCredentials = $ucsCredential
           UcsConnectionString = $ucsConnString
           RefUcsCredentials = $refUcsCredential
           RefUcsConnectionString = $refUcsConString
           Ensure="Present"
           Identifier ="3"
           Hierarchy=$true
           Dn = "org-root/ls-SPTSqlServer"
         }


         UcsSyncMoWithReference SyncLDAPGroups
         {
           UcsCredentials = $ucsCredential
           UcsConnectionString = $ucsConnString
           RefUcsCredentials = $RefUcsCredential
           RefUcsConnectionString = $refUcsConString
           Ensure="Present"
           Identifier ="4"
           DeleteNotPresent=$true
     #Sync all the LDAP groups by specifying the DN and Hierarchy true
           Hierarchy=$true
           Dn="sys/ldap-ext" 
           
         }

In the above example I have specified the DN of the SP, SP Template, and the LDAP group. By specifying Ensure=”Present” the UcsSyncMoWithReference resource ensures that the MOs are created on the UCS domain. You can also specify what action to take in case if there are additional MOs than compared to the MOs present on the reference UCS. If you want to delete the additional MOs, you need to specify DeleteNotPresent= $true as done in the LDAP sync configuration in the above example.  Refer to UCS Manager PowerTool User Guide for more details on the properties of the Resource.

UcsManagedObject Custom Resource

This is a generic resource provided to configure any MO in UCS Manager. To use this resource, you need to be familiar with the MO definitions and properties. One way to make use of this resource is by generating this configuration automatically as explained in the earlier section.  If you are writing the configuration manually refer to the UCS Manager XML API Programmer’s Guide.  Below is an example configuration of creating an Org in the UCS Manager. There are few key things to consider while creating the configuration, you need to specify the DN, XML API Class ID, and the Property Map.

UcsManagedObject CreateOrganisationDemo
 {
    Ensure = "Present"
ModifyPresent = $true
ClassId= "orgOrg"
Dn = "org-root/org-DSCDemoOrg"
PropertyMap= "Descr = test for DSC with certificate `nName = DSCDemoOrg"
UcsCredentials = $ucsCredential
UcsConnectionString = $connectionString
Identifier = "2"
 }

You need to specify the properties of the managed object as key value pairs using the below format

`<key1>=<value1> `<key2>=<value2>

UcsScript Custom Resource

This is a generic resource provided to execute UCS Manager PowerTool cmdlets in a script. You can use this resource in cases where the configuration is complex and needs to be written as a script. You can also generate the configuration automatically for this resource as explained earlier.  Below is an example configuration of renaming a Service Profile.

UcsScript RenameServiceProfileDemo
        {
            Ensure = "Present"
            Dn = "org-root/ls-dscdemo"
            Script = "Get-UcsOrg -Level root | Get-UcsServiceProfile -Name 'TestSP' -LimitScope | Rename-UcsServiceProfile -NewName 'dscdemo' "
            UcsCredentials = $ucsCredential
            UcsConnectionString = $connectionString
            Identifier ="1"
        }

If the configuration script is complex you can specify multiple DNs in a comma separated format.

Configuration Example

This section details how you can put together all the things in a DSC configuration document.

For all the examples mentioned above you need to specify environment settings, UCS Connection details and Credentials.

UCS connection string needs to be specified in the following format.

Name=<ipAddress> [`nNoSsl=<bool>][`nPort=<ushort>] [`nProxyAddress=<proxyAddress>] [`nUseProxyDefaultCredentials=<bool>]

UCS Manager credentials needs to be specified as a PSCredential object. You can use certificates for encrypting the credentials to keep it secure. For information on using certificates for encryption refer to Microsoft DSC documentation.

Below is an example configuration which has all the details.

$ConfigData=    @{
        AllNodes = @(    
                        @{ 
                            # The name of the node we are describing
                            NodeName ="10.105.219.128"
                            # The path to the .cer file containing the
                            # public key of the Encryption Certificate
                            # used to encrypt credentials for this node
                             CertificateFile = "C:\Certificate\MyCertificate.cer"


                            # The thumbprint of the Encryption Certificate
                            # used to decrypt the credentials on target node
                            Thumbprint = "558CF40844CDC6303D25494FB007189F75BEE060"
                        };
                    );   
    } 

Configuration AutoGeneratedConfig
{
    param(
        [Parameter(Mandatory=$true)]
        [PsCredential] $ucsCredential,

        [Parameter(Mandatory=$true)]
        [string] $connectionString
        )
    
        Import-DSCResource -ModuleName Cisco.Ucs.DesiredStateConfiguration
        
        
        Node "10.105.219.128"
        {
         
            LocalConfigurationManager
            {
                CertificateId = $node.Thumbprint
                ConfigurationMode = 'ApplyOnly'
               RefreshMode = 'Push'
               
            } 

            UcsManagedObject UcsManagedObject1
            {
                Ensure = "Present"
                ModifyPresent = $true
                ClassId= "equipmentLocatorLed"
                Dn = "sys/chassis-1/blade-1/locator-led"
                PropertyMap= "Id = 1 `nBoardType = single `nAdminState = on"
                UcsCredentials = $ucsCredential
                UcsConnectionString = $connectionString
                Identifier = "1"
            }
           

           UcsManagedObject ucsManagedobject2
           {
               Ensure = "Present"
               ModifyPresent = $true
               ClassId= "orgOrg"
               Dn = "org-root/org-SubOrg2"
               PropertyMap= "Descr = test for DSC with certificate `nName = SubOrg2"
               UcsCredentials = $ucsCredential
               UcsConnectionString = $connectionString
               Identifier = "2"
           }

        }

 }

try
{
${Error}.Clear()
       $credential = Get-Credential

    AutoGeneratedConfig  -ConfigurationData $ConfigData `
                         -ucsCredential $credential `
                         -connectionString "Name=10.65.183.5" `
                         -OutputPath "C:\DscDemo\AutoGeneratedConfig"  
}
Catch
{
    Write-Host ${Error}
   
    exit
}

When you run this script PowerShell will generate the corresponding MOF files. You can deploy the configuration based on the LCM configuration mode. If it is set to Push mode, then you can enact the configuration using the below syntax.

Start-DscConfiguration -Path "C:\DscDemo\AutoGeneratedConfig\" -Wait -Verbose -force

When using any configuration management platform or a tool, the imperative resource modules, that “make it so”, are the most important. The declarative configuration documents combine the resource definitions and the imperative scripts get called behind the scenes to do the job. Now, it can be very tempting to write a resource for everything going by the resource granularity principle. But, this is where we need to apply some filters. There are many such filters but let’s first start with the configuration vs orchestration filter which is the subject of this article.

Take a look at this flowchart for an example.

The flowchart above is about deploying OS through automated processes in my lab. This is very high-level and abstracted. Notice the step before complete OS deploy. We need to set the bare metal system to perform one time PXE boot so that the WDS server kick starts the WinPE boot and completes required OS deployment steps. In a scenario where you have artifacts developed for even bare metal configuration, it is certainly possible to put this one time PXE boot also as a resource configuration. After all, it is just BIOS attribute configuration.

I have put together an example for this.

$configData = 
@{
    AllNodes = 
    @(
        @{
            NodeName = 'localhost'
            PSDscAllowPlainTextPassword = $true
        }
    )
}

Configuration PEPXEBootDemo
{
    Import-DscResource -ModuleName DellPEWsManTools -Name PEOneTimeBoot
    Import-DscResource -ModuleName PSDesiredStateConfiguration

    Node $AllNodes.NodeName {
        PEOneTimeBoot PEPXEBootDemo
        {
            Id = 'Node17-UniqueBoot'
            DRACIPAddress = '172.16.100.17'
            DRACCredential = (Get-Credential)
            OneTimeBootMode = 'Enabled'
            OneTimeBootDevice = 'NIC.PxeDevice.1-1'
        }
    }
}

PEPXEBootDemo -ConfigurationData $configData

The above configuration document uses the PEOneTimeBoot resource from the DellPEWSManTools resource module. The PEOneTimeBoot is a proxy DSC resource.

Here is what it does when we enact this configuration.

All is well and good. Once the bare metal system restarts, it boots into PXE and completes OS deployment. Now, here is the interesting part. If I use the Test-DscConfiguration cmdlet to check if my node is in desired state or not, here is what I see.

There is a configuration drift. And, it will always be like this. This is because the PEOneTimeBoot resource is configuring a BIOS attribute that is short lived. It gets reset or gets disabled after the configuration is complete. So, when we check the attribute after the configuration enact is complete, the Test-DscConfiguration will always find that this resource has drifted from the expected configuration. When we aggregate the resource desired state at the complete system level, this drift in PEOneTimeBoot will roll up as drift at the whole system level. And, this is what makes this, PEOneTimeBoot, an incorrect choice for creating a DSC resource.

In practice, this is an orchestration step and not a configuration item. As a part of the deployment process orchestration, we would need an orchestration step that PXE boots the bare metal system for OS deployment to complete and then perform any other post-OS configuration tasks using DSC.

Therefore when designing and developing a DSC resource module, apply this filter to check if the resource configuration in question is short lived. In other terms, check if the resource configuration implies to be an orchestration step or a configuration item.

Stay tuned for more in this series!

There are several custom DSC resource modules available either via the official PowerShell team’s DSC resource kit or via other community repositories. There are guidelines on contributing to the official resource kit. This includes a set of requirements for creating High Quality DSC Resource modules and the style guidelines that must be followed when creating these High Quality Resource Modules.

These guidelines only discuss how you should structure the DSC resource modules, how and what type of tests you should include, how you must document your module, and what coding style guidelines should be followed, and so on but not how you should design your resource module and what a resource within the module should represent. This is not just about logic in the resource module’s imperative scripts but what and how those imperative scripts configure. In other words, the granularity of resource configuration should be one of the design considerations when writing resource modules.

Let’s look at an example to understand this.

Here is the xVMHyperV resource in xHyper-V module. This resource has a bunch of properties and supports creation of a VM and some of its related components such as network adapters.

If I look at the properties listed here, I see a few issues with the way this resource is designed.

• This resource takes an array of switch names (SwitchName property) and then attaches a network adapter for each switch that is specified in the resource instance configuration. This takes care of adding new adapters in future through the same configuration. However, it fails when you want to remove an adapter. Even if you want to implement that logic in the same resource, it becomes complex.
• While there is support for multiple network adapters, there is no VLAN ID configuration available in the resource. There is no way we can configure other network adapter settings such as bandwidth settings, DHCP Guard, and other supported configuration settings.
• This module does a lot of heavy lifting when it comes to virtual hard drive and dynamic memory configurations. However, there is no VHDX Q0S configuration that is possible.

While this resource takes care of certain VM settings, it excludes many of the other VM settings. Adding all this support within the same xVMHyperV resource module will only increase its complexity. Instead, separating out the configuration into multiple smaller resource modules would be a good design.

One of the things that I consider when starting out with developing a new resource module is to first understand the resource itself. Let us look at the VM example again. Here is how I would represent the VM and its related components in a hierarchical manner.

Once I have this representation, I look at what items in the hierarchy are best suited to be resource on their own. For example, processor in the VM context need not a resource on its own. When we create a new VM, we generally assign number of virtual CPU and that is it. But, the processor settings such as Resource Control, NUMA Topology and so on can be packaged into a different resource so that the VM resource need not worry about having all the logic to manage these special configuration settings. The same thought process applies to VM memory as well. I want to be able to manage the dynamic memory settings only when I need and not package them into the VM resource configuration.

However, when it comes to a network adapter, I want to add or remove network adapters without really touching the VM resource configuration. And, configure other settings for these network adapters when I need them. So, I would create separate resources for network adapters and their associated settings.

Simply put, the more complex your resource becomes, the more complex your tests need to be. Without proper tests for a complex resource, you end up creating something that is substandard. Period.

The granular resource design gives me flexibility while reducing the complexity. You may argue that the configuration documents tend to become very long and difficult to write with so many granular resources just to create a single VM. Yes, if you are doing this only one time, it shouldn’t matter. But, if you plan to reuse these configurations, composite resources solve this exact problem. I combine long and complex configurations into composite resources and use them very often. These composite resources become my infrastructure blueprints.

Here is how my SimpleVM resource looks like.

Note: This is not yet in my public release of cHyper-V module. If you want to give this a try, just give a shout and I will be able to invite you to try a few more things along with this.

This SimpleVM resource provides functionality that is good enough to create a VM that has a VHD attached and the required memory and CPU configuration. You can choose to leave the default network adapter or remove it if you want to use the xVMNetworkAdapter to add one or more resources separately.

Using this method for resource design also helped me create resources that are used for different types of components. Consider my xNetAdapterRDMA resource for an example. This resource is used to enable or disable RDMA on a network adapter. This is a very simple resource. It just flips the RDMA configuration on a network adapter.

I have also written the xVMNetworkAdapter resource which adds/removes VM network adapters in the management OS or to the virtual machines. So, when I have to add a VM network adapter that is RDMA capable, I could have implemented the logic to make it so in the xVMNetworkAdapter itself. But, that design, while making the resource complex, would prevent me from using the same functionality with physical adapters. So, if I were to configure physical adapter RDMA settings, I would have ended up writing a different resource or do it some other way. Instead, I chose to separate the RDMA configuration into its own module so that I can share that functionality across different types of network adapters.

To summarize, when designing a resource module, you should look at how and what the resource would do and structure your module in a way that allows flexible resource configurations.

Stay tuned for more in this series.

In this article, we will look at how binary files can be interpreted in PowerShell by looking at a real world challenge as an example.

The challenge

In System Center Data Protection Manager (DPM), there is an agent installed on protected computers for managing backup and restore operations. If there is a lot of DPM servers in the environment and you do not know what DPM server is protecting an agent, you need a way to find this information on the local machine where the DPM agent is installed on. You might want to look at all the DPM servers, but the agent might be inactive and left over from a decommissioned DPM server.

There aren`t any official references on this topic besides some guidance in a forum thread on the TechNet Forums:

Open an administrative command prompt, then run:

     C:\> type “C:\Program Files\Microsoft Data Protection Manager\DPM\ActiveOwner*.

The beginning of the first line returned will be the FQDN in Unicode of the DPM Server owning the agent on the protected server/client

Let’s try:

This gives us the information we want, but not in a very convenient format. Optimally, we would like to gather this information remotely via PowerShell remoting and get an object back with information about the DPM agent. This could be a function containing version information in addition to the name of the DPM server(s) an agent is attached to.

In PowerShell we can use Get-Content (or its alias type) to get the same information:

At this point, I was thinking that regular expressions might be an appropriate way to solve this challenge. I presented the challenge to my colleague Øyvind, which had experience working with binary files like this in PowerShell.

Working with binary files

Usually you would want to have some kind of documentation of the file format in question before trying to parse a binary file format. Unfortunately, we couldn’t find any for this file type. However, it seems from reading the file contents raw, that the information we want is right at the beginning of the file format. If you look at the raw format representation of the string we want to extract, you see that each character has a space between them. This tells us that it’s a Unicode encoded string.

It’s useful to use a dedicated hex editor when working with binary file types. In the following screenshot, I’m using the 010 Editor, and as you can see the editor have confirmed our suspicion that the text is in Unicode format.

What we don’t know is length of this field, so we must do some guess work. Since this information is referring to a hostname or a domain name, we can assume it’s not going to contain any spaces. We also hope that there will be at least one space between this field and the next one. Building on these assumptions, we can create a do-while loop that keeps reading bytes until we encounter a byte that when converted to a Unicode string equals to an empty string. The resulting string should be the data that we are after.

Note that since we are reading Unicode-encoded strings, we need to read 2 bytes in each pass.

The way to read data from a binary file is to set up a BinaryReader object. This class has the ReadBytes method that we will use to read bytes from the binary stream.

We also need some way of converting the binary data to something meaningful. Since we already have identified the data as Unicode string, we can use the Unicode.GetString static method in the System.Text.Encoding class in .NET for this.

That’s all we really need for this particular case. You can find the full code at https://gist.github.com/janegilring/afec8213d3d14e4f436d0f9d88804f74

If you are interested in learning more about how to parse data from binary file formats, Øyvind did a talk about this at PowerShell Conference Europe 2017. The video recording of the talk is available here.

Within the Infrastructure as Code (IaC) practices, there is enough emphasis on the repeatable and reusable automation enabled using configuration management tools. This is referred to as configuration as Code. Configuration as Code enables consistent methods to configure IT systems. And, when we integrate these processes into DevOps practices, we can ensure that the configuration across different stages of the deployment (Development / Test / Staging / Production) can be done in a efficient and consistent manner.

One of the best practices in Configuration as Code practice is to ensure that the configurations that we deploy are made reusable. This means that the configurations are parameterized and have the environmental data separated from the structural configuration data.

PowerShell Desired State Configuration (DSC) supports the separation of environmental configuration from structural configuration using the configuration data artifacts. And, sharing of parameterized configurations is done using the composite configuration modules or what we call composite resources. Composite configurations are very useful when a node configuration requires a combination multiple resources and becomes long and complex.

For example, building a Hyper-V cluster includes configuring host networking, domain join, updating firewall rules, creating/joining a cluster, and so on. Each node in the cluster should have this configuration done in a consistent manner. Therefore, the configurations that are applied for each of these items can be made reusable using the composite configuration methods.

Also, a real-world deployment pipeline implemented using IaC practices should also have validation of the infrastructure configuration at various stages of the deployment. In the PowerShell world, this is done using Pester. Within DSC too, Pester plays a very important role in validating the desired state after a configuration is applied and in the operations validation space.

Infrastructure Blueprints

The infrastructure blueprints project provides guidelines on enabling reusable and repeatable DSC configurations combined with Pester validations that are identified by Operations Validation Framework. As a part of this repository, there will be a set of composite resource modules for various common configurations that you will see in a typical IT infrastructure.

Infrastructure Blueprints are essentially composite resource packages that contain node configurations and Pester tests that validate desired state and integration after the configuration is applied.

This repository contains multiple composite configuration modules. Each module contains multiple composite resources with ready to use examples and tests that validate the configuration.

The following folder structure shows how these composite modules are packaged.

• Diagnostics folder contains the Simple and Comprehensive tests for performing operations validation.
  • Simple: A set of tests that validate the functionality of infrastructure at the desired state.
  • Comprehensive: A set of tests that perform a comprehensive operations validation of the infrastructure at the desired state.
  • For ease of identification, the test script names include Simple or Comprehensive within the file name.

The Operations Validation Framework can be used to retrieve the list of tests in this module and invoke the relevant ones.

Once you know the composite resource that is applied on the system, you can invoke either simple or comprehensive tests using the Invoke-OperationValidation cmdlet.

• Examples folder contains a sample configuration data foreach composite configuration and also a configuration document that demonstrates how to use the composite resource.
• CompositeModuleName.psd1 is the module manifest for the composite configuration module.

This manifest contains the RequiredModules key that has all the required modules for the composite configuration to work. This is listed as a module specification object. For example, the RequiredModules key for Hyper-VConfigurations composite module contains the following hashtable.

# Modules that must be imported into the global environment prior to importing this module
RequiredModules = @(
    @{ModuleName='cHyper-v';ModuleVersion='3.0.0.0'},
    @{ModuleName='xNetworking';ModuleVersion='2.12.0.0'}
)

These composite modules are available in the PowerShell Gallery as well. And, therefore, having the RequiredModules in the module manifest enables automatic download of all module dependencies automatically.

PS C:\> Install-Module -Name Hyper-VConfigurations -Force -Verbose
VERBOSE: Using the provider 'PowerShellGet' for searching packages.
VERBOSE: The -Repository parameter was not specified.  PowerShellGet will use all of the registered repositories.
VERBOSE: Getting the provider object for the PackageManagement Provider 'NuGet'.
VERBOSE: The specified Location is 'https://www.powershellgallery.com/api/v2/' and PackageManagementProvider is 'NuGet'.
VERBOSE: Searching repository 'https://www.powershellgallery.com/api/v2/FindPackagesById()?id='Hyper-VConfigurations'' for ''.
VERBOSE: Total package yield:'1' for the specified package 'Hyper-VConfigurations'.
VERBOSE: Performing the operation "Install-Module" on target "Version '1.0.0.0' of module 'Hyper-VConfigurations'".
VERBOSE: The installation scope is specified to be 'AllUsers'.
VERBOSE: The specified module will be installed in 'C:\Program Files\WindowsPowerShell\Modules'.
VERBOSE: The specified Location is 'NuGet' and PackageManagementProvider is 'NuGet'.
VERBOSE: Downloading module 'Hyper-VConfigurations' with version '1.0.0.0' from the repository 'https://www.powershellgallery.com/api/v2/'.
VERBOSE: Searching repository 'https://www.powershellgallery.com/api/v2/FindPackagesById()?id='Hyper-VConfigurations'' for ''.
VERBOSE: Searching repository 'https://www.powershellgallery.com/api/v2/FindPackagesById()?id='cHyper-v'' for ''.
VERBOSE: Searching repository 'https://www.powershellgallery.com/api/v2/FindPackagesById()?id='xNetworking'' for ''.
VERBOSE: InstallPackage' - name='cHyper-V', version='3.0.0.0',destination='C:\Users\ravikanth_chaganti\AppData\Local\Temp\1037779645'
VERBOSE: DownloadPackage' - name='cHyper-V', version='3.0.0.0',destination='C:\Users\ravikanth_chaganti\AppData\Local\Temp\1037779645\cHyper-V\cHyper-V.nupkg', uri='https://www.powershe
llgallery.com/api/v2/package/cHyper-V/3.0.0'
VERBOSE: Downloading 'https://www.powershellgallery.com/api/v2/package/cHyper-V/3.0.0'.
VERBOSE: Completed downloading 'https://www.powershellgallery.com/api/v2/package/cHyper-V/3.0.0'.
VERBOSE: Completed downloading 'cHyper-V'.
VERBOSE: InstallPackageLocal' - name='cHyper-V', version='3.0.0.0',destination='C:\Users\ravikanth_chaganti\AppData\Local\Temp\1037779645'
VERBOSE: InstallPackage' - name='xNetworking', version='3.2.0.0',destination='C:\Users\ravikanth_chaganti\AppData\Local\Temp\1037779645'
VERBOSE: DownloadPackage' - name='xNetworking', version='3.2.0.0',destination='C:\Users\ravikanth_chaganti\AppData\Local\Temp\1037779645\xNetworking\xNetworking.nupkg', uri='https://www
.powershellgallery.com/api/v2/package/xNetworking/3.2.0'
VERBOSE: Downloading 'https://www.powershellgallery.com/api/v2/package/xNetworking/3.2.0'.
VERBOSE: Completed downloading 'https://www.powershellgallery.com/api/v2/package/xNetworking/3.2.0'.
VERBOSE: Completed downloading 'xNetworking'.
VERBOSE: InstallPackageLocal' - name='xNetworking', version='3.2.0.0',destination='C:\Users\ravikanth_chaganti\AppData\Local\Temp\1037779645'
VERBOSE: InstallPackage' - name='Hyper-VConfigurations', version='1.0.0.0',destination='C:\Users\ravikanth_chaganti\AppData\Local\Temp\1037779645'
VERBOSE: DownloadPackage' - name='Hyper-VConfigurations', version='1.0.0.0',destination='C:\Users\ravikanth_chaganti\AppData\Local\Temp\1037779645\Hyper-VConfigurations\Hyper-VConfigura
tions.nupkg', uri='https://www.powershellgallery.com/api/v2/package/Hyper-VConfigurations/1.0.0'
VERBOSE: Downloading 'https://www.powershellgallery.com/api/v2/package/Hyper-VConfigurations/1.0.0'.
VERBOSE: Completed downloading 'https://www.powershellgallery.com/api/v2/package/Hyper-VConfigurations/1.0.0'.
VERBOSE: Completed downloading 'Hyper-VConfigurations'.
VERBOSE: InstallPackageLocal' - name='Hyper-VConfigurations', version='1.0.0.0',destination='C:\Users\ravikanth_chaganti\AppData\Local\Temp\1037779645'
VERBOSE: Installing the dependency module 'cHyper-V' with version '3.0.0.0' for the module 'Hyper-VConfigurations'.
VERBOSE: Module 'cHyper-V' was installed successfully.
VERBOSE: Installing the dependency module 'xNetworking' with version '3.2.0.0' for the module 'Hyper-VConfigurations'.
VERBOSE: Module 'xNetworking' was installed successfully.
VERBOSE: Module 'Hyper-VConfigurations' was installed successfully.

As you can see in the above Install-Module cmdlet output, the required modules are downloaded from the gallery. Thanks to Chrissy for this tip.

You can contribute to this project by submitting a pull request. All you need a set of composite resource modules packaged as a PowerShell module with DSC resources. Of course, ensure you add clear examples and tests.

As part of my work in an engineering team, I am tasked with writing scripts which will validate the underlying infrastructure before the automation (using PowerShell DSC) kicks in to deploy the solution.

Below are the different generic phases which comprise the whole automation process:

• Pre-deployment – getting the base infrastructure ready, the bare minimum required for automation. For example – network configuration on the nodes is needed.
• Deployment – deployment of the solution leveraging PowerShell DSC.
• Post-deployment – scripts/runbooks configuring or tweaking the environment.

What I meant by validating underlying infrastructure above, is that the compute and storage physical hosts/nodes have a valid IP configuration, connectivity to the AD/DNS infrastructure etc. the key components that we required to be tested and validated to get confidence in our readiness to deploy the engineered solution on top of it.

Note – Our solution had scripts in place that would configure the network based on some input parameters and record this in a manifest XML file. After the script ran, we would assume that everything is in place. These assumptions at some points cost us a lot of efforts in troubleshooting.

In short, initial idea was to have scripts validating, what the scripts did in an earlier step. So it began, I started writing PowerShell functions, using workflows (to target parallel execution on nodes). This was a decent solution until there were requests to add validation tests for entirely everything in the solution stack e.g. DNS configuration, network connectivity, proxy configuration, disks (SSD/HDD) attached to the storage nodes etc.

Phew! It was a nightmare maintaining it.

Rays of hope: Pester, PoshSpec, and Remotely!

We went into looking at how to use some of the open source PowerShell modules into helping us perform operations validation. At this time in community, Pester was gaining traction for the operations validation.

Using Pester

We moved away from using standalone scripts for the operations validation and started converting our scripts into Pester tests. It is not surprising to see that many operations people find it easier to relate to using Pester for Ops validation, since we have been doing this validation for ages manually. Pester just makes it easy to automate all of it.

For example, in our solution each compute node gets three NIC cards, pre-deployment script configures them. If we had to test whether the network adapter’s configuration was indeed correct, it would look something like below using Pester:

Describe "TestIPConfiguration" {
    It "Should have a valid IP address on the Management NIC" {
        (Get-NetIPAddress -AddressFamily IPv4 -InterfaceAlias 'vEthernet(Management)' | Select-Object -ExpandProperty IPAddress) |
            Should be '10.10.10.1' 
    }

    It "Should have a valid IP address on the Storage1 NIC" {
       (Get-NetIPAddress -AddressFamily IPv4 -InterfaceAlias 'vEthernet(Storage1)' | Select-Object -ExpandProperty IPAddress) |
            Should be '10.20.10.1'  
    }

    It "Should have a valid IP address on the Storage2 NIC" {
        (Get-NetIPAddress -AddressFamily IPv4 -InterfaceAlias 'vEthernet(Storage1)' | Select-Object -ExpandProperty IPAddress) |
            Should be '10.30.10.1'  
}

Using PoshSpec & Pester

PoshSpec added yet another layer of abstraction on our infrastructure tests by adding yet another DSL.
Below is how our tests started looking with usage of Pester and PoshSpec.

Note – For validation of IPv4 Address, another keyword named IPv4Address was added to PoshSpec which would essentially call Get-NetIPAddress and spit out the IPv4 address assigned on the NIC interface with specified alias.

Describe "TestIPConfiguration" {
    Context "Validate the Management NIC " {
        IPv4Address 'vEthernet(Management)' {Should be '10.10.10.1'} 
    }

   Context "Validate the Storage1 NIC" {
        IPv4Address 'vEthernet(Storage1)' {Should be '10.20.10.1'} 
    }

    Context "Validate the Storage2 NIC" {
        IPv4Address 'vEthernet(Storage2)' {Should be '10.30.10.1'}
    }
}

By using Pester and PoshSpec to write tests, it sure made maintaining these tests easy, but we still have a problem at hand.  How do we target our above tests to all the nodes in the solution?

Remotely??

At some point Ravi was tinkering with this particular PowerShell module and suggested to take a look at it. It was promising to begin with as he added support for passing Credential hash to Remotely. We would have to specify a hash table with the computer name as key and credential as value to Remotely and it would take care of connecting to those nodes, executing the script block in the remote runspace. At this point things started falling in place for what we had in mind. Our tests started looking nice and concise:

$CredHash = @{
    'ComputeNode1' = Get-Credential
    'ComputeNode2' = Get-Credential
}
Describe "TestIPConfiguration" {
    Context "Validate the Management NIC " {
       Remotely ComputeNode1, ComputeNode2 {IPv4Address 'vEthernet(Management)' {Should be '10.10.10.1'}} 
    }

   Context "Validate the Storage1 NIC" {
        Remotely ComputeNode1, ComputeNode2 {IPv4Address 'vEthernet(Storage1)' {Should be '10.20.10.1'}} 
    }

    Context "Validate the Storage2 NIC" {
        Remotely ComputeNode1, ComputeNode2 {IPv4Address 'vEthernet(Storage2)' {Should be '10.30.10.1'}} 
    }
}

Soon we realized that the Assertions above e.g. {Should Be ’10.10.10.1’} are to be dynamically created by reading the manifest XML file which drives the whole deployment. It contains what is the expected configuration on the remote nodes.

We wanted our tests to be generic so that we could target them to all nodes part of the solution. We were looking to have our tests organized like below, where of course a node-specific details e.g. $ManagementIPv4Address etc. would be read from the manifest file and created on the fly either on the local machine or remote node :

$CredHash = @{
    'ComputeNode1' = Get-Credential
    'ComputeNode2' = Get-Credential
}
Describe "TestIPConfiguration" {
    Context "Validate the Management NIC " {
       Remotely ComputeNode1, ComputeNode2 {IPv4Address 'vEthernet(Management)' {Should be  $ManagementIPv4Address}} 
    }

   Context "Validate the Storage1 NIC" {
        Remotely ComputeNode1, ComputeNode2 {IPv4Address 'vEthernet(Storage1)' {Should be $Storage1IPv4Address}} 
    }

    Context "Validate the  Storage2 NIC" {
        Remotely ComputeNode1, ComputeNode2 {IPv4Address 'vEthernet(Storage2)' {Should be $Storage2IPv4Address}}  
    }
}

The above syntax looks quite descriptive and decouples the validation tests and environment details too.

But there were some downsides to the above approach.

• Requires us re-writing our existing tests to accommodate keyword Remotely for executing script block on remote and running assertions locally.
• Remotely connects each time to all the nodes to run each PoshSpec based ops validation tests. Results in lot of overhead to run a large number of validation tests.
• Trouble passing environment specific data to the remote nodes e.g. in the above tests passing the expected IPv4 address to the remote node.
• For running Pester/PoshSpec tests on the remote nodes, these modules need to be present on the remote node, to begin with.

The existing Remotely framework was meant to execute script block against a remote runspace but it was not specifically built to perform operations validation remotely.

Enter PSRemotely

After trying to integrate Remotely with Pester/PoshSpec based tests, we had a general idea on what we needed from a framework/DSL, if it was to provide us with the capability of orchestrating operations validation remotely on the nodes. Below are some of those features we had in mind along with the arguments for these to be implemented:

• Target Pester/PoshSpec based operations validation tests on the remote nodes.
• Allow specifying environment data separately from the tests, so that same tests could be applied across on nodes.
  We decided on the ability to use DSC Style configuration data here for specifying node specific environment details.
• Easier debugging on the remote nodes, in case tests fail.
  If something failed on the remote node during validation, we should be able to connect to the underlying PowerShell remoting session and debug issues.
• Allow re-running specific tests on the remote nodes.
  In case a test failed, performing a quick remediation action and validating that specific test passed it is a good to have feature when you have lot of tests in your suite.
• Self-contained solution.
  Have the framework bootstrap the remote nodes with required modules version (Pester & PoshSpec) under the hood. Remote nodes might not have internet connectivity here.
• Allow copying required artifacts to remote nodes.
  For our solution, we require a manifest file with details about the deployment to be copied on each node.
• Use PowerShell remoting as underlying transport mechanism for everything.
• Return bare minimum JSON output, if everything passes. If a test fails then return the error record thrown by Pester.

And, PSRemotely was born!

So after a lot of discussions with Ravi, I finally got insight on how the remote operations validation DSL should look like:

$CredentialHash = @{
    'ComputeNode1' = Get-Credential
    #'ComputeNode2' = Import-CliXML # If a node is missed here, current user creds are used with PSRemoting
}

# Configuration data, can be generated separately or specified from a .psd1 or .json file
$ConfigData = @{
    AllNodes = @(
        @{
            NodeName = '*' 
            DomainFQDN = 'dexter.lab'
        },
        @{
            NodeName = "ComputeNode1"
            ManagementIPv4Address = '10.10.10.1'
            Storage1IPv4Address = '10.20.10.1'
            Storage2IPv4Address = '10.30.10.1'
            Type = 'Compute'
        },
        @{
            NodeName = "ComputeNode2"
            ManagementIPv4Address = '10.10.10.2'
            Storage1IPv4Address = '10.20.10.2'
            Storage2IPv4Address = '10.30.10.2'
            Type = 'Compute'
        }
    )
}

# PSRemotely tests, specify CredentialHash and Configuration data to the PSRemotely
PSRemotely -ConfigurationData $ConfigData -CredentialHash $CredentialHash {
    Node $AllNodes.Where({$_.Type -eq 'Compute'}).NodeName {
        
        # Below are the existing Pester/PoshSpec-based tests, with changes on how node specific data is supplied
        Describe 'TestIPConfiguration' -Tag IP {
            
            Context "Validate the Management NIC" {
                    # pre-deployment script always creates NICs with these names
                    IPv4Address 'vEthernet(Management)' {Should be  $Node.ManagementIPv4Address}
                }

            Context "Validate the Storage1 NIC" {
                    IPv4Address 'vEthernet(Storage1)' {Should be $Node.Storage1IPv4Address} 
                }

            Context "Validate the Storage2 NIC" {
                IPv4Address 'vEthernet(Storage2)' {Should be $Node.Storage2IPv4Address}
            }
        }       
    }
}

After having a clear idea on the features required in the framework and how we wanted the DSL to look like, I started working on it. This post has set up the context on why we began working on something entirely new from scratch.

Join me in the second post where I try to explain how to use PSRemotely to target remote nodes for operations validation.

While you find plenty of PowerShell beginners classes to teach fresh admins the fundamentals, it’s much harder to find places for experienced admins to learn relevant new information and be on equal height with speakers and other delegates. psconf.eu is such a place and lightened up last year as a completely new PowerShell event format.

“Can’t wait for another #psconfeu!”, “You don’t want to miss this”, “best conference”, “highlight of the year” is just some feedback when you search Twitter for #psconfeu. Let’s take a look at what makes psconf.eu so special, and what to expect this year!

This year, the conference has an unprecedented speaker lineup: five of the six powershellmagazine.com editors will be there and speaking. PowerShell inventor Jeffrey Snover opens the conference with his keynote “State of the Union”. Microsoft sends six PowerShell team members covering PowerShell 6, Open Source, DSC, PowerShell for VSCode, and more. So you’re not just getting answers – you are talking directly to the people actually doing these things. And get first hand information, including where the PowerShell journey will go next.

psconf.eu covers all major areas of PowerShell, and the agenda information below uses color-coding to help you better find your favorite topics:

Special Trainings Ops

Right before the main conference starts, you have the option to participate in our optional preconf special trainings day. These are definitely no MOC-style courses. Workshops aim to bring experienced people up to speed. You can polish fundamentals, dive into topics that are new to you (like DSC, GUIs, or JEA Security), and warm up. These “Special Trainings” make sure you get the most out of the main conference sessions, and prepare yourself for topics that may be new to you – like DSC, or JEA, or building GUI tools. Workshops are available in English and in German, and you can suggest more topics by contacting us via the contact form at the bottom of psconf.eu.

Opening Day

On day 1, we open the conference with delegate registration and then start together in the great historic Leibniz Saal. PowerShell inventor Jeffrey Snover delivers the keynote “State of the Union”, summarizing where PowerShell stands today and where it is heading in the light of being open source now and available on Linux and OS X.

We’ll then warm up with “Quiz and Quirks”: I extracted some of the funniest and strangest questions asked in our internal MVP forum, and you get the chance to test your knowledge and learn new things: would you have been able to answer the MVP questions easily? We are all in this conference together, and this session encourages interaction, asking questions, and being part of it.

Will “Harmj0y” Schroeder finally ends the morning with an awesome security presentation. His presentation last year was crammed, so we wanted him to deliver it in the largest room we have: listen closely where attackers can sneak into your systems, and how PowerShell can be used both as a forensic and penetration testing tool.

We all then have a good lunch together, and move into the workshop area. After lunch, we fan out into four tracks, giving you the choice. Track 1 will be in German, and tracks 2, 3 and 4 in English.

In the afternoon, you have the chance to meet many of the renowned speakers in person, and make connections. After the first round of presentations, we all come back to Leibniz Saal for the “Ask the Speakers Roundtable”.

Then, we walk over to the Hannover Zoo to enjoy the Evening Event.

Evening Event in the Zoo

The evening event is absolutely unique: we have “Yukon Bay”, an ancient gold digger town, all to us. We’ll have great food and drinks, beer and wine, and the chance to hang loose and make new connections and friendships. Everyone will be there, including all speakers. You may want to continue to talk about PowerShell, but you may just as well just kick back and enjoy the evening.

Conference Day 2 + 3

On the remaining two days, we fan out in 5 parallel tracks. Track 1 stays German, and the other four tracks deliver English sessions. 19 Microsoft MVPs, six PowerShell team members, former MVPs, Engineers from Microsoft Germany, and many other awesome PowerShell experts deliver presentations covering pretty much all areas that are relevant to PowerShell.

Here’s the preliminary agenda for day 2:

To give delegates the chance to even better connect to speakers, we’ll have designated “Ask the Experts” areas during lunch so you can socialize with the people from your favorite field of interest.

Net length for all presentations is 45 minutes sharp. We want presentations to be concise and on the point, and prefer demos over endless slides. At the end of each slot, you have 15 minutes of Q&A. If you are hungry for more, you can ask the speaker to extend: we have breakout-session rooms available where you have all the time you need for extensive Q&A, and where you can hold spontaneous whiteboard sessions or meet with user groups.

The conference ends on May 5 at 16:30h (4:30 pm). We provide a baggage room to leave your stuff. Here is the preliminary agenda for day 3:

Of course, we are working on conference memorabilia to take home – these are secret until the conference starts. Except for our famous “PowerShell container”: you’ll get one of these limited collectors items of course.

Register Now – Seats are Limited

We want this conference to stay personal. So even though we decided to further increase number of speakers and tracks, there’s room for a maximum of roughly 250 delegates. Half of these seats were taken before we published the agenda, and last year’s event sold out. We’d love to see you join this fun PowerShell event!

When you register on www.psconf.eu, your seat is immediately reserved for 30 days, and you receive an invoice. Pay it within 30 days, and your seat is secured.

If you have any additional questions, please use the contact form at the bottom of www.psconf.eu to get in touch. Looking forward to seeing you soon!

Tobias