Felipe Binotto's Blog

Building a hub between your lab and your internet devices using Azure

2021-11-09T15:44:00.001+10:00

I decided to make this post for a couple reasons. The first reason is to demonstrate how you can quickly build a hub between your own lab and your internet devices using Azure and how easy it is. No more port forwaring in your router, public IP addresses in your VMs, everything will route through the Azure gateway and you will get an any-to-any type of connectivity. The second reason is to demonstrate some important concepts such as:

Site-to-Site VPN
Point-to-Site VPN
BGP
Azure AD authentication for VPN

Note that everything I will demonstrate here can also be done using Azure vWAN. However, this is cheaper and fit for lab and demonstration purposes.

The following is the architecture overview of what we are trying to achieve.

The following are the prerequisites which I will not cover in this post and you should already have in place before you start:

Azure subscription
On-premises Windows Server 2016 or higher VM with 2 network cards and internet access (the 2 network cards are only required if you want to route traffic to different subnets otherwise 1 network card should do)
Enable Azure AD authentication on the VPN gateway. Instructions are documented HERE. This will create a new Enterprise Application in your Azure tenant

The following are the high-level steps on what we will do and the order we will do it:

Install required modules
Connect to Azure
Create a Resource Group
Create VNet and subnets
Create a Local Network Gateway
Create a Virtual Network Gateway
Create a Connection
Set up the on-premises VM as a router
Create a Site-to-Site interface
Set up BGP Router
Configure BGP Peering
Start the VPN connection
Download the P2S VPN configuration from Azure
Create a VM for testing
Demonstrate any-to-any connectivity

Now we will start to take a look on how you can fully automate that deployment. The full script can be downloaded from HERE but I will break it down in this post so you understand what is happening. You must run this script from your on-premises VM if you don't want to make any modifications. However, once you understand it, you should be able to split the commands and play around.

Note: in the scripts I defined the Subscription Name and Tenand Id as parameters. Here I will use them as variables.

Note: disable Internet Enhanced Security Configuration (IEESC) for the administrator or you will have issues when authenticating to Azure. You can do that from Server Manager or using the following script. Restart your PowerShell session after running it.

function Disable-IEESC {
    $AdminKey = "HKLM:\SOFTWARE\Microsoft\Active Setup\ `
    Installed Components\{A509B1A7-37EF-4b3f-8CFC-4F3A74704073}"
    Set-ItemProperty -Path $AdminKey -Name "IsInstalled" -Value 0
    Stop-Process -Name Explorer
}

OK, let's get started. Let's start with the basics. The following lines of code will:

Install the required modules
Set some variables which I will explain when we are looking at the commands which use them. You are welcome to change their values as long as you do what your doing. You must provide values for $subName and $tenantId
Prompt you for credentials to connect to your Azure subscription

# Install required modules
Install-PackageProvider -Name NuGet -Force
Install-Module Az.Resources,Az.Compute,Az.Network -Force -Confirm:$false

# Set variables
$subName                = ''
$tenantId               = ''
$rgName                 = 'Connectivity'
$location               = 'AustraliaEast'
$vnetName               = 'Global-Connect'
$subnetName             = 'IaaS'
$subnetPrefix           = '172.16.0.0/24'
$gatewaySubnetPrefix    = '172.16.255.0/27'
$myPIP                  = (Invoke-WebRequest myexternalip.com/raw).content
$vnetAddressRange       = '172.16.0.0/16'
$vpnAddressPool         = '172.17.0.0/24'
$sharedKey              = 'mySuperS3cr3t123'
$vmPassword             = 'myC0mplexP@ssword'
$localASN               = 64512
$remoteASN              = 65515

# Connect to Azure subscription
Connect-AzAccount -Subscription $subName

Next we will start creating the foundation resources in this order:

Create Resource Group

Define Azure subnets

Create VNet

# Create Resource Group
$rg = @{
    Name = $rgName
    Location = $location
}
New-AzResourceGroup @rg

# Define subnets
$subnet1 = New-AzVirtualNetworkSubnetConfig -Name 'GatewaySubnet' `
-AddressPrefix $gatewaySubnetPrefix
$subnet2 = New-AzVirtualNetworkSubnetConfig -Name $subnetName `
-AddressPrefix $subnetPrefix

# Create Virtual Network
$vnet = New-AzVirtualNetwork -Name $vnetName -ResourceGroupName $rgName `
-Location $location -AddressPrefix $vnetAddressRange -Subnet $subnet1, $subnet2

So far so good, nothing special.

Now we are going to create the Local Network Gateway. This resource represents your on-premises router configuration. Note how we are not specifying our on-premises subnets. This is because they will be known to Azure via BGP peer route exchange.

# Create Local Network Gateway
$localIP = (Get-NetIPAddress | Where-Object {$_.InterfaceAlias -eq 'Ethernet' `
-and $_.AddressFamily -eq 'ipv4'}).IpAddress
$localGW = New-AzLocalNetworkGateway -Name HomeLab -ResourceGroupName $rgName `
-Location $location -GatewayIpAddress $myPIP -Asn $localASN `
-BgpPeeringAddress $localIP

Let's break down the important parameters being used in this command:

GatewayIpAddress: this is your on-premises public IP address. Notice that when we define the variables in the beginning we are dynamically retrieving it using a HTTP request

Asn: ASN stands for Autonomous System Number. This is key for BGP routing. ASNs are assigned to organisations but if you don't have one like I don't, you can use one from the private range 64512 - 65534

BgpPeeringAddress: this is your local IP address used for BGP. BGP peers exchange routing information between them via BGP sessions. We will use the IP address of your first Ethernet device. You can also use APIPA addresses. By the way, BGP stands for Border Gateway Protocol

Next we create the Virtual Network Gateway. This is the Router representation in the Azure side.

# Create Virtual Network Gateway
$gwpip = New-AzPublicIpAddress -Name "$vnetName-GWIP" -ResourceGroupName $rgName `
-Location $location -AllocationMethod Dynamic
$vnet = Get-AzVirtualNetwork -Name $vnetName -ResourceGroupName $rgName
$subnet = Get-AzVirtualNetworkSubnetConfig -Name 'GatewaySubnet' -VirtualNetwork $vnet
$gwipconfig = New-AzVirtualNetworkGatewayIpConfig -Name gwipconfig1 -SubnetId $subnet.Id `
-PublicIpAddressId $gwpip.Id
$vnetGW = New-AzVirtualNetworkGateway -Name "$vnetName-GW" -ResourceGroupName $rgName `
-Location $location -IpConfigurations $gwipconfig -GatewayType Vpn -EnableBGP $true `
-VpnType RouteBased -GatewaySku VpnGw1 -Asn $remoteASN `
-VpnClientProtocol OpenVPN -VpnClientAddressPool $vpnAddressPool `
-AadTenantUri "https://login.microsoftonline.com/$tenantId" `
-AadIssuerUri "https://sts.windows.net/$tenantId/" `
-AadAudienceId '41b23e61-6c1e-4545-b367-cd054e0ed4b4'

Let's focus on the creation of the Virtual Network Gateway because there is where the important bits are. The following is the break down of the important parameters being used in this command:

GatewayType: here we are specifying this will be a Site-to-Site VPN gateway as opposed to an Express Route gateway

EnableBGP: here we specify we will be using BGP

VpnType: RouteBased as PolicyBased doesn't support P2S or BGP

GatewaySku: you cannot use the Basic SKU as it doesn't support the OpenVPN or IKEv2 protocols for Azure AD authentication as well as BGP

Asn: this is the ASN in the Azure side

VpnClientProtocol: we are going to use OpenVPN as it uses TLS and can penetrate firewalls, and it is the protocol that supports most Operating Systems such as Windows, Linux, MacOS, iOS and Android

VpnClientAddressPool: this is the pool of addresses that will be used by the VPN clients

AadTenantUri: this is the URI which contains your tenant details

Next, let's create a connection between our on-prem router and the Azure VPN gateway. Note that at this point the connection won't be established as we haven't yet configured the on-prem router.

# Create Virtual Network Gateway Connection
New-AzVirtualNetworkGatewayConnection -Name "$($vnetGW.Name)-$($localGW.Name)" `
-ResourceGroupName $rgName -Location $location -VirtualNetworkGateway1 $vnetGW `
-LocalNetworkGateway2 $localGW -ConnectionType IPsec -RoutingWeight 10 `
-SharedKey $sharedKey -EnableBgp $true

Everything above is pretty self explanatory, just worth mentioning that we are enabling BGP in the connection.

Now it is time to configure the local server. We first install the required Windows Features and then install site-site VPN and BGP Routing. The script sleeps for 3 seconds to allow the service to start before we run the next command.

# Install the required Windows Features and Install Remote Access
Install-WindowsFeature -Name RemoteAccess, Routing, RSAT, RSAT-Role-Tools, `
RSAT-RemoteAccess, RSAT-RemoteAccess-PowerShell
Install-RemoteAccess -VpnType VpnS2S
Start-Sleep 3

Once we have those prerequisites in place, we can create the S2S connection from the on-premises side. Remember we have already created one in Azure and it is waiting for a connection from the other side.

# Create S2S Connection
$gwpip = Get-AzPublicIpAddress -Name "$vnetName-GWIP" -ResourceGroupName $rgName
Add-VpnS2SInterface -Name 'Azure' $gwpip.IpAddress -Protocol IKEv2 `
-AuthenticationMethod PSKOnly -SharedSecret $sharedKey `
-IPv4Subnet @("$($vnetAddressRange):100" ,"$($vpnAddressPool):100")

Let's take a look in the important parameters from the command above:

$gwpip.IpAddress: this is the public IP address of the Azure VPN Gateway

IPv4Subnet: these are the Azure subnets which will be routed through that interface

Now it is time for the on-prem BGP configuration. We first create a BPG Router followed by a BGP Peer.

# Create BGP Router and Peer
Add-BgpRouter -BgpIdentifier $localIP -LocalASN $localASN
Add-BgpPeer -Name AzureS2S -LocalIPAddress $localIP `
-PeerIPAddress $($vnetGW.BgpSettings.BgpPeeringAddress) `
-LocalASN $localASN -PeerASN $remoteASN -PeeringMode Automatic

Let's take a look in the important parameters from the command above:

BgpIdentifier: this is the local on-prem IP, the same we have configured as the BGP Peer in the Azure Local Network Gateway

LocalASN: this is the on-prem ASN, the same we have configured as the BGP Peer in the Azure Local Network Gateway

LocalIPAddress: this is the local on-prem IP, the same we have configured as the BGP Peer in the Azure Local Network Gateway

PeerIPAddress: this is the Azure BGP Peering address which we retrieve from the Virtual Network Gateway

RemoteASN: this is the Azure ASN that we set when creating the Virtual Network Gateway

And finally, we can establish the connection.

Connect-VpnS2SInterface -Name 'Azure'

You can run the following commands to check everything is working:

Get-VpnS2SInterface

The output will be similar to the following:

Get-BgpPeer

The output will be similar to the following:

Now let's deploy an Azure VM so we can test connectivity between your router and the Azure VM.

# Deploy Azure VM
$vmAdminUser = "LocalAdminUser"
$vmSecurePassword = ConvertTo-SecureString $vmPassword -AsPlainText -Force
$nic = New-AzNetworkInterface -Name 'VM01-nic' -ResourceGroupName $rgName `
-Location $location -SubnetId $vnet.Subnets[1].Id
$credential = New-Object System.Management.Automation.PSCredential `
($vmAdminUser, $vmSecurePassword)
$vm = New-AzVMConfig -VMName 'VM01' -VMSize 'Standard_DS1_v2'
$vm = Set-AzVMOperatingSystem -VM $vm -Windows -ComputerName 'VM01' `
-Credential $credential -ProvisionVMAgent -EnableAutoUpdate
$vm = Add-AzVMNetworkInterface -VM $vm -Id $nic.Id
$vm = Set-AzVMSourceImage -VM $vm -PublisherName 'MicrosoftWindowsServer' `
-Offer 'WindowsServer' -Skus '2016-Datacenter' -Version latest

From the router VM you should be able to RDP to the Azure VM and vice-versa.

From the router VM:

From the Azure VM (make sure RDP is enabled in your router VM):

Cool, S2S is working. Now we need to download and configure the Azure VPN client to test P2S using Azure Authentication. First let's download the configuration file using our current authenticated session in the server.

# Downloads VPN Client Config
$vpnClient = New-AzVpnClientConfiguration -ResourceGroupName Connectivity `
-ResourceName $vnetGW.Name
Invoke-WebRequest -Uri $vpnClient.VpnProfileSASUrl -OutFile .\vpnclientconfig.zip

Note that a ZIP file has been download to the current directory.

Next you need to download the Azure VPN client from HERE. Install it in a desktop, laptop or any device which is not connected to the router you have just configured. Copy and extract the ZIP file to this device.

We can now configure the VPN Client as follows:

And finally, you should be able to connect using your Azure AD credentials (Conditional Access and MFA will apply if applicable).
You can also see you got an IP from the pool we have configured before and you got the default routes.

Now from your internet VPN client device you should be able to RDP to both the Router and the Azure VM. But what if you want to route to other devices on-premises which are in different subnets?

No problem. We just need to advertise the new routes and the BGP Router will let Azure know about them. In this example, I have a second network card (Ethernet 2) which routes traffic to the 10.0.2.0/24 subnet. So we can advertise the route with the following command.

# Add custom route
Add-BgpCustomRoute -Interface "Ethernet 2" -Network 10.0.2.0/24

Once you reconnect the VPN, you will notice you have new routes as per below.

You have the 10.0.2.0/24 route and you also get the gateway (10.0.2.45/32) and broadcast (10.0.2.255/32) addresses.

From this point you can RDP from any-to-any as long as you have RDP enabled.

I hope this was useful for you and that now you have a better understanding on how to use VPN S2S with BGP and P2S with Azure AD authentication.

Log Analytics - Cross-workspace queries

2020-09-17T17:07:00.004+10:00

Log Analytics allows you to query up to 10 workspaces simultaneously as long as you have at least read permission on them. This is very handy in scenarios when you want to get the results from multiple workspaces at the same time, or when you want to correlate data living in different workspaces or in general when you are managing multiple workspaces even if they hosted in different subscriptions.

The way you do this from the Log Analytics workspace is using a query similar to the one below.

union OfficeActivity, workspace('secondworkspace').OfficeActivity, workspace('thirdworkspace').OfficeActivity, workspace('ninethworkspace').OfficeActivity | where TimeGenerated > ago(1d)

The problem with the query above is that you need to retype the workspace names every time and you have to specify the table name as well.

One possible solution would be to save the query to reuse it in the future but this is not the ideal solution considering the table is hardcoded in the query and most likely you will want to query different tables from time to time. Furthermore, if you add new workspaces, you will need to update the query.

Due to this existing limitation I have come up with the following PowerShell script which will retrieve all workspaces across all subscriptions you have access to and it will execute the query against the table you specify at runtime.

param (
    [Parameter(Mandatory)]$table,
    [Parameter(Mandatory)]$inputQuery
    )

$subscriptions = Get-AzSubscription
$workspaces = @()
$buildQuery = $null
$workingWorkspaceId = "Specify here the workspace id of the workspace you usually work from."

foreach ($subscription in $subscriptions) {

    Select-AzSubscription $subscription
    $workspace = Get-AzOperationalInsightsWorkspace | Select-Object -ExpandProperty Name
    $workspaces += $workspace
    $workspaces = $workspaces -notmatch "specify here the name of the workspace you usually work from so that it is not queried twice"

}

foreach ($workspace in $workspaces) {

    $buildQuery += ",workspace(`"$($workspace)`").$table"

}

$query = @"
union $table$buildQuery$inputQuery
"@

$output = (Invoke-AzOperationalInsightsQuery -WorkspaceId $workingWorkspaceId -Query $query).Results

$output

The script is also available from my Github.

To execute the script simply provide the table name and query you would like to use. For example:

.\Invoke-CrossQuery.ps1 -table "OfficeActivity" -inputQuery "| where Time Generated > ago(1d)"

Azure Bicep

2020-09-11T23:12:00.006+10:00

Azure Bicep is an open source project for deploying Azure resources declaratively. It is an Azure Resource Manager (ARM) Domain Specific Language (DSL). Its main purpose is to simplify the authoring of ARM templates with a cleaner syntax and be easier to understand.

The way you compile Bicep code is by using the Bicep CLI. A Bicep VS Code extension is also available to author the Bicep code. A Bicep file has a .bicep extension.

You can install Bicep on Windows by running the following PowerShell script.

$installPath = "$env:USERPROFILE\.bicep"

$installDir = New-Item -ItemType Directory -Path $installPath -Force

$installDir.Attributes += 'Hidden'

(New-Object Net.WebClient).DownloadFile("https://github.com/Azure/bicep/releases/latest/download/bicep-win-x64.exe", "$installPath\bicep.exe")

$currentPath = (Get-Item -path "HKCU:\Environment" ).GetValue('Path', '', 'DoNotExpandEnvironmentNames')

if (-not $currentPath.Contains("%USERPROFILE%\.bicep")) { setx PATH ($currentPath + ";%USERPROFILE%\.bicep") }

if (-not $env:path.Contains($installPath)) { $env:path += ";$installPath" }

bicep --help

And you can install the VS Code extension by running the following PowerShell script.

$vsixPath = "$env:TEMP\vscode-bicep.vsix"

(New-Object Net.WebClient).DownloadFile("https://github.com/Azure/bicep/releases/latest/download/vscode-bicep.vsix", $vsixPath)

code --install-extension $vsixPath

Remove-Item $vsixPath

For Mac or Linux visit https://github.com/Azure/bicep/blob/master/docs/installing.md

OK now that you have it installed let's dig into some Bicep code. The example below generates an ARM template to create a storage account with a unique name, provides the option for LRS or GRS, creates a container and output some values. This is a more complex example so that I can demonstrate how it matches the ARM template parameters, variables and functions.

param location string = resourceGroup().location

param namePrefix string = 'sa'

param globalRedundancy bool = true

var storageAccountName = '${namePrefix}${uniqueString(resourceGroup().id)}'

var storageSku = 'Standard_LRS'

resource storageAccount 'Microsoft.Storage/storageAccounts@2019-06-01' = {

location: location

kind: 'Storage'

sku: {

}

resource blob 'Microsoft.Storage/storageAccounts/blobServices/containers@2019-06-01' = {

}

output storageId string = storageAccount.id

output computedStorageName string = storageAccount.name

output primaryEndpoint string = storageAccount.properties.primaryEndpoints.blob

OK so let's break it down. Note that I won’t cover every single line, only the ones which need to be understood.

First we define a parameter and assign the resource group's location value.

param location string = resourceGroup().location

In JSON this would be equivalent to:

"parameters": {

"location": {

"type": "string",

"defaultValue": "[resourceGroup().location]"

}

Then we assign a value to the storageAccountName variable based on the value of the namePrefix variable concatenated with a unique string based on the resource group id. Note the use of interpolation ${}${} which is much simpler than using concat or format in JSON.

var storageAccountName = '${namePrefix}${uniqueString(resourceGroup().id)}'

In JSON this would be equivalent to:

"variables": {

"storageAccountName": "[format('{0}{1}', parameters('namePrefix'), uniqueString(resourceGroup().id))]"

}

Then we declare the storage account. Note that we give it a name of storageAccount. This is not the storage account name, this is a name which we can reference the object later. The storage account name is specified inside the {}. Then we specify the 'Resource Provider\Resource Type@API version'. Also note the use of the ternary operator ? to specify a condition which is equivalent to if true then Standard_GRS else Standard_LRS.

resource storageAccount 'Microsoft.Storage/storageAccounts@2019-06-01' = {

location: location

kind: 'Storage'

sku: {

}

In JSON this would be equivalent to:

"resources": [

{

"type": "Microsoft.Storage/storageAccounts",

"apiVersion": "2019-06-01",

"name": "[variables('storageAccountName')]",

"location": "[parameters('location')]",

"kind": "Storage",

"sku": {

"name": "[if(parameters('globalRedundancy'), 'Standard_GRS', 'Standard_LRS')]"

}

Then we declare a new container which when compiled will automatically add the dependsOn property because we are referencing a property of the resource via the symbolic name. On ARM templates you must manually declare the dependencies.

resource blob 'Microsoft.Storage/storageAccounts/blobServices/containers@2019-06-01' = {

}

In JSON this would be equivalent to:

{

"type": "Microsoft.Storage/storageAccounts/blobServices/containers",

"apiVersion": "2019-06-01",

"name": "[format('{0}/default/logs', variables('storageAccountName'))]",

"dependsOn": [

"[resourceId('Microsoft.Storage/storageAccounts', variables('storageAccountName'))]"

]

}

And finally we have the outputs. The first line demonstrates a shortcut for the resourceId property, the second line demonstrates calling a property of the storage account using the symbolic name and the last line demonstrates what would be equivalent to the reference function in the ARM template.

output storageId string = storageAccount.id

output computedStorageName string = storageAccount.name

output primaryEndpoint string = storageAccount.properties.primaryEndpoints.blob

In JSON this would be equivalent to:

"outputs": {

"storageId": {

"type": "string",

"value": "[resourceId('Microsoft.Storage/storageAccounts', variables('storageAccountName'))]"

"computedStorageName": {

"type": "string",

"value": "[variables('storageAccountName')]"

"primaryEndpoint": {

"type": "string",

"value": "[reference(resourceId('Microsoft.Storage/storageAccounts', variables('storageAccountName'))).primaryEndpoints.blob]"

}

You can use the Bicep Playground to play around with Bicep code and see it translated to JSON in an ARM template.

Once you have your code sorted, you can compile it running the following command which will generate a JSON file.

bicep build ./storage.bicep

Then you can deploy it as you would normally do.

New-AzResourceGroup -Name storageAccounts -Location australiaeast

New-AzResourceGroupDeployment -TemplateFile ./storage.json -ResourceGroupName storageAccounts

If you have already authored many ARM templates, don't worry as a decompiler is in the roadmap to allow the templates to be translated into Bicep code.

To learn more visit https://github.com/Azure/bicep

Automate Azure Sentinel Deployment

2020-09-01T15:59:00.002+10:00

Let's have a look how we can automate an Azure Sentinel deployment.

In this post I will show how you can automate the deployment of the following components using PowerShell and ARM templates:

Resource Group

Log Analytics workspace

Azure Sentinel

Alert Rules

Workbooks

Runbooks

In the end, as a bonus, I will show how you can put all together as part of an Azure DevOps pipeline.

OK, let's get started. First thing we have to do is to create the Resource Group where we are deploying the Sentinel workspace and all Sentinel related resources.

Note: all scripts and templates used in this post can be found here in my GitHub. From now on I will assume you have cloned it to your computer and your PowerShell window is set to the Sentinel directory.

New-AzResourceGroup -Name Workspaces -Location australiaeast

And the following is the output.

Next, let's deploy the Log Analytics workspace.

New-AzResourceGroupDeployment -Name SentinelWorkspace -ResourceGroupName Workspaces -TemplateFile .\Onboard\law-Sentinel.json -TemplateParameterObject @{workspaceName='SentinelWorkspaceAzureGear'}

If successful, you should see something similar to the figure below.

Once we have an Azure Log Analytics workspace with the Azure Sentinel solution deployed, we can start deploying Sentinel artifacts such as alert rules, workbooks and runbooks.

Let's start by deploying Alert Rules. If you have already manually imported Alert Rules you know how tedious it can be. Execute the command below which will install the AzSentinel module created by Pouyan Khabazi and will import all Alert Rules inside the AlertRules folder to the nominated workspace.

.\Scripts\ImportAlerts.ps1 -workspaceName SentinelWorkspaceAzureGear -folderName .\AlertRules\

Once the command completes, you can count your Alert Rules and you should see something like in the figure below.

Now it is time to import the workbooks. Run the PowerShell command below.

.\Scripts\ImportWorkbooks.ps1 -ResourceGroup workspaces -WorkbooksFolder .\Workbooks\ -WorkbookSourceId /subscriptionId/4858bb33-d8c5-4a26-86dd-d85c9cf73bda/resourceGroup/workspaces/workspaceName/sentinelworkspaceazuregear

Note: to retrieve the workspaceId you can navigate to the Azure Sentinel workspace, click on Workbooks and copy the URL starting at /subscriptionId/…

Once all workbooks are imported you will see something like the figure below.

And last but not least, let's import the Playbooks. Run the PowerShell command below.

.\Scripts\ImportPlaybooks.ps1 -ResourceGroup workspaces -PlaybooksFolder .\Playbooks\

Note: we are just importing the playbooks but you will still need to connect some of the steps with the required services such as Azure AD and Office 365.

The output should look like the figure below.

That's it. You have just shorten an Azure Sentinel deployment from a couple hours to just a few minutes.

Now to the bonus part. Let's consolidate all this steps in an Azure Pipeline so that we can do smart things such as:

Require approval for deploying all or any of the steps

Automatically deploy new artifacts based on new code committed to the repository

Create dependencies between the tasks

Track changes and document changes

Redeploy to any environment

Note: in this post I will not cover how you can achieve any of the above as it is beyond the scope of what I want to cover. Perhaps in a future post.

First step is to create a repository in your Azure DevOps project based on what is available in my GitHub like displayed below.

Next create a new Release Pipeline and create new stages in a way they are setup like the below.

For the Create Resource Group stage, add an ARM template deployment task and configure it with the following arguments.

For the Create Sentinel stage, add an ARM template deployment task and configure it with the following arguments.

For the Import Alert Rules, add an Azure PowerShell script task and configure it with the following arguments. Make sure you set it to use PowerShell Core.

For the Import Workbooks, add an Azure PowerShell script task and configure it with the following arguments. Make sure you set it to use PowerShell Core.

For the Import Runbooks, add an Azure PowerShell script task and configure it with the following arguments. Make sure you set it to use PowerShell Core.

Last step is to create a Variable Group with the variables like in the Figure below. Once created, don't forget to link it to the pipeline. And we are done!

Create a new Release for your pipeline and you should get the same results as when you run the scripts but now you can start adding the smarts :)

Azure WAF Policy

2020-08-27T21:56:00.001+10:00

Azure Web Application Firewall (WAF) on Azure Application Gateway provides centralized protection to web applications from common exploits and vulnerabilities. Web applications are increasingly targeted by malicious attacks that exploit commonly known vulnerabilities. SQL injection and cross-site scripting are among the most common attacks.

Azure WAF is getting better and better since V1 was released back in March 2017 as an additional capability to the Azure Application Gateway. In April 2019, V2 was made public available and WAF policies were released after that which extended the WAF capabilities.

In this post, I will provide some tips on how to fine tune OWASP rules from the WAF perspective and also how to create custom rules, geo-fencing rules to be more specific. I say from the WAF perspective because we are going to disable specific rules which are blocking legit traffic. In an ideal world you would remediate your application so that whatever is in the request doesn't match what is in the rule.

So let's go ahead and create a new WAF Policy with a Custom rule.

Specify the basic details. In this example I will associate the policy with an Application Gateway.

Now specify the Policy settings. If you want to block traffic that matches the rule, make sure you set the Mode to Prevention.

Important: Currently the maximum request body size supported is 128KB. Anything larger than that will generate an error 413 Request Entity Too Large. To my knowledge, from the WAF side, the only way to avoid this error is to disable the request body inspection however in my view this defeats the purpose of the WAF. It will still evaluate header and cookie requests though.

In the managed rules section select one of the rule sets. I will select OWASP_3.0 and later in this post I will demonstrate how to fine tune the available rules.

OK so now we got to the custom rules, where we are going to configure geo-fencing. Go ahead and click to Add a custom rule.

Configure the rule like in the figure below. I have set priority 1 so that it is always the first rule to be evaluated. I have also selected Australia and New Zealand as the countries. This rule will deny any traffic which is not coming from an IP Address in Australia or New Zealand.

Now associate the policy with an Application Gateway or HTTP listener and we are done.

You can validate the custom rule is working by checking the App Gateway WAF logs. In the example below we can see the rule was evaluated and it blocked the traffic.

OK now that we are done with the custom rule, let's see how we can fine tune that OWASP 3.0 ruleset. If you expand the ruleset, you will see that all rules are enabled and they have an ID. For example, the rule "Method is not allowed by policy" has an ID 911100.

So what I recommend you do is to place your WAF in detection mode, get an application user to execute common tasks and get the public IP from where the tasks are being executed while you monitor the App Gateway WAF logs.

In the example below, we searched for the public IP address and we found that traffic was getting blocked because it was matching the rule "SQL Comment Sequence Detected" with ruleId 942440.

Now with the ruleId in hand, you can go back to the rule set and disable that specific rule.

In the next example we have 2 more matches.

Continue the process until you don't have any more matches in your logs.

As I said before, in an ideal world you would remediate your application so that it doesn't match any of the OWASP rules but in the real world that is often not possible therefore you have to disable some of the rules so that your application can function correctly while still having protection from the rules which are still enabled.

Multi-tenant or multi-workspace Azure Sentinel Workbooks

2020-08-21T10:39:00.002+10:00

By default, workbooks in Azure Sentinel display data from a single tenant and a single log analytics workspace. This can be a problem when you are managing multiple Sentinels.

Javier Soriano created a nice article which can be found HERE, where he details how to make a workbook multi-tenant or multi-workspace.

In this post, I will skip the explanations he makes and focus on a step-by-step configuration on making the workbook multi-tenant or multi-workspace enabled.

Make sure you have access to the workspaces you are trying to visualize.

Let's get started. The first step is to go to the Workbooks and save one of the templates if you haven't already.

Once saved, click to view the saved workbook.

Click on Edit.

Click on Edit again but this time at the right side of the screen as displayed below.

Click on Add Parameter

Configure the new parameter as follows:

Parameter name: Workspace
Display name: Workspace
Parameter type: Resource picker
Required: True
Allow multiple selections: True
Get data from: Query

For the query, select Azure Resource Graph as the Data source and on the Subscription dropdown click on Load all subscriptions if you want to select more than one subscription. Then select the subscriptions.

Now enter the following query and it should return all the available workspaces. Select to include all in the dropdown. Hit Save.

resources | where type =~ 'Microsoft.operationsmanagement/solutions' | where name contains 'SecurityInsights' | project id = tostring(properties.workspaceResourceId)

Click on Done editing.

Once the workspace resource picker is done, we need to go to each of the visualizations / queries and change them to use the new workspace resource picker we have created.

Click on Edit on the visualization / query you want to change.

Click on change query.

From the Log Analytics workspace dropdown, select Worskspace.

Click on Done Editing.

Once we have updated all visualizations / queries, click on Done Editing.

Save it.

Now you should be able to select multiple workspaces from the same tenant or across tenant as shown below.

Why Azure Sentinel?

2020-07-27T15:24:00.000+10:00

Azure Sentinel is a great product and it has so much capability. In this post, I will try to summarise what I know and what I have learned so far but by no means it will cover all its extent.

Before we start talking about Sentinel, it is worth noting that Microsoft is investing heavily on security (over $1B annual investment), with 3500+ global security experts and trillions of diverse signals for unparalleled intelligence.

Azure Sentinel is a cloud-native Security Information and Event Management (SIEM) and as a cloud solution it provides virtually infinite scalability. It also leverages Microsoft provided AI and automation to improve effectiveness. In addition, it is very simple to onboard and it provides immediate benefits.Sentinel is essentially a Log Analytics solution, it is built on top of Log Analytics.So now let's get to know more about where all the work with Sentinel takes place:

Azure Sentinel workspace
The figure below is how a workspace looks like.

Think about the workspace as a container for Azure Sentinel which contains the event database, rules and incidents. However not everything related to Azure Sentinel is part of a workspace. Playbooks which are Logic Apps are not part of a workspace as well as Azure Monitor Workbooks. They can still be managed from a workspace but they are separate resources.

Sentinel also supports multiple workspaces. The following are some of the reasons to have multiple workspaces.

Data owners need access to their data (although a better solution is to use RBAC instead of multiple workspaces in this case)
Global SOC and Local SOC
MSSP and customers
Data ownership or sovereignty
Multiple Azure tenants
Fine grained retention setting (table level retention settings can be used instead of multiple workspaces)
Separate billing
However the rule is the fewer workspaces the better. Use one workspace for each tenant, geo and subsidiary.

Before we go any further, let's take a look in the roles of a SIEM.

Collect
Detect
Investigate
Respond

OK now that we covered the roles of a SIEM, let's go in more detail on how Sentinel provides excellent value for each one of them. I will provide an overview of key components which make up Azure Sentinel and I will map each of them to the roles above which will, most of the time, match the flow on how you would work with Azure Sentinel.

Data connectors

First thing you do with a SIEM is collect data. Sentinel can collect data from a variety of sources such as:

On-premises (Linux, FW, Windows, DNS, DHCP, files, etc)
Office 365
Security Center
Azure AD
SaaS Applications
AWS
and the list goes on.

To send the data from systems to Azure Sentinel, agents are the recommended option and a collector proxy can be used if there isn't a direct connection to the cloud.

For Windows systems if the agent can't be installed, a WEF (Windows Event Forwarding) connector can be used. For other systems where an agent can't be installed, a CEF or Syslog connector can be used. Other options are LogStash which a plugin is also available and custom connectors using PowerShell, Logic Apps or REST API using Ruby, Python, PHP, C# and Azure Functions.

Workbooks

Querying the data ingested from your data sources can feed information to workbooks which provide visibility of your data and power interactive dashboards. Many workbooks are already available from the gallery, they can be customised or you can create your own. Workbooks are used for visualisation of your data and are good for investigation and reporting

The workbooks are based on Azure Monitor Workbooks to provide you with enhanced customisation and flexibility in designing your own workbook.

Analytics

Once you have the data, you can use the data for analysis, investigation and to hunt for issues, vulnerabilities and threats.

There are 100+ built-in analytics rules and these rules are also customisable. In addition, you can create your own rules using the KQL query.

These rules, once enabled, detect anomalies and suspect activity. The alerts generated from these rules create incidents.

Incidents

Incidents can be used to collect related alerts, events and bookmarks. Bookmarks are how you mark notable data then you can start an investigation from a bookmark or add to an existing incident. Then you can assign those incidents and keep track of them. Tags and comments can be added to incidents. In addition, you can also integrate with your ticketing system.

Hunting

Azure Sentinel provides many built-in hunting queries to get you started proactively looking for threats and anomalies that weren't detect by other mechanisms.

For advanced hunting you can use Jupyter notebooks to query Azure Sentinel data or bring external data sources. However, you must use programming languages such as Python, SQL, KQL, R.

Playbooks

Finally, you can trigger automated playbooks powered by Logic Apps based on alerts or incident investigations generated from the rules. From the playbooks you can do actions such as open a ticket in ServiceNow, trigger a Defender ATP investigation or trigger some sort of remediation.

Threat Intelligence

Sentinel also provides Threat Intelligence via Data Connectors. Threat Intelligence is information such as malicious URLs, IP addresses, file hashes, etc. Currently you can push threat intelligence data to Azure Sentinel from Threat Intelligence Platforms or from TAXII servers. Once you have that information, as an example you could create a rule to generate an incident if one of the IPs from the Threat Intelligence feed matches the IP of an Azure Activity event.

Notebooks

Notebooks are used for advanced hunting and investigation. They allow API connections through programming languages such as Python and C# to allow for data manipulation and enrichment of the data.

Jupyter notebooks extend the scope of what you can do with this data. It combines full programmability with a huge collection of libraries for machine learning, visualization, and data analysis.

Azure Sentinel Pricing

Azure Sentinel provides very competitive prices. There are two ways to pay for the Azure Sentinel service: Capacity Reservations and Pay-As-You-Go.

As you would imagine there are big discounts when you do capacity reservation, between 50-60% discount comparing to pay-as-you-go prices.

You pay for the data ingested into Azure Sentinel and also the data stored in the Log Analytics workspace. The data can be retained for free for 90 days, longer than that you are charged.

In addition, you may be charged for Logic Apps if you run any automated responses and for Azure Databricks and/or Azure Machine Learning Studio if you bring your own machine learning modules.

Tips

Before I make my conclusion, I have a few tips that are good to know:

When you assign a Service Principal to create a connection from Logic Apps to Azure Sentinel, make sure you assign the Service Principal the Azure Sentinel Reader role
You can create the Logic App with the Azure Sentinel connector using ARM templates
ARM templates can be used to deploy Azure Sentinel
You can use Azure Lighthouse to manage Azure Sentinel across multiple tenants
You can find on GitHub a PowerShell module to enable cmdlets for many Azure Sentinel operations
Office 365 audit logs, Azure activity logs and alerts from Microsoft Threat Protection can be ingested for free

So why Azure Sentinel?

Azure Sentinel provides everything traditional SIEM tools provide and more. Leverage the several built-in connectors, pre-baked templates for workbooks, hunting and rules or make your own with full flexibility.

Hackers out there are always inventing new ways to crack through the fences. Azure Sentinel not only provide you with the tools to investigate and block these threats but also the latest technology and a team of security specialists behind the scenes.

Another great advantage of Azure Sentinel is that machine learning is built in the product to detect anomalies . Once you connect your data sources the learning begins. You can also bring your own machine learning modules.

All that in addition to the competitive prices makes it a very strong candidate to disrupt the SIEM landscape.

28 ways to reduce your Azure costs

2020-02-02T17:52:00.005+10:00

As an Azure consultant, the following are some examples of what I have helped our clients with:

Cloud architecture
Cloud foundation
Solution design
Cloud deployment
Cloud remediation
Moving workloads to the cloud
Consolidating subscriptions
Automation
Governance

Those are all very common and important types of engagements, however what has been in high demand for the last couple of years is the optimization of cloud spending. CIOs and managers are often challenged to explain why cloud spending is high and justify if the move to the cloud was the right call usually based on a single measurement – COST.

This article is all about ways to reduce and optimize the costs with workloads deployed specifically to Microsoft Azure.

The following are 28 ways to decrease your cloud spending without compromising performance, availability or scalability.

1. The first and most obvious way of saving on your Azure spending is to keep your subscription(s) clean. Periodically perform a clean-up and remove resources which are not in use anymore.

2. Stop VMs on times when they are not required. Most dev and test workloads are not required to be running 24/7. Leverage Azure automation and resource tagging to automate VM shutdown and start up.

3. Build dev and test workloads on Dev/Test subscriptions. This type of subscription provides a discount on the price of selected resources such as Windows Virtual Machines. Basically, you pay the price of a Linux VM as you are not paying for the Windows license.

4. Leverage the Azure Hybrid Benefit. This allows you to use your spare Windows Server Standard licenses on your Azure Windows Servers or concurrently for Windows Server Datacentre licenses. This is also applicable for Azure SQL Servers and SQL Servers running on Azure VMs. Save up to 40%.

5. VM Reservations. Get a big discount when buying 1- or 3-years reservations for your VMs. Pay upfront or monthly. If you know some workloads will be running for at least 1 or 3 years buy reservations and save big!

6. Blob Storage Reservation. Buy reserved capacity in chunks of 100TB or 1PB for your storage accounts and Azure Data Lake storage and save up to 38%. Reservations are available for all storage tiers for 1 year or 3 years.

7. Azure SQL Database. Save on SQL Database compute costs by buying reserved vCores for 1 or 3 years. Note this is only available with the vCore model not DTU.

8. Azure SQL Data Warehouse. Save up to 65% on SQL Data Warehouse costs by buying reserved capacity for 1 or 3 years.

9. Azure Database for MySQL, PostgreSQL and MariaDB. 1-year reservations are available providing savings of up to 51%!

10. Azure Data Explorer. Save up to 30% with reservations for Azure Data Explorer.

11. Azure Cosmos DB. Save up to 65% on Cosmos DB by buying reserved throughput capacity for 1 or 3 years.

12. Premium SSD Managed Disks. Save up to 5% on 1-year reservations. It may not appear too much of a saving but premium disks are expensive and if you have a lot of them when adding up it will be a big cost cut. Reservations are only available for disks sizes of 1TB and larger.

13. Azure Databricks. Save on your Azure Databricks costs by pre-purchasing DBUs for 1 or 3 years.

14. Azure Red Hat OpenShift. Save on Red Hat OpenShift costs by pre-purchasing plans for 1 year.

15. Azure SUSE Linux. Save on SUSE Linux enterprise server costs by pre-purchasing SUSE software for 1 or 3 years.

16. Azure Red Hat Plans. Save on Red Hat costs by pre-purchasing plans for 1 or 3 years.

17. Azure VMware Solution by CloudSimple. Save on VMware CloudSimple Node Instance costs by buying reserved instances for 1 or 3 years.

18. App Services. Save on App Service Isolated Stamp Fee by buying a 3-years Reserved Instance.

19. Use Lifecycle policy on your storage accounts. This feature allows you to enforce when blobs should be moved to cheaper tiers such as the cold and archive tiers or even when blobs should be deleted. If you know your files are accessed infrequently after 3 months and rarely after 6 months you could create a policy which would automatically move those files reducing your storage costs significantly.

20. Backup your SQL workloads to block blob instead of page blob. Beginning with SQL 2016, you can now perform SQL backups to block blob which is much cheaper than page blob.

21. Resize VMs which are not consuming much of the allocated resources. Azure Advisor will usually tell you if you have any VMs which should be resized due to over allocation of resources.

22. Azure App Services. If you don’t need to scale your websites independently, consider consolidating them in the same App Plan.

23. Azure SQL Elastic Pools. Combine databases which are not often used or databases which have peak resource consumption in different times of the day to allow them to share the available resources and generate extra savings on database costs.

24. Use B-Series VMs. This VM family is cheaper than other VM families with same size. The B-series VMs are ideal for workloads that do not need the full performance of the CPU continuously, like web servers, proof of concepts, small databases and development build environments. These workloads typically have burstable performance requirements. The B-series provides you with the ability to purchase a VM size with baseline performance and the VM instance builds up credits when it is using less than its baseline. When the VM has accumulated credit, the VM can burst above the baseline using up to 100% of the vCPU when your application requires higher CPU performance.

25. Leverage Tags to keep track of your resources. Easily allocated cost codes to your resources and remove them together by searching by tag.

26. Azure Backup. When setting up Azure Backup policies, set the retention of backups to meet your requirements. Don’t set it forever just because you can.

27. Recovery Vaults. By default, Recovery Vaults are created with GRS storage which means backups stored there are replicated to another region. If you don’t require Geo Replication, change it to LRS which replicates it 3 times in the same datacentre. This is often enough for most companies and it cost half the price.

28. Log Analytics. Be careful with what time of events you collect. It is tempting to collect everything but as you know there is a cost associated with that collection and it is not cheap.

Azure DevOps - App Deployment

2018-12-13T09:06:00.004+10:00

Welcome back! On part 1, I have shown you how to deploy your Azure resources with the power of Azure DevOps. Now it is time to deploy an application on top of that infrastructure. To make this more interesting I will setup continuous deployment so that we first deploy to a staging slot and if successful automatically deploy to production.

If you haven't seen part 1 yet, you can find it here: Azure DevOps - Infrastructure Build

For this example, I will use the simple-feed-reader App available from the Azure samples in GitHub. Simple-Feed-Reader App

Clone it and make it available in your own repo.

From the Azure DevOps project created earlier:

Click on Builds
Click on New pipeline

Click on Use the visual designer

Click on GitHub. A Service Point should already when you have set it up in Part 1
Select the simple-feed-reader repo
Select the master branch
Click on Continue

Select ASP.NET

Nothing has to be changed here.

Click on Save & queue (3 times)

Click on the link for the current build

After a couple minutes you will see all the steps have completed successfully and now you have an artifact which will be used by the Release pipeline to deploy the app to App Services. The artifact is just a zip file with your compiled application.

Now it is time to create the Release pipeline.

Click on Release, New Release

Apply the Azure App Service deployment

Name it Staging

Click on the jobs and tasks links on the Staging stage

If not already selected, select Staging
Select your subscription
Select Web App as the App type
Select your App Service created earlier

Select the Deploy Azure App Service task
Tick the box to Deploy to slot
Select the Resource Group where the App Services is located
Select the Slot staging

Click on Save
Return to the Pipeline page

Great, the Staging stage is done now. If you wanted we could deploy right now to the staging slot. But first let's create the production slot so that we can see the CD happening.

Hover your mouse on the Staging stage and click on Add

Select Apply for the Azure App Service deployment

Name it Production

Now repeat all steps we did for Staging but without ticking the box to Deploy to a slot. Once you do that, return to the pipeline and it should look like the picture below.

Click on Release and Create a Release. Then click on Create

Once that is done you should see both stages completed successfully and you should be able to browse to your website in staging and production.

Hopefully by now you know the basics of Azure DevOps and the power it provides to you. Keep tuned because soon I will post another article on how to deploy Azure Policies with Azure DevOps.

Azure DevOps - Infrastructure Build

2018-12-13T09:06:00.003+10:00

Azure DevOps former VSTS is so powerful that I have decided to write and share a bit on what you can achieve with it. In this Part 1, I will show how you can leverage Infrastructure-as-code (IaC) and Azure DevOps to build your entire infrastructure in a secure and modular fashion. In Part 2, we will then use Azure DevOps to build and deploy an application to the new infrastructure.

I will make it simple so our infrastructure will consist of:

1x Azure KeyVault
1x Azure App Service
1x Azure SQL DB

All the templates can be cloned from my GitHub repo.

First thing we need to do is to go to the Azure portal, create a new Azure DevOps project and browse to the project homepage. We also have to manually create an Azure KeyVault so that we can authorize it to be used by the project. We will use KeyVault to store our secrets. I have made available the KeyVault templates from the repo as well so that you can deploy them using PowerShell.

Now that we have a DevOps project we can create our first Release pipeline. We don't need a Build pipeline as we are not compiling any code. That said, if you are using an external repo which is not available to be used in the Release pipeline, you can use the Build pipeline to create an artifact based on the source files from the repo and make it available to the Release pipeline.

The following pictures display the steps required to create the Release pipeline to build your infrastructure.

First thing let's authorize the KeyVault we created earlier to be used in our pipelines.

Click on Library
Click to add a Variable Group

Name the variable group
Enable the Link secrets from an Azure key vault as variables
Select your subscription
Select the KeyVault
Click on Authorize

Click on Add
Select the available secrets
Click on OK
Click on Save

Now let's create a new Release. Click on Releases and New pipeline.

Click on Empty job.

Name the stage "Production"
Name the pipeline Infrastructure Deployment
Click on Add an artifact so we can link our repo

Select GitHub
Choose the source repository
Choose the branch
Choose the version
Add it.

Note that external repos like BitBucket are not available. In this case you have to create a Build pipeline linked to an external repository and then use the artifact created by selecting Build in the picture above.

Now let's create the tasks which will deploy our resources.

Click on the link for jobs and tasks

The first task we will add is a task to retrieve our secrets so that we can query them when building our resources.

Click on the + symbol over the Agent job
Search for key vault
Click on Add twice. We will need to retrieve the secrets twice as we are placing the connecting string of the DB in the KeyVault and we will need to fetch it before building the WebApp.

Select your subscription
Select the KeyVault

Configure both tasks the same.

Now let's add the tasks to build the database and the WebApp.

Click on the + symbol on top of the Agent job
Search for resource
Add 2 Azure Resource Group Deployment

Select the new Azure Resource Group Deployment
Name it Provision DB
Select your subscription
Select a Resource Group or type it if you want a new one provisioned
Select the location
Select the DB template
Select the DB parameter template
Enter -administratorLoginPassword $(sqlpass) in the Override template parameter. Here we are passing the sql admin password stored in KeyVault as a parameter

Now let's configure the WebApp.

Select the next Azure Resource Group Deployment task
Name it Provision WebApp
Select your subscription
Select a Resource Group or type it if you want a new one provisioned
Select the location
Select the WebApp template
Select the WebApp parameter template
Enter -connectionString "$(connectionString)" in the Override template parameter. Here we are passing the connection string to the DB stored in KeyVault as a parameter

Note that when you are passing variables as parameters you must enclose them in $() and if there are spaces in the value stored in the variable you must enclose them in "$()"

Move one of the KeyVault tasks after the Provision DB
Click on Save. Click OK
Click on Release. Create a Release.

Select the version (usually you leave the default which is the latest commit)
Click on Create

Now you will see a link to the Release job where you can follow the progress and see logs.

Click on it

Once it is finished you will see the screen below. You can click on Logs to investigate any issues or to see real-time output of the tasks.

If you want to enable continuous deployment you can set it to trigger a new build when new code is committed to your repo. As an example you could add a new slot to your WebApp template and once committed the deployment is automatically triggered.

You can also deploy on a schedule.

And you can define pre-deployment approvals. The deployment only starts once one or more approvers give the go ahead.

That's it! As you can see this is a simple example but it can expand as much as you want. If you want to deploy it to a different subscription you can clone the pipeline and just update the subscription references.

See you on Part 2 where we are going to deploy the app to the new infrastructure.

Setting up SAP BusinessIntelligence BI Platform SAML single sign on with Microsoft Azure AD as the Identity Provider

2018-08-15T21:26:00.004+10:00

I have co-author this guide with Francisco Almeida, BI Lead Consultant for DXC Oxygen.
There is already a very good guide on how to set this on up, but in here we have emphasized the practical side of the setup and focused on the order they should be performed:

https://blogs.sap.com/2018/03/01/saml-integration-between-microsoft-azure-portal-and-sap-business-intelligence-platform/

There are some small differences in the order and steps, and this is because we found this approach was most likely to return the expected result as we implemented this in a real World scenario. There are also a few practical suggestions you may find useful.

This configuration has been applied to BI 4.2 SP05 and SP06. Not all screenshots were taken from the same system, so some may have been pixelated to avoid confusion.

BOE preparation configuration

1. If your deployment is already configured for HTTPS, you can skip this step; otherwise, you can contract a third-party certificate authority, or create a self-signed certificate (which will throw a warning in your browser). To go with the self-signed certificate, run the command below, which will create a keystore and alias:

keytool -genkey -alias thisAlias -keyalg RSA -keystore C:\keystore\thisKeystore -keysize 2048

Please note thisAlias and C:\keystore\thisKeystore should be changed to reflect your organisation’s naming conventions, and you will need to create a password and fill out details for like company name, state, etc.

2. Locate <BOE Install Dir>\tomcat\conf\server.xml and create a backup of the file, then edit it to enable the https connector. To do that, uncomment it and change it to look like this:

Note: thisAlias, C:\keystore\thisKeystore and thisKeystorePassword will need to match the values you used in the previous step

3. Restart tomcat and make sure you can connect to tomcat using https://myserver

Note: myserver here is the name of the host where tomcat is installed. Do not proceed if you cannot.

4. Copy all .jar files from:

<BOE Install Dir> \SAP BusinessObjects Enterprise XI 4.0\SAMLJARS\
to:
<BOE Install Dir>\tomcat\webapps\BOE\WEB-INF\lib\

Note: the changes below will make the BI and BILaunchpad apps inaccessible. If you are on 4.2 SP06, and want to include CMC in the SAML SSO, do not configure it now. Come back to configure CMC only after you have gone through all configuration steps and tested that it works, otherwise you may find yourself locked out of the CMC.

5. Create a backup copy of the file below, and edit the original to uncomment the apps you want to use. In the case below, we have removed the leading “” from OpenDocument and BILaunchpad (this is the “fiorified” one). The line for the classic BI Launchpad was already uncommented. Save the changes but you can keep this file open as we will return to it later.

<BOE Install Dir>\tomcat\webapps\BOE\WEB-INF\securityContext.xml

6. Locate (or, if they do not exist, create) the files below, and make sure the property saml.enabled=true is configured in both:

a. <BOE Install Dir>\tomcat\webapps\BOE\WEB-INF\config\custom\BIlaunchpad.properties

b. <BOE Install Dir>\tomcat\webapps\BOE\WEB-INF\config\custom\FioriBI.properties

7. At this point BI and BILaunchpad should be inaccessible.

8. Locate <BOE Install Dir>\tomcat\webapps\BOE\WEB-INF\web.xml and uncomment the SAML parameters:

9. Launch a command line shell and navigate to <BOE Install Dir>\SAP BusinessObjects Enterprise XI 4.0\win64_x64\sapjvm\bin and execute the command below:

keytool -genkeypair -alias analias -keypass Password1 -keystore "D:\Program Files (x86)\SAP BusinessObjects\tomcat\webapps\BOE\WEB-INF\sampletestKeystore.jks" -validity 735

Define the alias (we used analias in the example above), give it a password (we used Password1) and chose a pre-existing keystore to generate it (we used the sampletestKeystore.jks). You will be asked the password of the keystore (the password for sampletestKeystore.jks is Password1) and further details.

10. Now go back to that open file <BOE Install Dir>\tomcat\webapps\BOE\WEB-INF\securityContext.xml and update it with the keystore name, alias and password:

11. If you use BOE tomcat’s hostname to access BOE (such as https://myserver:8443/BOE/BI) you can skip this step; if you have a more sophisticated setup with a load balancer and two or more tomcat hosts, you need to declare that in the same securityContext.xml file. Comment out the generic context provider and create a new one as below (you need to replace boe.itrust.co.nz from the screenshot below with your URL and the port is defined in your tomcat’s server.xml configuration file):

Also here:

12. You can finally save and close this file and replace the existing one (you have a backup, right?)

13. Now is time to set up the trust between tomcat and the CMS. Create the file <BOE Install Dir>\tomcat\webapps\BOE\WEB-INF\config\custom\global.properties (or change if existing) to contain the following text:

sso.enabled=true

trusted.auth.shared.secret=MySecret

trusted.auth.user.param=MyUser

trusted.auth.user.retrieval=WEB_SESSION

14. Log on to the CMC and generate a new shared secret by clicking in the “New Shared Secret” after you set the validity period and request timeout. Please note SAP recommends that the request timeout is not set to zero for security reasons.

15. Click on “Download Shared Secret” to download the file TrustedPrincipal.conf in your browser, and copy it over to the server(s) where the CMS service runs, into the following directories, so that the CMS will trust tomcat:

a. <BOE Install Dir>\SAP BusinessObjects Enterprise XI 4.0\win64_x64\

b. <BOE Install Dir>\SAP BusinessObjects Enterprise XI 4.0\win32_x86\

16. Preparation is done, now you need to stop Tomcat, and navigate to the work directory, where you should find the directories below, and delete these cached files (they will be recreated when you start tomcat again):

<BOE Install Dir>\tomcat\work\Catalina\localhost\

17. Start tomcat. Any errors in removing the comments, pointing to the keystore and password typos in the XML configuration files could cause tomcat not to rebuild the /BOE app and 404 page not found errors or even 500 ones.

18. Download the metadata XML by accessing the URL below, so the file can be sent to the Azure Specialist:

https://boe.itrust.co.nz/BOE/saml/metadata

Note: if you download the XML prior to this step, it may return a page not found error or it may download an XML with the wrong path in it – hence why this should be a late step.

If you get an error 404 page not found or similar, that means you should revisit the configuration steps, especially the on the securityContext.xml file. Open it in a browser to see that it looks right.

19. Now you need to wait for the Azure specialist to work his/her magic.

Azure AD Configuration (Identity Provider)
From the Azure Portal:

1. Click on Azure Active Directory

2. Click on Enterprise applications

1. Click on All applications

2. Click on New application

1. Click on Non-gallery application

2. Give the application a name

3. Click on Add

1. Click on Single sign-on

2. Select SAML-based Sign-on

3. Select the metadata file exported from the application

4. Click on Upload

1. Make sure user.userprincipalname is selected as the User Identifier

2. Download the Certificate (Base64)

3. Download the Metadata XML

4. Click on Save

Resume BOE configuration

1. The Azure specialist will download the metadata XML file and send you that file. Whatever the name of the file, you need to rename it to idp-meta-downloaded.xml and placed under <BOE Install Dir>\tomcat\webapps\BOE\WEB-INF\

This is because your securityContext.xml file expects to find it there:

2. Manually create a BOE account in CMC using the format used by Azure (usually firstname.lastname@domain.com) and try to log on to BI or BILaunchpad as normal. You should be greeted by the IdP (Azure) screen:

3. If you get the password right, this should redirect you to the BI / BILaunchpad, and automatically log you in – success!

4. We have seen some errors here where you get the Microsoft logon error or a 404 on URL /BOE/saml/metadata not found. These were caused by errors in the xml configuration file or failing to use the load balancer specific configuration mentioned above

5. We have also noticed errors when we moved on to configure other environments and tried to re-test the previous ones. This was resolved by clearing the cache of the browser. Make sure you do that every time you test SAML SSO.

6. Another possible issue you may run into is with the BILaunchpad – please remember to update the CMC > Applications > RESTful Web Services to the URL you are using, if different from a single server installation

7. If everything works, copy the changed configuration files to <BOE Install Dir>\SAP BusinessObjects Enterprise XI 4.0\warfiles\webapps\BOE\ otherwise the configuration will be lost in the next patch or service pack deployment

8. The SSO/SAML configuration may be finished, but for every account, an enterprise alias will have to be created. This can be done manually for each account, or by importing a file on CMC > Users and Groups > Manage > Import. Still, someone will need to map an S4 or BW account to the enterprise alias because they will most likely not match. If this is not your scenario, then your work is done.

9. If you need to find a way to automate the user creation and association, and you have the user’s email associated with his/her SAP account, you can develop an SDK script to use that information to automatically create an Enterprise alias and associate it with the SAP alias. We have developed a jar file (AddEntAlias) based on Business Objects’s SDK that reads an existing SAP account email property, creates and Enterprise alias, associate both accounts and renames the account to that email address, so that an SAP account like S4D~100/FALMEIDA would end up looking like francisco.almeida@itrust.co.nz, and mapped to the S4D alias automatically. This jar file can be scheduled to run after the SAP alias update.

10. If you want to use it, please read this disclaimer first. This SDK-based jar has been superficially tested with an SAP BI 4.2 SP05 system, but we cannot guarantee that it will work or that it will not cause any issues. So before using any SDK for the first time (and even if you use it all the time), make sure you have a backup of your hosts, your bootstrap files, your CMS database and your File repositories before proceeding. Also, please try this in a non-production environment before you try that in live production systems. To use it, download the file from here (password=readChomsky*), then log on to your CMC > Folders, select a location for it and click Manage > Add > Program File > click on Choose File select the file which you have just downloaded, select Java then hit OK.

11. Make sure you add the Classname to the Properties > Program Parameters of that file:

12. Now every time you want to run it, you need to use the Arguments: field (which can be changed on every scheduled job) to allow the script to log in and get its session with the CMS. So you need to specify the CMS to log the script in to, the user account you want to use to log the jar file into the system, the password, and the group that the script should run for. This group will be the only group that should be affected.

13. Each argument is separated by a space character; below is an example of how the Arguments: field could be filled out:

WIN-A2302R6200I administrator Ap@ssw0rd Everyone

This would log in to CMS WIN-A2302R6200I as the administrator and change all users because the group Everyone is specified. The following users are ignored: Administrator, Guest, users that already have an Enterprise account and SAP accounts that do not have an email associated with them. If there are 2 identical email addresses or 2 users could be created with the same ID, the script will throw an error and fail.

Azure ARM Templates with conditional logic and complex objects

2018-05-30T08:50:00.001+10:00

Azure templates have been used to automate the build of all sorts of Azure resources from a single virtual machine to an entire application composed of several virtual machines, networking resources, storage, etc.

For a long time Azure engineers have used linked templates or nested templates to build more complex solutions including multiple resource types and several instances of the same resource. These are still used today and can achieve the required outcome but it can lead to a proliferation of templates that are hard to maintain. The real power of Azure templates become clear when combining them with complex objects and conditional logic.

Conditional logic allows the use of conditions such as if a statement is true if deploys a resource and if it is false the resource creation is skipped. It also allows the creation of X resource or Y resource depending on the statement evaluated.

Complex objects can be used to build a resource with several nested resources each with several properties and iterate through them during the build process.

Combine conditional logic and complex properties and the sky is the limit.

I have built an ARM template to build any number of Virtual Machines with any number of resources such as data disks and network interfaces.

The deploy and parameter templates can be found in my GitHub in the link below.

https://github.com/fbinotto/101-vm-multi-disk-nic

Now let me break down the important bits which make them flexible and powerful.

The following is a snippet of the parameter template which provides the main template with a complex object. The object is made of two servers. The first server is of size Standard_DS1, with two NICs in different subnets and two disks being one standard and one premium. The second VM is of size Standard_A1 and has one disk and one NIC.

                "servers": [
                    {
                        "VMName": "server01",
                        "VMSize": "Standard_DS1",
                        "NIC": [
                            {
                                "Name": "NIC1",
                                "Allocation": "Dynamic",
                                "SubnetName": "Core"
                            },
                            {
                                "Name": "NIC2",
                                "Allocation": "Dynamic",
                                "SubnetName": "Edge"
                            }
                        ],
                        "Disk": [
                            {
                                "Name": "Disk1",
                                "Size": "1024",
                                "StorageAccountType": "Premium_LRS"
                            },
                            {
                                "Name": "Disk2",
                                "Size": "1024",
                                "StorageAccountType": "Standard_LRS"
                            }
                        ]
                    },
                    {
                        "VMName": "server02",
                        "VMSize": "Standard_A1",
                        "NIC": [
                            {
                                "Name": "NIC1",
                                "Allocation": "Dynamic",
                                "SubnetName": "Edge"
                            }
                        ],
                        "Disk": [
                            {
                                "Name": "Disk1",
                                "Size": "1024",
                                "StorageAccountType": "Standard_LRS"
                            }
                        ]
                    }
                ]

Now take a look in the code below which is part of the deploy template. It deploys the first NIC by default and then check if a second NIC is present by checking the length of the NIC object which is nested in the server object.

    {
      "apiVersion": "2018-04-01",
      "type": "Microsoft.Network/networkInterfaces",
      "name": "[concat(parameters('serverSettings').servers[copyIndex()].VMName, '-', parameters('serverSettings').servers[copyIndex()].NIC[0].Name)]",
      "location": "[resourceGroup().location]",
      "copy": {
        "name": "NICCopy",
        "count": "[length(parameters('serverSettings').servers)]"
      },
      "dependsOn": [],
      "properties": {
        "ipConfigurations": [
          {
            "name": "ipconfig",
            "properties": {
              "privateIPAllocationMethod": "[parameters('serverSettings').servers[copyIndex()].NIC[0].Allocation]",
              "primary": "true",
              "subnet": {
                "id": "[concat(variables('vnetID'),'/subnets/', parameters('serverSettings').servers[copyIndex()].NIC[0].SubnetName)]"
              }
            }
          }
        ]
      }
    },
    {
      "apiVersion": "2018-04-01",
      "condition": "[greater(length(parameters('serverSettings').servers[copyIndex()].NIC),1)]",
      "type": "Microsoft.Network/networkInterfaces",
      "name": "[concat(parameters('serverSettings').servers[copyIndex()].VMName, parameters('serverSettings').servers[copyIndex()],'-NIC1')]",
      "location": "[resourceGroup().location]",
      "copy": {
        "name": "NICCopy",
        "count": "[length(parameters('serverSettings').servers)]"
      },
      "dependsOn": [],
      "properties": {
        "ipConfigurations": [
          {
            "name": "ipconfig",
            "properties": {
              "privateIPAllocationMethod": "[parameters('serverSettings').servers[copyIndex()].NIC[1].Allocation]",
              "primary": "true",
              "subnet": {
                "id": "[concat(variables('vnetID'),'/subnets/', parameters('serverSettings').servers[copyIndex()].NIC[1].SubnetName)]"
              }
            }
          }
        ]
      }
    },

I don't know if you noticed but the second NIC has a conditional statement. It checks if the number of NICs for each of the servers is greater than one. It will only create the new NIC if there are more than one NIC in the object.

"condition": "[greater(length(parameters('serverSettings').servers[copyIndex()].NIC),1)]",

Note that if you need a third NIC a new resource would be required with a condition to deploy only if the NIC object length inside the server object is greater than 2. NIC resources are not as easy to deploy as for instance data disks. We will see shortly that multiple data disks can be deployed much easier.

We now have the NICs created and it is time to attached them to the VMs. To do that the copy function is used which iterates through each server and attach the new NICs.

        "networkProfile": {
          "copy": [
            {
              "name": "networkInterfaces",
              "count": "[length(parameters('serverSettings').servers[copyIndex('VMCopy')].NIC)]",
              "input": {
                "id": "[resourceId('Microsoft.Network/networkInterfaces', concat(parameters('serverSettings').servers[copyIndex()].VMName, '-NIC', copyIndex('networkInterfaces')))]",
                "properties": {
                  "primary": "[parameters('serverSettings').servers[copyIndex('VMCopy')].NIC[copyIndex('networkInterfaces')].Primary]"
                }
              }
            }
          ]
        }

Next it is the code to deploy the data disks. Notice how easier it is to deploy multiple disks. We just iterate through the disks objects which are part of the server object and can even dictate the size of disk, the disk type and other properties.

          "copy": [
            {
              "name": "dataDisks",
              "count": "[length(parameters('serverSettings').servers[copyIndex('VMCopy')].Disk)]",
              "input": {
                "diskSizeGB": "[parameters('serverSettings').servers[copyIndex('VMCopy')].Disk[copyIndex('dataDisks')].Size]",
                "lun": "[copyIndex('dataDisks')]",
                "createOption": "Empty",
                "managedDisk": {
                  "storageAccountType": "[parameters('serverSettings').servers[copyIndex('VMCopy')].Disk[copyIndex('dataDisks')].StorageAccountType]"
                }
              }
            }
          ]

This was just a snippet of what can be accomplished. The templates can be modified to build several additional objects with different configuration.

Managed Service Identity (MSI) for Azure Resources

2017-09-16T02:23:00.000+10:00

Microsoft recently announced the public preview of this new great feature named Managed Service Identity (MSI) and it is free!

https://docs.microsoft.com/en-us/azure/active-directory/msi-overview

Basically, with MSI you can make API calls to Azure using the VM identity to retrieve the token to be used as the authorization header. DevOps is just getting easier!

I lost count of how many times I was asked on how to authenticate your code to Key Vault without having hard coded credentials or certificate thumbprint. The good news is that now you can!

CI/CD tools like Jenkins, Puppet, Chef, etc which are running in an Azure VM don’t need to use Service Principals exposing secrets in its pipeline anymore!

A-W-E-S-O-M-E

Perhaps the extension will also be available for computers which are Azure AD joined in the future? I hope so!

There are a few tutorials on Microsoft Docs on how to access Azure Resource Manager, Azure storage and a non-Azure AD resource (Key Vault).

In this article, I will provide steps on how to get a token using a Windows VM with MSI to authenticate the code in a PowerShell script I have recently developed to manage Azure VM Scale Sets.

The full code can be found at:

https://github.com/fbinotto/AzureScaleSetManagement

First of all, we have to enable MSI on the VM where we will run the script. As of today (16/09/17), this cannot be done via the Azure portal here in Australia. I used PowerShell but you can also use ARM templates or CLI.

All steps to create a new Windows VM or update an existing Windows VM with MSI can be found in the following links:

https://docs.microsoft.com/en-us/azure/active-directory/msi-qs-configure-powershell-windows-vm
https://docs.microsoft.com/en-us/azure/active-directory/msi-qs-configure-template-windows-vm
https://docs.microsoft.com/en-us/azure/active-directory/msi-qs-configure-cli-windows-vm

I’m updating my existing Windows VM using PowerShell. The commands are as follows:

Login-AzureRmAccount
$vm = Get-AzureRmVM -ResourceGroupName sydprodrg01 -Name sydprodvm01
Update-AzureRmVM -ResourceGroupName sydprodrg01 -VM $vm -IdentityType "SystemAssigned"
$settings = @{ "port" = 50342 }
Set-AzureRmVMExtension -ResourceGroupName sydprodrg01 -Location AustraliaEast -VMName sydprodvm01 -Name "ManagedIdentityExtensionForWindows" -Type "ManagedIdentityExtensionForWindows" -Publisher "Microsoft.ManagedIdentity" -TypeHandlerVersion "1.0" -Settings $settings

Next we assign the contributor role to new Service Principal account created to the VM at the resource group scope where the resources to be managed are:

$SP = Get-AzureRmADServicePrincipal -SearchString 'sydprodvm01'
New-AzureRmRoleAssignment -ObjectId $SP.Id.Guid -ResourceGroupName sydprodrg02 -RoleDefinitionName Contributor

Now we RDP to the VM, open PowerShell and run the following command to retrieve the token:

$response = Invoke-WebRequest -Uri http://localhost/50342/oauth2/token -Method GET -Body @resource="https://management.azure.com/"} -Headers @{Metadata="true"}
$content = $response.Content | ConvertFrom-Json
$ArmToken = $content.access_token

With that token in hands now we can retrieve a resource such as a scale set from the resource group we have been granted permissions. For example:

$Uri = "https://management.azure.com/subscriptions/YOUR_SUBSCRIPTION_ID_HERE/providers/Microsoft.Compute/virtualMachineScaleSets?api-version=2017-03-30"
$scalesets = Invoke-RestMethod -Method Get -Headers $ArmToken -Uri $Uri

No more excuses to have hard coded credentials and secrets in your code!

Azure VM Scale Set Management

2017-09-09T01:42:00.004+10:00

I have made this tool in PowerShell inspired by the tool Guy Bowerman made coded in Python.

You can download it from https://github.com/fbinotto/AzureScaleSetManagement

You can check Guy's tool at https://github.com/gbowerman/vmssdashboard

My tool has been coded in PowerShell using mostly the Azure REST API. The only exception is the login to Azure which is part of the AzureRM module. I made it this way to be easier for anyone to use it. In addition the required modules are automatically installed if not already installed. PowerShellGet is required to be installed or PowerShell 5.0 and above. Run as administrator.

When running the Azure Scale Set Management tool, it will search for all Azure VM Scale Sets across all subscriptions the account you signed in has access to and provide that as a list to be selected.

The following are the functionalities once a scale set is selected:

Start scale set VM instance(s)
Deallocate scale set VM instance(s)
Restart scale set VM instance(s)
Reimage scale set VM instance(s)
Delete scale set VM instance(s)
Upgrade scale set VM instance(s)
Roll out upgrade to all VM instances in batches
Scale out / Scale in

The following is an overview of the user interface:

The following is an overview on how to perform a Roll Upgrade.

The following shows that instances are selectable and actions can be triggered for that selection.

And the following shows that the instances colors change accordingly to their state. The state is pulled every 30 seconds.

Azure Stack TP3 POC - My experience

2017-06-14T00:19:00.000+10:00

I always wanted to play with Azure Stack but never had the compute power to do so. Recently I bought a new desktop/server and thought I would be fine to run a POC but I was mistaken. So I had to dig the web for some workarounds and eventually (after a few rebuilds) I was able to make it work. I was excited to have it working and playing in my own Azure portal on premises but unfortunately I was not able to create any VMs as I did not have any more resources available.. Anyway.. It was good for the experience and the challenge of getting it working.

From this point I will describe what I have done to make it work.

My POC computer setup was:

1x CPU socket with 6 cores
64Gb of RAM
1x 1TB SSD for the OS
3x 1TB Data disks

Steps:

1. First thing download and extract the Cloudbuilder.vhdx from here.
2. Make sure you have .NET Framework 4.6 or later installed
3. Copy the Cloudbuilder.vhdx to the root of the C:
4. Mount the VHDX file
5. Browse to DRIVELETTER:\CloudDeployment\NuGetStore
6. Copy Microsoft.AzureStack.Solution.Deploy.CloudDeployment.1.0.422.0.nupkg to a temp location and rename it as Microsoft.AzureStack.Solution.Deploy.CloudDeployment.1.0.422.0.nupkg.zip
7. Extract the files in the same location.
8. Browse to \content\Configuration\Roles\Fabric\VirtualMachines and open the file OneNoteRole.xml using notepad

What worked for me was the following configuration changes. Locate each VM name and adjust the values.

MAS-DC01, StartUpMemoryBytes="2147483648", ProcessorCount="2", DynamicMemory="False"
MAS-NC01, StartUpMemoryBytes="2147483648", ProcessorCount="2", DynamicMemory="False"
MAS-SLB01, StartUpMemoryBytes="2147483648", ProcessorCount="2", DynamicMemory="False"
MAS-GWY01, StartUpMemoryBytes="4294967296", ProcessorCount="2", DynamicMemory="False"
MAS-CON01, StartUpMemoryBytes="2147483648", ProcessorCount="2", DynamicMemory="False"
MAS-SQL01, StartUpMemoryBytes="4294967296", ProcessorCount="2", DynamicMemory="False"
MAS-WAS01, StartUpMemoryBytes="5368709120", ProcessorCount="4", DynamicMemory="False"
MAS-WASP01, StartUpMemoryBytes="5368709120", ProcessorCount="4", DynamicMemory="False"
MAS-ACS01, StartUpMemoryBytes="5368709120", ProcessorCount="4", DynamicMemory="False"
MAS-XRP01, StartUpMemoryBytes="5368709120", ProcessorCount="4", DynamicMemory="False"
MAS-ERCS01, StartUpMemoryBytes="5368709120", ProcessorCount="4", DynamicMemory="False"
MAS-ADFS01, StartUpMemoryBytes="2147483648", ProcessorCount="2", DynamicMemory="False"
MAS-CA01, StartUpMemoryBytes="4294967296", ProcessorCount="2", DynamicMemory="False"
MAS-BGPNAT01, StartUpMemoryBytes="4294967296", ProcessorCount="2", DynamicMemory="False"

9. Now browse to \content\Configuration\Roles\Infrastructure\BareMetal and open the file OneNoteRole.xml using notepad

Change the following values to match your requirements.

    <ValidationRequirements>
      <MinimumOperatingSystemVersion>10.0.14300</MinimumOperatingSystemVersion>
      <MinimumNumberOfCoresPerMachine>4</MinimumNumberOfCoresPerMachine>
      <MinimumPhysicalMemoryPerMachineGB>32</MinimumPhysicalMemoryPerMachineGB>
      <MinimumNumberOfAdaptersPerMachine>0</MinimumNumberOfAdaptersPerMachine>
      <MinimumAdapterSpeedBitsPerSecond>0</MinimumAdapterSpeedBitsPerSecond>
      <MinimumSizeOfDataDisksGB>50</MinimumSizeOfDataDisksGB>
      <MinimumSizeOfSystemDiskGB>50</MinimumSizeOfSystemDiskGB>
      <MinimumNumberOfDataDisksPerMachine>2</MinimumNumberOfDataDisksPerMachine>
      <MaximumNumberOfMissingDisksPerMachine>0</MaximumNumberOfMissingDisksPerMachine>
      <SizeComparisonTolerancePercentage>1</SizeComparisonTolerancePercentage>
    </ValidationRequirements>

10. Copy both OneNoteRole.xml files to the ZIP file replacing the originals
11. Rename the file back to Microsoft.AzureStack.Solution.Deploy.CloudDeployment.1.0.422.0.nupkg and copy it back to the original location replacing the original file
12. Unmount the VHD
13. On the POC host, run the following PowerShell script to download the Azure Stack support files.

$Uri = 'https://raw.githubusercontent.com/Azure/AzureStack-Tools/master/Deployment/'
$LocalPath = 'c:\AzureStack_SupportFiles'
New-Item $LocalPath -type directory
( 'BootMenuNoKVM.ps1', 'PrepareBootFromVHD.ps1', 'Unattend.xml', 'unattend_NoKVM.xml') | `
foreach { Invoke-WebRequest ($uri + $_) -OutFile ($LocalPath + '\' + $_) }

The support files will be downloaded to C:\AzureStack_SupportFiles

14. Now open an elevated PowerShell, browse to C:\AzureStack_SupportFiles and run the following:

.\PrepareBootFromVHD.ps1 -CloudBuilderDiskPath C:\CloudBuilder.vhdx -ApplyUnattend

15. Provide the Administrator password for the new AzureStack OS (inside the CloudBuilder.vhdx) so you can log in once the computer is rebooted
16. When prompted say Yes which will restart the computer. Now the POC computer will boot into the OS of the CloudBuilder.vhdx
17. Log in as Administrator using the password provided earlier
18. Open the network connections and disable ipv6. Also make sure you only have 1 NIC enabled if you have multiple
19. Rename the computer to a more descriptive name
20. A lot of drivers will probably be missing if you look at device manager but they should be all available in the old OS which should be accessible through one of the volumes. To install all the drivers in one go, create the following batch file

forfiles /p %1 /s /m *.inf /c "cmd /c pnputil -a @Path"

21. Now run it from an elevated command prompt. For example if you created the file InstallDrivers.cmd run the following (Make sure you point the drive letter to the correct volume which should be the old OS volume):

InstallDrivers.cmd E:\Windows\System32\DriverStore\FileRepository

22. Make sure you drives are not partitioned otherwise they won't be considered as available

23. Now it is time to start the installation. I have chosen to use Azure AD for authentication but ADFS can also be used. To start the installation run the following from an elevated command prompt

cd C:\CloudDeployment\Setup 
$adminpass = ConvertTo-SecureString "〈LOCAL_ADMIN_PASSWORD〉" -AsPlainText -Force
$aadpass = ConvertTo-SecureString "<AAD GLOBAL ADMIN ACCOUNT PASSWORD>" -AsPlainText -Force
$aadcred = New-Object System.Management.Automation.PSCredential ("<AAD GLOBAL ADMIN ACCOUNT>", $aadpass)
.\InstallAzureStackPOC.ps1 -AdminPassword adminpass

Now this will take hours. In the first hour it will reboot and that is the only time it will reboot. Once it reboots you can log in as AzureStackAdmin.

If there are any issues first thing to try it to re-run it as displayed below:

.\InstallAzureStackPOC.ps1 -Rerun

I had a few issues with ADFS where I had to restart the server. I also had issues with a watchdog monitoring service. I had to start the service on both the host and the server where the issue was happening,

If for any reason I have rebooted the POC server, restart the VMs in the following order:

MAS-DC01, MAS-BGPNAT-1, MAS-SLB01, MAS-SQL01
Wait 5 minutes
MAS-ACS01, MAS-XRP01
Wait 5 minutes
MAS-WAS01, MAS-WASP01
Wait 5 minutes
MAS-CON01, MAS-CA01, MAS-ADFS01, MAS-GWY01
Wait 5 minutes
MAS-NC01, MAS-ERCS01

Once the build is complete you should see the following:

You can also see all the VMs running from the Hyper-V or Failover clustet console.

Now it is time to active the portals.

From the POC host navigate to https://adminportal.local.azurestack.external/guest/signup and accept. Now navigate to https://portal.local.azurestack.external/guest/signup and accept again.

Now change the password expiration to 180 days running the following cmdlet.

Set-ADDefaultDomainPasswordPolicy -MaxPasswordAge 180.00:00:00 -Identity azurestack.local

And last if you want to import an image so you can create a VM run the following:

Import-Module .\Connect\AzureStack.Connect.psm1
Import-Module .\ComputeAdmin\AzureStack.ComputeAdmin.psm1

Add-AzureStackAzureRmEnvironment `
  -Name "AzureStackAdmin" `
  -ArmEndpoint "https://adminmanagement.local.azurestack.external"

$TenantID = Get-DirectoryTenantID `
  -AADTenantName "<myDirectoryTenantName>.onmicrosoft.com" ` #CHANGE WITH YOUR TENANT NAME
  -EnvironmentName AzureStackAdmin

$ISOPath = "<Fully_Qualified_Path_to_ISO>" #PATH TO ISO

$UserName='<Username of the service administrator account>' #AZURE ADMIN
$Password='<Admin password provided when deploying Azure Stack>'| ` #PASSWORD
  ConvertTo-SecureString -Force -AsPlainText
$Credential=New-Object PSCredential($UserName,$Password)

# Add a Windows Server 2016 Evaluation VM Image.
New-Server2016VMImage `
  -ISOPath $ISOPath `
  -TenantId $TenantID `
  -EnvironmentName "AzureStackAdmin" `
  -Net35 $True `
  -AzureStackCredentials $Credential

That's all.. what a lengthy post!!

Create Intune iOS Policy using Microsoft Graph API and Powershell

2017-03-05T22:59:00.002+10:00

This is my first post in a while but I'm returning in great style with a very interesting topic to talk about.

The Microsoft Graph API is a very powerful way to consume, integrate and interact with Microsoft Cloud services. In this post, I will run you through the steps required to create an iOS compliance policy in Intune using the Graph API.

My intention here is to make you realize how important and ground breaking this is. Imagine yourself having to configure an Intune tenant. All the compliance policies, configuration policies, applications, etc. Now imagine you can automate everything and set up an Intune tenant in minutes instead of hours or even days. This is the new reality. This is what the Microsoft Graph API empowers us to do.

Assumptions:

Intune tenant in the new Azure portal (portal.azure.com)
An app in Azure registered with the correct delegations to create policies in Intune. Refer to https://docs.microsoft.com/en-us/azure/active-directory/develop/active-directory-integrating-applications

Once you have your app registered, run the following code in PowerShell to get an access token:

Add-Type -AssemblyName System.Web
$graphURL = "https://graph.microsoft.com"
$appID = "Replace-with-AppicationID"
$appKey = "Replace-with-Key-Value"
$appUri = "https://localhost:8000"
$appIDEncoded = [System.Web.HttpUtility]::UrlEncode($appID)
$appKeyEncoded = [System.Web.HttpUtility]::UrlEncode($appKey)
$appUriEncoded =  [System.Web.HttpUtility]::UrlEncode($appUri)
$graphEncoded = [System.Web.HttpUtility]::UrlEncode($graphURL)
$delEncoded = [System.Web.HttpUtility]::UrlEncode("https://outlook.office.com/user.readwrite.all")

Function Get-AuthCode {
    Add-Type -AssemblyName System.Windows.Forms

    $form = New-Object -TypeName System.Windows.Forms.Form -Property @{Width=440;Height=640}
    $web  = New-Object -TypeName System.Windows.Forms.WebBrowser -Property @{Width=420;Height=600;Url=($url -f ($Scope -join "%20")) }

    $DocComp  = {
        $Global:uri = $web.Url.AbsoluteUri        
        if ($Global:uri -match "error=[^&]*|code=[^&]*") {$form.Close() }
    }
    $web.ScriptErrorsSuppressed = $true
    $web.Add_DocumentCompleted($DocComp)
    $form.Controls.Add($web)
    $form.Add_Shown({$form.Activate()})
    $form.ShowDialog() | Out-Null

    $queryOutput = [System.Web.HttpUtility]::ParseQueryString($web.Url.Query)
    $output = @{}
    foreach($key in $queryOutput.Keys){
        $output["$key"] = $queryOutput[$key]
    }

}

$url = "https://login.microsoftonline.com/common/oauth2/authorize?response_type=code&redirect_uri=$appUriEncoded&client_id=$appID&resource=$graphEncoded&prompt=admin_consent&scope=$delEncoded"
Get-AuthCode

$regex = '(?<=code=)(.*)(?=&)'
$authCode  = ($uri | Select-string -pattern $regex).Matches[0].Value

$body = "grant_type=authorization_code&redirect_uri=$appUri&client_id=$appId&client_secret=$appKeyEncoded&code=$authCode&resource=$resource"
$Authorization = Invoke-RestMethod https://login.microsoftonline.com/common/oauth2/token `
    -Method Post -ContentType "application/x-www-form-urlencoded" `
    -Body $body `
    -ErrorAction STOP

$accesstoken = $Authorization.access_token

$body = "grant_type=authorization_code&redirect_uri=$redirectUri&client_id=$clientId&client_secret=$clientSecretEncoded&code=$authCode&resource=$resource"
$Authorization = Invoke-RestMethod https://login.microsoftonline.com/common/oauth2/token `
    -Method Post -ContentType "application/x-www-form-urlencoded" `
    -Body $body `
    -ErrorAction STOP

$accesstoken = $Authorization.access_token

Now that you have the access token, run the following code to create a new iOS compliance policy in Intune. We interact with the Microsoft Graph API using JSON objects. In the example below we convert a hashtable to a JSON object before feeding it to the Invoke-RestMethod cmdlet.

$odata = "@odata.type"
$policy = 
@{
  $odata = "#microsoft.graph.iosCompliancePolicy"
  "description"= "Created using Microsoft Graph"
  "lastModifiedDateTime"= "2017-01-01T00:00:35.1329464-08:00"
  "displayName"= "iOS Compliance Policy"
  "version"= 7
  "passcodeBlockSimple"= $true
  "passcodeExpirationDays"= 6
  "passcodeMinimumLength"= 5
  "passcodeMinutesOfInactivityBeforeLock"= 5
  "passcodePreviousPasscodeBlockCount"= 2
  "passcodeMinimumCharacterSetCount"= 0
  "passcodeRequiredType"= "alphanumeric"
  "passcodeRequired"= $true
  "osMinimumVersion"= "8.0"
  "osMaximumVersion"= "10.0"
  "securityBlockJailbrokenDevices"= $true
  "deviceThreatProtectionEnabled"= $true
  "deviceThreatProtectionRequiredSecurityLevel"= "low"
} | ConvertTo-Json

Invoke-RestMethod -Headers @{Authorization = "Bearer $accesstoken"} `
                         -Uri  https://graph.microsoft.com/beta/deviceManagement/deviceCompliancePolicies/ `
                         -Method POST `
			 -Body $policy `
                         -ContentType "application/json"

Wow! So powerful. And this is just one example! I will provide 2 more examples to wrap this up.

The following code will list of Intune enrolled devices or Azure AD joined devices:

Invoke-RestMethod -Headers @{Authorization = "Bearer $accesstoken"} `
                        -Uri  https://graph.microsoft.com/v1.0/devices `
                        -Method Get | select -expand value

The following code will create a new user in Azure AD:

$user = @{
  "accountEnabled" = $true
  "displayName" = "Donald Trump"
  "mailNickname" = "donald.trump"
  "userPrincipalName" = "donald.trump@felipebinottohotmail.onmicrosoft.com"
  "passwordProfile" = @{
    "forceChangePasswordNextSignIn" = $false
    "password" = "P@ssw0rd1!"
  }
  
} | ConvertTo-Json

$newuser = Invoke-RestMethod -Headers @{Authorization = "Bearer $accesstoken"} `
                         -Uri https://graph.microsoft.com/v1.0/users `
                         -Method POST `
                         -Body $user `
                         -ContentType "application/json"

As you can see there is so much you can do leveraging the Microsoft Graph API. Some APIs are still in Beta but I'm sure Microsoft will soon make them production ready and expand their functionality. Very exciting to say the least.

DirectAccess - External Adapter Forwarding State

2014-11-11T18:23:00.001+10:00

Recently I have encountered this error message in my DirectAccess set up. It just came out of the blue. Everything was working OK until it was not. I have also noticed both network cards were assigned the domain profile.

I had a MuliSite DA running on Windows Server 2012 R2 with 2 NICs with 2 consecutive public IP addresses.

I did some research and I found a few possible solutions but they were all for Forefront UAG 2010.

I will list all of them here but only one worked for me.

1. Check that forwarding is not enabled in your external adapter. If forwarding is enabled traffic could have been forwarded from the external to the internal network adapter eventually reaching a domain controller. You can check that with the following command:

netsh int ipv6 show int external

where "external" is the name of your external network interface.

The parameter "Forwarding" should be set to disabled.

2. Create a firewall rule to block traffic from your external network adapter to all domain controllers. This could actually work but I decided this would be my last resort so in the end I didn't have to use it.

3. Check your routes. Check there is no entry where the interface is your external adapter's IP and the network destination is an IP or IP Subnet where a DC is located.

4. The real deal. Clear NLA registry entries for the external adapter, disable it and re-enable it.

Locate the following key and delete any entries where you see the IP(s) of the external network adapter.

HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\NetworkList\Nla\Cache\IntranetAuth

Now disable and re-enable the external network adapter and DirectAccess should be happy again.

SCCM 2012 - Automatically uninstall unused applications

2014-10-03T18:19:00.000+10:00

In this post I will explain how to automate the uninstall of applications that are not used in a number of days.

This is useful for a number of reasons but mostly to save on licenses.

The following is the workflow until it gets to the automatic uninstall of the application.

Once you follow all the steps of this post no further interaction is required. When a condition is met the application will automatically be uninstalled from the computer.

Let's get started. I'm going to use as example the automatic uninstall of WinZip (a licensed application) if it's not used for 90 days or more.

First thing create a Software Metering rule for WinZip.

Once the rule is created, select it and from the Summary at the bottom of the console take note of the Internal Rule ID which will be used in a Collection Query later.

Now create the following collections:

INSTALL WinZip

The membership can be Direct or Query based. In this case I will chose query based which is the most complex but most common scenario. Limit the membership to All Systems.

Let's assumed that my domain is ITSELF and the AD Groups is named WinZip. The following query will retrieve all member of the ITSELF\WinZip AD Group.

select SMS_R_SYSTEM.ResourceID,SMS_R_SYSTEM.ResourceType,SMS_R_SYSTEM.Name,SMS_R_SYSTEM.SMSUniqueIdentifier,SMS_R_SYSTEM.ResourceDomainORWorkgroup,SMS_R_SYSTEM.Client from SMS_R_System where SMS_R_System.SystemGroupName = "ITSELF\\WinZip"

This collection will have the WinZip install deployed to it.

UNINSTALL WinZip

The membership should be Query based. Limit the membership to All Systems. The following query will retrieve all computers which have WinZip installed.

select SMS_R_SYSTEM.ResourceID,SMS_R_SYSTEM.ResourceType,SMS_R_SYSTEM.Name,SMS_R_SYSTEM.SMSUniqueIdentifier,SMS_R_SYSTEM.ResourceDomainORWorkgroup,SMS_R_SYSTEM.Client from SMS_R_System inner join SMS_G_System_AppClientState on SMS_G_System_AppClientState.MachineName = SMS_R_System.Name where SMS_G_System_AppClientState.ComplianceState = 1 and SMS_G_System_AppClientState.AppName like "%WinZip%"

This collection will have the WinZip uninstall deployed to it.

USED IN THE LAST 90 DAYS WinZip

The membership should be Query based. Limit the membership to the INSTALL WinZip collection.

The following query will retrieve the Software Metering data collected for WinZip using the Rule ID I told you to take note in the beginning of this post. But it doesn't get all computers which have used WinZip, it only retrieve those which have used it in the last 90 days.

select SMS_R_SYSTEM.ResourceID,SMS_R_SYSTEM.ResourceType,SMS_R_SYSTEM.Name,SMS_R_SYSTEM.SMSUniqueIdentifier,SMS_R_SYSTEM.ResourceDomainORWorkgroup,SMS_R_SYSTEM.Client from SMS_R_SYSTEM inner join SMS_MonthlyUsageSummary on SMS_R_SYSTEM.ResourceID = SMS_MonthlyUsageSummary.ResourceID INNER JOIN SMS_MeteredFiles ON SMS_MonthlyUsageSummary.FileID = SMS_MeteredFile.MeteredFileID WHERE DateDiff(day, SMS_MonthlyUsageSummary.LastUsage, GetDate()) < 90  AND SMS_MeteredFiles.RuleID = 16777317

There is nothing deployed to this collection.

AUTOMATIC UNINSTALL WinZip

This collection should be limited to the INSTALL WinZip collection.

The membership should be the inclusion of the INSTALL Winzip Collection and the exclusion of the USED IN THE LAST 90 DAYS WinZip collection.

There is nothing deployed to this collection also.

OK now we have all the collections required and they should reflect the correct numbers.

Next step is to create a daily scheduled task which will run the script below. The scheduled task should run using an user which has both permissions to remove members of SCCM collections and remove members of AD groups.

The script below will go through all the AUTOMATIC UNINSTALL collections and it will retrieve a list of its members. Then it will search for those members in the INSTALL collection and in the AD Group and it will remove them from there.

Once they are removed from the collection the Uninstall Deploy will trigger and uninstall the application.

Note that we must have a good name convention for your collections and AD groups. They should all match in order for this to work. For example if the collection is INSTALL Winzip, your AD group should be named WinZip. Of course you can adapt the script to fit your environment.

#Gets current date to append to log
$date = (get-date).ToShortDateString().replace("/","")

#Log location and log name
$logfile = "C:\Temp\AutomaticUninstall$date.log"

#Writes first line to log
"Starting AutomaticUninstall.ps1..." | Out-File $logfile

#Imports SCCM module
Import-Module ($Env:SMS_ADMIN_UI_PATH.ToString().SubString(0,$Env:SMS_ADMIN_UI_PATH.Length - 5) + "\ConfigurationManager.psd1")

#Imports AD module
Import-Module ActiveDirectory

#Gets SCCM Site Code
$SMSProvider = get-wmiobject sms_providerlocation -namespace root\sms -filter “ProviderForLocalSite = True”
$SiteCode = $SMSProvider.SiteCode

#Change location to SCCM drive
Set-Location "$SiteCode:"

#Get a list of all collections which starts with AUTOMATIC UNINSTALL
$collections = Get-CMDeviceCollection -Name "AUTOMATIC UNINSTALL*" | select Name, MemberCount

#for each collection
foreach($collection in $collections){

#if current collection has more than 1 member
    if($collection.MemberCount -gt 0){

#Write current collection name to log
    "Collection: $($collection.name)" | Out-File $logfile -append

#Get members of the current collection
    $devices = Get-CMDevice -CollectionName $collection.Name | Select -ExpandProperty Name

#Get name of AD Group
    $group = $collection.Name.Replace("AUTOMATIC UNINSTALL ","")

#Get name of the current INSTALL collection
    $InstallCollection = "INSTALL $group"

#for each computer in the current collection
    foreach($device in $devices){

#Write computer name to log        
 "Member removed: $device" | Out-File $logfile -append

#Remove computer from AD group
        Try{Remove-ADGroupMember -Identity $group -Members (Get-ADComputer $device) -Confirm:$False -ErrorAction SilentlyContinue}
        Catch{}

#Remove computer from current INSTALL collection
        Remove-CMDeviceCollectionDirectMembershipRule -CollectionName $InstallCollection -ResourceName $device -Force
        }
    }
}

Sequencing VMware View Client 5.x with broker history using App-V - Recipe

2014-08-31T17:45:00.000+10:00

In this demonstration I will be using App-V Sequencer 5 SP2 running on a Windows 7 SP1 Enterprise 64-bit.

Start the sequencer and select Create a New Virtual Application Package.

Select Create Package (default) and click on Next.

Make sure there is nothing to be resolved before start sequencing.

Select the installer.

Give the package a name and select the installation directory.

Once the installer starts make sure you change the destination folder to match the installation directory you chose in the last step.

Leave this blank.

When prompt to restart select No.

Once the installation is finished select I am finished installing and click on next.

Run Selected. Wait until VMware View Client stats and then close it.

Now open the Windows Registry and browse to HKCU\Software\VMware, Inc.\VMware VDM\Client, right click it and select New Multi-String Value.

Name it BrokerHistory and in the value data type the server names which should be available from the drop down list in VMware View Client. Click OK and close the Windows Registry. Click on Next.

If you left the default options you should see the below "Unsupported driver detected". This is related to the USB redirection. Either install the driver manually or disable USB redirection. Click on next.

Select Stop now. Click on next.

Select Save the package now. Type the location where you want the package to be saved and click on Create.

Once you deploy the package you should see the list of connection servers available for selection.

Customize Windows 8.x Start Menu from Task Sequence

2014-08-29T16:10:00.001+10:00

Very simple task.

First from a reference computer where all applications which will be in the deployed image are installed customize the start menu as required.

From an elevated powershell run the following:

Export-StartLayout -as BIN -Path C:\Temp\StartLayout.bin

There are many ways of placing it to the task sequence. I like to include it to the MDT Toolkit (in this example I will place it in the scripts folder) so I don't have to create an extra package and then from the task sequence create a new Run Command Line with the following command:

C:\Windows\System32\WindowsPowershell\v1.0\powershell.exe -noprofile -windowstyle hidden -command Import-StartLayout -LayoutPath %SCRIPTROOT%\StartLayout.bin -MountPath %SYSTEMDRIVE%\

Just make sure you create the Run Command Line when the MDT Toolkit is cached in the computer.

SCCM 2012 - Unable to delete application or edit task sequence

2014-08-29T15:56:00.001+10:00

A few days ago I experienced something unusual where I was unable to delete any applications or edit a task sequence.

I have seen some posts where they recommend to query the SCCM database to check if something is locked and if it is then delete the row. This is not supported and it didn't work for me but I will post here just for the sake of completeness.

Open the SQL Server Management Studio and browse to the SCCM database.
Click on New Query.

Type the following and click on Execute:

Now if you have something locked it should show up. Get the ID of the locked object and replace it in the command below. Click on Execute.

OK now to the solution that worked for me.

Just restart the WMI service or alternatively restart the server.

net stop winmgmt && net start winmgmt

Application streaming failing when using SCCM 2012 with App-V integration

2014-05-01T20:05:00.001+10:00

UPDATE: Hotfix 4 released. Download it from HERE.

And yet another Microsoft bug...

If you were pulling your hair off as I was a few hours ago then you understand the frustration of spending hours of troubleshooting and knowing that everything looks fine and finding that the problem you were experiencing was actually a bug.

Well luck for you if you have found this blog and this fix.

You can identify the problem by looking into the AppEnforce.log and finding the following:

Publish-AppvClientPackage : Failure to communicate with Application Virtualization Service. Operation attempted: Publish AppV Package. HRESULT: 800706BE

Now look in the event viewer and find the following:

The Microsoft App-V Client service terminated unexpectedly.  It has done this 2 time(s).

Faulting application name: AppVClient.exe, version: 5.0.3361.0, time stamp: 0x52781536Faulting module name: AppVEntStreamingManager.dll, version: 5.0.3361.0, time stamp: 0x52781643

If you monitor the App-V Service you will see that it stops every time SCCM start caching the initial files for streaming.

If you are experiencing the above then there are 2 workaround. The manual and the automated:

Manual:

Open powershell as admin and type:

Import-Module AppvClient; Set-AppvClientConfiguration –IgnoreLocationProvider 1

Automated:

Create a GPP registry item with the following:

HKLM\SOFTWARE\Microsoft\AppV\Client\StreamingName: IgnoreLocationProvider Value: 1 Type: DWORD

Happy streaming!

Install and Configure Exchange 2013 Step by Step

2014-04-21T14:22:00.002+10:00

I will perform the installation on a Windows Server 2012 R2. If you are installing on Windows Server 2008 R2 make sure you download and install:

.NET Framework 4.5
Windows Management Framework 3.0

Now run setup.exe

Check if there are updates available. Click Next.

Click on Next to copy the files and initialize the setup.

Click on Next in the Introduction screen.

Accept the license and click on Next.

Choose whether you want to send information to Microsoft or not. Click on Next.

Select the roles you want to install in the server. In this example I'm going to install the CAS and Mailbox role in the same server. Click on Next. In case you are wondering where is the hub transport role it has been retired and it's functionality split and integrated in the CAS and Mailbox role.

Choose the installation location and click on Next.

Choose if you want Malware protection enabled or not and click on Next.

Now it will automatically configure some of the Exchange pre-requisites. Now a restart is required.

Once you are back download and install:

Microsoft Unified Communication Managed API 4.0
Microsoft Office 2010 Filter Packs
Microsoft Office 2010 Filter Packs SP1

Now run the setup again and repeat all steps above until you get to the Readiness check screen. It should look like the screen below. Click on Install.

Once the installation is completed, click on finish and restart the server.

The installation is now completed. Let's do the basic configurations. First be aware the Exchange Management Console doesn't exist anymore and it has been replaced by the Exchange Admin Center which is web based.

So open IE and type https://SERVERNAME/ecp

If you still have another exchange server in your environment the link above may take you to the web app instead. In order to access the Admin Center use the following link https://SERVERNAME/ecp?ExchClientVer=15

If it's the first time you are signing in you may get the OWA screen as below:

Once you sign in, select the language and time zone then you will see the admin center screen like the one below:

Ok now click on servers node and in the virtual directories tab and configure the websites for each of the virtual directories and authentication if required.

The example below show the ActiveSync configuration for the internal and external URLs. In my case they are the same as both external and internal DNS records are also the same.

To configure Outlook Anywhere still in the severs node, double click on your exchange server (or select it and click on Edit) and click on Outlook Anywhere. Now specify the internal and external hostname that users will use to connect to Exchange.

Make sure you also have all your receive and send connectors configured.
Click on Mail Flow then on receive connectors. The most important part is the scoping.

Client frontend scoping:

Client Proxy scoping:

Default frontend scoping:

Default scoping:

Outbound Proxy frontend scoping:

Now click on send connectors.

Internet scoping:

Once you test all is working and you still have old Exchange servers in the environment you can start migrating the mailboxes. Click on recipients and from the mailboxes select the mailbox you want to migrate and click on Move Mailbox To another database.

Give the job a name and select the target database(s). Click on Next.

Select the mailboxes which will be sent a notification once the batch is completed and click on New.

If you have a large number of mailboxes to migrate it's easier to use the Management Shell. The following example would migrate all mailboxes from one database to the other. Use the Get-MailboxDatabase to get the names of the source and target databases or look for them in the Exchange admin center.

Once you migrate everything else and check all the functionality is working you can shut down your old exchange servers leave them there for a couple weeks and then decommission them.

SCCM 2012 - DDM.BOX Millions of DDRs

2014-04-13T10:53:00.000+10:00

If you have thousands or even millions of records in the inboxes\auth\ddm.box folder your SCCM is suffering from DDR backlogs.

Usually this is caused by some corrupted record or/and too aggressive Discovery Methods.

There are a few symptoms of DDRs backlogs:

AD group membership doesn't work properly
Some computer records don't have all information (i.e. AD site, IP Address, etc)
You can't browse to the inboxes\auth\ddm.box folder through Explorer
Some computers don't show up in the SCCM console

Your inboxes\auth\ddm.box folder should be clean like the one below:

To count the number of DDRs, open powershell as administrator, browse to the ddm.box folder and type:

(Get-ChildItem).count

Note that depending on the number of items it can take several hours to complete.

In average SCCM process 1 DDR per second so as an example if you have 5 million DDRs it can take 2 months to process all of them (considering you disable all discovery methods!!)

I imagine you can't wait 2 months, can you?

That's how you fix it:

Adjust your Discovery Methods to a more realistic interval. You can leave the delta discoveries to 5 minutes but your full discoveries shouldn't be less than 7 days. If you need to decrease to less than 7 days, make sure you have the horse power for this and keep an eye in the ddm.box folder.
Browse to the SCCM folder\bin\X64 and open compmgr.exe.
Enter the site server name.
Expand Components, click on SMS_EXECUTIVE and in the right pane right click the SMS_EXECUTIVE and click on Query.
Right click it again and click on Stop.
Now from a command prompt or powershell go to the inboxes\auth folder and type:

ren ddm.box ddm.old

Go back to the Configuration Manager Service Manager and start SMS_EXECUTIVE.

Now SCCM will recreate the ddm.box folder and everything should start working as expected.

Install Adobe Photoshop Elements 11 silently

2014-04-09T19:29:00.000+10:00

First of all create a file named 'application.xml.override' with the content below and place one copy in the root folder of the source files and another copy under the folder ElementsOrganizer.

<?xml version="1.0" encoding="utf-8"?>
<Configuration>
<Payload>
<Data key="EULA">Supress</Data>
<Data key="Registration">Supress</Data>
<Data key="Updates">Supress</Data>
</Payload>
</Configuration>

Once you have the files and place just run the following command:

"Setup.exe" /UL1033 /V"SERIALNUMBER=XXXX-XXXX-XXXX-XXXX-XXXX-XXXX COUNTRY=183 AgreeToLicense=Yes"

The following is a list of all country codes:


Afghanistan	3
Albania	6
Algeria	4
American Samoa	10
Andorra	8
Angola	9
Anguilla	300
Antarctica	301
Antigua and Barbuda	2
Argentina	11
Armenia	7
Aruba	302
Ascension Island	303
Ashmore and Cartier Islands	304
Australia	12
Austria	14
Azerbaijan	5
Bahama, The	22
Bahrain	17
Baker Island	305
Bangladesh	23
Barbados	18
Belarus	29
Belgium	21
Belize	24
Benin	28
Bermuda	20
Bhutan	34
Bolivia	26
Bosnia and Herzegovina	25
Botswana	19
Bouvet Island	306
Brazil	32
British Indian Ocean Territory	114
Brunei	37
Bulgaria	35
Burkina Faso	245
Burundi	38
Cambodia	40
Cameroon	49
Canada	39
Cape Verde	57
Cayman Islands	307
Central African Republic	55
Chad	41
Channel Islands	308
Chile	46
China	45
Christmas Island	309
Clipperton Island	310
Cocos (Keeling) Islands	311
Colombia	51
Comoros	50
Congo	43
Congo (DRC)	44
Cook Islands	312
Coral Sea Islands	313
Costa Rica	54
Côte d’Ivoire	119
Croatia	108
Cuba	56
Cyprus	59
Czech Republic	75
Denmark	61
Diego Garcia	314
Djibouti	62
Dominica	63
Dominican Republic	65
East Timor	7299303
Ecuador	66
Egypt	67
El Salvador	72
Equatorial Guinea	69
Eritrea	71
Estonia	70
Ethiopia	73
Falkland Islands (Islas Malvinas)	315
Faroe Islands	81
Fiji Islands	78
Finland	77
France	84
French Guiana	317
French Polynesia	318
French Southern and Antarctic Lands	319
Gabon	87
Gambia, The	86
Georgia	88
Germany	94
Ghana	89
Gibraltar	90
Greece	98
Greenland	93
Grenada	91
Guadeloupe	321
Guam	322
Guantanamo Bay	323
Guatemala	99
Guernsey	324
Guinea	100
Guinea-Bissau	196
Guyana	101
Haiti	103
Heard Island and McDonald Islands	325
Honduras	106
Hong Kong S.A.R.	104
Howland Island	326
Hungary	109
Iceland	110
India	113
Indonesia	111
Iran	116
Iraq	121
Ireland	68
Israel	117
Italy	118
Jamaica	124
Jan Mayen	125
Japan	122
Jarvis Island	327
Jersey	328
Johnston Atoll	127
Jordan	126
Kazakhstan	137
Kenya	129
Kingman Reef	329
Kiribati	133
Korea	134
Kuwait	136
Kyrgyzstan	130
Laos	138
Latvia	140
Lebanon	139
Lesotho	146
Liberia	142
Libya	148
Liechtenstein	145
Lithuania	141
Luxembourg	147
Macau S.A.R.	151
Macedonia, Former Yugoslav Republic of	19618
Madagascar	149
Malawi	156
Malaysia	167
Maldives	165
Mali	157
Malta	163
Man, Isle of	15126
Marshall Islands	199
Martinique	330
Mauritania	162
Mauritius	160
Mayotte	331
Mexico	166
Micronesia	80
Midway Islands	21242
Moldova	152
Monaco	158
Mongolia	154
Montserrat	332
Morocco	159
Mozambique	168
Myanmar	27
Namibia	254
Nauru	180
Nepal	178
Netherlands Antilles	333
Netherlands, The	176
New Caledonia	334
New Zealand	183
Nicaragua	182
Niger	173
Nigeria	175
Niue	335
Norfolk Island	336
North Korea	131
Northern Mariana Islands	337
Norway	177
Oman	164
Pakistan	190
Palau	195
Palestinian Authority	184
Palmyra Atoll	338
Panama	192
Papua New Guinea	194
Paraguay	185
Peru	187
Philippines	201
Pitcairn Islands	339
Poland	191
Portugal	193
Puerto Rico	202
Qatar	197
Reunion	198
Romania	200
Rota Island	340
Russia	203
Rwanda	204
Saipan	341
Samoa	259
San Marino	214
São Tomé and Príncipe	233
Saudi Arabia	205
Senegal	210
Serbia and Montenegro	269
Seychelles	208
Sierra Leone	213
Singapore	215
Slovakia	143
Slovenia	212
Solomon Islands	30
Somalia	216
South Africa	209
South Georgia and the South Sandwich Islands	342
Spain	217
Sri Lanka	42
St. Helena	343
St. Kitts and Nevis	207
St. Lucia	218
St. Pierre and Miquelon	206
St. Vincent and the Grenadines	248
Sudan	219
Suriname	181
Svalbard	220
Swaziland	260
Sweden	221
Switzerland	223
Syria	222
Taiwan	237
Tajikistan	228
Tanzania	239
Thailand	227
Tinian Island	346
Togo	232
Tokelau	347
Tonga	231
Trinidad and Tobago	225
Tristan da Cunha	348
Tunisia	234
Turkey	235
Turkmenistan	238
Turks and Caicos Islands	349
Tuvalu	236
Uganda	240
Ukraine	241
United Arab Emirates	224
United Kingdom	242
United States	244
Uruguay	246
Uzbekistan	247
Vanuatu	174
Vatican City	253
Venezuela	249
Vietnam	251
Virgin Islands	252
Virgin Islands, British	351
Wake Island	258
Wallis and Futuna	352
Yemen	261
Zambia	263
Zimbabwe	264