Cultural Practices

• Setting up your environment
• Development
• Provisioning for Dev/Test
• Deployment
• Provisioning a Continuous Integration (CI) server
• CI and CD with Jenkins
• Scaling the app

Parts Unlimited

VorlonJS – A Journey to DevOps

Wait, there’s more

• DevOps Fundamentals – to get you started with DevOps practices
• Publishing node.js web apps to Azure – Using Visual Studio Team Services
• Node.js app, js test, Travis CI integration – Example web app with deployment scripts to Azure web app

• The inspiring and entertaining keynote by Alaska Airlines CIO Veresh Sita discussing how Alaska is transforming the customer’s airplane experience (it made me proud that we share greater Seattle as our companies’ headquarters).
• GE Digital’s story on how they’re transforming GE’s extensive portfolio of industrial products with software and ServiceOps.
• How Westpac Bank is delivering ideas at velocity to stay ahead of the curve in the constantly evolving financial services industry.

• Industry analyst Ben Kepes provides insights on how DevOps can help drive organizational agility and innovation.
• RunasRadio podcaster Richard Campbell discusses the strained dynamic between devs and ops and how to turn it around to create high-performance, collaborative teams.
• DevOps evangelist Cads Oakley describes how to jumpstart a DevOps initiative within your organization.

• Chef Principal Engineer and Microsoft MVP Steven Murawski, explaining how, by using Chef and PowerShell Desired State Configuration (DSC) together, you can streamline the change management process and successfully deploy code and infrastructure on-demand and in-compliance.
• Microsoft engineers Narayanan Lakshmanan and Boris Scholl elaborating on the technical details around DSC, microservices, and DevOps.
• Nirmal Mehta, Chief Technologist at Booz Allen Hamilton, expounding on how you can accelerate compliance using Infrastructure as Code.

Diving In

> ssh docker@xxx.xxx.xxx.xxx

Welcome to Docker!

dockerswarm-manager0:~$

dockerswarm-manager0:~$ docker -v

Docker version 1.12.0, build 8eab29e, experimental

dockerswarm-manager0:~$ docker node ls

ID                           HOSTNAME                    STATUS  AVAILABILITY  MANAGER STATUS

6p1xtdlrfwf3suzoqzws3fldh    _dockerswarm-worker-vmss_0  Ready   Active

bmzw0jiufycg23p3u3pfixsih *  _dockerswarm-manager0       Ready   Active        Leader

dockerswarm-manager0:~$ docker images

REPOSITORY             TAG                   IMAGE ID            CREATED             SIZE

docker4x/agent-azure   latest                b2b22beefac8        4 days ago          94.13 MB

docker4x/init-azure    azure-v1.12.0-beta4   dd6652cf2f87        7 days ago          35.32 MB

docker4x/controller    azure-v1.12.0-beta4   d61704e07424        10 days ago         22.46 MB

docker4x/guide-azure   azure-v1.12.0-beta4   8dca840e0fc0        2 weeks ago         35.15 MB

dockerswarm-manager0:~$ docker ps

CONTAINER ID        IMAGE                                      COMMAND

eec4385cd9a4        docker4x/controller:azure-v1.12.0-beta4    "loadbalancer run --d"

334c9369735a        docker4x/guide-azure:azure-v1.12.0-beta4   "/entry.sh"

c50f292b7dc6        docker4x/agent-azure                       "supervisord --config"

• Docker4x/agent-azure: this is Azure Linux Agent which "manages interaction between a virtual machine and the Azure Fabric Controller". This is basically responsible for communication between the VM and Azure for diagnostics, provisioning etc.
• Docker4x/guide-azure: I did not find a lot of info on this one. It is running a cron task that call `buoy` , a custom tool, which seem to be used for logging important events in the swarm.
• Docker4x/controller: This is the most interesting one. It is running `loadbalancer`, another custom tool from docker (which doesn't seem to be open source at the time of this writing). This container is actually managing the load balancing rules of our external load balancer, more about that following.

dockerswarm-manager0:~$ docker service create --name nginx --publish 80:80 nginx

dockerswarm-manager0:~$ docker service ps nginx

ID                         NAME     IMAGE  NODE                        DESIRED STATE  CURRENT STATE           ERROR

6lnn6ucp4hg0f5l3dnm47fluq  nginx.1  nginx  _dockerswarm-worker-vmss_0  Running        Running 13 minutes ago

• Docker Swarm information page
• The Containers Channel video series on @ch9

The current state of things (spoiler: git-flow)

Enter GitHub Flow

Making large-scale changes

• Feature flags: Split your work in smaller self contained changes and release them behind a feature flag. You can still get true CI and fast feedback from a staging environment for example, without exposing your work in progress in production.
• Branch by abstraction: We branch so our modifications are isolated and don't impact others and vice-versa. Developers already have another tool do the job: abstraction. Hide the component you want to refactor behind an interface and incrementally swap the old implementation with the new. You should still be able to release in production at anytime without breaking anything. Here is a nice read on the subject by Martin Fowler.
• Split your work (microservices): This won't apply to most cases, but when creating a new component, ask yourself if you could make it stand alone. A series of smaller and independent components are much easier to change safely.

Going One step further

• Deploying before merging: GitHub deploys in production directly from the feature branch and if everything looks good, it is then merged into master. This means that master becomes a record of known good production code.
• Scheduled release: GitHub.com is deployed continuously, but GitHub Enterprise's updates are shipped every few weeks. To achieve that, GitHub creates release branches from master (say release-2.6) and cherry-pick the features they want to ship in that version. No development happens on this release branch.

When GitHub Flow doesn't make sense

• The excellent "Understanding the GitHub-flow" by GitHub themselves.
• "Enabling Trunk Based Development with Deployment Pipelines" by Vishal Naik from ThoughtWorks

• DevOps Practices
• DevOps Fundamentals Video Series
• DevOps Getting Started Cheat Sheet

• Read more from Gene Kim, co-author of The Phoenix Project and the upcoming DevOps Handbook
• Read the State of DevOps Report
• Read more from Jez Humble, author of Continuous Delivery and Lean Enterprise
• Read Large Scale Agile by Gary Gruver
• Visit Simon Wardly's blog to learn more about organization and cell structure
• Visit The ITSM Crowd channel for regular chats on all things service management

We all know there is no way you can buy DevOps-in-a-Box, but we thought it should be possible to at least produce DevOps experts, off-the-shelf, factory style. How did this factory come about and how would a DevOps factory do that? Let me explain.

True to a DevOps approach we setup a multi-disciplinary team of subject matters experts and had them brainstorm and build this new virtual factory from ground up. No pre-conditions for the team except two: It had to be a DevOps learning experience spanning Microsoft and OSS tools and products and it had to be a fun experience. Out came the DevOps Factory: A gaming, competition, and learning experience enabling people to learn about a diverse set of technologies and practices around DevOps.

As much as learning new stuff always is fun, we wanted it to be more. So we added a competitive angle to the experience of the DevOps Factory. And while you check out the different floors and rooms of the factory, you will learn new things at every step.

Learn and Earn

The whole factory consists of different floors and each floor has a number of practice rooms. Before you leave any one room you want to test your knowledge and earn well deserved points. While you wander throughout the factory floors, automated factory staff will make it easy for you to collect points and pick up badges to demonstrate your growing knowledge and expertise in DevOps practices. At any time you will be able to see how you stack up against peers and compare your results, points, and badges.

Later on, before you leave, don’t forget to check your point balance. While it is rewarding and fun to learn and compete, the points you collect are also worth something else. The DevOps Factory is connected to a rewards system that turns your points into real world rewards like gift cards, concert tickets, software, devices, and more. If you register with the factory, you will keep your points across subsequent visits to the practice rooms.

The Tour

The factory consists of several production floors where individual rooms on each floor are associated with key DevOps practices. Let me take you on a tour through the factory. The current iteration covers the following practices in detail:

Automated Testing

Conveniently located on the second floor, in Automated Testing, you learn about the benefits, tools, and best practices automating test infrastructure provides. Automated tests allow for repeatable test and comparable results among other benefits. Learn about Automated Testing on the second floor.

Continuous Integration

Sharing code among developers of the same and/or different teams requires a common repository being used. Checking in code allows for automating builds and testing. This and more you learn at the factory’s ground level.

Infrastructure as Code

Are your releases delayed due to inadequate error tracking and version control? You might want to look at the DevOps practice, Infrastructure as Code, conveniently located on ground level, next to Continuous Integration.

Application Performance Management

What do you know about your apps performance? Are customers happy with the response time? Is your app performance diagnosis time consuming and incomplete? Check and learn about APM on level 2.

Continuous Deployment

Pushing a new known good build to a single environment automatically via automation capabilities removes human error and enables for faster and more predicable outputs to dev, test, and even production environments. The factory’s third floor is all over this.
By the way, don’t confuse Continuous Deployment and Continuous Delivery. They are often confused – partially because people tend to throw around those terms interchangeably.

Release Management

We have dedicated a high ceiling floor to this. Release Management, RM for short, is about managing, planning, and controlling builds throughout the stages of the software lifecycle. RM includes things like testing and deploying of software releases. Lots of good materials to look at in this section.

Configuration Management

Last but not least, Configuration Management. On the top floor of our little factory you will dive into details about how to establish and maintain consistency of a product, tool, environment, or service over its lifetime.

In Closing

Our factory will grow over time and we are constantly working on new and improved content. I would be very interested in your feedback, ideas and suggestions. Please use the comment section of this post to share.
And now go and hit the factory floor.

Have fun,

@volkerw

Introduction

Configure a Linux Build Agent

curl -skSL https://aka.ms/xplatagent | bash

./run.sh

• username: this field is ignored when using personal access toke, so you can type any random username
• password: enter your personal access token
• agent name: the name of the agent (will be used to display the agent the VSTS agent queues)
• pool name: you can leave default
• server url: your VSTS account URL (https://youraccount.visualstudio.com)

Create the build definition to create a Docker image

# use the node argon (4.4.3) image as base FROM node:argon # Set the Vorlon.JS Docker Image maintainer MAINTAINER Julien Corioland (Microsoft, DX) # Expose port 1337 EXPOSE 1337 # Set the entry point ENTRYPOINT ["npm", "start"] # Create the application directory RUN mkdir -p /usr/src/vorlonjs # Copy the application content COPY . /usr/src/vorlonjs/ # Set app root as working directory WORKDIR /usr/src/vorlonjs # Run npm install RUN npm install

#!/bin/bash # get version from package.json appVersion=$(cat package.json | jq -r '.version') echo "Building Docker Vorlon.JS image version $appVersion" docker build -t vorlonjs/dashboard:$appVersion . docker login --username="$1" --password="$2" echo "Pushing image..." docker push vorlonjs/dashboard:$appVersion docker logout exit 0

docker run -d -p 80:1337 vorlonjs/dashboard:0.2.1

This post is part of the series “VorlonJS – A Journey to DevOps”

In the previous post of this series we discussed about how Visual Studio Team Services and Release Management can be used to automate the creation of environments and the deployment of an application in these different environments.

If you’ve gone deeper into the Azure Resource Management templates that are used to deploy Vorlon.JS you may have noticed that we are using a different template for the production environment. This is because we are not deploying de application directly in the production web app but we are using a really nice feature of Azure Web App called “deployment slots”.

What are deployment slots?

Deployment slots are available if you are using the Standard or Premium App Service application plan. This feature allows to be able to deploy an application on a separate slot than the production one. Technically, a slot is just another web app running on the same service plan, with its own URL.

Using slots enable features like instant swapping or tests in production. Swapping is a really useful feature that make possible the upgrade to a new version of an application without service interruption / down time.

When deploying a new application, you always have a warm up time that could be more or less important and could impact the experience of your users. Using slots, it’s possible to deploy a new version of the application on a preview or staging slot, boot the application and then use the swap function that will swap the virtual IP address of the slots at the load balancers configuration level.

In the case of Vorlon.JS, the ARM template that describes the production environment is responsible for creating a Web App slot named staging and we use Release Management to deploy the application on this slot. After a deployment has occurred, we have two applications running in the production web app:

In this example, the production of Vorlon.JS (version 1.4) is deployed and used on the production slot of the Azure Web App named vorlonjs-production and accessible at http://vorlonjs-production.azurewebsites.net. The new version is 1.5 has been deployed on the staging slot and is accessible at http://vorlonjs-production-staging.azurewebsites.net.

Once the application is booted and we are done with our last checks on the staging slot we use the swap function to make the 1.5 version available on the production URL. As explained before, swapping does not deploy the 1.5 version to the production slot but only update the network configuration so it’s done in only a few seconds with no down time!

The swap function is available in the Azure Portal or using the command line (PowerShell or Azure Cross Platform CLI):

What are tests in production?

Test in production is a common practice that consist to redirect (transparently) a bunch of application’s users to a new version to get more feedback before making the application available for all the users.

With Azure App Service and deployment slots, implementing test in production is super easy. Once you have two versions of an application on different slots (see 1.4 and 1.5 versions of Vorlon.JS above), you just have to go in the Azure portal, in the routing settings and select the Traffic Routing option:

Then, you can configure Azure Web App routing to automatically redirect a percentage of users to another slot. For example, you can choose to redirect 30% of your production traffic on the staging slot so 30% of your regular users will use the new version of the application:

Note: once a user has been redirected to a given slot using traffic routing, a cookie is automatically set to ensure that he will not be redirected to another version the next time its browser sends an http request.

Conclusion

As explained in this article, Azure App Service deployment slots and traffic routing are really nice features to simplify the way you can upgrade an application with no service interruption. Tests in production, used with technologies like Azure Application Insights that allows to get information and metrics about application usage can be really helpful to determinate if a new version of the application will be really appreciated by the users or not.

Julien