Latest Articles from Joe Pruitt | F5 DevCentral

Getting Started with iControl: History

Mon, 13 Jun 2016 04:00:00 -0700

tl;dr - iControl provides access to BIG-IP management plane services through SOAP and REST API interfaces.

The Early Days

iControl started back in early 2000. F5 had 2 main products: BIG-IP and 3-DNS (later GTM, now BIG-IP DNS). BIG-IP managed the local datacenter's traffic, while 3-DNS was the DNS orchestrator for all the BIG-IP's in numerous data centers. The two products needed a way to communicate with each other to ensure they were making the right traffic management decisions respective to all of the products in the system. At the time, the development team was focused on developing the fastest running code possible and that idea found it's way into the cross product communication feature that was developed. The technology the team chose to use was the Common Object Request Broker Architecture (CORBA) as standardized by the Object Management Group (OMG). Coming hot off the heels of F5's first management product SEE-IT (which was another one of my babies), the dev team coined this internal feature as "LINK-IT" since it "linked" the two products together.

With the development of our management, monitoring, and visualization product SEE-IT, we needed a way to get the data off of the BIG-IP. SEE-IT was written for Windows Server and we really didn't want to go down the route of integrating into the CORBA interface due to several factors. So, we wrote a custom XML provider on BIG-IP and 3-DNS to allow for configuration and statistic data to be retrieved and consumed by SEE-IT.

It was becoming clear to me that automation and customization of our products would be beneficial to our customers who had been previously relying on our SNMP offerings. We now had 2 interfaces for managing and monitoring our devices: one purely internal (LINK-IT) and the other partially (XML provider). The XML provider was very specific to our SEE-IT products use case and we didn't see a benefit of trying to morph that so we looked back at LINK-IT to see what we could to do make that a publicly supported interface. We began work on documenting and packaging it into F5's first public SDK.

About that time, a new standard was emerging for exchanging structured information. The Simple Object Access Protocol (SOAP), which allows for structured information exchange, was being developed but not fully ratified until version 1.2 in 2003. I had to choose to roll our own XML implementation or make use of this new proposed specification. There was risk as the specification was not a standard yet but I made the choice to go the SOAP route as I felt that picking a standard format would give us the best 3rd party software compatibility down the road. Our CORBA interface was built on a nice class model which I used as a basis for an SOAP/XML wrapper on top of that code. I even had a great code name for the interface: X-LINK-IT! For those who were around when I gave my "XML at F5" presentation to F5 Product Development, you may remember the snide comments going around afterwards about how XML was not a great technology and a big mistake supporting. Good thing I didn't listen to them...

At this point in mid-2001, the LINK-IT SDK was ready to go and development of X-LINK-IT was well underway. Well, let's just say that Marketing didn't agree with our ingenious product naming and jumped in to VETO our internal code names for our public release. I'll give our Chief Marketer Jeff Pancottine credit for coining the term "iControl" which was explained to me as "Internet Control". This was the start of F5's whole Internet Controlled Architecture messaging by the way. So, LINK-IT and X-LINK-IT were dead and iControl CORBA and iControl SOAP were born.

The Death of CORBA, All Hail SOAP

The first version of the iControl SDK for CORBA was released on 5/25/2001 with the SOAP version trailing it by a few months. This was around the BIG-IP version 3 time frame. We chugged along for a few years through the BIG-IP version 4 life and then a big event occurred that was the demise for CORBA - well, it's actually 2 events. The first event was the full rewrite of the BIG-IP data plane when TMOS was introduced in BIG-IP, version 9 (we skipped from version 4 to version 9 for some reason that slips my mind). Since virtually the entire product was rewritten, the interfaces that were tied to the product features, would have to change drastically. We used this as an opportunity to look at the next evolution of iControl. Until this point, iControl SOAP was just a shim on top of CORBA and it had some performance issues so we worked at splitting them apart and having SOAP talk directly to our configuration engine. Now we had 2 interface stacks side by side. The second event was learning we only had 1 confirmed customer using the CORBA interface compared to the 100's using SOAP. Given that knowledge and now that BIG-IP and 3-DNS no longer used iControl CORBA to talk to each other, the decision was made to End of Life iControl CORBA with Version 9.0. But, iControl SOAP still used the CORBA IDL files for it's API definitions and documentation so fun trivia note: the same CORBA tools are still in place in today's iControl SOAP build tools that were there in version 3 of BIG-IP. I'm fairly sure that is the longest running component in our build system.

The Birth of DevCentral

I can't speak about iControl without mentioning DevCentral. We had our iControl SDK out but no where to directly support the developers using it. At that time, F5 was a "hardware" company and product support wasn't ready to support application developers. Not many know that DevCentral was created due to the popularity of iControl with our customer base and was born on a PC under my desk in 2003. I continued to help with DevCentral part time for a few years but in 2007 I decided to work full time on building our community and focusing 100% on DevCentral. It was about this time that we were pushing the idea of merging the application and infrastructure teams together - or at least getting them to talk more frequently. This was a precursor to the whole DevOps mentality so I'd like to think we were a bit ahead of the curve on that.

Enter iControl REST

In 2013, iControl was reaching it's teenage years and starting to show it's age a bit. While SOAP is still supported by all the major tool vendors, application development was shifting to richer browser-based apps. And with that, Representational State Transfer (REST) was gaining steam. REST defined a usage pattern for using browser based mechanisms with HTTP to access objects across the network with a JavaScript Object Notation (JSON) format for the content. To keep up with current technologies, the PD team at F5 developed the first REST/JSON interface in BIG-IP version 11.5 as Early Access and was made Generally Available in version 11.6. With the REST interface, more modern web paradigms could be supported and you could actually code to iControl directly from a browser! There were also additional interface based tools for developing and debugging built directly into the service to give service listings and schema definitions.
At the time of this writing, F5 supports both of the main iControl interfaces (SOAP and REST) but are focusing all new energy on our REST platform for the future. For those who have developed SOAP integrations, have no fear as that interface is not going away. It will just likely not get all the new feature support that will be added into the REST interfaces over time.

SDKs, Toolkits, and Libraries

Through the years, I've developed several client libraries (.Net, PowerShell, Java) for iControl SOAP to assist with ramp-up time for initial development. There have also been numerous other language based libraries for languages like Ruby, PHP, and Python developed by the community and other development teams. Most recently, F5 has published the iControl library for Python which is available as part of our OpenStack integration. DevCentral is your place to for our API documentation where we update our wiki with API changes on each major release. And as time rolls on, we are adding REST support for new and existing products such as iWorkflow, BIG-IQ, and other products yet to be released that will include SDKs and other reference material.

F5 has a strong commitment to our end users and their automation and integration projects with F5 technologies. Coming full circle, my current role is overseeing APIs and SDKs for standards, consistency, and completeness in our Programmability and Orchestration (P&O) team so keep a look out for future articles from me on our efforts on that front. And REST assured, we will continue to do all we can to help our customers move to new architectures and deployment models with programmability and automation with F5 technologies.

DevOps 101 - Automation

Thu, 24 Sep 2015 13:08:31 -0700

In my last few articles, I've begun the discussion on the 6 pillars of DevOps and have covered Management, Integration, Communication, and Collaboration:

The Fifth Pillar

Next on the list is Automation. Automation is defined as:

"The technique, method, or system of operating or controlling a process by highly automatic means, as by electronic devices, reducing human intervention to a minimum."

Back in the day when deployment meant copying code to the one server you had connected directly to the internet, you could get away with manual processes, but it was still fraught with potential issues. That is definitely not true with today's rats nest of wires, servers, virtual machines, proxies, load balancers, firewalls, and all the other components needed to host a scalable and secure network.

In the early 2000's when I developed iControl (F5's first web service based management interface), I saw the pain that developers and network ops faced when dealing with application deployment. I talked about this in my "DevOps - A Brief History of Time" article I wrote earlier this year. Developers needed a better way to get their code pushed to the end user and Network ops needed a better way to understand how the applications worked and trust the development team to be a part of the deployment process. I talked a lot about the brick wall the existed between Dev and Ops and that automation was critical in leveraging the talents from both sides.

Automation - The Glue That Binds Dev to Ops

It's taken over 10 years, but DevOps has been the catalyst to blowing up that brick wall and coordinating the two teams together. But, network ops would never hand the keys to the network kingdom over to the developers so something was going to have to be put in place that was testable, reliable, and free from the risk of fat-fingering a config option that could take down the network.

Automation is the key to providing the trust and reliability into the equation and acts as the glue that binds the two teams together during the production phase. Automation fits into all aspects of the deployment process including development, testing, deployment, and all of the sub-processes within. Care needs to be made to ensure that all components in your system can be automated via software integrations or APIs. The build system, the testing tools, and the network components (load balancers, firewalls, application delivery devices, security appliances, etc) all need to be able to be scripted or programmed.

When dealing with 2 or 2000 developers, tools are needed to ensure that the collaborative work done can be integrated with success to achieve the desired results. Continuous Integration is the process of merging all development code into a shared codebase on frequent intervals. Merges aren't complete with out validation to ensure the merge didn't break anything so integrated testing is also a critical component. When merges occur, automated tests are triggered so that the developers will know right away if their work needs a little bit more TLC . When check-ins occur to the revision control system, automated build servers compile the code and run the automated tests. In addition to functional testing, performance and optimization testing is also occurring during this process.

Build tools

For the system to flow together, a centralized build system is needed that will interact with the various test and deployment systems. Vendors/Tools in the "Build" space are

Automated Testing

Integrating testing into the development and build process both locally and at the central build server is done to ensure an issue-free code base. Tools in this space include

Continuous Integration

CI should occur frequently enough that no errors can arise in the compilation or testing of the code without developers noticing and correcting them immediately. Typical practice is to trigger a build every time code is committed to a revision control system, rather than periodic scheduled builds. To achieve this, CI relies on a code repository, build automation, build self-testing, commits auto-build, and having a replica of the production environment to perform tests. Vendors and tools in the CI space include:

Continuous Delivery

Continuous Delivery (CD) is the process of building a deployment pipeline where a set of tests are performed that must pass before a piece of software is allowed to be released. Teams produce software updates in short cycles to ensure the software can be reliably released at any time. CD builds on CI by adding regular deployments to production as part of the build process. A healthy CD process aids in accelerated time to market, improved productivity, reliability, quality, and customer satisfaction. Vendors in the CD space include:

It Doesn’t Stop There

Automation can take place at virtually every step of the application delivery process. A successful implementation of automation within the DevOps methodology will lead to better products and thus much happier customers. For those just tackling DevOps, start small by automating with scripts manual processes that are time consuming. Then move into software solutions that assist with the build/test/deploy areas, then look on how to tie them together. The hard part is going to be picking which solutions to use to achieve this goal but luckily there are many vendors and products in this space so it’ll be just a matter of you finding what works for your environment.

Node.js ABC’s - P is for POST

Mon, 21 Sep 2015 14:19:05 -0700

With the web services that are popping from your corporate networks to the wild-internet, accessing HTTP based resources is becoming a critical component to any and all application development. One of the more popular API application design styles is Representational State Transfer (or REST). REST implementations are typically developed on top of HTTP with the objects you are effecting being in the URI and the actions, or verbs, being the HTTP methods. The HTTP methods follow the four basic methods of persistent storage as referenced in the CRUD acronym of Create, Read, Update, and Delete acronym.

The Verbs (Actions)

POST - Create new object
GET - Read the contents of an existing object
PUT - Update an existing object
DELETE - Delete an object

The Nouns (Objects)

While the HTTP method defines the action you take, the URI defines the object you wish to act on. The URI path can be used to specify hierarchy as illustrated in the following set of resources

POST /article - create an article
PUT /article/10 - update article #10
POST /article/10/comment - create a comment on article #10
DELETE /article/10/comment/1 - delete comment #1 on article #10

In the above examples, the id's of the objects were included in the URI paths to identify the various specific objects. This is a typical pattern in API design but it's also possible to include those values as part of the payload in the requests

Sample Client and Server

If you are implementing a Node.js based solution, you will likely come into a spot where you need to write either a HTTP client, HTTP server, or both. While the different verbs have subtle differences in how you pass data (GET and DELETE pass parameters in the URI, while POST and PUT utilize payload data within the request), I'll focus this article on how to build a simple client and server that accept POST data (for new object creation).

The Server (http-server.js)

The server code for this article is a basic HTTP listener that supports both GET and POST requests. It logs the request content and then returns a message to the calling application.

var http = require('http');

function requestHandler(request, response) {
  console.log("--------------------------");
  console.log(request.method + ": " + request.url);
  if ( request.method == "POST" ) {
    request.setEncoding('utf8');
    var data = "";
    request.on("data", function(chunk) {
      data += chunk;
    });
    request.on("end", function() {
      console.log("DATA: " + data);
    });
  }
  response.end("Thank you for " + request.method + "ing the uri '" + request.url + "'");
}

var server = http.createServer(requestHandler);

server.listen(8080, function() {
  console.log("Listening on http://localhost");
});

The Client (http-client.js)

The client application illustrates how to make a simple GET request and a POST request with payload data. The GET request is meant to retrieve a listing of articles, and the POST request will create a comment on an existing article.

var http = require("http");

// GET
var get_options = {
  host: "localhost",
  port: 8080,
  path: "/article"
};

http.get(get_options, function(response) {
  response.setEncoding("utf8");
  console.log("Response status code: " + response.statusCode);
  response.on("data", function(data) {
    console.log("Data: " + data);
  });
});

// POST
var post_data = "{'content': 'Blah Blah Blah'}";
var post_options = {
  host: "localhost",
  port: 8080,
  path: "/article/10/comment",
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    "Content-Length": post_data.length
  }
};

var request = http.request(post_options, function(response) {
  response.setEncoding("utf8");
  console.log("Response status code: " + response.statusCode);
  response.on("data", function(data) {
    console.log("Data: " + data);
  });
});
request.write(post_data);
request.end();

Running the code

The client and the server programs can be run by loading them with the node runtime

$ node http-server.js
Listening on http://localhost
--------------------------
GET: /article
--------------------------
POST: /article/10/comment
DATA: {'content': 'Blah Blah Blah'}

$ node http-client.js
Response status code: 200
Data: Thank you for GETing the uri '/article'
Response status code: 200
Data: Thank you for POSTing the uri '/article/10/comment'

Conclusion

With the built-in http node package, calling a HTTP POST method is very seamless and you can be well on your way to building a client or server as part of your architectural solution.

Node.js ABC’s - O is for Object

Wed, 16 Sep 2015 15:12:42 -0700

Objects are one of the core components of the JavaScript language and something that you will interact with every time you look at a pice of JavaScript code. The easiest way to explain what a JavaScript Object is would be to compare it to an "object" in real life. In JavaScript, an Object is a single entity with properties and a type. Compare that to a car. A car is a standalone object with properties too. A car has weight, design, color, materials, passenger capacity, etc. In the same way, a JavaScript object can have it's own set of properties which define it's characteristics.

Constructing and Object

There are several ways to create an object from scratch. You can use the Object constructor with the new operator or you can use the alternate object literal syntax by using curly-braces. These are called "Object Initializers".

Object Initializers

// create object from Object's constructor
var car = new Object();
// create object with object literal syntax
var car = {};

Object literal syntax is almost always preferred as it allows you to easily pre-populate the Object with properties. The following example creates a new "car" object that represents a blue 2015 Honda Accord.

var car = {
  make: "Honda",
  model: "Accord",
  year: 2015,
  color: "blue"
};

Object Literal Notation looks a lot like JSON (JavaScript Object Notation) but differs in that wrapping property names in single or double quotes is optional, while with JSON it's mandatory. Certain JSON libraries allow for single quotes, but for broadest compatibility, I'd suggest using double quotes whenever possible.

This alternate declaration from above is both valid Object Literal Notation as well as JSON.

var car = {
  "make": "Honda",
  "model": "Accord",
  "year": 2015,
  "color": "blue"
};

Constructor Functions

To use this method, you create a function named the type of Object you want, and in the function body, assign property values to the "this" property. The following example creates a "Car" constructor function and creates a new Car object of make "Honda".

// create a "Car" constructor function
function Car(make) {
  this.make = make
};
var car = new Car("Honda");

The Object.create Method

Objects can also be created using the "Object.create()" method. This method behaves similarly to the constructor function, but allows you to choose a prototype object for the object without defining a constructor function.

var Car = {
  "make": "Unknown",
  "getMake": function() { return this.make; }
};
var honda = Object.create(Car);
honda.make = "Honda";

Object Properties

As you have seen above, a JavaScript object can have properties associated with it. Think of a property of an object as a variable attached to it. Object properties are just like variables except that they are scoped within the context of that object. In the above example, I created a "car" object with "make", "model", "year", and "color" properties.

Properties are not limited to strings and integers. They can be any JavaScript type object including Functions and Objects themselves.

var interior = {
  "upholstery": "leather",
  "color": "tan",
  "mud-guards": "all-weather"
};
var car = {
  "make": "Honda", // String
  "model", "Accord", // String
  "year", 2015, // Integer
  "color": "blue", // String
  "interior": interior, // Object
  "mileage": function() { // Function
    return 12345;
  }
}

Adding Properties To An Existing Object

If you have an existing object and would like to add a property to it, it's as simple as assigning it a value with the "dot" or "bracket" notation

car.transmission = "manual";
car["sunroof"] = true;

An Object property name can be a string, or anything that can be dynamically converted to a string. But, if the property name starts with, or contains special characters, you must use the "bracket" notation.

var prop_name = "1CoolProperty";
car[prop_name] = "some value";

Enumerating Properties Of An Object

There are three native ways to enumerate object properties:

"for..in" loops which traverse all enumerable properties of an object (and it's prototype chain)
The "Object.keys(o)" method which returns an array with the enumerable properties of the object (not the prototype chain)
The "Object.getOwnPropertyNames(o)" method which returns an array with the property names of an object.

Below is an example of iterating through all the properties of an object (including it’s inheritance chain) and filtering out only those properties that belong directly to that object.

for (var prop_name in car) {
  if (car.hasOwnProperty(prop_name)) {
    console.log(prop_name + " = " + car[prop_name] + "\n");
  }
}

Inheritance and Prototypes

Above I mentioned the word "prototype". All Objects in JavaScript inherit from at least one other object. This inherited object is known as the prototype and all inherited prototypes can be accessed with the "prototype" object of the constructor. Discussing prototypes and inheritance is more than a single article in itself, so I'll point you to the Mozilla documentation on Inheritance and the Prototype Chain for further reading on that topic.

Deleting Properties

You can remove a property from an object by using the "delete" operator. Keep in mind that you can only delete properties that are in scope with the current object (not inherited). The following code shows you how to remove a property:

var car = {
  "make": "Honda",
  "model": "Accord",
  "year": 2015,
  "color": "blue"
};
delete car.color; // Deletes the color property for the car object.

DevOps 101 - Collaboration

Thu, 03 Sep 2015 10:02:48 -0700

In my previous articles on the history of DevOps, I outlined the following pillars of the DevOps methodology

Management
Integration
Communication and information sharing
Collaboration
Automation
Measurement

The Fourth Pillar

The fourth pillar is Collaboration. Collaboration is defined as:

"working with others to do a task and to achieve shared goals."

In traditional separated organizations, there is not often collaboration between the design, development, testing (QA), and deployment (IT) functions that go with the application delivery lifecycle.

As I mentioned in my last article, Communication is essential for successful for cross-departmental teams to work together effectively to deploy applications. But, without collaboration tools to put some structure and process around those communications, it will be very difficult to get a project completed on time, on schedule, and with any degree of accuracy. Collaboration tools typically fall into three categories depending on the levels of collaboration:

Communication - An unstructured exchange of information such as email, a phone call, or hallway conversation.
Conferencing - Interactive work, such as brainstorming or voting, that leads to a shared goal.
Coordination - Complex interdependent work that leads to a combined goal.

Collaboration tools

The main shared attribute of tools in the collaboration space is that of helping to facility and manage group activities. From electronic calendars, workflow systems, knowledge management system, social software, and project management systems, there are literally thousands of tools that fall into the category of collaboration.

Below is a list of some of the vendors in the collaboration space.

Asana - plan and manage projects and tasks.
Basecamp - to-do lists, web-based text documents, milestone management, file sharing and time tracking.
Brightpod - task management, milestone tracking, task lists, file sharing, workflows, calendar, and messaging.
CollaborateCloud - social work management with workflows, forms, and virtual workrooms.
Confluence - enterprise unified wiki with social network notifications and integration of multimedia files.
DeskAway - team based project and task management, milestones, documentation, and file sharing.
Github - distributed revision control and source code management.
Intellinote - meeting collaboration, task management, and idea capturing.
Jive - online communities, micro-blogging, social networking, discussion forums, wikis, and instant messaging.
LiquidPlanner - online project management.
SharePoint - content and document management, personal cloud, social networking, enterprise search, and workflow management.
Trello - project management.
Wrike - project management and work collaboration.
Zoho - word processor, spreadsheets, calendar, contacts, and CRM.

Node.js ABC’s - N is for npm

Tue, 01 Sep 2015 15:32:18 -0700

Whether you are ready to start coding your first Node.js project, or if you are a seasoned veteran and need finish up a new exciting project, odds are you will need some functionality that has been written many times before by others. One of the beauties of the internet is the ability to create and share things. Node.js is a great language set in itself, but one key selling point is the vast amount of code that is available for free download.

The Node Package Manager, also known as npm, is a software system that automates the process of installing, upgrading, configuring, and removing Node.js packages. As of Node.js version 0.6.3, npm is bundled and installed with the environment so once you have access to the node command line tools, you also have access to the vast number of shared packages available to you.

As of this writing, there are over 180,000 packages available in the npm. You can find anything from credit card validators, to full language parsers. If you have a task to perform, odds are there is a 3rd party package to help you along. And the best feature I see is that packages are distributed as source script so you have full access to modify or extend the packages you pull from the repository.

In my article on Loading, I discuss how you use the "require" command to load a module

Installing NPM

As I mentioned above, npm is included with Node.js versions 0.6.3 and above. If you are using an older version of node, I'll leave it as an exercise to the reader to find the distribution package. Better yet, I'd suggest upgrading to a more recent version of Node.js!

npm is itself distributed as a package through npm so you can have npm upgrade itself. npm is updated quite frequently, so it's a good idea to upgrade npm once in a while

$ sudo npm install npm -g

Installing a package

Node.js has the concept of "local" and "global" packages. Local packages are installed as part of your current project and are not available to other projects in your system. Global packages are installed at the system level, so distributions that include command line tools (like npm) that you will want to use "globally", then you will want to include it in global scope.

Local Package Installation

To install a package locally, you'll use the "install" npm sub command. Most often you will just need to specify the package name as an argument.

$ npm install package-name

Global Package Installation

To install a package globally, you'll use the "-g" argument. Most often you will need to run this under "sudo" to allow for permissions in system folders.

$ sudo npm install -g package-name

Updating Packages

Since packages can be installed locally or globally, there is an update procedure for each as well. Local updates are the easiest and can be completed with the "update" npm command. This command will update all the packages in the local installation.

$npm update

To update packages globally, you add the "-g" option like you did above.

$npm update -g

But, in some cases you won't want to update all global packages at once. You can use the "outdated" command to get a listing of global packages that need updating:

$ npm outdated -g --depth=0

and then you can update the global package with the same install command you used above

$ npm install -g package-name

Removing Packages

If the package you recently downloaded isn't living up to your expectations, you can use the "uninstall" npm command to remove the package from a local or global installation. For a global installation, use the "-g" option as above

$ npm uninstall package-name
$ npm uninstall -g package-name

Creating Packages

In my article on Node.js Modules, I go over in detail the package creation process. I'd suggest you read that article for details on creating your own package.

Publishing Packages

Once you have a great package put together, you may want to share that with the world. The process for doing so is very easy. You must first have a npm user account. You can create a new user with the "npm adduser" command, or alternately login if you have an existing account with the "npm login" command.

Once you are authenticated, you can use the "publish" npm command to push your package to the npm repository. use the folder name of your package in the "my-cool-package" parameter below:

$npm publish my-cool-package

Conclusion

Check out the Getting Started documentation on the npm website for more information on getting the most out of the Node Package Manager.

Node.js ABC’s - M is for Modules

Mon, 24 Aug 2015 14:11:54 -0700

As any one who has created a software system can tell you, after some point you get to a point in time where putting all your code in one big location becomes unmanageable and leads to maintenance issues. The logical first step is to try to separate code that is related into smaller distinct entities and "include" them in the project in question. But, what happens if two included file "A" and file "B" both define a "doSomething()" method? Global Scope is sometimes convenient, but more often a cause for troubles when development and enhancing a product.

Different languages handle these "namespace" collisions in different ways. Some make them optional, some a bit more strict. Node avoids potential "global scope" issues by not offering an easy way to accidentally over-stuff the global scope. This is done in the form of "Modules".

At a high level, modules are just an easy way to group similar code together. Every file in Node.js is considered a module, although modules can be a bit more complex than a single file. Node modules allow you to select which functions and variables that are in the file are exposed to the calling application and, consequently, which ones are private.

Modules can be packaged and published to an online repository (most likely the Node Package Manager or npm) without those using the modules having to worry about one module conflicting with another in the project.

CREATING MODULES

For the sake of simplicity, we'll focus the following examples on a single file module solution. To create a new module, just create a new file with the name of your module as it's file name. Inside that file, there is a special object calls "exports". It is with this object, that you specify what you want exposed to the caller.

/* indexGenerator.js */
var counter = 0;
exports.nextIndex = function() {
  return counter++;
}
exports.reset = function() {
  counter = 0;
}

In this example module titled "indexGenerator", I've exposed the following functions:

"nextIndex" that will return the next 0-based index in the set.
"reset" that will reset the index to 0

That's it, you've created your first module! The next step is to figure out how to load it and get access to it's functionality. That is done with the "require" command. I went into this in detail in my "L is for Loading" article. I'll point you to that article on the ins and outs of loading a module.

/* test.js */
var generator = require("./indexGenerator.js");
console.log("INDEX: " + generator.nextIndex() + "\n");
console.log("INDEX: " + generator.nextIndex() + "\n");
console.log("resetting index\n");
generator.reset();
console.log("INDEX: " + generator.nextIndex() + "\n");

will produce the following output

INDEX: 0
INDEX: 1
resetting index
INDEX: 0

MODULE PATTERNS

The above example illustrates how to return a set of independent functions from a module. More often then not, you will want to package your functions into an object and return that to the caller. There are several methods for returning objects from modules: The Factory and The Constructor models

The Factory Model

With the factory module, a creation function is returned in the "exports" object that, when called, will create a new instance of the object in question. The following code illustrates this:

/* hello.js */
function helloObject() {
  this.sayHi = function() { return "Hi"; }
  this.sayHowdy = function() { return "Howdy"; }
}
exports.createHello = function() {
  return new helloObject();
}

/* test.js */
var helloFactory = require("./hello.js");
var helloObj = helloFactory.createHello();
console.log("HI: " + helloObj.sayHi());

The benefit of this approach is that you can expose many numbers of factories by adding each of them to the exports object along with all other variables and exported standalone functions.

The Constructor Model

Another way to return an object from a module is to override the exports object with the new object in question. With this approach, you can only return a single item (in this case an object) from your module.

/* hello.js */
function helloObject() {
  this.sayHi = function() { return "Hi"; }
  this.sayHowdy = function() { return "Howdy"; }
}
module.exports = helloObject;

/* test.js */
var helloClass = require("./hello.js");
var helloObject = new helloClass();
console.log("HI: " + helloObj.sayHi());

MODULE FOLDER FORMAT

As I mentioned above, modules can be more than just a file. You can create a folder structure which contains meta-data, version information, and the module implementation and test files.

/my-module
/package.json
/lib
    /module_impl.js
    /file2.js
    ...

package.json
{
"name": "my-module",
"version": "1.0.0",
"main": "./lib/module_impl.js"
}

More information on the Folder Module and the loader rules can be found in my article on Node.js Loading.

PUBLISHING

So you've written a pretty awesome index generator module and you want to share it with the rest of the world. The Node Package Manager makes it really simple to do so:
- The first step is to thoroughly test out your module.
- Ensure your package.json file has all the info you want to include (help can be found with the "npm help json" command).
- Create an account in the npm registry servers with the "npm adduser" command.
- run "npm publish" from within the module directory.

Pretty simple isn't it! If you need to manage your package within the repository, you can do so with the "npm unpublish" command.

For more information on Node.js modules, check out the Modules section of the Node.js API reference.

Node.js ABC’s - L is for Loading

Mon, 03 Aug 2015 14:50:02 -0700

One of the great benefits of utilizing a language like Node.js is the wide assortment of 3rd party modules that are at your disposal. From calculating the validity of credit card numbers, to managing network connections, the Node Package Manager (npm) repository has hundreds of thousands of packages available to you. This article will explain how you can load modules into your project.

Loading and Referencing a Module

In Node.js, modules are accessed either by their name or the file path on the system. Non-system modules referenced by name will eventually be mapped into a file system path.

Each module exposes a public interface that can be used after the module is imported with the "require" into the current script.

var mod = require("some_module_name");

The previous command will import and return an object that represents the external API exposed by the module. The value of that object is dependent on the module and can include an object, an array, a function, or any other type of JavaScript object.

Developing and Exporting Modules

For the module developer, you must use the CommonJS Module System to make your modules available to the outside world. Let's create a sample module “person.js” that exports a Person constructor.

/* File person.js */
function Person(name) {
  var _name = name;
  function name() {
    return _name;
  }
}
module.exports = Person;

In the above example, the Person function is defined and then specified with the "module.exports" object that the module will export to other scripts that require this module.

As mentioned above, more complex objects can be exported by adding onto the module.exports object with whatever values you deem fit for your module.

/* file mymodule.js */
function sayHi() { console.log("HI!!!"); }
function sayBye() { console.log("BYE!!!"); }
module.exports.sayHi = sayHi;
module.exports.sayBye = sayBye;

Below is the code to load the module and call the sayHi and sayBye functions.

var mymodule = require("./mymodule");
mymodule.sayHi();
HI!!!
mymodule.sayBye();
BYE!!!

Loading a Module

There are a couple of ways to reference modules depending on their kind. There are core modules, 3rd party modules, or local modules.

Core Modules

Since core modules are part of the distribution, you need only refer to it by it's name (not the path). Core modules with name collisions to local modules take precedence. The following line will load a instance of the core "http" module.

var http = require('http');

File Module

A file module is a one that is defined in a single file on the file system. The person.js example above is an example of a file module. When loading a file module, you supply a path (full or relative) to the module file, optionally omitting the .js extension.

var mymodule = require("/home/joe/node/modules/mymodule");
var mymodule = require("./mymodule");

In those examples the loader will look for a file named "mymodule.js" in the specified directory.

Folder Modules

Like a file module, when loading a folder module, you specify the absolute or relative location of the folder. The Node loader will assume this folder is a package and attempt to look for a package definition file (package.json) in which the name and location of the module file are located. If no package.json file exists, the Node loader will look for a file named index.js.

Loader search procedure

If the module name is not a path and is not a core module, the loader will attempt to find it inside the special "node_modules" folder. If the module doesn't exist there, it will continue up the path, folder by folder, looking within node_modules folders on those directories until either finds the module or hits the root of the file system.

Conclusion

That's about all there is to it for exporting and loading a Node.js module. Creating modules is as easy as writing the module code and specifying what is returned with the module.exports object. Then a simple call to require("module") is all the client has to write to make use of it.

DevOps 101 - Communication

Thu, 30 Jul 2015 09:03:34 -0700

In my previous article on the history of DevOps, I outlined the following pillars of the DevOps methodology

Management
Integration
Communication and information sharing
Collaboration
Automation
Measurement

The Third Pillar

The third pillar in the DevOps stack is Communication (and Information Sharing). Back when DevOps was Development and Operations, the two teams were typically separated by both priorities, organization, and most often geographic locations. These separations led to many issues, one of the biggest being communication.

The Development Silo

As I mentioned in my article on DevOps History application releases used to be very spread out over long periods of time with a lot of changes and features. The application team lived in a silo in that they built the application, tested it and then tossed it over the wall to the operations team to handle security testing and ultimately deployment to production environments. Communication consisted of change order tickets that sat in a queue for weeks while operations could fit it into their queue of work. And, when they were able to get to it, if any issue arose, it was sent back to the development team to fix and resubmit.

With the advent of Agile programming where the development teams worked on more frequent point releases, the "change order" process began to feel like the "TPS report" in Office Space. A new agile method of deployment was needed as "tickets" became a hindrance and wasted a lot of time for both sides of the Dev+Ops fence. Communication and Information Sharing became a critical aspect in moving to a more agile deployment cycle.

Communication Tools

Each organization has it's own set of tools for allowing teams to communicate. The trick was getting these included in the application deployment lifecycle. Some popular tools include

Email solutions including Exchange, Gmail, Yahoo, Hotmail/Office 365, and others.
Instant Messenger and chat applications such as Lync, Skype, Slack, Google Hangouts , and IRC systems.
Content management and social systems such as Jive and Lithium.
Task based systems such as Asana, Basecamp, and Trello.

As the "Dev" gets closer to "Ops" in departmental and procedural areas, communication becomes a bit easier but is still critical to act as the connective tissue keeping DevOps together.

Node.js ABC’s - K is for Koa

Mon, 20 Jul 2015 13:35:00 -0700

Node.js is an open source runtime environment for server-side and network based applications. Node.js contains all the components needed to build a functioning website. But, you need to think of Node.js as a big tool box. As with any other art form, going to the tool box over and over to perform the same task is not productive. The concept of tool reuse is used in all forms of artistry and development is no exception.

What is a Web Framework

In general, frameworks are code libraries that are created to save people a lot of time and unnecessary work for repetitive tasks. A framework is a universal environment that has several key features that distinguish it from a regular library:

Inversion of control - The control flow of the application is dictated by the framework, not the code using the framework.
Default behavior - Provide a default set of behaviors yielding a consistent outcomes.
Extensibility - Provides a layer of extensibility where the user can selectively override the default behaviors of the framework.
A non-modifiable framework - The core framework software is not intended to be modified.

Frameworks can allow you to create a server in just a few lines of code and make creating a REST API very simple.

Koa

Koa is a web framework that is being worked on by the team that wrote the Express Node.js framework. Koa differentiates itself by not using any middleware within the core of the framework. From Koa's website

Koa is a new web framework designed by the team behind Express, which aims to be a smaller, more expressive, and more robust foundation for web applications and APIs. Through leveraging generators Koa allows you to ditch callbacks and greatly increase error-handling. Koa does not bundle any middleware within core, and provides an elegant suite of methods that make writing servers fast and enjoyable.

When building an application on top of Koa, you are creating generator functions that are composed and executed upon request. Koa is similar to other middleware systems like Rack for Ruby.

Koa Hello World

It can't finish up talking about a framework without presenting the obligatory "Hello World" example. So, here it is in Koa:

var koa = require('koa');
var application = koa();

application.use(function *() {
  this.body = "Hello world!!!"
});

var server = application.listen(8080, function() {
  console.log("Koa is listening on http://localhost:8080");
});

Summary

Koa is a fairly small set of code so it does not cause unnecessary bloat to your application. It is basically a bare-boned framework where the developer can decide on the middleware they want to run instead of relying on that of the framework. The current state of Koa right now is that it's pretty early on in development so if you decide to investigate it, I'd avoid being bleeding edge and stick with the latest stable release.

Other Popular frameworks

The benefit of the open source community is that there are always alternatives to choose from. Here is a list of some of the more popular Node.js frameworks out there:

Express - http://expressjs.com/
Locomotive- http://locomotivejs.org/
Total.js - http://www.totaljs.com/
Derby - http://derbyjs.com/
Meteor - https://www.meteor.com/
Mojito - https://developer.yahoo.com/cocktails/mojito/
Nombo - http://nombo.io/
Restify - http://mcavage.me/node-restify/
Loopback - http://strongloop.com/mobile-application-development/loopback/#automatically-generate-apis-to-data-models
ActionHero - http://www.actionherojs.com/
Frisby - http://frisbyjs.com/
DocPad - http://docpad.org/

DevOps 101 - Integration

Thu, 02 Jul 2015 10:52:56 -0700

In my previous articles on the history of DevOps, I’ve outlined the following pillars of the DevOps methodology

Management
Integration
Communication and information sharing
Collaboration
Automation
Measurement

The Second Pillar

The second pillar in the DevOps stack is that of Integration. DevOps integration targets quality testing, feature development, and product delivery. Integration, or more specifically, Systems Integration is the process of linking together different computing or component systems and software applications physically or functionally to perform as a single consolidated unit.

Systems integration requires a wide skill set in areas such as network architecture, protocols, systems, software engineering, documentation and communication. A DevOps Engineer needs to have a unique skill set that combines that of both the traditional software developer with that of the IT engineer. Typically DevOps engineers are either developers who get interested in the deployment process of their applications or system administrators who have a passion for scripting and software development.

The Methods of Systems Integration

System integration traditionally falls into three methods. Vertical, Star, and Horizontal (or Enterprise Service Bus).

Vertical Integration

Vertical integration involves integrating systems according to their functionality by creating functional components referred to as silos. One of the key benefits of this method is performed quickly and typically is cheaper in the shorter term. But, a negative to this methods is that the cost of ownership is often higher and scaling the system would likely require implementing another silo. Also subsystem reuse for new functionality is often not possible.

Star Integration

Star (or Spaghetti) Integration is the method of integration where multiple systems are interconnected to each other (like a plate of spaghetti). The cost varies depending on the types of interfaces the system use. Costs and time are greatly increased when adding additional systems. This method is very flexible and provides high levels of reuse.

Horizontal Integration

Horizontal Integration (or Enterprise Service Bus) is a method where a dedicated system is used to communicate to other subsystems. This allows for only one connection per subsystem to the service bus and the ESB translates the interface to a different system. With this method, it is designed to allow for you to replace one subsystem with another easily by updating the ESB with the new systems interface.

Conclusion

Integration is just one piece of the DevOps puzzle. In my next article I’ll discuss how communication is integral in the DevOps process.

Node.js ABC’s - J is for JavaScript

Mon, 29 Jun 2015 09:24:00 -0700

Node.js is an open source runtime environment for server-side and network based applications. At it's core, Node.js uses the Google V8 JavaScript engine to execute client code and a large portion of the core basic modules are written in JavaScript. Since, for all practical purposes, writing Node.js applications is writing JavaScript applications, it makes sense to try to understand the gist of what JavaScript is.

What is JavaScript

JavaScript is primarily a client-based dynamic scripting language most commonly used within web browsers as client-side scripts to interact with the user, browser, and communicate asynchronously to servers.

If you have been part of any web-based development, odds are you have worked with JavaScript in one form or another. In this article, I'll focus on the aspects of JavaScript that are relevant within the Node.js environment.

Types

Node.js has only a few core object types including boolean, number, object, and string. There are three complex types consisting of the array, function, and object. And, finally, null and undefined are special objects that are treated in their own way.

> typeof 10;
'number'
> typeof "HI";
'string'
> typeof function() { }
'function

Constants

While Node.js does support the const keyword, is it not widely used. A standard practice is to use all upperscore characters when defining a constant.

> var HOURS_PER_DAY = 24;

Numbers

All numbers in JavaScript are 64-bit 754 double-precision floating-point numbers. For numbers that can be expressed in 2&53 bits, their type behaves much like an integer in any other language.

> 100
100
> 1/3
0.3333333333333333

There are functions that can help you convert strings to numbers

> parseInt("12345")
12345
> parseFloat("1.2345")
1.2345
> parseInt("1.2345")
1

Booleans

The boolean type can have values of true or false. false, 0, empty strings, NaN, null, and undefined all evaluate to false and all other values evaluate to true.

Strings

Strings are sequences of Unicode characters that can be wrapped in single or double quote characters. The string type has built-in properties such as "length" to give you access to attributes about the string

> var s = "hi there"
> s
'hi there'
> s.length
8

Objects

JavaScript resolves around objects and are something you will use in virtually any JavaScript-based coding effort. They are extensible allowing you to add and remove properties and methods. To create an Object, you can use the new keyword or use the preferred object literal syntax

> var o = new Object();
undefined
> var o = {};
undefined

One of the benefits of Object-Literal syntax is that you can specify the contents of an object at initialization time

> var person = {
  first_name: "Joe",
  last_name: "Pruitt"
};

You can also add a new property to an object like this

> person.weight = 180;
180
> person['weight'] = 180;
180
> user
{ first_name: 'Joe',
  last_name: 'Pruitt',
  weight: 180 }

To delete an object or property, you can use the keyword:

> delete user.weight
true
> user
{ first_name: 'Joe',
  last_name: 'Pruitt' }

Arrays

The array type is a special implementation of the object type. To create arrays, you can use the traditional notation, or the array-literal syntax

> var a1 = new Array();
undefined
> a1
[]
> var a2 = [];
undefined
> a2
[]

To add an item to an array, you can do one of the following

> var a = []
undefined
> a.push("hi")
1
> a
[ 'hi' ]
> a[a.length] = "there";
'there'
> a
[ 'hi', 'there' ]

Functions

JavaScript is a functional programming language in that functions are fully typed objects and can be extended, manipulated, and used as data.

A simple function is defined with the function keyword

> function hi(name) {
  console.log("Hey " + name + "!");
}
> hi("Joe");
Hey Joe!

Parameters are declared in the parameter list after the function name. If too few parameters are passed in, the missing ones will be undefined. If too many parameters are passed in, then the extras are ignored.

You can create an anonymous function without a name by using the following syntax

> var hi = function(name) {
  console.log("Hey " + name + "!");
}
> hi("Joe");
Hey Joe!

It is recommended to use named functions whenever possible as debugging becomes more difficult without a named context in exception chains.

Special Global Objects

Node.js has a couple of global variables that can come in handy.

console - The console object contains the well known log function as well as warn, time, timeEnd, and assert.
global - Similar to how a browser's JavaScript model has the window object which is global in scope, Node.js has the special global object. Anything attached to it is available anywhere in your node application.
process - The process global variable contains information and methods about the current's environment.

Summary

It's beyond the scope of this article to get into all of the aspects of the JavaScript programming language. For more details into the concepts I've covered, as well as others in the language, check out the JavaScript reference on w3schools.com - http://www.w3schools.com/jsref/default.asp.

Node.js ABC’s - I is for Inheritance

Fri, 26 Jun 2015 11:51:00 -0700

The ability to create a class and then extend it is the basis for object-oriented programming. Inheritance in object-oriented programming is when an object, or class, is derived from another object using the same core implementation.

Inheritance in JavaScript is a little different than other languages as there is no explicit class keyword and a mechanism for specify a derived relationship. In JavaScript classes are all declared as functions as in this example

function MyStuff(myThings) {
  this.things = myThings;
}

In the above example, the function MyStuff is a way to hold onto a bunch of things. the things property is used to store the value of myThings which is passed in through the constructor.

Prototypes

Since there is no explict "class" keyword to declare class and inheritance, all objects (functions) in JavaScript have a special "prototype" property, which is used to add methods and properties available to instances of that function.

If one wanted to add on a method to print the contents of the MyStuff object, you could do so like this

MyStuff.prototype.print = function() {
  console.log(this.things);
}

Now you can create a new object and access the print method like this

var coolStuff = new MyStuff("Lots of cool things here");
coolStuff.print();

The above showed the use of the prototype "property". The second aspect of prototypes in JavaScript is the prototype "attribute". The prototype attribute, also commonly referred to as the "prototype object", points to the objects "parent" that the object inherited it's properties and methods from. The prototype attribute is set automatically when you create a new object.

In the above example, the coolStuff's prototype attribute is MyStuff.prototype. All objects created with the Object constructor, inherit from Object.prototype.

Inheritance

All JavaScript objects inherit the methods and properties of their prototypes. In the following example, I'll create a Pet object and then a Dog object that derives from Pet by using the parent constructor.

var Pet = function() {
}
// Add a "feed" method to Pet.prototype
Pet.prototype.feed = function() {
  console.log("The pet has been fed.");
}
// Define a Dog constructor
function Dog() {
  // Call the parent constructor, making sure that "this" is set correctly
  Pet.call(this);
}
Dog.prototype = Object.create(Pet.prototype);
Dog.prototype.constructor = Pet;
// Add a sleep method
Dog.prototype.sleep = function() {
  console.log("The dog is sleeping...");
}
var pet = new Pet();
var dog = new Dog();
// Call Pet's feed method
pet.feed();
// Call Dog's feed method 
dog.feed();
// Call Dog's sleep method
dog.sleep();
console.log(dog instanceof Pet);

$ node inheritance.js
The pet has been fed.
The pet has been fed.
The dog is sleeping...
true

Since every object has a prototype, one can chain objects upon objects to create a hierarchy chain.

Node.js ABC’s - G is for Globals

Mon, 01 Jun 2015 14:18:00 -0700

A variable is a storage location paired with an associated name, which contains the information known as a value. Nearly every programming language under the sun supports variables in some form or another.

Within the context of a single piece of code, the use of variables are straight forward: Store a value, retrieve a value, use the value.

var sum = 1 + 2;
console.log("The sum is " + sum);

The concept is nice and tidy within the context of a single code segment. But not many software programs are a limited to a single instance. With object oriented and modular programming where code logic is broken into smaller segments to enable code reuse and inheritance, sharing variables can be cumbersome.

Variable Scope

The concept of where variables are available and accessible is referred to as the variables context, or scope. When you declare a variable, the section of code it is included in is referred to as local scope. If you want to share a variable with different contexts (local scopes), you will need to use a global variable that lives within global scope. Different languages use different constructs to get at global variables.

Javascript will first look for a local scoped variable and if not found look for a global scoped one and use that. The following example shows a global "sum" with various functions taking in as parameters,

var sum = 3;
console.log("SUM 1: " + sum);
function printSum2() {
  var sum = 5;
  console.log("SUM 2: " + sum);
}
printSum2();
console.log("SUM 2.5: " + sum);
function printSum3(sum) {
  sum = 6;
  console.log("SUM 3: " + sum);
}
printSum3(sum);
console.log("SUM 3.5: " + sum);
function printSum4(sum) {
  var sum = 7;
  console.log("SUM 4: " + sum);
}
printSum4(sum);
console.log("SUM 4.5: " + sum);

function printSum5() {
  sum = 9;
  console.log("SUM 5: " + sum);
}
printSum5();
console.log("SUM 5.5: " + sum);

Will result in the following output:

SUM 1  : 3
SUM 2  : 5
SUM 2.5: 3
SUM 3  : 6
SUM 3.5: 3
SUM 4  : 7
SUM 4.5: 3
SUM 5  : 9
SUM 5.5: 9

A problem arises when you have a local variable and a global variable with the same name and you wish to control which variable you are accessing. Node.js has a solution for this with the "global" object.

The Global Object

Just as JavaScript in web browsers has the "window" object that is accessible to all code contexts as a global object, Node has a similar global-level object but is aptly named "global".

global.sum = 3;
function printSum() {
  var sum = 5;
  console.log("Local Sum: " + sum);
  console.log("Global Sum: " + global.sum);
}

Local Sum: 5
Global Sum: 3

With the global object, you can initialize the variable in one context (function) and access it in another like the following:

function printSum() {
  global.sum = 3;
  console.log("Global Sum 1: " + global.sum);
}
function printSum2() {
  console.log("Global Sum 2: " + global.sum);
}
printSum();
printSum2();

Global Sum 1: 3
Global Sum 2: 3

There are semi-religious wars going on as to whether the global variables should never be used or are a necessary evil. I fall in the later camp but do make sure you use them wisely and understand the context from which your variables are run to avoid debugging headaches later down the road.

Node.js ABC’s - F is for Functions

Thu, 07 May 2015 10:04:00 -0700

JavaScript, and thus Node.js, is at it's heart a functional programming language. Functions are fully typed objects that can be modified, extended, and used as data objects.

A function is a block of code designed to perform a specific task and is much the same as a procedure or subroutine in other programming languages. Node.js takes advantage quite extensively of this capability allowing the developer to define code once and reuse it many times.

Function Syntax

A JavaScript function is defined with the "function" keyword, followed by a name, followed by parentheses. A function name has the same rules as variables in that they can contain letters, digits, underscores, and dollar signs. The code to be executed in the function is placed inside curly braces after the function declaration.

function name(param1, param2, ...) {
  code to be executed
}

As I mentioned above, functions can be used as data objects in code and other function calls. When passing a function in as a parameter, one could do something like this

function passInFunc(f) {
  f("in myfunc");
}
function myfunc(param) {
  console.log(param);
}
passInFunc(myfunc);

When control hits the "passInFunc" function, the name of the "myfunc" function is not relevant. this is where "Anonymous functions" come into play. Anonymous functions are commonly used as callback parameters when using the Event Emitter pattern described in my article on Node.js Events.

An anonymous function can be declared either as a standalone variable

function passInFunc(f) {
   f("in myfunc");
}
var f = function(param) {
  console.log(param);
}
passInFunc(f);

or more concisely as

function passInFunc(f) {
  console.log("in myfunc");
}
passInFunc(function(param) {
  console.log(param);
});

It is this later format that you see used extensively in the Event Emitter pattern. Anonymous functions have the drawback in that when exceptions are thrown, the debugger does not have a name context for the call stack which makes life a bit more difficult when debugging.

Function Parameters

To declare parameters, you simply list them in the parentheses separated by commas. Since JavaScript is a loosely typed language, there is no checking of these parameters at runtime. If a function is declared with two parameters, but the caller does not include them, the values of the parameters within the function will be set to undefined. Within the scope of the function, parameters have local scope to the function, meaning that their values are not accessible from a global scope or from other functions.

function func(param1, param2, param3) {}
func("one");  // param2, and param3 are undefined
func("one", "two"); // param3 is undefined
func("one", "two", "three"); // all parameters are defined

Function Return Values

Functions can include return statements. When a return statement is reached, the function will stop executing and control will be returned to the code after the statement that invoked the function call. To return a status back to the caller, the return statement can take an optional value parameter which the caller can use to make logic decisions.

function add(a, b) {
  return a+b;
}
var sum = add(1,2)  // the sum variable will contain the value 3.

Function Invocation

To cause the function to execute, you must include the () operator after the function name. Referencing just the function name refers to the function object.

function myfunc(param) { }
# f contains the myfunc object
var f = myfunc;

# f contains the return value from the myfunc 
var f = myfunc(param);

Function Scope

When a function is called, a new variable scope is created. Variables declared in the calling scope are available to the function, but variables declared within the new function scope are not available when the function completes it's processing and returns.

var best_language = "JavaScript";
function print_best() {
  console.log(best_language);
  best_language = "c++";
  console.log(best_language);
}
print_best();
console.log(best_language);

The above code will print the following to the console

JavaScript
c++
JavaScript

Combining scope with anonymous functions can yield some interesting ways to do work with private variables that disappear when the anonymous function exits.

DevOps 101 - Management

Wed, 06 May 2015 13:50:00 -0700

In my previous article on the history of DevOps, I outlined the following pillars of the DevOps methodology

Management
Integration
Communication and information sharing
Collaboration
Automation
Measurement

The First Pillar

In this article I'll touch on the first pillar of DevOps: Management. Dictionary.com's definition for the term "management" gives a good direction to focus on to cover this topic.

"Management : The act or manner of managing; handling, direction, or control"

Managing is a bit ambiguous, but handling, direction, and control are right on target with regards to DevOps. DevOps is a set of tools and methodologies to help Developers and IT Operations work together to push software updates through the system faster and more reliably.

:%s/software updates/fliers/

If you think about it, DevOps is very similar to an airline company. The goal of an airline company is to get a person from point "A" to point "B" faster and more reliably. Much like the developer and IT teams try to get software from the developers workstation to the application server hosting the application for your user base.

The Three Managements

Many of the concepts involved in DevOps come from three types of management processes: Software Configuration Management, Enterprise Systems Management, and Software Application Release Management.

Software Configuration Management

Software configuration management (SCM) is the task of handling and controlling changes in software by utilizing revision control and the use of code stability baselines. Revision control gives a view into who changed what and when and can also help with determining how to replicate configurations across servers. SCM provides identification, control, auditing, build and process management, teamwork, and defect tracking.

There are literally hundreds of free and commercially available software solutions to help with revision control. Wikipedia has a nice list under "List of revision control software".

Enterprise Systems Management

Once the organization has control of the software, it must be able to control the administration of the servers and distributed systems used to build and deploy the software. Systems management includes taking software and hardware inventories, monitoring and metrics, anti-virus and anti-malware software, system and network capacity and utilization monitoring, storage, and security. These tasks fall into the following management functions

Fault - troubleshooting, logging, and recovery
Configuration - hardware and software inventory, provisioning, and software deployment
Account - billing and statistics
Performance - software metering and event/metric monitoring
Security - identity and policy management.

As with the software configuration tools, there are more options here than I can list. I'll refer you to another Wikipedia listing on systems management systems.

Application Release Management

Release management is the process giving direction to software releases from development, to test, and finally through to final software release. Release managers are key to this process and they typically handle being a facilitator between business units, a gatekeeper to builds and systems, and being a coordinator of source locations, projects, and release process teams.

The advent of Agile software development has brought on a much higher frequency of releases which have led to increased dependency on the release management teams to track and execute on complex release processes. Release teams have used methodologies such as Information Technology Infrastructure Library (ITIL) to help achieve these needs.

The list of tools in the release management space is not as broad as it's configuration and systems management counterparts. Popular solutions includes ones from Distelli, Chef , Puppet Labs, and Octopus Deploy.

Conclusion

Management is just one component in the DevOps "stack". In the next article I'll be discussing how Integration fits into the DevOps puzzle.

Node.js ABC’s - E is for Events

Thu, 16 Apr 2015 15:25:29 -0700

Node.js has it's basis in asynchronous programming. Also known as Event-driven programming, asynchronous development gives the programmer a way to write a program in such a way as to not block the progress of the program while waiting for a long-lived task to complete. Input/Output processing for local resources such as file system access or for remote network based content such as database queries or web requests is a prime example of a operation that can take an extended period to complete.

The Event Driven Programming Style

Events are a way for the underlying system to notify the calling program that a certain long lived task is finished and ready for the calling program to act on that operation.

In traditional synchronous programming, if you were to make a database request, you'd do something like this

result = query('select * from very_large_table where complex_criteria...')
process_result(result);
...
do_more_work

In this example, the calling process will block and not be able to perform any other processing until the result is returned from the query. A more elegant way to do this would be to use a callback, or Event.

query_finished = function(result) {
 process_result(result);
}
query('select * from very_large_table where complex_criteria...', query_finished)
...
do_more_work

In this second example, we are using event callbacks. The calling program passes in a function to the query method to call back to when the long lived operation is completed. Meanwhile the program can continue doing work and when the query is completed, the query_finished function is called and the result can be processed.

Standard Callback Pattern

Events require 2 components, a "emitter" and a "receiver". The receiver indicates to the emitter which events it would like to receive, and the emitter sends those events to the receiver when the event has occurred. In Node, many objects emit events. For instance, a TCP connection can emit a "connected" event when a new connection is made, or a HTTP request can emit a "data" event each time a new batch of data is read from it's connection.

In my first example above, a return value is used to pass back the status of the operation. The asynchronous alternative is to use what is called "Continuous-passing style" or "CPS". For a CPS styled method, the function takes an extra argument which is a "continuation" for the completion of the function.

Event Emitter Pattern

The standard callback pattern shown above works well when you want to let the client know of command completion, but it not optimal when several events take place during the execution, or if the events happen numerous times. This is where the Event Emitter pattern comes into play. There are two pieces of the Event Emitter pattern, the event emitter and the event listeners (where there can be more than one).

To illustrate this, let's consider the above example of a long lived SQL query. If the result contains a very large record set, it would likely not be optimal to return the whole result set in a single response from the function call. Best practice would be to break the data up into smaller chunks. For a SQL query, that could equate to rows in the resulting table. In this case, we'd want to have an event that was emitted for each row in the result set and then one event for when the result set is complete. This could be accomplished with something like this:

process_row = function(row_data) {
 // do something with row
};
process_end = function() {
 // do something to when data is complete
};
query('select * from very_large_table where complex_criteria...', function(response) {
 response.on("row", function(row_data) {
   process_row(row_data);
 });
 response.on("end", function() {
   process_end();
 });
});

As a general rule of thumb, you should use CPS when you want to regain control after an operation completes and use the emitter pattern when an event can happen more than once per method call.

DevOps 101 - A Brief History Of Time

Thu, 02 Apr 2015 12:55:00 -0700

If you have anything to do with developing products or working in IT helping to deploy and run them, chances are you have heard the term "DevOps" in one form or another. Just like the ubiquitous "Cloud" floating out in the Internet somewhere, DevOps has become a catch-all phrase for anything that is Developer or Operations related.

Before jumping into what DevOps is, I think it's helpful to look back at the evolution of software development which will make it clear why DevOps was inevitable.

First Came The Developer And QA

I'll start be looking back a few years when software was primarily developed and targeted as software installs on end users computers with physical media (DVDs, CDs, or dare I date myself and say Floppy Disks). Development teams would work tirelessly 6+ months on the next release of their product. When the development team gave it the green light, it would go to the quality assurance engineers to perform end user testing, then back to Dev to fix things, then back to test and so on. When QA gave it the green light, it would go to pressing and packaging and ultimately end up at the end user.

Next Came The Internet

Once companies discovered they could distribute their applications in "soft" copy, they eliminated the overhead and costs for physical media, paper manuals, and shipping costs while providing almost instantaneous access for their customers to their latest and greatest software.

Then The End Users Got Greedy

I'm not sure who first thought of a "hotfix", but whoever did is somewhat responsible for the current need for the agile philosophies in application development. Users no longer had to wait 6 months for a "bug" to be fixed in their local copies of the software. Hotfixes brought an interesting dilemma as they only contained a fraction of the code needed for a major release and, consequently, needed to be produced on a much shorter timeframe and at a much higher frequency. Developers and QA need to come up with a better way to handle the more frequent development cycles and came up with a solution with development methodologies like Agile and Scrum. That helped them maintain sanity in their development processes. But Dev and QA needed to come up with solutions to make the delivery process more seamless.

Then Came The Web Application And Network Ops

Someone sat in a meeting in some conference room and said "We have a website right? Why don't we build a version of our product to run on the web?" Reducing the users need to physically install software is a big bonus as they always have access to the latest and greatest. If the hotfix wasn't a turning point, then the migration of apps from standalone to web-based definitely was. Network Operations (Net Ops) came into the scene here as they were needed facilitate distribution of the downloadable images. This helpful IT folks make it their life's goal to have the bits accessible and the download as fast as possible to all users across the internet. They took the files, put them in their distribution channels such as ftp/web servers and Content Delivery Networks, checked the box on the release document and waited until the next request. (Ok, I know I'm simplifying things and IT does a lot more than than, but this is going somewhere I promise).

Then Came The Pain

What that guy, or gal, in that conference room didn't think of was the issues that would arise with the deployment of an application from physical media to the network. No longer are the Ops folks just responsible for the "bits" passed to them to be accessible, they were now responsible about minor things like user experience, access controls, hackers, availability, compliance, etc. Up until this point, the development and release process was fairly painless but now it just got a whole lot messier. The Ops team's jobs are now at stake for maintaining all of those "minor" things and much more. No longer can they just take the word from the development team that a product is "ready" for distribution and blindly "check" the box on the release.

Then Came The Brick Wall

Needless to say, things went from good to worse. It started enforcing process that infused them with the dev-test-production migration. Change tickets, request queues, and delays ensued. A brick wall was being built between Dev and Ops. Dev would toss a release over the wall to Ops. A response would be tossed back over the wall to Dev with either the successful QA and deployment or rejection to fix something and try again. Dev started seeing Ops as those trying to stifle their innovation while Ops saw Dev as reckless with no regard for how the apps run in the real world. Distrust then led to a slow development process with logs of back and forth.

Introducing Some Sanity

Since the Agile development process was so successful in helping put sanity in the code production process, at the Agile 2008 conference, Andrew Clay Shafer and Patrick Debois discussed the new concept of "Agile Infrastructure" and afterwards created the "Agile System Administrators" group on Google. The term "DevOps" was then born and made popular through a new "DevOps Days" set of conferences that started in 2009 in Belgium. DevOps takes the Agile development processes to the next level by adding Ops into the mix. Welcome to the club Ops, now you are a developer too!

Enter DevOps

To put it succinctly DevOps is a set of tools and methodologies to help Dev and Ops work together to enable pushing more software updates out faster and more reliably. Regular communication, small batches of work, and an iterative approach are just a few pieces of the puzzle. DevOps methods stress the following pillars which I'll refer to as the MICCAM model:

Management
Integration
Communication and Information Sharing
Collaboration
Automation
Measurement

By now you see how the DevOps methodology came to be and the problems that it is trying to solve. It's important to keep in mind that "DevOps" is not a end-all-be-all solution that fits everyone to a “T”. Just as Agile and Scrum are tweaked and adjusted by developers, the tools and execution will vary based on the companies requirements and dependencies. To help with the execution, several companies like Chef, Ansible, and Puppet Labs are building products and solutions to help companies adopt these ideas and become productive with application development and deployment.

In upcoming articles, I'll dig more deeply into the different pillars of DevOps, how companies are implementing them, and the various companies that are helping with the execution.

Node.js ABC’s - D is for Debugger

Tue, 09 Dec 2014 15:57:00 -0800

When developing new software, a good debugging tool is essential to making sure your code is functioning as expected. The console is most often used as a poor-mans debugging system by including extra code in your script that prints out status to the hosted console. Is some simple cases this is suitable, but with Node.js's asynchronous nature, you cannot rely on the synchronous nature of log statements in helping you diagnose issues with your code. Fortunately, the Node.js runtime has built in tools to help.

Node.js is built on the JavaScript engine "V8" built for Google Chrome that was open sourced by Google in 2008. V8 includes with it an extensive debugging system for use when development your Node.js applications. The debugger can be accessed by a TCP based TCP DebuggerProtocol .

Having an API to access the debugger is useful for 3rd party developer tools but is a bit overkill for the average developer trying to diagnose an issue with a script. Fortunately, Node.js also has a built-in client to access the debugger.

THE BUILT-IN CONSOLE DEBUGGER

To use this client, start Node with the "debug" argument.

$ node debug sample.js
< debugger listening on port 5858
connection... ok
break in sample.js:1
1 var str = "";
2 for(var i=0; i<10; i++) {}
3   str += i.toString();
debug>

COMMANDS REFERENCE

Stepping

cont,c	Continue execution
next,n	Step to next statement
step,s	Step in to function
out,o	Step out to calling function
pause	Pause running code similar to the pause button in the browser developer tools.

$ node debug sample.js 
< debugger listening on port 5858
connecting... ok
break in sample.js:1   1 var str = "";
  2 for(var i=0; i<10; i++) {   3     str += i.toString();
debug> n
break in sample.js:2   1 var str = "";
  2 for(var i=0; i<10; i++) {
  3     str += i.toString();
  4 }
debug> n
break in sample.js:3
  1 var str = "";
  2 for(var i=0; i<10; i++) {
  3     str += i.toString();
  4 }
  5 console.log(str);

Breakpoints

setBreakpoint(),sb()	Set a breakpoint on the current line
setBreakpoint(line),sb(line)	Set a breakpoint on the specified line
setBreakpoint('fn()'),sb('fn()')	Set a breakpoint on the first statement in a given functions body
setBreakpoint('script.js',n),sb('script.js',n)	Set a break point on the nth line of script.js
clearBreakpoint,cb()	Clear the current breakpoint

$ node debug sample.js 
< debugger listening on port 5858
connecting... ok
break in sample.js:1
  1 var str = "";
  2 for(var i=0; i<10; i++) {
  3 	str += i.toString();
debug> setBreakpoint(3)
  1 var str = "";
  2 for(var i=0; i<10; i++) {
* 3 	str += i.toString();
  4 }
  5 console.log(str);
  6 
debug> c
break in sample.js:3
  1 var str = "";
  2 for(var i=0; i<10; i++) {
* 3 	str += i.toString();
  4 }
  5 console.log(str);

Information

backtrace,bt	Print the callstack for the current execution frame.
list(n)	List scripts source code with n line context (n lines before and after)
watch(expr)	Add an expression to the watch list
unwatch(expr)	Remove an expression from the watch list.
watchers	List all watchers and their values (which are also automatically listed on each breakpoint).
repl	Evaluate code remotely for evaluation in the debugging scripts context.

$ node debug sample.js 
< debugger listening on port 5858
connecting... ok
break in sample.js:1
  1 var str = "";
  2 for(var i=0; i<10; i++) {
  3 	str += i.toString();
debug> watch("str")
debug> setBreakpoint(3)
  1 var str = "";
  2 for(var i=0; i<10; i++) {
* 3 	str += i.toString();
  4 }
  5 console.log(str);
  6 
debug> c
break in sample.js:3
Watchers:
  0: str = ""

  1 var str = "";
  2 for(var i=0; i<10; i++) {
* 3 	str += i.toString();
  4 }
  5 console.log(str);
debug> c
break in sample.js:3
Watchers:
  0: str = "0"

  1 var str = "";
  2 for(var i=0; i<10; i++) {
* 3 	str += i.toString();
  4 }
  5 console.log(str);
debug> c
break in sample.js:3
Watchers:
  0: str = "01"

  1 var str = "";
  2 for(var i=0; i<10; i++) {
* 3 	str += i.toString();
  4 }
  5 console.log(str);

Execution Control

run	Run a script
restart	Restart a script
kill	Kill the current scripts execution

Misc

scripts	List all loaded scripts
version	Display V8's version.

THIRD PARTY SOLUTIONS

If Console debugging doesn't do it for you, there are several great third party node packages out there that make use of the Debugger API. I prefer the node-inspector package that is maintained by the folks at StrongLoop. If you have done any debugging with the built-in debugger in FireFox, then the user interface will look very familiar. Installing the package will create a new command node-debug on your system that will start your default browser and connect to a local hosted web app that runs the debugger interface.

$ node-debug program.js

Node.js ABC's - C is for Callbacks

Mon, 24 Nov 2014 13:54:36 -0800

With synchronous programming, when you make a call to function, your code is blocked until the method you are calling has processed it's work and returned it's status to you. A good example is the "C" fopen and fread commands.

FILE *pFile = fopen("file.txt", "r");  // Wait...
char buf[101];
if ( NULL != pFile ) {
  fread(buf, 1, 100, pFile); // Wait...
  buf[(sizeof buf)-1] = 0;
  printf("%s", buf); // Wait...
  fclose(pFile); // Wait...
}

In this code, the vast amount of time used when this code is ran is waiting for the underlying file system to access and read from the file. It is true for most I/O based applications such as ones that utilize databases or connect to external services, your code will spend a majority of it's time sitting around waiting.

One of the key components of Node.js is the concept of non-blocking I/O and asynchronous programming. If you have seen or used the setTimeout() function in JavaScript, you already have seen how a non-blocking call works. With the setTimeout() function, you pass in a function to call and a time after which the function should be called.

setTimeout(function() { console.log("Ring-a-ling...")}, 5000);
console.log("I'm Waiting for the phone to ring...");

If you run the preceding code, you'll see:

I'm waiting for the phone to ring...
Ring-a-ling...

The code sets a timeout of 5000ms (or 5 seconds), passing in the function to call when it fires, and then it continues with the execution of the script.

The Callback Function

One of the primary patterns you will see for asynchronous programming is by using a callback function you pass to asynchronous functions. It has at least one parameter which is to pass information about whether the call succeeded or failed. It also frequently has a second parameter for passing back additional details about the call (a file system or network socket handle, contents of a object read requests, or the output from a database call).

The following example illustrates how the readFile function in the file system fs core module works with a callback to determine the size of the file.

var file = "somefile.txt";
var fs = require("fs");
rs.readFile(file, function(err, data) {
  if ( err ) { throw err; }
  var len = data.toString().split("\n").length -1;   console.log("File Length: " + len);
});

Controlling Asynchronous Calls

One can imagine when you have a complicated script that makes many asynchronous calls, you could get into some trouble in handling the callbacks in any kind of predictable order. One handy third-party tool is the "async" module. The async module comes with a series of helper functions for asynchronous calls and response control such as map, reduce, filter, each, etc.

async.map(['f1', 'f2', 'f3', 'f4'], fs.stat, function(err, results) {
  // results contains an array of stats for each input item
});

// call functions at the same time and then call callback when they have all completed.
async.parallel([
  function1() { ... },
  function2() { ... },
  function3() { ... }
], callback);

// call functions one at a time in series one at a time.
async.series([
  function1() { ... },
  function2() { ... },
  function3() { ... }
])

Synchronous Versions of Core Library Calls

Several of the core libraries include synchronous versions of their asynchronous counterparts. This is primarily evident in the File System module (http://nodejs.org/api/fs.html). Look for functions ending in "Sync" with the omitted callback parameter. These are useful for command line scripting where the processing delays associated with synchronous calls is not a factor.

Node.js ABC’s - B is for Buffers

Wed, 19 Nov 2014 10:43:00 -0800

Core JavaScript uses Unicode friendly interfaces for storing string data but when dealing with network protocols or file I/O that are using raw data, it's necessary to handle that data as binary streams of data. Node has several options for creating, consuming, and interacting with octet streams, one of which is a Buffer.

Node.js stores raw data in instances of a Buffer class which is similar to an array of integers but stores raw memory allocations outside the heap. A Buffer is a region of a physical memory storage used to temporarily store data while it is being manipulated. A Buffer class is global in scope, thus not needing a require to use it, and cannot be resized.

Converting Between String Objects and Buffers

To convert between a Buffer and a String Object requires you to supply an explicit encoding method. The following is a list of the different string encodings available:

ascii	To be used for 7-bit ASCII data only and limited to the ASCII character set. This is a very fast encoding method but will strip the high-bit off if set.
utf8	Multi-byte encoded Unicode characters typically used in many document formats including web based content.
utf16le,ucs2	2 or 4 bytes, little endian encoded Unicode characters. It can encode only Basic Multilingual Plane U+0000 - U+FFFF (BMP).
base64	Base64 encoded strings
binary	A deprecated encoding from raw binary data into strings by only using the first 8 bits of each character. This should not be used as it will be removed from Node.js in the future.
hex	Encode each byte as two hexidecimal characters.

Creating Buffers in Node.js

The Buffer object has the following constructors for creation:

 // Allocate a new Buffer of “size” octets new Buffer( size // number; size of the buffer );

 > var buf = new Buffer(5); // Create a buffer of size 5. > console.log(buf); <Buffer 00 00 00 00 00>

 // Allocates a new Buffer using the supplied Array of octets new Buffer( array // array; user supplied array of data );

 > var buf = new Buffer([1,2,3,4,5]); // Create a buffer of size 5 with the numbers 1-5. > console.log(buf); <Buffer 00 01 02 03 04 05>

 /// Allocates a new buffer containing the string “str”.  The encoding defaults to “utf8”. new Buffer(   str,      // string; A input string to store in the Buffer   encoding  // string (optional); Character encoding defaulting to utf8. );

 >var buf = new Buffer(“Hello World!”, “utf8”); // Create a Buffer containing the string “Hello World!” >console.log(buf); <Buffer 48 65 6c 6c 6f 20 57 6f 72 6c 64 21>

Static Methods

The Buffer object contains the following static methods for use when interacting with Buffer objects.

 // Test if encoding string is valid Buffer.isEncoding( encoding // string; Encoding string to verify ); // Test if object is a Buffer Buffer.isBuffer( obj // object; object to test ); // Calculate the byte length of a string Buffer.byteLength( string, // string; input string to get length of encoding // string (optional); encoding string defaults to utf8. ); // Concatenate a list of Buffer objects into a single Buffer Buffer.concat( list, // array; Array of Buffer objects totalLength // number (optional); total length of buffers when concatenated );

Writing To Buffers

Buffers aren’t of much use unless you can actually manipulate the data in them. The constructors can be used to create new Buffers with content and the write method allows you to write a string to an existing buffer. You can also use the [] accessor to set individual octets in the Buffer.

 // Write a string to a Buffer buf.write( str, // string; A supplied string offset, // number (optional); The index of the buffer to start writing to (default is 0) length, // number (optional); The number of bytes to write (defaults to buf.length – offset. encoding // string (encoding); The encoding to use (defaults to utf8). );

 > var buf = new Buffer(“Hi There); > buf.toString(); ”Hi There” > buf.write(“HI”); > buf.toString(); ”HI There” >buf.write(“H”, 4); >buf.toString(); ”HI THere” >buf[7] = “E” >buf.toString(); ”HI THerE”

Reading From Buffers

Reading from Buffers can be done with the index [] accessor and the toString() method to convert the Buffer to a string

 // decode and return a string from a Buffer buf.toString( encoding, // string (optional); The encoding to use when converting to a string (defaults to “utf8”). start, // number (optional); The index in Buffer to start at (defaults to 0). end // number (optional); The index in Buffer to end at (defaults to buf.length). );

 >var buf = new Buffer(“Hi There”); >buf.toString(); ”Hi There” >buf[3]; ”T” >buf.toString(“utf8”, 3); ”There”

Since JSON is at the heart of JavaScript, there is the following method to convert a buffer to it’s JSON array format

 // Return a JSON-representation of the Buffer instance. buf.toJSON();

 >var buf = new Buffer(“Hi There”); >buf.toJSON(); [ 72, 105, 32, 84, 104, 101, 104, 101 ]

Further Exploration

To explore the full list of features of the Buffer object, check out the Buffer documentation on nodejs.org.

Node.js ABCs - A is for About Node.js

Thu, 13 Nov 2014 14:16:00 -0800

Back in 2008, I started a series of articles on Networking basics titled the "Networking ABC’s" in which I set out to document a networking Term for each letter of the alphabet. After some good feedback, I followed that up with The "Social Media ABC’s" and then the "PowerShell ABC’s".

After a hiatus on my ABC series, I decided to start it back up again, this time focusing on the up and coming development language Node.js. With F5's release of the Linerate Application Proxy earlier this year, F5 officially added Node.js to it's portfolio of application APIs. For those new to Node.js, this series aims to take 26 of the basic terms or concepts and hopefully give you some insight into how they support the language and runtime.

For the first article in this series I'm going to try to explain what Node.js is. For that, I'll start with the definition on the Node.js website.

What Is Node.js?

"Node.js is a platform built on Chrome's JavaScript runtime for easily building fast, scalable network applications. Node.js uses an event-driven, non-blocking I/O model that makes it lightweight and efficient, perfect for data-intensive real-time applications that run across distributed devices."

Not unlike Java, Python, or PHP, the Node.js runtime can be used to build client and server applications. Due to it's light footprint, it is also ideal for embedded systems such as F5's Linerate Application Proxy.

Node.js has a base built-in library of standard runtime functions (I/O, environment, etc) and, like Perl's massive public repository of support libraries, Node.js has the same that can be managed with the Node Package Manager. The online repository for Node.js packages can be found at https://www.npmjs.org/. With a whopping 104,144 packages that have been contributed by the user community, you will likely find help for any task you want to accomplish.

Getting Up And Running

Download the Node.js runtime from the Downloads section of the Nodejs.org website. Pre-built packages are available for Windows, OSX, Linux, SunOS. A source package is available as well if you want to build it yourself.
Install the runtime distribution. This will provide you with the "node" and "npm" applications.
Open a console window and run the “node” (or “node.exe on windows) command to start the runtime interpreter.
Read-up and run through some intro tutorials. http://nodeschool.io has some very good introduction materials including the "javascripting", "learnyounode", "git-it", and "stream-adventure" npm packages. You can use these to learn Node.js directly from within the node runtime.

Your First Node.js Program

For you first timers out there, here’s your typical “Hello World!” program written for node. I’ll use the Javascript “console.log” command to print output to the console. The first line echo’s the code into the script called prog.js. Once the file exists, passing it as a command to the node runtime will cause the script to be executed.


$ echo “console.log(‘Hello World\!’);” > prog.js
$ node prog.js
Hello World!

When you are interested in tackling some more interesting problems, check out the “learnyounode” node package linked to above. The first few are fairly basic and the last ones can be a bit of a challenge to the intermediate level developer as well.

Be on the lookout for the next article in this series where I talk about Buffers.

Shellshock mitigation with BIG-IP iRules

Thu, 25 Sep 2014 16:34:00 -0700

Yesterday, NIST released information on a new network exploitable vulnerability in the GNU Bash shell as demonstrated by vectors involving parts of OpenSSH sshd, the mod_cgi, and mod_cgid modules in the Apache HTTP Server, scripts executed by DHCP clients, and other situations where setting the environment around a bash process occurs. Vulnerability CVE-2014-6271, or Shellshock as everyone is calling it, allows remote attackers to execute arbitrary code on the target systems. Redhat has a great overview in their security blog posting on the topic.

What makes this vulnerability serious is the fact that server-side CGI applications running under the Apache Web Server with mod_cgi or mod_cgid are susceptible.

F5's Jeff Costlow is maintaining an article on the Shellshock vulnerability so make sure you check back to that article for F5's official stance.

The first step to mitigation is to setup a plan to patch the bash shell on all of your systems. In the interum, if you believe that any of your backend servers are vulnerable (if they are Unix-based then they likely are) and you are fronting them with a BIG-IP, below is an iRule solution you can apply to your Virtual Servers that will protect your servers against attacks.

Block-Shellshocked

The Block-Shellshock iRule searches for the pattern "*() {*" in the URI and in the HTTP headers. It is rare that this pattern of characters will occur in a URI or HTTP Header so we believe the number of false-positives will be minimal. If the pattern is found, an audit entry will be written to the system log with the client ip as well as the URI, and optionally the HTTP header, that the attack was detected. I chose to issue a reject on the connection. If you want to be more polite to the hackers, you can substitute the "reject" with a 403 - Forbidden.

when HTTP_REQUEST {
  set pattern "*() \{*";  
  if { [string match $pattern [HTTP::uri]] } {
    log local0. "Detected CVE-2014-6271 attack from '[IP::client_addr]' in URI '[HTTP::uri]'";
    reject;
  } else {
    foreach header_name [HTTP::header names] {
      foreach header_value [HTTP::header values $header_name] {
        if { [string match $pattern $header_value] } {
          log local0. "Detected CVE-2014-6271 attack from '[IP::client_addr]' in HTTP Header $header_name = '$header_value'; URI = '[HTTP::uri]'";
          reject;
          break;
        }
      }
    }
  }
}

Block-Shellshock-full

The first iRule gives a more detailed audit if the attack is detected. In the case where that is not a concern and you are more concerned with performance, the HTTP::request method can be used to perform a single search across all of the request header information. This does not cover the POST body, but at this point there is no indication that the exploit is exposed within the body portion of a POST request. This irule reports the client IP along with the requested URI that is the source of the attack.

when HTTP_REQUEST {
  if { [string match "*() \{*" [HTTP::request]] } {
    log local0. "Detected CVE-2014-6271 attack from '[IP::client_addr]'; URI = '[HTTP::uri]'";
    reject;
  }
}

We will be monitoring this vulnerability over the coming weeks and update the iRule solution if a more optimal one becomes available. Please leave a comment if you have any suggestions. And, make sure you monitor Jeff's article for updates.

Special Considerations

Please be aware that this iRule will reject any connections containing the string "() {" in the URI or header. While we are confident that this is not a valid pattern for standard headers, there could be situations where custom application values contain this string pattern and this iRule would cause issues to those applications. We suggest running the longer iRule and actively monitoring the system log for false positives. If you find a case where your custom application is being blocked, you can customize the first iRule to exclude the header your application relies on.

Introducing a RESTful interface for iControl

Thu, 20 Jun 2013 11:30:00 -0700

iControl isn’t just SOAP anymore…

No, iControl isn’t getting lazy. While taking it easy is an important part of life, I’m talking about the other kind of REST. REST, or “REpresentational State Transfer” for you technically inclined, is a style of architectural principals with which you can design web services that focus on a system’s resources. It also defines how resource states are addressed and transferred over the network. REST is really a “style” of getting and setting resources and doesn’t define the underlying communications. Most implementations out there make use of HTTP and JSON as a content format, which is what we’ve chosen to do as well.

There are plenty of articles on the web that compare and contrast SOAP and REST, so I won’t get into those here. I’m also not going to go really deep into the principals of REST as you can find dozens of those easily with a web search. In this article, I’ll discuss how we’ve chosen to implement our REST interface for iControl and give you some examples on how to use it.

Oh, and don’t get any ideas that our SOAP based interface is going anywhere. We are creating our REST interfaces as an alternate method for performing automation and monitoring. We don’t currently have plans on ceasing development on our SOAP interface.

iControl-REST

The REST interface for iControl was introduced in BIG-IP version 11.4. For this release, we are considering the feature as “Early Access”. Call it beta or whatever you want. But what that really means is that it will change in our next release. We are using this release for feedback from the users out there to find out what works and what doesn’t. Several key features are not implemented yet (for example versioning) which we have targeted for an upcoming product release when we finalize the implementation.

Ok, with that said, now we can get into the details.

iControl-REST, like it’s SOAP counterpart, is implemented on HTTPS and uses the same authentication and authorization roles for user access. We support the following HTTP commands:

GET - for retrieving (ie. Querying the status of a Pool)
POST - for creating (ie. Creating a Pool Member)
PUT - for updating (ie. Changing the load balancing method on a Pool)
DELETE - for deleting (ie. Deleting a Virtual Server)

And, the format of the requests and responses is JSON.

Starting the iControl REST Service (icrd)

Run the “modify sys service icrd” TMSH command to add and start the iControl REST service. Notice the nice “EA” warning.

root@(BIG-IP1)(…)(tmos) # modify sys service ircd add
WARNING: This early-access feature comes with minimal
documentation and testing.  Version control is not implemented;
therefore any scripts that you write using this API version
may not work with subsequent releases.
root@(BIG-IP1)(…)(tmos) #

Once the service is running, you can use the “show sys service”, “stop sys service” and “start sys service” TMSH commands to monitor and control the status of the iControl REST service.

Writing Your First Script

Since iControl-REST is just using HTTPS, you can use any scripting technology you heart desires. For this article, we’ll use the command line tool cURL. It’s cross platform, so you should be able to wrap a bash script for Unix, Mac, etc or PowerShell for Windows around it very easily.

Use the “-u” parameter to pass in the user credentials, the “-X” parameter to specify the HTTP method, and the uri for the resource you wish to access.

curl -k -u user:pass -X http-method uri

The URI format is as follows

Module URI

https://management-ip/mgmt/tm/module

This access all of the sub-modules and/or components under the given module (ltm, gtm, etc).

Sub-module URI

https://management-ip/mgmt/tm/module/sub-module

This access all of the sub-modules and/or components under the given sub-module.

Component URI

https://management-ip/mgmt/tm/module[/sub-module]/component

This accesses the details about the given component. The tmsh Traffic Management Shell Reference documents the hierarchy of modules and components, and identifies the details of each component.

Example

This curl command will query all of the information about the ltm module.

$ curl -k -u user:pass -H “Content-Type: application/json” -X GET https://bigip_ip/mgmt/tm/ltm
{
  "currentItemCount": 22,
  "items": [
    {
      "reference": {
        "link": "https://bigip_ip/mgmt/tm/ltm/auth"
      }
    },
    {
      "reference": {
        "link": "https://bigip_ip/mgmt/tm/ltm/data-group"
      }
    },
    {
      "reference": {
        "link": "https://bigip_ip/mgmt/tm/ltm/dns"
      }
    }
    ...
  ],
  "kind": "tm:ltm:ltmstate",
  "pageIndex": 1,
  "partition": "/Common/",
  "selfLink": "https://bigip-ip/mgmt/tm/ltm",
  "startIndex": 1,
  "totalItems": 22,
  "totalPages": 1
}

User Guide

For this release, we have provided a User Guide to assist in getting started with using the new REST interface. It can be downloaded from this link: iControlRest-UserGuide.pdf.

We would love to hear your feedback on using iControl-REST. Any and all comments and questions should be posted to the iControl group.

-Joe

Project Acceleration: Programmatic Performance Testing with HTTPWatch

Tue, 09 Apr 2013 06:00:00 -0700

Note: As of 11.4, WebAccelerator is now a part of BIG-IP Application Acceleration Manager.

This is article nine of ten in a series on DevCentral’s implementation of WebAccelerator. Join Colin Walker and product manager Dawn Parzych as they discuss the ins and outs of WebAccelerator. Colin discusses his take on implementing the technology first hand (with an appearance each from Jason Rahm and Joe Pruitt) while Dawn provides industry insight and commentary on the need for various optimization features.

In a recent performance project, I was asked to come up with a way to automate the page-level metric collection for various optimization profiles we were testing on DevCentral. The idea is that we would like to come up with a simple way to compare the many performance settings side-by-side and, hopefully, give us a better view as to which would benefit the end users who are accessing the site.

I’ve done some automation work with HttpWatch in the past, so I thought that would be a good starting point. I’ve blogged in the past about the HttpWatch COM interface in Windows, but I will give a little refresher again to give some context to the work done in this script.

Dawn Says...

There is a specific process that needs to be followed to get accurate, actionable metrics in relation to web page performance. It can be a very time consuming process and is prone to errors. Generally speaking you want to run a minimum of 3 test runs for each scenario and take an average of the 3. My preference is to run 10 test runs but time typically doesn’t allow for that. If you are running 4 scenarios and need to run a minimum of 3 times that is at least 12 test runs. Unfortunately I don’t think I have ever been able to run just 12 as I always make mistakes. Take the scenario of running as a first time user with a clean cache. For every test run you need to open the browser, clear the cache, launch a measurement tool of some kind like HttpWatch, access the page, save the report and close the browser. Then repeat. Mistakes that are common to make in this process (at least I pretend they are common so I don’t feel like I’m the only one that does this).

Forgetting to clear the cache when running as a first time user. I see the objects being returned with a response code of “Cache” and I want to scream.
Loading the page before starting to record with the measurement.
Closing the browser and forgetting to save the report

Having a way to automate the process eliminates these mistakes and enables the running of multiple test runs without wasting a lot of time. I would much rather go get a cup of tea come back and have all my testing completed.

I chose to implement the solution in PowerShell as that gave me an easy way to interact with the HttpWatch COM object from a scripting environment available on all recent versions of Windows.

Script Requirements

My colleague that offered this great opportunity to me, supplied me with the following functional steps that needed to occur

Baseline: VIP:w1.x1.y1.z1; Domain:devcentral.f5.com; URL: /

1. Test Initial Visits
    - Empty browser cache
    - Access test page with an empty cache
    - Record time and data transfer metrics
    - Close browser
    - Repeat “x” number of times and calculate averages

2. Test Repeat visits
    - Reopen browser
    - Access page as repeat visitor
    - Record time and data transfer metrics
    - Close browser
    - Repeat “x” number of times and calculate averages

Optimization: VIP:w2,x2,y2,z2; Domain:devcentral.f5.com: URLs:/tcp, /compress, /ibr, /img

1. Repeat baseline logic for each URL.

To allow for us to dynamically change optimization profiles for a given connection, we implemented a nifty bit of iRules that would trap the “/tcp”, “/compress”, “/ibr”, and “/img” URLs and issue a HTTP 301 redirect to the test page along with a cookie containing the corresponding BIG-IP profile to apply. An iRule sitting in front of the site looked for that cookie and then enabled the associated profile. The optimization requests then contained one more request (the 301 redirect) that will need to be accounted for and subtracted from the totals (more on that later).

HttpWatch.Controller

The core object is the HttpWatch.Controller. It contains the following members:

FireFox –Returns a reference to the FireFox object. Use this property if you want to be using FireFox in your testing.
IE – Returns a reference to the Internet Explorere object. Use this property if you want to use IE for your testing.
IsBasicEdition – Returns true if the product is the basic edition, false for Pro.
OpenLog(logFileName) – This allows existing HttpWatch log files to be opened and examined using the automation library.
Wait(plugIn, timeOutSecs) – Wait for a page to be fully loaded and is normally used after the GotoURL method.
Version – A string containing the current version of HttpWatch.

The PowerShell code to create this object looks something like this.

$script:HTTPWATCH = New-Object -ComObject “HttpWatch.Controller”;

And by accessing the FireFox.New(), or IE.New() methods, you’ll now have access to control the HttpWatch interface. For more info on the HttpWatch API, check out the HttpWatch Automation Overview.

Main Application Logic

The script takes in the following parameters

num_retries - the number of times to perform each test for averages
browser - use “ff” for FireFox, or “ie” for Internet Explorer. The default is “ff”
WA - an optional switch to determine whether the script is run on the regular site or optimization profiles.
SubtractRedirects - If supplied, this parameter will cause the 301 redirects mentioned above to be subtracted from the metrics on the optimization tests.

param(
  $num_retries = 5,
  $browser = "ff",
  [switch]$WA = $false,
  [switch]$SubtractRedirects = $False
);

$script:SUBTRACT_REDIRECTS = $SubtractRedirects;
$script:LOGDIR = "$(Split-Path $MyInvocation.MyCommand.Path)\WATests";
if ( -not [System.IO.Directory]::Exists($script:LOGDIR) ) { mkdir $script:LOGDIR; }

$script:HTTPWATCH =  New-Object -ComObject "HttpWatch.Controller";

#----------------------------------------------------------------------------
# Main App Logic
#----------------------------------------------------------------------------

$tests = @("/");
if ( $WA )
{
  $tests = @("/tcp", "/compress", "/ibr", "/img");
}
$results = Do-DCPerfTest -num_retries $num_retries -browser $browser -tests $tests;
Run-Report.CSV -results $results;

Functional Test Logic

The Do-DCPerfTest() function performs various tests with the specified number of retries. The difference between initial and return visits is the clearing cache step performed for the initial visits.

function Do-DCPerfTest()
{
  param(
    $num_retries = 5,
    $browser = "ff",
    $tests = @("/")
  );
  
  $results = @();
  
  foreach( $test in $tests )
  {
    Write-Host "------------------------------------------------------------";
    Write-Host "- Testing https://devcentral.f5.com${test} initial visit";
    Write-Host "------------------------------------------------------------";
    
    for($i=0; $i -lt $num_retries; $i++)
    {
      Clear-BrowserCache -browser $browser;
      
      $plugin = New-Plugin;
      $results += Run-TimingTest -plugin $plugin -domain "devcentral.f5.com" -url $test -index $i -note "I";
      $plugin.CloseBrowser();
    }

    Write-Host "------------------------------------------------------------";
    Write-Host "- Testing https://devcentral.f5.com${test} return visitor";
    Write-Host "------------------------------------------------------------";
    
    for($i=0; $i -lt $num_retries; $i++)
    {
      $plugin = New-Plugin;
      $results += Run-TimingTest -plugin $plugin -domain "devcentral.f5.com" -url $test -index $i -note "R";
      $plugin.CloseBrowser();
    }
  }
  
  $results;
}

Single Timing Test

The Run-TimingTest() function will first clear the HttpWatch internal log, load the URL and wait for the page to render, save the HttpWatch log (.hwl file) locally for future analysis, and then process the logs to build a PowerShell object with the results.

function Run-TimingTest()
{
  param(
    $plugin = $null,
    $domain = $(Throw "domain required!"),
    $url = $(Throw "url required!"),
    $index = "",
    $note = "",
    $is_redirect = $null
  );
  
  $results = @();
  if ( $null -ne $plugin )
  {
    Write-Host "Clearing HttpWatch Log...";
    $plugin.Clear();
  
    # Load the url
    Load-WebPage -plugin $plugin -url "https://${domain}${url}";

    # save logs
    $ip = Get-IPFromHostname -hostname $domain;
    
    $logname = "${ip}-${domain}-${url}-${note}-${index}".Replace("/", "");
    $log = "$($script:LOGDIR)\${logname}.hwl";
    Write-Host "Saving log '$logname'";
    $plugin.Log.Save($log);

    # Process logs
    $results += Process-Logs -plugin $plugin -note "${logname}";

    Write-Host "Clearing HttpWatch Log...";
    $plugin.Clear();
  }
  $results;
}

Controlling HttpWatch

The New-Plugin() function will use the HttpWatch Controller to instantiate a new instance of the plugin for the given browser. The instance of that plugin is returned to the calling code.

function New-Plugin()
{
  param($browser = "ff");
  
  $plugin = $null;
  
  if ( $browser = "ff" ) { $plugin = $script:HTTPWATCH.FireFox.New(); }
  else { $plugin = $script:HTTPWATCH.IE.New(); }

  $plugin;
}

For this test, we wanted to isolate any issues of caching between plugin sessions, so we opted to have new plugins created for each functional step. This forced a refresh of the plugin and browser objects. The Clear-BrowserCache() function creates a new plugin instance, Clears the browsers cache with the plugin.ClearCache() method and then closes the browser with the plugin.CloseBrowser() method.

function Clear-BrowserCache()
{
  param($plugin = "ff");
  
  $plugin = New-Plugin -browser $browser;
  if ( $null -ne $plugin )
  {
    Write-Host "Clearing browser cache...";
    $plugin.ClearCache();
    $plugin.CloseBrowser();
  }
}

The Load-WebPage() function first initiates a transport level recording with the plugin.Record() method. It then calls the asynchronous plugin.GotoURL() method and waits for that load to complete with the plugin.WaitEx() method. Recording is then stopped with the plugin.Stop() method and control is returned back to the calling code.

function Load-WebPage()
{
  param(
    $plugin = $(Throw "Plugin parameter required!"),
    $url = $(Throw "url parameter required!"),
    [switch]$ClearCache = $false
  );
  
  if ( $null -ne $plugin )
  {
    if ( $ClearCache )
    {
      $plugin.ClearCache();
    }

    Write-Host "Loading page '${url}'...";
    $plugin.Record();
    $plugin.GotoURL($url);
    #$HttpWatch.Wait($plugin, -1) | Out-Null;
    $wait = $script:HTTPWATCH.WaitEx($plugin, -1, $true, $false, 1);
    $plugin.Stop();
  }
}

The last piece of the puzzle is processing the HttpWatch log, that is captured during the page load, and extracting all the required data. There is a lot going on here but are the main things to look at

$plugin.Log - This is where all the log data is stored from recording start to stop.
$plugin.Log.Pages - Each URL loaded with the GotoURL() method, will get it’s own Page item.
$plugin.Log.Pages.Item(0) - There was only one URL loaded so I’ll look at the first Page only.
$plugin.Log.Pages.Item(0).Events - Some aggregate data for the page including DOMLoad, PageLoad, HTTPLoad, and RenderStart times.
$plugin.Log.Pages.Item(0).Entries - This is a list of all the items requested for the given GotoURL request. This includes redirects, embedded images, scripts, etc.

Once you get those items down, the Process-Logs() function should hopefully make sense.

function Process-Logs()
{
  param(
    $plugin = $null,
    $note = ""
  );
  
  $page = $plugin.Log.Pages.Item(0)
  
  $summary = $page.Entries.Summary;
  $events = $page.Events;
  $entry = $page.Entries.Item(0);
  
  $url = $entry.URL;
  $redirectTime = 0;
  $redirectBytesSent = 0;
  $redirectBytesReceived = 0;
  $redirectRoundtrips = 0;
  
  if ( $entry.IsRedirect -eq $true )
  {
    $url += " -> ";
    $url += $entry.RedirectURL;
    $redirectTime = $entry.Time;
    $redirectBytesSent = $entry.BytesSent;
    $redirectBytesReceived = $entry.BytesReceived;
    $redirectRoundTrips = 1;
  }
  
  $DOMLoad = $events.DOMLoad.value;
  $PageLoad = $events.PageLoad.value;
  $HTTPLoad = $events.HTTPLoad.value;
  $RenderStart = $events.RenderStart.value;
  
  $roundTrips = $summary.RoundTrips;
  $bytesReceived = $summary.BytesReceived;
  
  if ( $script:SUBTRACT_REDIRECTS )
  {
    $PageLoad -= $redirectTime;
    $HTTPLoad -= $redirectTime;
    $roundTrips -= $redirectRoundTrips;
    $bytesReceived -= $redirectBytesReceived;
  }
  
  $compressionSavedBytes = $summary.CompressionSavedBytes
  
  Write-Host "TIMING: $note - rt: $RoundTrips; pl: $PageLoad";
  
  $o = 1 | select Note, URL, RenderStart, PageLoad, HTTPLoad, BytesReceived, CompressionSavedBytes, RoundTrips, RedirectTime
  $o.Note = $note;
  $o.URL = $url;
  $o.RenderStart = $RenderStart;
  $o.PageLoad = $PageLoad;
  $o.HTTPLoad = $HTTPLoad;
  $o.BytesReceived = $bytesReceived;
  $o.CompressionSavedBytes = $compressionSavedBytes;
  $o.RoundTrips = $roundTrips;
  $o.RedirectTime = $redirectTime;
  $o;
}

Running the Script

Since the requirements called for different IP addresses for the two virtuals, I had to make a host entry before each run of the script to make sure that the browser was going to the correct VIP.

#for VIP 1 (substitute w.x.y.z for real IP address)
w1.x1.y1.z1 devcentral.f5.com
#for VIP 2
w2.x2.y2.z2 devcentral.f5.com

Then the script can be run like the following:

PS> # Change IP Address to VIP 1
PS> .\DC-WATest.ps1 > results.csv
PS> # Change IP address to VIP 2
PS> .\DC-WAText.ps1 -WA >> results.csv
PS> type results.csv
BytesReceived,CompressionSavedBytes,HTTPLoad,Note,PageLoad,RedirectTime,RenderStart,RoundTrips,URL
1519542,467368,4.713,w1.x1.y1.z1-devcentral.f5.com--I-0,4.716,0,1.046,54,https://devcentral.f5.com/
1514991,467368,4.432,w1.x1.y1.z1-devcentral.f5.com--I-1,4.435,0,1.044,54,https://devcentral.f5.com/
1515020,467368,4.548,w1.x1.y1.z1-devcentral.f5.com--I-2,4.55,0,1.051,54,https://devcentral.f5.com/
...
BytesReceived,CompressionSavedBytes,HTTPLoad,Note,PageLoad,RedirectTime,RenderStart,RoundTrips,URL
1350376,637177,5.378,w2.x2.y2.z2-devcentral.f5.com-tcp-I-0,5.381,0.555,1.06,54,https://devcentral.f5.com/tcp -> https://devcentral.f5.com/
1345751,637177,4.401,w2.x2.y2.z2-devcentral.f5.com-tcp-I-1,4.403,0.637,1.171,54,https://devcentral.f5.com/tcp -> https://devcentral.f5.com/
1345662,637177,4.538,w2.x2.y2.z2-devcentral.f5.com-tcp-I-2,4.54,0.606,1.155,54,https://devcentral.f5.com/tcp -> https://devcentral.f5.com/
...

The raw data can then be imported into a spreadsheet and processed to determine the overall effectiveness of the various optimization profiles.

Conclusion

HttpWatch proves to be a very valuable tool to integrate into your automation and monitoring environments. For more information on HttpWatch go to their httpwatch.com. The documentation for their Automation interface is located at the HttpWatch Automation Overview site.

Source Code

The full source for this script can be found in the Advanced Design and Config wiki under “Programmatic Performance Testing with HttpWatch”.