Headwired

Decrease Indent in LoadRunner

Joel Deutscher — Tue, 08 May 2012 11:29:11 +0000

To increase the indent on a section of code in VUGen, you can simply press the TAB key. This works as expected. To decrease the indent, the SHIFT + TAB keys work as they do in other programs such as MS Word and even VIM.

Hopefully everyone already knows this shortcut, though as its not documented in the menu, some people may have missed it.

Bandwidth Emulation for Mac

Joel Deutscher — Tue, 01 May 2012 09:27:13 +0000

Apple introduced a useful tool with Lion that allows you to test application performance under various network conditions. The Network Link Conditioner (NLC) is primarily designed for testing iOS apps, though it can also be used for testing websites via Safari in the iOS simulator.

Once you install XCode on Lion, you will find the NLC preference pane installer located within /Developer/Applications/Utilities/Network Link Conditioner/. A reboot is required before the NLC daemon will run.

Once you have installed the Network Link Conditioner you can find the NLC icon in the “other” section of the system preferences. Once activated the NLC tool applies to the network interface of your Mac so it will impact ALL traffic on your computer when you turn it on. The nice thing about using NLC is that you can change the bandwidth profile on the fly.

The NLC is a nice addition to OS X and might just come in handy when running performance profiling from Lion.

Stress testing by end users

Joel Deutscher — Mon, 23 Apr 2012 11:45:17 +0000

Performance testing is an expensive exercise. It takes specialist skills and a significant amount of planning and understanding of the application in order to replicate the hundreds or thousands of users you are expecting. Another option of course is to let your end users do the performance testing for you. Take the announcement from Blizzard below for Diablo III.

Beginning this Friday everyone is invited to log in and help us put the game and servers through their paces in this three day stress test as we march toward the game’s release.

It is really viable to get your end users to stress test your application?

I am not sure how viable it truly is, though it appears to be getting more popular of late. One other example is ‘Things’, a popular task manager for the Mac and iOS devices .

Since we announced the beta, more than 35,000 people have signed up to become test pilots. Over the past months, we’ve gradually been inviting more and more people. […] Some of our users have adopted the beta entirely and created exceptionally large databases. In the coming weeks we are going to work out some kinks and performance bottlenecks related to such large databases.

In many ways, this makes a lot of sense. Why spend countless hours simulating your users activities when you can just let them do it themselves. I think this approach works well for companys that have a loyal and enthusiastic fan base. People are so excited to play a Blizzard game, or get access to cloud syncing on a tool that they will suffer through any issues without much complaining. After all, here is Blizzards answer to that very question.

What if the service is down/laggy/disconnecting me?

It’s very possible that players connecting to the stress test could experience issues with the service. While not ideal, this is exactly why we’re having a stress test. We want to catch and analyze as many bugs as possible during this stress test period, so that we can try to ensure a smooth launch

In reality, there are very few companies with a loyal enough fan base that will put up with a slow or interrupted service just to be one of the first to use a new product or feature. More importantly, there are not many brands that would be happy to risk their image by doing so.

Stress testing by end users it seems is then out of reach for most organisations. This is most likely not helped by the media attention that occurs when it goes wrong, like when the Australian Government launched the MySchool website and it was brought down an hour after the launch.

Inevitably, it all comes down to risk. What are the risks of the site not being able to handle the load generated by our users? What is the impact if it is the public that finds the breaking point? Finally, how will they react when they find the breaking point?

I am yet to come across a project where it’s acceptable to let the public stress test the application like Blizzard, though maybe I just haven’t worked for a company as fun as Blizzard yet.

Visual Representation of Process

Joel Deutscher — Wed, 08 Feb 2012 09:17:55 +0000

In most organisations, you will find that while they have a process, nobody seems to know it exactly, or even where to go to find it. The problem, it seems is with the way in which processes are documented. Process documents are usually lamented over at the time of their writing, then shelved without much thought at all. The reason for this I believe is that there is primarily only two times when a process document is actually referenced:

When a new employee joins the team, and is shown how things are done
When a higher manager asks “how does your team operate”

In my mind, I would much prefer a simpler process flow that is actually used by staff, even if it doesn’t cover every possible eventuality along the way. The visual process document provides the most effective way of presenting the flow of how we go about completing our tasks. Its typically printable on one page (though it might have to be A3), it’s pinnable to your office cubicle, and sometimes as importantly, can be pasted into powerpoint presentations for the business.

So how do you present your testing process?

An Alternative to System Info Wallpaper

Joel Deutscher — Wed, 21 Dec 2011 05:04:33 +0000

I have setup many machines to control and generate load. My initial experiences was with LINUX based systems, then as the tools progressed, the operating systems regressed onto the Windows platform. It seemed that back in the days of KVM’s, setting a new background colour for each machine was the way to tell them apart. Next came the system information imprinted on a wallpaper with tools like BgInfo from SysInternals. Unfortunately for this approach, a majority of RDP clients are setup to remove wallpapers (and should be if they aren’t). So in 2011 do we have a better way of displaying machine information on the desktop?

Enter Rainmeter. Anyone who reads Lifehacker and seen their Featured Home Screen section would be very familiar with what this tool can do. Best of all, for my purpose at least, is that Rainmeter shows up over RDP even when the wallpapers are turned off. My initial intention was to simply display the system name and possibly the IP address, though after playing with the tool, I was able to quickly create something that would be a little more useful. Using the Welcome example from the default illustro skin, I was able to provide a brief summary of the current system metrics as well as some short-cut links to my tools and directories, and given how hard is it to find anything in Windows Server 2008, this is a good thing. This also gets rid of creating stupid bitmap wallpapers on each machine.

While I might have gone a little bit overboard with this one, keep Rainmeter in mind if you are trying to display the system information without having to worry if your users have wallpapers turned on or off.

Building a realistic load profile with Analytics

Joel Deutscher — Mon, 12 Dec 2011 23:13:30 +0000

One of the most important parts of a successful Performance Test is getting the requirements right. No matter what tool you use or how well you script, your test won’t succeed if you are not accurately simulating end-user activity on the application. Traditionally, the user volume has been a mix of current load levels from log files and projected load from the business. More recently, analytics have been so easily implementable that every site these days, even small blogs, have analytics information.

So how can we use Analytics to help build a realistic load profile for performance testing?

A post by Suraj Sundarrajan (@perfengineering) highlighted some of the current uses of GA in modelling user behaviour for performance testing. These include the calculation of user idle time, 3rd party web components like Flash, Silverlight & AJAX and a current area of interest for me, cache.

  Industry reports suggest that ‘Cached pages’ can account for up to one-third of all page views. Due to its obvious performance benefits, browser caching mechanism is being extensively used by application developers. Web server logs do not (and will not be able to) capture user activity metrics for cached pages as no request is made to the Web server. On the contrary, Web Analytics tools track visits to cached pages (as they track usage from users’ browser) and thus provide a more accurate picture of “browser caching” on the web application. Performance testers can use this information to determine what percentage of total application usage is being cached and emulate this browser behavior in the load test scenario. – Suraj Sundarrajan

Suraj does a great job of describing some of the elements of Analytics that will help performance testers build a realistic load profile. Though I hear you saying “Analytics? So what, Google Analytics have been publicly available since 2006, and there were many options before then”. This is exactly where I was headed. Though I am impressed with the new features of GA, one of the most valuable additions isn’t actually a feature at all. It’s the fact that they have been around so long, and that we now have enough history to start to evaluate trends. Combining this data with projected business usage provides another invaluable source of information to help ensure a successful performance test.

Using mldap_modify & mldap_delete functions in LoadRunner

Joel Deutscher — Tue, 20 Sep 2011 02:43:06 +0000

I have written before about using the LDAP protocol using LoadRunner. This time, I had a requirement to reset the passwords for all my test users before an execution. Luckily for me, there was already some shell scripts setup that utilised ldapmodify, so it was just a matter of converting them over to LoadRunner scripts.

After binding to the LDAP server, the first command was to delete the users password history to get around the password policy. The batch file called the ldapmodify function like below

ldapmodify -h ldap.headwired.com -p 389 -D 'cn=Directory Manager' -w PASSWORD <
As the action is actually a delete, this translates to the mldap_delete function in LoadRunner.
// Clear Password History
mldap_delete("LDAP Delete", 
      "DN=uid={USERNAME},ou=people,dc=headwired,dc=com ", 
      "AttrName=passwordHistory", 
      LAST );

The next step was reseting the actual password, and here is the script again. Note that as the script must authenticate every time it calls ldapmodify. 
ldapmodify -h ldap.headwired.com -p 389 -D 'cn=Directory Manager' -w PASSWORD <
This time, we will use the mldap_modify function to change the userpassword value
// Update Password
mldap_modify("LDAP Modify", 
      "DN=uid={USERNAME},ou=people,dc=headwired,dc=com", 
      "AttrName=userpassword",   // Name of attribute 
      "AttrValue={PASSWORD}",    	// New Value 
      LAST );

While the task to be performed here is very trivial, it’s a simple demonstration of the mldap_modify and mldap_delete functions. The full script is below.
Action()
{
  // LDAP Bind
  mldap_logon("TRANSACTION_NAME",
        "URL=ldap://cn=Directory Manager:PASSWORD@ldap.headwired.com:389/ ",
        "Version=3",
        "Mode=Async",
        LAST);

  // Clear Password History
  mldap_delete("LDAP Delete", 
        "DN=uid={USERNAME},ou=people,dc=headwired,dc=com ", 
        "AttrName=passwordHistory", 
        LAST );

  // Update Password
  mldap_modify("LDAP Modify", 
        "DN=uid={USERNAME},ou=people,dc=headwired,dc=com", 
        "AttrName=userpassword",   // Name of attribute 
        "AttrValue={PASSWORD}",    	// New Value 
        LAST );

  return 0;
}

Performance Testing SOAP with LoadRunner – Basics

Joel Deutscher — Wed, 10 Aug 2011 06:28:58 +0000

Web services are more commonly becoming the targets of Performance tests. This post aims to provide an introduction to testing SOAP with LoadRunner without using the Web Services VUser type. Why not use the Web Services Vuser? Errors like this

System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.Xml.XmlException: 'xsi' is an undeclared namespace.

Using the custom web request (as suggested by Wilson Mar, Stuart Moncrieff and many others) bypasses the XML validation steps of LoadRunner, which is a good thing. This is possible because SOAP is just XML over HTTP, so lets take a look.

Understanding SOAP

A typical SOAP message is made up of header and body encapsulated in a SOAP envelope. The SOAP header contains application-specific information (like authentication, payment, etc) about the SOAP message. The header is optional, yet if present, must be the first child element of the Envelope. The SOAP body element contains the actual message.
As SOAP uses HTTP as an application protocol, it utilises a request / response model. A SOAP request is sent to the endpoint via a HTTP POST and the returned SOAP message uses the same response codes for HTTP (200 OK, etc). This request is to get the stock price for the ticker symbol HWB (Headwired Bank), with the response returning a price of $460.75 as the last traded price.

SOAP Request

POST /StockQuote HTTP/1.1
Host: headwired.com
Content-Type: text/xml; charset="utf-8"
Content-Length: 1452


    
        
            HWB

SOAP Response

HTTP/1.1 200 OK
Content-Type: text/xml; charset="utf-8"
Content-Length: 1298


    
        
            460.75

SOAP as a Web Service

As a Web Service, the structure of all supported SOAP requests and responses are provided in a WSDL (Web Service Definitions Language). A WSDL provides the following information:

The interactions service provided
The arguments and results are involved in the interactions
The network addresses are used to locate the service
The communication protocol that should be used
The data formats that messages are represented in

In our example SOAP web have a structure for our request in the SOAP body. We have a GetLastTradePrice tag, with a child tickerSymbol . How can we determine that this is what we need to submit by looking at the WSDL? The following WSDL definition declares what elements should be in GetLastTradePrice input request.

This specifies that our input should contain the “ TradePriceRequest” element which is also defined in the WSDL. The definition is for one element “tickerSymbol” that must be a string value.

The WSDL will also specify the response that we should expect from this request.

This specifies that our input should contain the “TradePrice” element which is also defined in the WSDL. The definition is for one element “price” that will contain a float value.

The WSDL below is a rough example of the stock service that would be used by the example SOAP.



  
    
      
        
          
        
      
      
        
          
        
      
    
  
  
    
  
  
    
  
  
    
      
      
    
  
  
  
        
           
           
           
        
    
    
        My first service

SOAP With LoadRunner Custom Requests

SOAP in its most basic form is XML + HTTP. Therefore, we can use the LoadRunner web_custom_request function to manually specify our SOAP message. This method is more verbose than a web_service_call script, yet as it is a web request, there is no XML parsing performed by LoadRunner. This can be very useful if your web_service_call scripts are returning errors.

Scripting a web service is different a web page. Firstly, unless you have a client application, you will not be able to record and replay, instead you will have to recreate the web service calls based on logs, interface specification documents or a WSDL. It is also important to understand how the operations of the web service will be used in production.

Setup

This section discusses the steps to setup your LoadRunner script to commence scripting.

Script Type

A web_custom_request script for SOAP should be created as a “Web (HTTP/HTML)” script. This allows for Internet Protocol Run Time settings to be configured for handling authentication in a more effective way (read: web_set_sockets_option).

Authentication

The web service you are using may require HTTP level Authentication in order to access the web service operations. This can be performed using a simple web_set_user command in LoadRunner. To avoid a Authentication Error on your first request, the web_set_sockets_option can be configured along with the “Enable Integrated Authentication” run-time setting to avoid this issue.

// Requires "Enable Integrated Authentication" to be set to "yes"
web_set_sockets_option("INITIAL_BASIC_AUTH","1");
// Authenticate
web_set_user("USERNAME","PASSWORD","www.headwired.com:443");

To enable integrated authentication, open “Run-time Settings > Internet Protocol > Preferences > Set Advanced Options” and select yes for “Enable Integrated Authentication”. More information about authentication can be found here.

There may also be the requirement to login using multiple authentication pairs throughout the test. This is possible through the use of the web_cleanup_cookies function. This function will remove all cookies related to the VUser, including NTLM authentication cookies.

// Clear previous logins
web_cleanup_cookies();

Scripting

Making the Request

In order to use the web_custom_request function for SOAP, we must specify that we will be submitting text/xml. The web_add_header function is used to specify this.

// Set Headers to plain text / xml
web_add_header("Content-Type", "text/xml; charset=utf8");

The web_custom_request is pretty straightforward. You specify your URL, Method and Body of the request. The SOAP message, including XML specification and Envelope is placed in the Body section of the request.

web_custom_request("Transaction_Name", 
	"URL=https://www.headwired.com/StockQuote", 
	"Method=POST", "TargetFrame=", "Resource=0", "Referer=", 
	"Body=" 
	
	"Snapshot=t1.inf",
	LAST);

Verifying the Response

The simplest way to verify the response of a SOAP request is to use the web_reg_find function. This allows the specification of text that must be matched within the response text.

// Ensure the response contains correct text
web_reg_find("Text=GetLastTradePriceResponse",LAST);

To extract data from the response and save it as a parameter, you may also use the web_reg_save_param function. In the example below, the entire XML response, including HTTP headers is saved in the “response” variable.

// Save entire response as a string
web_reg_save_param("response","LB=","RB=",LAST);

Debugging Tips

Perhaps the easiest way to debug a web services script that utilises the web_custom_request function is to enable “Visual test results”. This feature can be enabled via the VuGen General Options, Replay Tab

The visual test results will display a report after replaying a script and provide you a script tree view, and allow you to view the XML response from the SOAP server.

Example Script

The following is a LoadRunner script that will send our sample SOAP request to the server. It will ensure that the response contains the term “GetLastTradePriceResponse”. As SOAP uses HTTP, any HTTP level errors (401, 500) will also automatically be failed by LoadRunner.

// Ensure the response contains correct text
web_reg_find("Text=GetLastTradePriceResponse",LAST);

// Set Headers to plain text / xml
web_add_header("Content-Type", "text/xml; charset=ISO-8859-1"); 

// Submit SOAP Request
lr_start_transaction("SOAP_GetLastTradePrice");
web_custom_request("SOAP_GetLastTradePrice", 
	"URL=http://www.headwired.com/stock", 
	"Method=POST", "TargetFrame=", "Resource=0", "Referer=", 
	"Body=" 
""
  ""
    ""
      "HWB"
    ""
  ""
""
	"Snapshot=t1.inf",
	LAST); 
lr_end_transaction("SOAP_GetLastTradePrice", LR_AUTO);

Links and References

W3Schools SOAP Tutorial
WSDL Authentication
Authentication

Quick Tip – LoadRunner Download Filters and Proxy Servers

Joel Deutscher — Tue, 24 May 2011 11:40:20 +0000

A previously mentioned the move from a download filters black list to a white list to avoid hitting servers outside of our testing scope.

One interesting outcome of this change was a nasty C compile error.

Error: C interpreter run time error: Action.c (39):  Error -- memory violation : Exception ACCESS_VIOLATION received.

As it turns out, as we were using a proxy auto-config (PAC) file and this error occurs if your download filters are set to block your pac files. Luckily, LoadRunner does report that the PAC file was “Not downloaded due to a filter”, yet on the very next line reports that it was “downloaded and evaluated successfully”

Starting to download the proxy automatic configuration script, URL="http://www.headwired.com/proxy.pac"  	[MsgId: MMSG-27097]
Warning -26554: Not downloaded due to a filter, URL="http://www.headwired.com/proxy.pac"  	[MsgId: MWAR-26554]
Proxy automatic configuration script (URL="http://www.headwired.com/proxy.pac") downloaded and evaluated successfully  	[MsgId: MMSG-27096]

Surely if iframes are considered a critical resource, then the PAC file should cause an error and not just a warning, especially when it causes such a poorly handled memory violation error.

Luckily adding the pac file location to your download filters include list fixes this problem.

LoadRunner and iframes

Joel Deutscher — Tue, 24 May 2011 09:56:11 +0000

After a recent application update, I noticed that all of our scripts started to fail. It turns out that the AUT now included a piece of tracking code, and our load generators did not resolve the domain name. The error message that came up was:

Error -26627: HTTP Status-Code=404 (Not Found) for "http://tracking.headwired.com/track.asp"  	[MsgId: MERR-26627]

So why would a 404 cause an error and not just a warning? Turns out that iframes are treated as a critical resource. By default, images, scripts, css are all treated as non-critical resources resulting in a warning only, however iframes are considered to be critical and hence cause an error. It makes sense for iframes to be a critical resource… in 1999 when the virtual mall was a good idea, yet in 2011, iframes are used for tracking services only. Rants aside, I could find the option to treat images, scripts and css as critical (pictured below), however I could not find the option to treat iframes as non-critical.

By turning this option off, all 404’ed resources are treated as errors as well, as you can see in the following output log

Found resource "http://localhost/notfound.css" in HTML "http://localhost/iframe.html"  	[MsgId: MMSG-26659]
Found resource "http://localhost/notfound.js" in HTML "http://localhost/iframe.html"  	[MsgId: MMSG-26659]
Found resource "http://localhost/notfound.png" in HTML "http://localhost/iframe.html"  	[MsgId: MMSG-26659]
Detected non-resource "http://localhost/notfound-iframe.html" in "http://localhost/iframe.html"  	[MsgId: MMSG-26574]
Found resource "http://localhost/notfound-object.html" in HTML "http://localhost/iframe.html"  	[MsgId: MMSG-26659]
Error -26627: HTTP Status-Code=404 (Not Found) for "http://localhost/notfound.js"  	[MsgId: MERR-26627]
Error -26627: HTTP Status-Code=404 (Not Found) for "http://localhost/notfound.css"  	[MsgId: MERR-26627]
Error -26627: HTTP Status-Code=404 (Not Found) for "http://localhost/notfound.png"  	[MsgId: MERR-26627]
Error -26627: HTTP Status-Code=404 (Not Found) for "http://localhost/notfound-iframe.html"  	[MsgId: MERR-26627]
Error -26627: HTTP Status-Code=404 (Not Found) for "http://localhost/notfound-object.html"  	[MsgId: MERR-26627]
web_url("web_url") highest severity level was "ERROR", 1627 body bytes, 1438 header bytes  	[MsgId: MMSG-26388]

So back to the problem, a new domain that I needed to block had surfaced and if the domain name resolved, then it would have been included in the tests without my knowledge. While there are two schools of thought here, in the particular testing that I was doing, the customer did not want their tracking sites being hit from the performance tests, it was time to change approach and move to the include only list.

Why was this important? Well now I don’t have to catch every new tracking feature in order to block it, and my ever expanding list of blocked hostnames was now a thing of the past and everything was coming up Milhouse.

My test HTML page is as follows for anyone who is interested.

Testing IFRAME

Your browser does not support iframes.