Objective of measuring the Transaction Response Time – to ensure that the application is working perfectly under heavy load. However, the definition of heavy load may vary from system to system. In Loadrunner by defining an initial acceptable response time, we can benchmark the application if it is performing as anticipated.

Measuring transaction response time is important as it gives an idea of how the aplication is performing in terms of time. This performance parameter can be related to the end users at the time of processing request. It helps in understanding how their application performed.

Transaction Response Time is the sum of

• Time taken for the request made to the web server
• Time taken by the web server to process
• Time taken by the application server to process
• Time taken by the database server to process
• Time taken for the procesed request to come back to the client / user through various routes
• Time taken for the request or data in the network transmission is also considered in this – Network latency between the servers and the client

Measuring Transaction Response Time in LoadRuner: It begins when the defined transaction makes a request to the application. From here, till the transaction completes before proceeding with the next subsequent request (in terms of transaction), the time is been measured and will stop when the transaction completes.

How transaction response time is different from Hits Per Seconds: Hits per Seconds is the number of hits made to the web server. These hits can be requests made to the web server for data or graphics. Hits per second measures the number of times the web server is being accessed. So, it does not represent well to users on how well their applications is performing.

Transaction Response Time helps us to find performance bottlenecks in the application. We can further drill down by correlation using other measurements such as the number of virtual users that is accessing the application at the time of measurement. Other system related metrics like CPU Utilization etc. can be used to identify the root cause.

In LoadRuner, after running a performance test, various measurements can be correlated to find trends and bottlenecks. Correlation can be done between the response time, the amount of load that was generated and the payload of all the components of the application.

Users may be unable to understand the meaning of CPU utilization or Memory usage and thus using a common language of time is ideal to convey performance related issues.

VuGen can be used for recording in windows platforms. But, a recorded Vuser script can also be run on Unix platform.

Developing Vuser Script is a five step process:

• Record a Vuser script
• Vuser Script Enhancement – by adding the control statements and other functions
• Run time Settings Configuration
• Running of Vuser Script on Stand Alone machine – Verify that the script runs correctly
• Integration of Vuser Script – into a LoadRunner scenario or Performance Center or Tuning module session or Business process monitor profile etc.

When you record a Vuser script, VuGen generates Vuser functions and inserts them into the script. There are two types of Vuser functions:

• General Vuser Functions and Protocol-Specific Vuser Functions

Both these functions together form the Mercury VuGen API. It enables Vusers to communicate directly with the server.

There are two types of Vuser scripts:

1. Single protocol Vuser script
2. Multiple protocol Vuser script. Here, we have the option to specify which protocols to record

Three sections of a Vuser script:

1. vuser_init – It is used when recording a login to a server and it is executed when the Vuser is initialized
2. Actions – It is used when recording a client activity and it is executed when the Vuser is in Running status
3. vuser_end – It is used when recording a logoff procedure and it is executed when the Vuser finishes or is stopped

• What is the response time of application for end users?
• Can application handle expected load?
• Will the application be able to handle the number of transactions required by business?
• How the application behave under unexpected loads?

LoadRunner helps us to get answers for these questions. Below are some benefits of LoadRunner over manual performance testing:

1. LoadRunner reduces the human resource requirement by replacing real human users with virtual users known as Vusers. Vusers emulate the real user behavior and operate the application in real.
2. LoadRunner reduces the hardware requirements by running multiple Vusers in a single computer.
3. All Vusers can be controlled from a single computer. It makes the performance testing easy and effective.
4. Using LoadRunner, we can monitor the application under testing online – we can fine tune the application during the performance test execution.
5. LoadRunner automatically records performance of application under test. LoadRunner provides lot of graph and reporting options for the analysis of performance test results.
6. Various kinds of delays can be identified using LoadRunner. During the test execution, LoadRunner monitors network and server resources continuously.
7. LoadRunner makes performance regression testing very easy – LoadRunner tests are fully automated.

IBM will conduct the Interviews on 05-Dec-09, Saturday.

Job Type : Permanent Job In IBM

Skills : LoadRunner, Mercury Performance Center, Rational Tools etc.

Experience : 5 – 7 yrs in Testing, 3 years relevant in Performance mandatory.

Date of Interview : 5-Dec-09

Interview Location : Gurgaon

Work Location : Bangalore, Chennai & Pune

Request you to send your updated CV at siddique@alpconsultants.com with the following details :

1. Current CTC
2. Expected CTC
3. Notice Period

• After each performance test run, clear the logs for database and application. Large log files may result in inaccurate performance results.
• To mirror the production environment, identify the correct software and hardware for server.

• When a load is being generated on the system; to capture end user response time, use a single GUI client. If you need to generate load by using multiple client computers, you should consolidate data at single client computer / machine and generate results based on the average values.
• During a load test, include a buffer time between the incremental increases of users.
• To create a more realistic load simulation, always Use different data parameters for each simulated user.
• Never overly stress the client. Monitor all systems involved in the test including client that generates the load.
• According to high volume transactions and critical functionalities, prioritize your scenarios.
• If you need to fire concurrent requests, then, to identify bottleneck issues, use think time as zero.
• Do stress testing for critical components of the system to assess their independent thresholds.

Don’ts:

• During load testing, never ever allow test system resources to cross resource threshold limits from a significant margin – Otherwise, it will distorts the data in your results.
• For performance testing, always use an isolated test environment, but that environment should be exact replica of production environment. There should not be any other traffic load.
• Do not put too much stress on the client test systems.

• Replacing the constant values with parameters (in Vuser script)
• Setting the properties and data source for the parameters

Limitation of Parametrization in LoadRunner – Text strings that are not function arguments can not be parametrize.

Creating Parameters:

• In the script View, select the string and right click on it. Then, select the replace with parameter.
• Type the Name of the parameter in the new opened small window / popup. You can also select it from drop down. Then, select parameter type from the parameter type list. You may select any of the available types: Date/Time, file, Group Name, Random number, Unique number, User defined function, or Vuser ID (whichever is appropriate). Next, you may select next row as:
  • Sequential
  • Random
  • Unique
  • Same line as <Pameter_Name>
• At last, you will need to update values on:
  • Each iteration - It instructs the Vuser to use a new value for each iteration
  • Each occurrence - It instructs the Vuser to use a new value for each occurrence of the parameter
  • Once - It instructs the Vuser to update the value of the parameter only once during the execution

Data wizard is a great option for Parametrization in LoadRunner.

• General Vuser Functions: The Vuser functions which have LR as prefix are general functions. These functions enhance the functionality of any Vuser Script to a great extent.

• Protocol specific Vuser Functions: These are the library functions used to enhance the script.
• Standard ANSI C functions: This is enhancing the Vuser script by adding general C functions like Adding Control statements, loops etc.
• Inserting Transactions into Vuser Script: Inserting Rendezvous point and Inserting Comments etc.
• Sending Messages to output:
  • For Log Messages – Lr_log_message
  • For Debug Messages – Lr_set_debug_message and Lr_debug_message
  • For Error and Output Messages – Lr_error_message and Lr_output_message
• Handling errors on Vuser Script during execution: When an error is encountered by Vusers, the Vuser stops the execution of the script. The function lr_continue_on_error can be used to override the continue on error runtime setting. The statements lr_continue_on_error(1); and lr_continue_on_error(0) can be used to mark the segment.
• Synchronizing Vuser Script: Synchronize applies only to RTE Vuser Scripts. The output from the application needs to be synchronized the execution of Vuser scripts.
• Emulate Vuser Think Time: The function lr_think_time is used to emulate user think time. (The time that a Vuser waits between performing successive actions is known as the Think Time). Think time can also be set at run time settings.

Apart from the above mentioned functions / methods, there are two other importants methods / concepts: Parameterization and Correlation. I’ll explain these soon in my next posts.

• Number of Vusers to emulate
• Actions that they perform
• Machines on which they run

Vusers: Vuser as the name implies – Virtual User. LoadRunner replaces human users with Vusers. During a scenario run, Vusers emulate the actions of human users to the server and the application. A scenario can contain multiple Vusers like tens, hundreds, or even thousands.

Vuser Scripts: Vuser scripts describe the actions to be performed by Vuser during the scenario run. During a scenario run, Vuser execute Vuser script which include functions to measure the performance of the server during that particular scenario.

Transactions: Transactions are defined in the Vuser script to measure the performance of the server. These are defined to capture the response time taken by the server for the task.

Rendezvous Points: Rendezvous points instruct multiple Vusers to perform a task or set of tasks at the same time to emulate a heavy user load on the server.

Controller: Controller is used to manage and maintain the scenarios from a single workstation.

Hosts: Host is a machine used to execute the vuser scripts. It enables the vusers to emulate the actions similar to a human user. At the time of scenario execution, the LoadRunner Controller distribute Vusers in the scenario to hosts.

Performance Analysis: A Vuser script includes functions that measure the system performance during load testing sessions. Network and server resources can be monitored during the scenario run. After the successful run of a scenario, performance analysis data can be viewed in reports and graphs.

• Application Deployment Solution: Citrix protocol
• Client – Server: MS SQL, ODBC, Oracle Web Applications 11i, DB2 CLI, Sybase Ctlib, Sybase Dblib, Windows Sockets, and DNS protocols
• Custom: C templates, Visual Basic templates, Java templates, Javascript and VBScript type scripts.

• Distributed Components: COM / DCOM, Corba – Java, and RMI – Java protocols.
• E-Business: FTP, LDAP, Palm, Web (HTTP / HTML), Web Services, and the dual Web / Winsocket protocols
• Enterprise Java Beans: EJB Testing and RMI – Java protocols
• ERP / CRM: Baan, Oracle NCA, Peoplesoft 8, Peoplesoft-Tuxedo, SAP-Web, SAPGUI, SAP GUI / SAP – Web dual and Siebel (Siebel – DB2 CLI, Siebel – MSSQL, Siebel – Web, and Siebel – Oracle) protocols
• Legacy: Terminal Emulation (RTE)
• Mailing Services: Internet Messaging (IMAP), MS Exchange (MAPI), SMTP and POP3
• Streaming: MediaPlayer and RealPlayer protocols
• Wireless: i-Mode, Voice XML, and WAP protocols
• Platforms: NT, 2000, XP, Sun, HP, IBM, Linux

Apart from these environments, LoadRunner also supports the following environments:

Adobe Action Message Format, Adobe Real-time Messaging Protocol, Adobe Flex 2 and Flex 3, Ajax, C Vuser, Citrix ICA and XenApp v4.5, DB2 CLI, Domain Name System, Enterprise JavaBeans, HP WinRunner software, HP QuickTest Professional software, Internet Message Access Protocol, Jacada, Java Vuser, Javascript Vuser, Java Message Service, Microsoft Windows Sockets, SQL Server Informix, MQSeries-Client, MQSeries-Server, Multimedia Messaging Service, Palm, Oracle, VB.NET Template, VBNet Vuser, Wireless Application Protocol etc.

• Does the application response time is adequate as expected by the intended users?
• How the application will handle the expected and unexpected user loads?
• How the application will handle the expected and unexpected transactions required by the business?
• To check the stability of application under heavy loads?

Difference between Functional Testing and Load Testing:

• Functional testing - to check the functionality – To check the functionality of business processes
• Load / Performance testing - to check Stability, Performance, Functionality under load. Examples:
  • Will 1000 concurrent hits crash the server or application?
  • Is response time is as per the specifications?

Performance Testing types:

• Component Testing - To find the performance of each component / tier.
• Load Testing - To find out whether the system can handle the expected load upon deployment in production environment.
• Stress Testing - It is performed to find the breaking point of the application. It is a type of testing that measures whether the application’s environment is properly configured to handle expected or unexpected high transaction volumes.
• Volume Testing - To check the stability of the system / application / server with under  large amount of data over extended time periods.

Benefits / Objectives of Performance Testing:

• To calculate the application Response Time.
• To access the stability of the application.
• To find out which configuration provides the best performance results.
• Capacity Planning – to identify degradation point in performance.
• Acceptance – To find out whether the system is acceptable under heavy volums, loads and stress
• Bottleneck Identification – To identify the bottlenecks in the application
• To identify best platform for the application.

Commonly used Load Testing Tools:

• LoadRunner – HP
• e-Load – Emprix
• Silk Performer – Borland
• QALoad – Compuware
• Rational Performance Tester – IBM Rational
• Web Load – Radview
• Neo Load – Neotys
• Open STA – It is an Open Source tool