Load Testing an AJAX Application
Part 1: A simple example using KnowledgeTree
Christopher L Merrill
©2007 Web Performance, Inc; July 17th, 2007; v1.1
Introduction
Most of us are relieved that web applications are starting to behave more like desktop applications - i.e. smarter and easier to use. A little wow factor here and there doesn't hurt either. Ease of use always has a cost, so it should come as no surprise that the AJAX applications are more difficult to develop. But you may be surprised that they can also be more difficult to load test.
The reason is fairly simple. Traditional web applications are relatively easy to model and simulate. The state of a user session, at any given time, is a combination of the client-side and server-side state of the session. The client-side state consists of the state of the browser (i.e. cookies) and the representation of the current web page (which includes form fields and the query parameters in each link). The server-side state does not require any effort on the part of a load testing tool if the client-side state is simulated correctly. AJAX adds an entirely new dimension to the client-side state. Javascript that runs after the page loads, either automatically or in response to user actions, can change the page structure, change the value of form fields, change links etc.Some classes of testing tools, such as functional testers, can simply control a real browser and force mouse movements and keystrokes into the browser - allowing the browser to function normally. Load testing tools, however, cannot afford this approach. Imagine trying to run 25 browsers on a single machine. Besides the fact that the performance would be a serious limitation, most browsers will share cookies, cache, etc. among multiple windows - making an accurate simulation of multiple individual users impossible.
As a result, most load testing tools simulate users at the HTTP layer - simulating the traffic that is generated by the browser, rather than simulating the entire browser. Web applications that utilize the traditional page-at-a-time design can be easily simulated with Load Tester™. Load Tester™ uses sophisticated state analysis algorithms to analyze and automatically configure the testcase so that dynamic field values (such as the _VIEWSTATE hidden fields used by the .NET framework) are simulated with little or no effort required from the tester. This is made easier by the fact that every browser submits forms using a standard format (application/x-www-form-urlencoded). Web applications that utilize AJAX methods, however can implement whatever data formats they choose to exchange information asynchronously with the server. JSON and XML are popular choices, but there are many others. While we are constantly expanding our format support and improving our analysis algorithms, there will always be testcases that require some manual configuration.
The purpose of this guide is to demonstrate an example of this configuration process using our load testing software (Load Tester™ 3.4) and a popular open-source application, KnowledgeTree. While this application is primarily a traditional page-based web application, it has some AJAX features that require custom configuration to be simulated correctly. Part 2 of this article will demonstrate a example of an fully AJAX application.
Prerequisites: This article assumes the reader is comfortable with the basic use of Load Tester™. At a minimum, you should be familiar with basic load testing concepts and the process of using Load Tester™ to record and configure a testcase. Our introductory videos and Load Testing 101 article are great ways to get started.
The KnowledgeTree scenario - add user to a group
In this scenario, an administrator logs into the system and adds a user to a group. In this example Joe Tester2 is added to the group named users (which already has Joe Tester as a member), When the administrator first arrives at the Add Users form, it looks like this:
As the administrator types in the Filter field, the list of available users matching the filter text is refreshed asynchronously, as seen below:
Eventually, as the administrator types more of the name, he sees the desired name, selects it in the list and uses the >> button to move it into the Assigned Users list. The changes are then saved and a logout is the final step of the testcase. Joe Tester2 is now in the group.
Create and prepare the testcase
The first step in testing an application is creating a testcase. In Load Tester™, like many tools, recording a testcase involves walking through the testcase steps in the browser -- just like a real user would do. After recording (which is referred to as a script in some tools), most testcases will require some configuration before they can be simulated correctly. This typically involves configuring cookies and dynamic fields that hold application state information on the client. Load Tester™ will automatically open the configuration wizard before any replay or load test is attempted, but it may also be invoked manually from the Edit->Configure menu.Replay before customization
After the recording is completed and the default replay configuration is applied (the wizard applies the default configuration rules), the testcase can be immediately replayed accurately. In Load Tester™, this can be accomplished by pressing the Replay button in the toolbar. As the replay proceeds, each page will be displayed in the content view. The replay will complete without any errors reported. However, since the state of the application has changed (user Joe Tester2 is now in the group users), the application returns this message in the page after the administrator has saved the change:
Load Tester™ has accurately replayed the testcase exactly as it was recorded. While this is a good thing (indeed, we strive to achieve exactly this result), it would not make for a very useful load test. Why? Because in the current state, using this testcase in a load test would simulate many users, all logging in using the same identity (username/password) and trying to add Joe Tester2 to the group users, which he is already a member of. To get an accurate simulation of real-world use, the load test should involve many unique users performing a similar action (adding different users to the group). Note that this is a crucial step in preparing for a load test. In nearly all applications, the testcase must be customized so that it may be used to simulate many users performing many similar actions. Note that this step is not specific to AJAX applications or to Load Tester™.
So, to simulate this scenario more accurately, two testcase customizations are required:
- simulate unique user identities
- simulate unique field entries
Simulate user identities
The first step is to change the testcase so that each simulated user (a.k.a VU or virtual user) enters a different username and password - so that the testcase can simulate multiple different users entering the system. This is easily accomplished using the User Identity wizard -- from the Edit menu, choose Configure->User Identity. After completing the wizard, you can see the result in the Fields view. In the image below, the username and password fields have been configured to pull values for the fields dynamically from a Dataset (the dataset must be filled with the appropriate values). After this customization has been performed, replaying the testcase simulates a user logging into the site using the usernames and passwords contained in the dataset.
Simulate unique field entries
Now we can move onto the next problem - adding a different user to the group. First, we need to determine what happens when the user types in the Filter field. If you have access to the application developers, asking them is probably the quickest way to answer the question. In many cases, you can easily find the answer just by inspecting the fields in the Fields view. In the screenshot below, you can see the fields named filter contain the text that was entered in the Filter field on the group member management page. Note that there are 3 instances - one for each request that was generated by the javascript as the text was entered in the Filter field.
In order to accurately simulate load on the name-filtering function, each VU should be sending different values for these fields - to simulate a real user entering parts of different people's names. The data to populate these fields will need to be supplied to Load Tester™ in a dataset (most tools have a similar capability). Each field can then be easily configured to use the data to supply values during the replay. Pressing the edit button for the field (or double-clicking the Modifier column, M?) will open the field editor. Here is an example that has been configured to use the third column from the FilterValues dataset.
The dataset to provide these values would look something like this:
In other tools, you may need to edit the text-based script to indicate that a field value should be supplied from a dataset (rather than the originally recorded value), but the concept is the same.
Simulate unique user additions
Now we are ready to address the final part - simulating the addition of a different user to the group. First we must locate the field that identifies which user is being added to the group. After a little digging we have located it:
Evidently the users_items_added field contains an identifier associated with the user...not the name that was displayed in the list box. This should not be surprising, but it does beg the question - how will we know the value of this field?
There are two obvious options for providing the value of the users_items_added field:
- provide the possible value of this field in the form of a dataset
- determine the value dynamically based on the content received from the server
But for many testcases, this may not be the most practical method. For a more flexible simulation (and to finally get to the AJAX part of this article), the next section will demonstrate how to extract the value dynamically from a previous response and use that value in place of the recorded value of the field. The first step in this task is to determine where the value "27" was used. To do this, select the user_items_added field in the Fields view and press the Display in Editor button at the top of the view. This will select the transaction in the testcase editor. In this case, it is the transaction labeled foward in the Group Management | Knowledge Tree [3] page.
The second step is finding the source of the value "27". Since it corresponds to a value in the application database, it MUST have been returned from the server in a response (if it wasn't, how would the browser know what value to send?). The most obvious place to look is the previous transaction. Indeed, opening the content and fields views (moved to side-by-side positions in the example below) and selecting that transaction (labeled admin.php [4]) yields the source:
When the value "tester2" was sent, the response contained the users full name and ID. The next step is configuring an extractor to pull the value "27" from the response and place it into a user state variable, so that it may be used in a later transaction.
- Open the Actors view
- Select the Extractors tab
- Press the Add extractor... button (+)
- Configure the extractor by supplying the prefix, suffix and variable name, as shown below
- Verify that the correct value (27) will be extracted
In other load-testing tools, you can accomplish the same behavior by editing the script - typically using a regular expression to locate the value for extraction.
As a result of this extractor, a user state variable named user_id will be created and after this transaction completes, an extractor will attempt to extract the required value from the content returned by the server. If the extractor cannot locate a value using the provided parameters, an error will be recorded.
After accepting this configuration, the final remaining step is to configure the users_items_added field to use the value in the user_id field:
- Return to the Fields view
- Select the transaction containing the users_items_added field (the first transaction in the Group Management | Knowledge Tree [3] page)
- Configure the field to use the user_id user state variable (this is similar to the step for configuring a field to use a dataset value, demonstrated earlier)
(again, you can accomplish this in most load-testing tools by editing your test script)
Replaying the testcase will now extract the appropriate value from the admin.php [4] transaction and use it to replace the value of the users_items_added field in the following request.
If we open the dataset editor and change the value of the third field to "fdummfi Tester" (an example of a user name - see the examples at the start of the article) in the first row, we can replay the testcase and the user named "fdummfi Tester" will be added to the group. During or after the replay, we can see this behavior by reviewing the confirmation page in the testcase:
We can also review the field values used in the replay to confirm the correct operation:
It's not intuitively obvious that "9" is the correct value of the users_items_added field when "fdummfi Tester" is the selected username -- you would need to verify this in your database. Alternatively, you can simply log into the application and verify that the correct user was added to the group.
Summary
This article has demonstrated how Load Tester™ can be easily used to simulate scenarios in applications that make use of AJAX for asynchronous page updates.The article shows examples of these testcase customization techniques:
- Use unique user identities (logins) via the User Identity wizard
- Use unique values in AJAX requests used in asynchronous page updates
- Extract dynamic values from AJAX responses and use that value in later pages to enable an accurate simulation of a simple AJAX implementation
Feedback & Comments
Comments about this report may be posted at the company blog post.
Version History
v1.0 - 1st public release (19 jul 2007)
v1.1 - email cleanup (23 Jan 09)