Leading up to the release of Load Tester 5.0, the Web Performance development team focused heavily on improving our capability to run massive load tests. Today, Load Tester 5.0 is specifically engineered to deliver as many as 1 million virtual users while controlling 500 remote load engines.
This is a “how to” article for Load Tester 5.0 users wanting to run their own massive load tests.
Before you Start
There are a few things you absolutely need. First and foremost is a modern workstation for the controller. By modern, I mean a 64-bit architecture with at least 7 GB of working memory. … Continue reading »
In Load Tester 4.2 we added a new fields view with vastly expanded options to configure each field. In Load Tester 4.3 we added support for JSON as well as ad-hoc custom regular expression and name-value delimited parsers. In Load Tester 5, we are adding support for XML automation. Hierarchical XML data structures that appear in form fields or as HTTP POST content will appear in the fields view, and Load Tester’s application state management (ASM) tool will automatically assign any XML value or attribute for which it can identify an appropriate data source.
Each XML field will be named after … Continue reading »
Load testing tools will report the performance of the system from the end-user perspective – i.e. how long pages take to load in the browser. This information crucial for determining if the site has a performance problem, but does not give a complete picture – it does not tell you why the site is slow. For that, you need performance data from your servers. This will allow you to, for example, correlate periods of slow response time to periods of high CPU utilization. Or find that the number of connections to the database server rises dramatically right before a flurry … Continue reading »
Unless you’ve specifically designed your load test to include them, the most accurate, consistent results will come from tests that do not compete for resources with other processes – particularly sporadic, uneven processes.
If the site’s peak traffic is at 2pm and the least traffic is at 2am, it may be tempting to run your load tests at 2a. Have you checked the backup schedule with the sysadmins? If your backups run at 2:30a, you could spend hours diagnosing strange performance anomalies that could not be reproduced in a second test at 4a. Sounds obvious, right? But it happens all the … Continue reading »
If you are testing in-house, then you will (hopefully) have the opportunity to run many tests. This gives you the opportunity to shake out your testing process ahead of time and to bring individuals in as needed to look at specific problems.
If, however, you are using an outside testing resource service or will be running one big test on a production system, then it is crucial to have the right people involved in the planning and execution of the test. This list includes:
Network administrators
System developers and/or integrators
Web- and app-server administrators
Database admins
Architects
Content Experts
3rd-party services, such as your hosting company
During the load … Continue reading »
When a website is placed under heavy load, it can fail in a wide variety of ways. The site may become unresponsive or return an HTTP status code indicating an error (500). Or it might return a web page that is mostly complete (the headers, footers and navigation are all present) but is missing a critical part that was related to the failure.
If the load test scenarios (testcases) do not validate page content well enough to catch these problems, the results could be misleading. I.e. the system could be failing but the results indicate success. In the process of helping … Continue reading »
In previous posts, I’ve talked a lot about the dangers of not accurately simulating the real-world usage. But there is another side to this story: obsessing too much over an accurate simulation can be very costly. There is one very important thing to understand about load testing: No matter how hard we strive to make tests accurate, they are still only an estimate.
Your load test will never match even one single hour of production usage, not even once, no matter how hard you try. Bold statement? Perhaps. Consider this: even if you could capture the exact usage of real-world customers … Continue reading »
So your license is obsolete, what exactly does that mean? By default a Load Tester license is not set to expire unless it is a temporary license. An obsolete license essentially means that the license version does not match the Load Tester version number.
When a license is mismatched, the following message appears:
As you can see on the License Key Summary, the version code is listed as 4.2, however the current version of Load Tester I have installed is 4.3.
When a license shows up as obsolete, you have two options:
Match up the license version with a downgraded version … Continue reading »
As I described in my previous post on choosing test scenarios, the early tests should focus on the high-traffic scenarios. After testing and optimizing those, the next step is to identify and test the rarely-used but high-risk operations. These are operations that may have an impact on performance that is not proportional to their frequency. Even though these are not as common, they can have large impacts due to their system-wide nature and can cause sporadic performance drops during production hours that defy explanation.
Examples include:
Complex searches
Updates that touch many joined tables at once
Mass changes to data that affect many … Continue reading »
When the first few testcases are ready, the next step is to put them together into a load test. How they are combined and executed will have a big impact on the accuracy of the test.
Testcase ratios – Make sure that each testcase is exercised in the correct ratio relative to the other testcases. For example, it is likely that many more people will be searching for products than buying them – the load configuration should reflect this ratio.
Page ratios – Re-visit the hit rate that will be exerted on each page based on the ratio of testcases and the … Continue reading »
