Design a Load Test that Measures What You Really Want to Know - Web Performance
Menu

Design a Load Test that Measures What You Really Want to Know

When the first few testcases are ready, the next step is to put them together into a load test. How they are combined and executed will have a big impact on the accuracy of the test.

Testcase ratios – Make sure that each testcase is exercised in the correct ratio relative to the other testcases. For example, it is likely that many more people will be searching for products than buying them – the load configuration should reflect this ratio.

Page ratios – Re-visit the hit rate that will be exerted on each page based on the ratio of testcases and the frequency of each page in the testcase. Be sure that this is still close to the expected usage.

Database size – Make sure you have an sufficient quantity of data in your system to reflect real-world usage. If your system is expected to have 100,000 rows in the “products” table and your test system only has 100, then the results will be questionable.

Number of users – There can be much confusion about the number of total users on a system and the number of simultaneous users, especially among the less-technical team members. Be sure that you are testing realistic load levels. A system with 100,000 registered users is likely to only have a fraction of that (typically less than 10,000) using the site at any one time. Testing beyond the expected load can be beneficial – just make sure everyone understands it is happening.

Load ramping – If a system is expected to support 1000 users during peak times, then a test that goes from zero to one thousand users in 30 seconds and then maintains that level for the entire duration of the test is most likely not measuring a realistic scenario. Most systems should ramp gradually to the peak load. There are exceptions, of course, but we frequently find that testers over-estimate the need for testing traffic spikes. In addition to being more accurate, a stepped ramp test configuration can generate much more useful data as well.

3rd-Party Systems – If your system includes resources from 3rd parties, carefully consider the impact of including them in the test. In some cases, these systems are crucial and must be included. But in many cases they are not – if not, leave them out. If these systems gather data, such as click-tracking systems, make sure the data from the testing period is purged or otherwise ignored by those who will analyze that data. If these systems have a financial impact, such as 3rd-party ad providers, make sure that you will not be incurring charges for your company (or worse – a third-party who is not directly associated with your organization). Be sure the 3rd-parties have granted permission to test their system and are aware of the testing schedule.

Simulate the available client-side bandwidth accurately – How will your users be accessing your site? From their home via cable modem? From a clogged corporate T-3 line? The better load testing tools make it easy to simulate different client-side bandwidth profiles – be sure you are using it!

The common theme between all the above topics is that when they are done wrong, they can lead a serious misinterpretation of the test data.  Make sure that what you are measuring is what you really want to know!

Chris Merrill, Chief Engineer

Add Your Comment

You must be logged in to post a comment.

Resources

Copyright © 2024 Web Performance, Inc.

A Durham web design company

×

(1) 919-845-7601 9AM-5PM EST

Just complete this form and we will get back to you as soon as possible with a quote. Please note: Technical support questions should be posted to our online support system.

About You
How Many Concurrent Users