Load Testing Metrics: Comparing to the Median
Once you’ve been using Load Tester for a while, it’s easy to find the minimum, maximum, and average page durations. But how does that compare against the median page duration, or percentiles?
Starting with Load Tester 3.6, these measurements can now be made, but it’s been made easier to find in Load Tester 4.1. First, you will need to enable “Detailed Page Durations”. In Load Tester 4.1, simply make sure the “Detailed Page Durations” option is checked on the “Data Collection” section of the Load Test Configuration editor.
With that setting enabled, just run your Load Test as normal. When examining the Load Test report, use the “Settings” button, and turn on the option to compare the Duration Goal against the Detailed durations. The median is compared to the goal by requiring only 50% of the pages to meet the goal – other goals can also be specified and analyzed.
Press OK, and the Load Test report will refresh to show you when each selected percentile has exceeded the configured page duration goal. By looking at the report for each page in the testcase, you can quickly get a feel for what percentiles of users were not receiving acceptable performance results.
We hope that this feature helps to give you a feel for your performance results. However, our friendly support staff is always willing to answer any questions you may have! Happy Testing!
-Frank
Engineer at Web Performance