Baseline the Performance of the Old System
If your project is replacing an existing system, it is immensely valuable to establish a baseline for the new system based on the existing system. Start by analyzing the usage patterns of the existing site. What operations are most common? What paths are users following through the site? How many users are accessing the system at various times throughout the day? Wherever possible, this data should come from system logs rather than assumptions and guesswork. Then start designing your test:
- Create a mix of scenarios that account for 70-80% of the site usage
- Add any other scenarios that are known, or suspected, of having poor performance or affecting the overall performance of the system.
- Run a test that simulates various load levels corresponding to the load levels seen throughout a typical day
- Verify that the results agree with actual performance of the system during real-world use. If it does not, then adjust the test design and scenario mix to better match the real-world usage. Return to step 3 and try again
Having an accurate benchmark of the existing system yields several benefits:
- Performance measurements of the existing system can be compared objectively to the new system
- Research on the usage patterns of existing users will greatly improve the estimates of usage of the new system – improving the design and accuracy of tests on the new system