Baseline the Performance of the Old System
If your project is replacing an existing system, it is immensely valuable to establish a baseline for the new system based on the existing system. Start by analyzing the usage patterns of the existing site. What operations are most common? What paths are users following through the site? How many users are accessing the system at various times throughout the day? Wherever possible, this data should come from system logs rather than assumptions and guesswork. Then start designing your test:
- Create a mix of scenarios that account for 70-80% of the site usage
- Add any other scenarios that are known, or suspected, of having poor performance or affecting the overall performance of the system.
- Run a test that simulates various load levels corresponding to the load levels seen throughout a typical day
- Verify that the results agree with actual performance of the system during real-world use. If it does not, then adjust the test design and scenario mix to better match the real-world usage. Return to step 3 and try again
Having an accurate benchmark of the existing system yields several benefits:
- Performance measurements of the existing system can be compared objectively to the new system
- Research on the usage patterns of existing users will greatly improve the estimates of usage of the new system – improving the design and accuracy of tests on the new system
When his dad brought home a Commodore PET computer, Chris was drawn into computers. 7 years later, after finishing his degree in Computer and Electrical Engineering at Purdue University, he found himself writing software for industrial control systems. His first foray into testing software resulted in an innovative control system for testing lubricants in automotive engines. The Internet grabbed his attention and he became one of the first Sun Certified Java Developers. His focus then locked on performance testing of websites. As Chief Engineer for Web Performance since 2001, Chris now spends his time turning real-world testing challenges into new features for the Load Tester product.