Load Test preparation will take longer than you think
Our first-time services customers greatly under-estimate the time required to get the first test configured and ready to run. In these cases, they have employed us to design the tests and develop the testcases – so that part goes pretty quick. What they, and other first-time load testers, don’t account for is the amount of time required for their people to:
- Decide what to test
- Create a set of test data in the system (e.g. test accounts with usernames and passwords)
- Install monitoring tools on servers
- Implement/execute backup and restore procedures
Deciding what to test may require interviews with end-users, analysis of log files and research into usage patterns for similar or related sites. All of that takes time.
Systems should be tested with databases fully populated to mirror the quantity of data expected when the system is live. Unless data can be easily pulled from an existing system, populating this data can take days.
Monitoring tools can be difficult to configure, particularly if they need remote access through firewalls or other special network privileges that only busy admins can provide.
When running a load test on a production system, it is imperative that good backup/restore procedures are in place and are executed before (and possibly after) every load test. Some junior testers are surprised to learn that this is almost as important when working on a test rig. In some cases, system performance is dictated by the data in the system. So if you cannot restore a snapshot of the data as it existed before a load test began, then you may not be able to reproduce the test results. That can be crucial to fixing the problem and is required to verify that system improvements had the desired effect.
If you are new to load testing, you should plan carefully how much time you need for the preparation. And then triple it!
Chris Merrill, Chief Engineer
When his dad brought home a Commodore PET computer, Chris was drawn into computers. 7 years later, after finishing his degree in Computer and Electrical Engineering at Purdue University, he found himself writing software for industrial control systems. His first foray into testing software resulted in an innovative control system for testing lubricants in automotive engines. The Internet grabbed his attention and he became one of the first Sun Certified Java Developers. His focus then locked on performance testing of websites. As Chief Engineer for Web Performance since 2001, Chris now spends his time turning real-world testing challenges into new features for the Load Tester product.