Here are a few quick pointers for checking your Load Test, to see if you are ready to run. Feel free to check back, as this list may be updated from time to time.
Have the appropriate parties been notified of the test?
Do any of the servers have scheduled jobs during the test window? Automatic backups or other maintenance windows that are scheduled during normal down-time or idle-time can have surprising results during a high volume test.
Is the firewall going to be an issue? When using a small set of external IPs to generate load, make sure the firewall administrators have been notified of the test, and have ensured the firewall will not throttle or block traffic from the test IPs.
Are all your engines enabled and ready? Check the Engines View (Window → Show View → Engines) to make sure all the test engines are enabled. If you need more engines, use the “Add Engine” button to connect to a running engine, or to start up engines in the cloud. If you have dedicated engines, it is advisable to make sure the “Local” engine is not enabled, since using dedicated engines is generally advantageous for accuracy and reliability.
Are all your servers enabled and ready? Check the Servers View (Window → Show View → Servers), and make sure that monitoring is enabled for each server used in your test.
Do all of your testcases have a recent & successful replay? If a testcase has not been used recently, run a replay to make sure that the testcase is still valid.
Has a “warm-up” test been run of the site? Warm-up tests exercise the server without applying excessive load, and give the servers a chance to compile & cache any resources that would normally get compiled just after the server had been started up. At the same time, it can test that all load engines are able to connect to the site – which is particularly useful for test sites that may use an IP rule to prohibit access to the server from arbitrary IP addresses.
Check to make sure that the server agent console is up and running. Windows users should either run the agent as a service, or verify that the console is up in a local session. In the later case, make sure sure not to log off from the server during the test, and check that the local security policy is not going to enforce an automatic logoff during the testing window.
For users who are running a server agent on a server which cannot be connected to from the Load Tester controller, go ahead and start manual data collection with the “start-log” command. Make sure the sample period (the “-p”) option is set to the same period as in your Load Configuration.
Except for Load Engines or Server Agents running as a Windows service, do check to make sure that any automatic log-off and automatic shut-down settings have been disabled during the test window. If the system shuts down or logs off while one of the test applications is running, then the application will close, and no further data will be collected from that machine during the test. Depending on how much data had been collected, this may even contribute to partial data loss.
Every system has different requirements, but I hope that this checklist serves as a good starting point!
-Frank, Engineer at Web Performance, Inc.