Checklist: Is your Load Test ready to run?
Here are a few quick pointers for checking your Load Test, to see if you are ready to run. Feel free to check back, as this list may be updated from time to time.
Test Planning:
-
Have the appropriate parties been notified of the test?
-
Do any of the servers have scheduled jobs during the test window? Automatic backups or other maintenance windows that are scheduled during normal down-time or idle-time can have surprising results during a high volume test.
-
Is the firewall going to be an issue? When using a small set of external IPs to generate load, make sure the firewall administrators have been notified of the test, and have ensured the firewall will not throttle or block traffic from the test IPs.
From the Controller:
-
Are all your engines enabled and ready? Check the Engines View (Window → Show View → Engines) to make sure all the test engines are enabled. If you need more engines, use the “Add Engine” button to connect to a running engine, or to start up engines in the cloud. If you have dedicated engines, it is advisable to make sure the “Local” engine is not enabled, since using dedicated engines is generally advantageous for accuracy and reliability.
-
Are all your servers enabled and ready? Check the Servers View (Window → Show View → Servers), and make sure that monitoring is enabled for each server used in your test.
-
Do all of your testcases have a recent & successful replay? If a testcase has not been used recently, run a replay to make sure that the testcase is still valid.
-
Has a “warm-up” test been run of the site? Warm-up tests exercise the server without applying excessive load, and give the servers a chance to compile & cache any resources that would normally get compiled just after the server had been started up. At the same time, it can test that all load engines are able to connect to the site – which is particularly useful for test sites that may use an IP rule to prohibit access to the server from arbitrary IP addresses.
For Server Agents:
-
Check to make sure that the server agent console is up and running. Windows users should either run the agent as a service, or verify that the console is up in a local session. In the later case, make sure sure not to log off from the server during the test, and check that the local security policy is not going to enforce an automatic logoff during the testing window.
-
For users who are running a server agent on a server which cannot be connected to from the Load Tester controller, go ahead and start manual data collection with the “start-log” command. Make sure the sample period (the “-p”) option is set to the same period as in your Load Configuration.
For Load Engines:
- Make sure the Load Engine console is up and running.
- For new engine installs, make sure the engine is configured with an optimal amount of memory. Using an Amazon EC2 Cloud Engine is automatically optimized and configured for use with Load Tester. If the server must be tested from inside the local network, download an “Instant Load Engine” ISO to automatically boot, configure, & run the load engine on an existing workstation without other applications running. Use the “Configure IPs” in the Engines View to configure which IPs each engine should use to originate load from.
- For Windows users, make sure either to run the engine as a service, or not to log-off from the engine while the test is running. For Linux users, we recommend launching the Load Engine with the “screen” command, which ensures that a session & console will remain open for the Load Engine application, while allowing you to logout of your primary console.
For all Windows Machines:
Except for Load Engines or Server Agents running as a Windows service, do check to make sure that any automatic log-off and automatic shut-down settings have been disabled during the test window. If the system shuts down or logs off while one of the test applications is running, then the application will close, and no further data will be collected from that machine during the test. Depending on how much data had been collected, this may even contribute to partial data loss.
Every system has different requirements, but I hope that this checklist serves as a good starting point!
-Frank, Engineer at Web Performance, Inc.