Identify Performance Goals Early…and Measure Them!
When a load test is complete, you will be asked “How did we do?” Do you know how will you answer that question? Our customers come to us and know, for example, that they need their site to handle 1000 users, but they frequently cannot tell us what “handle 1000 users” means to them. You will need to know which metrics are important and what the goals for those metrics are – preferably long before you start testing.
The first step is to determine what you should be measuring. For websites, you will typically be interested in page duration – how long it takes to display a page in the browser. This will usually be associated with another metric for the amount of load – the most common is simultaneous users. Other common metrics for load are hits/sec, transactions/min and page-views/hour.
Once you know what should be measured, you must assign goals to them. For example “All pages must load within 4 seconds under a load of 1000 users.” Without specific, concrete goals, you will not be able to give a specific, concrete answer to the “How did we do?” question.
Next, look for exceptions to these goals. Are there some pages that are ok to be slower, such as the final check-out page in a shopping cart or a site-wide search? Are there some that should be faster, such as the “Add to cart” button?
Knowing what to measure and what the goals are will allow you to design tests to accurately assess the system’s ability to meet those goals.
Chris Merrill, Chief Engineer
When his dad brought home a Commodore PET computer, Chris was drawn into computers. 7 years later, after finishing his degree in Computer and Electrical Engineering at Purdue University, he found himself writing software for industrial control systems. His first foray into testing software resulted in an innovative control system for testing lubricants in automotive engines. The Internet grabbed his attention and he became one of the first Sun Certified Java Developers. His focus then locked on performance testing of websites. As Chief Engineer for Web Performance since 2001, Chris now spends his time turning real-world testing challenges into new features for the Load Tester product.