How To: Run a Load Test to rule out Bandwidth Limitations
Bandwidth can be critical to the performance of a website. Since this is a rather obvious fact, it is common for our customers to watch for bandwidth as the limiting factor early in a testing effort. This is especially true when the load test results do not point immediately any other limitation, such as 100% CPU on the application server.
When faced with a poorly performing system but no obvious signs of limitations, it is natural to focus all energies on finding the cause of the problem. This might mean spending hours poring through firewall and router configurations, looking for anything that might be limiting through-put. Frequently it is more productive to use the process of elimination – work on proving that other factors are not the cause. Bandwidth is one that is particularly easy to eliminate since all that is required is a test result showing bandwidth consumption exceeding the result in a failing test. You do not need to determine the maximum bandwidth the site can supply…exceeding previous limitations is enough to prove that bandwidth was not the limiting factor.
Choosing a URL to test
So the question becomes: How do I configure a load test specifically designed to test bandwidth?
Here are some of the rules that we follow when running a bandwidth test:
- Avoid as much application logic as possible. The point is to test the bandwidth, not the application. A web page that requires database calls, for example, must be avoided.
- Test at the HTTP layer. It may be temping to test at a lower layer, but remember the goal – prove that bandwidth is NOT the limiting factor. Other protocols may receive different treatment by firewalls, load balancers and routers — which could result in the wrong conclusions based on the results. Testing at a lower level will help you find the problem AFTER determining that bandwidth IS the cause, so save that for a later step.
- Test URLs that represent static content. This is another way of saying #1, but it is important enough to mention twice. Most of the modern, dominant web servers (e.g. IIS and Apache) can serve static content with blinding speed – in most cases they are fast enough to overwhelm the network adapter on the host machine. If it is served by the web servers directly from disk, it is probably a good choice.
- Choose large URLs. If we were testing latency, lots of small HTTP transactions would be fine. We want to avoid, as much as possible, the latency of the initial connection. The test should test bandwidth consumption OUT of the server, so larger is better. Something in the 50-500kB range is good.
- Choose non-compressible content. If any network component along the way performs compression on the content, the process could affect the timing. One way to avoid this is to choose content types that are already compressed, such as multimedia files – e.g. JPG, PNG, MP3, MP4, etc. Network devices will see these as non-compressible and pass them through unchanged.
- Choose non-compressed content. This is only slightly different from #5 – check the response headers to ensure the web server is not compressing the content. Well-optimized web servers will typically be configured to compress text content, such as HTML, CSS and JS. If the server is also configured to cache the compressed versions, the effect will be negligible – but to be safe, avoid them anyway.
- Use persistent connections. Opening a new connection for each request adds additional latency to every request, thus skewing the test results. Check the headers returned from the server(s) to ensure that connections are being re-used. Note that HTTP 1.1 uses persistent connections by default, but servers can still be configured to close the connection after a few requests.
- Avoid SSL URLs. Encryption is a CPU-expensive process and will therefore introduce delays which invalidate the test results.
Creating a test configuration
If you have made it this far, you should have some URLs ready to test that meet the above criteria. The next step is to create a testcase in Load Tester utilizing one of those URLs. To run the test:
- Open Load Tester and start a recording
- Visit the web page containing the chosen URL, meeting the above criteria, in the browser
- Stop the testcase recording
- In the testcase editor, locate the chosen URL, then copy and paste it several times (5-10). Remove all other URLs from the page. You should now have a single page in the testcase with the test URL appearing 5-10 times in that page.
- In the Navigator view, select the testcase and choose Properties… from the context menu
- On the Restart Options tab, choose “Web Page” under the Restart test from and Run testcase to headings
- Under both of the above headings, choose the one (and only) web page from the drop-down that is now enabled
- Under Reset User State, choose Never
- Press OK to accept the updated testcase properties
- Returning to the Navigator view, choose New Load Configuration… from the context menu for the testcase
- Set the duration of the test: 1-2 minutes is sufficient for most cases
- Choose the number of VUs to start the test. If you are using a demo license key, choose 10. If you have a full Load Tester license, you may use more users as desired.
- In the first row of the testcase table below, set Speed to unlimited
- Select the Start button to start the test
Interpreting the results
While the test is running, watch the Bandwidth Out chart to determine the total outgoing bandwidth achieved by Load Tester. (note that this counts bytes at the HTTP layer, so TCP/IP overhead is not included. In most cases this is negligible – under 1%).
If the bandwidth reported exceeds the bandwidth achieved during the failed load test then congratulations are in order – you have proved that bandwidth between the load engine and the web server is NOT the bottleneck!
If the bandwidth reported in this test is similar to the limit seen in previous load tests, then the next step is to eliminate the possibility that the load engine itself has a bandwidth limitation. This is easy – run another test, but double the number of load engines used during the test. For instance, if the first test used only the local (built-in) load engine, install a second and use it, in addition to the local engine. If the results are the same, the engine is not the limiting factor. Something between the load engine and the web server is the bottleneck. The next step is to get your network administrator involved to help locate the point of congestion.
Summary
When faced with a system that has reached its capacity but the cause is not obvious, it can be difficult to decide what direction to investigate first. Running a simple bandwidth test can quickly eliminate bandwidth as a contender – allowing your team to investigate the remaining possibilities.
Chris Merrill, Chief Engineer at Web Performance
When his dad brought home a Commodore PET computer, Chris was drawn into computers. 7 years later, after finishing his degree in Computer and Electrical Engineering at Purdue University, he found himself writing software for industrial control systems. His first foray into testing software resulted in an innovative control system for testing lubricants in automotive engines. The Internet grabbed his attention and he became one of the first Sun Certified Java Developers. His focus then locked on performance testing of websites. As Chief Engineer for Web Performance since 2001, Chris now spends his time turning real-world testing challenges into new features for the Load Tester product.
Get web performance news
Occasional load-testing tips, performance-engineering notes, and product updates. No spam — unsubscribe anytime.