When diagnosing performance problems with our clients websites, it is not uncommon for the bandwidth to come into question. Very often, the bandwidth chart will show a distinct plateau and it can be difficult to determine if this is cause or effect. This example shows outgoing server bandwidth (green) as the users (blue) ramp up. It could be interpreted as a bandwidth limitation around 8Mbps. In this case it is not – the limitation is elsewhere in the system.
It can be challenging to prove that bandwidth is the problem. If the bandwidth is not reaching the known limits (i.e. what … Continue reading »
Nearly every useful web application is backed by some sort of database – usually a relational database. Many database administrators (DBAs) have never participated in a load-testing campaign and are sometimes even considered tangential to the load testing process.
However, your DBA is a crucial part of the load-testing team. In addition to the obvious skills of database performance analysis and tuning, the DBA brings other advantages to the project:
insight into how the application interacts with the database that may not be obvious from the perspective of other team members
ability to quickly backup and restore various database snapshots to increase consistency … Continue reading »
Is your load balanced website using more bandwidth under load than a single server would? In a previous article (“Status code 200 didn’t match expected: 304”), we discussed how a website may return full data content back to a client, even when the browser had the resource cached. If you believed that adding an extra server and a load balancer will increase your performance, this can be an unsettling surprise. The problem arises when two web servers are delivering different Entity Tags for the same static resource:
In this example, we have two requests made to the same public URL, … Continue reading »
I’m in Orland this week attending the 2010 StarEast conference for software testing, and was surprised at how much hadn’t changed. Its been over a decade since my last tech conference, and while the names have changed, the actual recommended practices haven’t changed much at all. Speakers are still preaching the values of early testing, automated test suites, and automated build processes that I learned as a young engineer at Sun Microsystems 20 years ago, but now they’re called “agile” and “ATDD“.
The only controversial statement of the day was in Jeff Payne’s opening keynote, where … Continue reading »
I ran across this article yesterday. It is nearly two years old, but the content is still entirely relevant. If you are building a system that needs to scale out to handle a large load, then you’ve probably already read this or something similar…if not, then it is worth your time.
What is the relevance to Load Testing, you ask? In section 5 author, Simon Brown, says”
“… set measurable goals throughout the system, verify and measure the real performance and consider performance at all stages of the project.”
For a project manager, a key part of the job is reducing risk … Continue reading »
I ran across these slides that present an introduction to load testing for developers. Being primarily a developer myself, I thought it might be worth a read. It is pretty light – it would be nice to get a video with the entire presentation. But it does raise some good points that many developers can benefit from. I particularly liked that the author (Simon Brown) differentiated between performance and scalability – a difference that is easy to miss. He also points out that you cannot predict system performance, either from diagrams and charts or by extrapolating results from past … Continue reading »
Load test, performance test, stress test, soak test…there are numerous of terms to describe these efforts and even within the testing community there is not unanimous agreement. These definitions line up well with the most common usages:
Performance Testing – Any testing that is focused on measuring the performance of a system. This is in contrast to functional testing, which is primarily focused on measuring the correctness of the system. When you use a stopwatch to measure page load times in your browser, you are performance testing.
Load Testing – Performance testing with the goal of measuring the performance of a system … Continue reading »
So you’ve got a shiny new site designed and ready for testing. You may get some questions like: who are your users, and what will they do while on your site? Sometimes questions like these can be answered with analytics, sometimes with educated guessing. However, for those who can’t answer these simple questions, some options still remain. Among those, one option may soon be a reality: monkeys – lots of monkeys.
In the past, deploying monkeys has been somewhat problematic, as it required extensive training to keep them stationary while using a keyboard & mouse while using computer. This limitation may … Continue reading »
This tutorial will show you how you can use performance goals to customize the reports to give you information that is relevant to the individual performance requirements of your system. The goal is to help you learn to draw better conclusions from the data and get more accurate results. You’ll also spend less time poring through the data.
We get this question a lot. I would really like to give an easy answer to that question. But since the answer depends heavily on the nature of your organization and the testing demands, I’m instead going to list some of the issues that we consider when helping our customers make these decisions.
Schedule – are you in a hurry? If you have an impending deadline and the performance testing needs to be completed immediately, then outsourcing may be the best choice. With an experienced partner, the first load test can be completed in less than a week. If the schedule … Continue reading »