Load Testing Blog
Systems never pass the first test
Out of our entire list of services customers, only a handful have satisfied their performance goals on the first test. Of those, all but one had been through a performance testing campaign with us in the previous year. If this is the first time for your project or your organization to undertake performance testing, you are virtually guaranteed to fail the first test.
There is a large list of things that can go wrong with modern web systems – firewalls, load balancers, databases, web servers and, of course, the code. One setting buried deep in an … Continue reading »
Load Test preparation will take longer than you think
Our first-time services customers greatly under-estimate the time required to get the first test configured and ready to run. In these cases, they have employed us to design the tests and develop the testcases – so that part goes pretty quick. What they, and other first-time load testers, don’t account for is the amount of time required for their people to:
Decide what to test
Create a set of test data in the system (e.g. test accounts with usernames and passwords)
Install monitoring tools on servers
Implement/execute backup and restore procedures
Deciding what to test may require interviews with end-users, … Continue reading »
Load Test Often!
Having performance testing results from frequent points in the development timeline can help developers understand the performance impact of various code and system changes. When testing early in the development process on a test rig that is not equivalent to production, the performance numbers are not valuable in their own right, but _changes_ in the performance numbers can extremely valuable. These changes can reveal newly-introduced performance bottlenecks that should be investigated.
For each new development iteration, the previous test (which does not exercise the new functions) should be repeated to determine if the changes had an effect on the performance of … Continue reading »
Site outage report for 27-28 October, 2011
As many of our customers have noticed, the Web Performance website was unavailable for most of the day on Thursday, October 27th. Much of the site was functional mid-day on Friday, but the support system remained inaccessible throughout the day.
Over the weekend all functionality has been restored and we believe that no data has been lost. However, if you have not received a response to any inquiries via e-mail, our contact form, or our support system, please contact us again so that we may both answer your inquiry and investigate the data loss.
Earlier in the week we experienced a hard … Continue reading »
Identify Performance Goals Early…and Measure Them!
When a load test is complete, you will be asked “How did we do?” Do you know how will you answer that question? Our customers come to us and know, for example, that they need their site to handle 1000 users, but they frequently cannot tell us what “handle 1000 users” means to them. You will need to know which metrics are important and what the goals for those metrics are – preferably long before you start testing.
The first step is to determine what you should be measuring. For websites, you will typically be interested in page duration – how long it … Continue reading »
Load Test Early!
Customers occasionally ask us “How early should we begin load testing?”
The answer is to test something, anything, as soon as the architecture is available. Performance problems have a wide variety of causes – from a single line of code to a load balancer setup; from a database schema to a server config file. Early in the development of the software you can catch simple coding problems and fundamental architectural limitations that are much easier to fix before a lot of code has been written.
Now, a word of caution: Testing against a scaled-down development or … Continue reading »
Performance Starts with the Developers
Performance starts with the developers as well as the server and network administrators. High-capacity websites do not happen by accident. To perform well and scale big, the system must be designed, built and configured for performance. That means it must be coded and configured with performance in mind, right from the start. Make sure the developer and admins all understand the levels to which the system must perform. Don’t make them guess what “it has to be fast” means.
Two of the most important tasks the test and project management teams to can do to help:
Identify the performance goals for the … Continue reading »
Bandwidth, latency and geographical distance in load testing
Because we usually talk about latency in tiny numbers (e.g. 20 milleseconds of latency) it is easy to overlook just how big an effect latency can have on the effective bandwidth between geographically distant locations. While running some recent tests to measure the available bandwidth from our cloud engines, I accidentally ran a test between a load engine and a server that were more than 2600 miles apart. Knowing that our server and engine should have both delivered better results, it took me a few minutes to realize that one mistaken click (where to start the load engine) had … Continue reading »
How much bandwidth can we expect from cloud engines during a load test?
I got into a discussion in the Performance Testing group on LinkedIn which raised a question that we had answered internally, but had neglected to share with our customers – how much bandwidth do our cloud engines have available?
Before I proceed, I must make this disclaimer: our cloud engines run on Amazon’s EC2 infrastructure, so the rules that apply there also apply to our cloud engines. Amazon does not make any guarantees for bandwidth, so anytime your test results look suspicious, we recommend doing a quick bandwidth test. Note that there is a Bandwidth Test wizard in Load … Continue reading »
How HTTP Authentication works and why load testers should care
The most commonly used authentication method for websites is a login form on a web page. We’ve all seen them – enter your username and password into fields on the web page and press the Submit or Login button. From the standpoint of the underlying technology, this is no different than submitting any other form – only the names of the fields distinguish them as login or password fields and the security mechanism is implemented within the web application.
Web Performance Consulting
Our experts … Continue reading »