Load Testing Blog
Testing tip: Make it Fail
We recently had a case of debugging a testcase where Load Tester reported an error with a page that looked and appeared just as it should, except it was missing a link the user needed to move forward. Debugging this case was easy: examine the replay two pages prior to the error, and the server had added a red error message to the page indicating the selected criteria was not valid.
So why didn’t Load Tester flag this error from the server? Beneath the error message, the page was exactly identical to the expect page: form fields were present and filled … Continue reading »
A Unique Browser approach to Fast: Opera Turbo
We’ve seen that modern browsers have made great strides to improve efficiency in rendering and javascript performance. But, the differences seem to be less noticeable when the network is stubborn. On a recent trip, I ran into just this problem. During the evening, the hotel’s local Wi-Fi seemed at peak capacity, the DNS servers were sluggish, and just getting connections opened often added a couple seconds. Getting onto common sites like Slashdot seemed to take 30 seconds or more from the overburdened local connection.
This seemed like a golden opportunity to try out one of Opera’s new features: Opera … Continue reading »
Tips for Successfully Testing from the Cloud
With Load Tester 4, we’ve made it easy to harness Amazon’s power for Load Engines quickly and inexpensively. Once you’ve familiarized yourself with the basics, here are a few rules of thumb:
1. Get to know the Amazon EC2 Console . The console will quickly tell you what instances you have running – and subsequently what you are getting billed for.
Note that the console shows you only data for a selected region, so you should check each region separately. Load Tester will automatically prompt you after a test and before shutting down Load Tester if you have running engines … Continue reading »
Load Testing Metrics: Comparing to the Median
Once you’ve been using Load Tester for a while, it’s easy to find the minimum, maximum, and average page durations. But how does that compare against the median page duration, or percentiles?
Starting with Load Tester 3.6, these measurements can now be made, but it’s been made easier to find in Load Tester 4.1. First, you will need to enable “Detailed Page Durations”. In Load Tester 4.1, simply make sure the “Detailed Page Durations” option is checked on the “Data Collection” section of the Load Test Configuration editor.
With that setting enabled, just run your Load Test as normal. When examining the … Continue reading »
Keeping Your Load Testing Cases Current (How To)
Maybe you’ve just finished your first crack at testing, handed some results up to the development team, and are just got a note that the dev team has revamped the site in preparation for the next test. Or maybe you completed testing last month and are ready to retest the site against any regressions. Are all the testcases setup from the last round of testing still going to work? Are you going to have to create all new testcases? Here are a few quick ways to find out:
1. Run a replay with your existing testcases. If the replay fails … Continue reading »
Simulated DNS Load Balancing Improves Testing Accuracy
With the release of Load Tester 4.1, we’ve been discussing some of the new features available in this release. One of those features is Simulated DNS Load Balancing. This feature offers improved testing accuracy over previous versions of Load Tester.
This feature is designed for systems which may use DNS to alias a single hostname to many different IP addresses, allowing the DNS system to provide load balancing. This works effectively when the users or load engines greatly outnumber the number of servers, and are well distributed and not in a common location. For users that wish to test their system … Continue reading »
Keeping your HOSTS files in sync with Load Tester 4.1
Load Tester 4.1 has a number of exciting new features, but in this post, let’s discuss one of the less obvious features: a hosts file which is automatically synchronized with all your Load Engines.
Consider how often this happens: the development environment is mirrored from your production environment, and then updated to an internal version of the application, ready for testing. Since the development environment is otherwise a mirror of the production system, it is configured with the same virtual hostnames as your production environment, and / or it may share the same SSL certificates which identify it with the hostname … Continue reading »
Is your Load Balancer increasing your Bandwidth usage?
Is your load balanced website using more bandwidth under load than a single server would? In a previous article (“Status code 200 didn’t match expected: 304”), we discussed how a website may return full data content back to a client, even when the browser had the resource cached. If you believed that adding an extra server and a load balancer will increase your performance, this can be an unsettling surprise. The problem arises when two web servers are delivering different Entity Tags for the same static resource:
In this example, we have two requests made to the same public URL, … Continue reading »
Status code 200 didn’t match expected: 304
Suppose you’ve got a testcase that either worked in the past, or just works sometimes, but now seems to be giving you an error: “The status code of the response (200) did not match the status code of the response in the Testcase (304)”. This particular error message is probably not a serious error, but it usually means that your test isn’t quite emulating what a real browser would do.
Fixing this problem in Load Tester is usually easy. Just right click on your testcase, select Properties, and then go to the “Restart Options” tab. Simply check the last checkbox: “Update … Continue reading »
Next Generation Testing: no test plan required
So you’ve got a shiny new site designed and ready for testing. You may get some questions like: who are your users, and what will they do while on your site? Sometimes questions like these can be answered with analytics, sometimes with educated guessing. However, for those who can’t answer these simple questions, some options still remain. Among those, one option may soon be a reality: monkeys – lots of monkeys.
In the past, deploying monkeys has been somewhat problematic, as it required extensive training to keep them stationary while using a keyboard & mouse while using computer. This limitation may … Continue reading »