Product Comparisons
Ziff Davis WebBench Load Testing Tool
The document compares the Ziff DavisWebBench software with Web Performance Load Tester™. WebBench was designed to help writers compare products for review in one of the Ziff Davis family of magazines, whileWeb Performance Trainer™ was designed to help test engineers and web developers find and fix bottlenecks, improve performance, measure performance, and stress test web sites. These are two very different goals, and as you will see below WebBench is good at comparing products, but lacks features required by most corporate customers.
Test Case Recording
Writing a fair product review requires that each product be tested in exactly the same way. Because of this goal, WebBench testing consists of two predefined test suites consisting of web pages, back-end processes,and test scripts. While these test suites can help tune the network and operating system, they do not exercise any of the customer's web pages, applications, or databases. The goal for most customers is to test their own web pages, application servers, and databases. While the user can create unique tests with WebBench, it isn't easy, as it requires creating test cases by hand in text files using a special format that must be learned.Web Performance Load Tester™ makes it easy to create test cases by simply accessing a web site with a browser and recording those interactions.
More Realistic Test Cases
The test cases in Web Performance Load Tester™ are organized around Business Cases, which are meaningful customer interactions unique to a business. Individual business cases can be grouped in new ways to simulate existing or newload patterns, called load profiles. Multiple business cases can run at the same time, each with different characteristics. For example, 20% of the users can connectat DSL speeds, while 80% connect at the speed of a 56K modem.
Organizing test cases in this way has two advantages. First, it puts a more realistic load on the web server, since the business cases are meaningful to the business' customers. Secondly, statistics can be gathered on the business case level, so you can view measurements relating to each separate activity on a site.
Dynamic Test Cases
Another advantage of the business oriented design of Web Performance Load Tester™is its easy to add dynamic behavior. For example, a test case may contain a form that is being filled out to purchase a product. Without dynamic data replacement, the same product would be purchased by each virtual user. Instead, with Web Performance Load Tester™ each virtual user can make a unique purchase, making for a more realistic simulation.
Modem Simulator
In the real world some computer users still use modems, while others have DSL or Cable, and others use a computer at work from the corporate LAN. Each of these different types of connections put a different type of stress on a web server. Web Performance Trainer™ has virtual users that can connect to a web server using any connection speed, and you can actually configure the mix of users, setting 50% of the virtual users to connect at 56K modem speed, 25% at DSL speeds, and 25% at LANspeeds, for example. The manual discusses modem connection speeds in the sections on load profile configuration and running a performance test.
Playback Computer Performance Monitoring
The accuracy of the load test is completely dependent on the ability of the computer generating the virtual users. If computer's CPU load gets too high, or it runs too low on memory, the simulated users become inaccurate and ruins the entire test. Web Performance Trainer™ monitors the performance of the computer generating the virtual users during a test, and limits the size of the simulation to what that computer can handle while insuring accurate results.
Graphing
Web Performance Load Tester™ has a graph built right in, so you can quickly get a picture of your test results.
Multiple Engine Support
In order to generate a large number of virtual users multiple computers must be used. While WebBench can use multiple computers to generate a load, it knows nothing about the capabilities of the computers involved, and can't monitor the performance during a test. This is crucial, since a wide variety of computers may be involved, with varying CPU power and memory size.Web Performance Load Tester™ dynamically monitors the performance of the load generating computers, and distributes more of the task to the more powerful computers, while insuring no computer is given more to do than it can handle accurately.
Support
WebBench is freeware, that is copyrighted by Ziff Davis, but available for free. Like most freeware, there is no support available since it isn't a product. You are free to contact Ziff Davis, but they have no incentive to support the program since they make no money from it.
Web Performance Load Tester™ comes with lifetime free email-based support, with the option of adding phone support. In either case, you will usually get a responsefrom Web Performance Inc. within two hours if submitted duringbusiness hours.
Another advantage of commercial software is regular bug fixes, upgrades,and feature enhancements.