Product Comparisons
Microsoft's Application Center
This page describes how Web Performance Load Tester™ compares with the Microsoft Application Center Test which is included with Visual Studio. If the new ACT program looks familar, that's because its almost identical to the old Microsoft WAS tool that hasn't been updated in years. The WAS program was simply copied into Visual Studio and renamed. Web Performance Trainer™ is a commercial, supported multi platform product with technical support and regular upgrades. The following sections describe some of the most importnt advantages of Web Performance Trainer™.This is what one customer had to say about using VisualStudio's built-in performance testing software:
"It was originally planned to use Microsoft Application Center Test (which come with VisualStudio.Net) for this purpose. But a combination of the limits of ACT and the architecture of the application make it unsuitable. ACT can be run through the various modules of the application and it will quantify a variety of performance factors. However, it is impossible to determine through ACT where in the course of the application the performance statistics were unsatisfactory."
"Web Performance Trainer (WPT), the target of this PO, will provide performance statistics on a page-by-page basis at various levels of simulated load. This will permit us to identify the location of a bottleneck and the effect of various load levels. The proposed license would allow us to simulate up to 100 simultaneous users."
"WPT has the added benefit of being independent of the development tool. While ACT is limited to the .Net environment, WPT is usable with any Web application or Web site."
"Compared to other software available and the PCCD project time constraints this would seem to be the best value and solution."
Ramping Load Generation
With Web Performance Trainer you can dynamically vary the number of virtual users hitting your site, so you can see how the performance varies according to load.ACT will only generate a static load, so you have to manually create a test for each number of virtual users you want to test, and create a graph by hand. In addition ACT only allows a test case to run once for each virtual user.
How Many Users Can Your Website Handle?
Web Performance Load Tester™ will tell you how many real users your website can handle, and even identify the specific web pages that are causing the slowdown.
SSL Test Case Recording
WAS doesn't support SSL recording of test cases. If your test case does require SSL, the only way to work with WAS is to disable the SSL security, record a test, re-enable SSL, and modify the VisualBasic test script by handle to add SSL support.Web Performance Trainer™ automatically supports SSL recording through any browser, and client certificates in version 2.6.
Three Levels of Statistics
When recording, Web Performance Trainer automatically divides the session into web pages and URLs. This gives Trainer the ability to then group playback statistics into a transaction, web page, or individual URL level. This puts statistics into a user-centric form such as "how long did it take on average for users to perform a search?", rather than just raw statistics on bytes transferred.ACT records only URLs, so you have to group the URL requests into higher level entities by hand. The result is you can only see statistics by the URL or "group", so if you want statistics at the web page or transaction level you have to figure it out for yourself.
Simultaneous Test Cases
Web Performance Trainer™ enables you to define transactions that are unique to your business, and allows you to group them in new ways to simulate existing or new load patterns. You can run multiple business cases at the same time, each with different characteristics creating a realistic load on a web site that is used for more than one simple transaction at a time. For example, you can have 20% of your users performancing a search, another 40% filling out a form, and then remaining 40% making a purchase.ACT requires each virtual user to run the same test case, making it impossible to support multiple simultaneous transactions needed to test for common performance issues such as database contention and other resource locking problems.
Network Speed Simulation
The most common cause of performance problems for users are web pages designed over a high speed LAN, but viewed over the internet. Furthermore, because lower bandwidth users keep HTTP connections open longer this creates a different type of affect on a web server than LAN users. Web Performance Trainer™ not only supports basic bandwidth simulation, but also allows virtual users to be run from different bandwidth connections simultaneously, creating more realistic loads and generating more realistic performance statistics.>
Multiple IP Addresses
Coming in version 2.6 the virtual users in Web Performance Load Tester™ can be generated from unique IP addresses. This is a requirement for many load balancing devices and tracking programs which use the IP address as a user identifier.
Run from Multiple Computers Automatically
ACT is a single program meant to be run from a single computer. While you can run larger tests by starting ACT from multiple computers at the same time, you have to do this by hand, and the resulting statistics aren't collated. Web Performance load Tester™ has been proven to realistically generate thousands of simulated users by automatically controlling dozens of other computers from a single console, and then collecting and merging the statistics.
Usernames, Passwords and Data Replacement
Both ACT and Web Performance Load Tester™ support the use of unique usernames and passwords for each simulated user. The different is that WPT supports unique values for generic forms in a way that doesn't require writing a Visual Basic program. WPT also supports sophisticated data replacement rules such as "scoping" and "reusing" of data making it easy to create sophisticated data replacement.
Use any Web Server, Operating System, or Web Browser
Because ACT is a Microsoft product it is designed to only work with Microsoft products such as IIS and Internet Explorer, and then only on the Windows operating system. Web Performance Load Tester™ not only supports those programs, but works with any web server, including IIS, Apache, ColdFusion, PHP, and all J2EE application servers such as BEA WebLogic. While ACT runs on only Microsoft operating systems, Web Performance Load Tester™ supports Linux, Unix, and Solaris in addition to Windows. WPT works with any web browser, including Mozilla and Netscape.