Load Testing Apache with IE8 and Firefox 3.5 - Web Performance
Menu

Load Testing Apache with IE8 and Firefox 3.5

How the latest browsers increase speed at the expense of web server performance

Christopher L. Merrill
©2009 Web Performance Inc; v1.0

Read and post comments

Introduction

The latest generation of browsers (IE 8.0, Firefox 3.5) have increased the number of simultaneous connections they will open to a single host during the rendering of a web page. For sites that experience simultaneous connection counts that are close to the maximum that the system will accept, increasing adoption of these new browsers will soon cause service interruptions to users of the site. These limitations can appear at numerous points in the architecture of a site, but the most common and most vulnerable limitation is at the web server. Many web servers, such as the Apache Web Server, have relatively low limits on the number of simultaneous connections they will accept. In the case of Apache, there is both a configured limit (in the httpd.conf file) as well as a limit that is hard-coded and compiled into the software.

In this report, we explore the impact on user capacity of an Apache web server under load when the number of connections used by the browser increases from two to six. The impact on every site will be different, but we hope that the results from this test will give webste administrators a rule-of-thumb that will help them estimate how soon they will need to address this problem. The goal is to measure the relative decrease in capacity when 6-connection-limited browsers are used compared with 2-connection-limited browsers.

Testing Considerations

The Apache Web Server has both configuration and compiled-in limits to the number of simulatenous connections it will accept. For our tests, we use the default Apache installation as distributed with Red Hat Enterprise Linux 5.0 (and CentOS 5.0). We chose this configuration because of any subset of the deployed configurations, the default configuration will be the most prevelant. The specific configuration chosen is largely irrelevant because the goal is to analyze the net change in capacity between different applied loads against a typical configuration.

Test Setup

Load Test Configuration

The testcase consists of a user browsing 10 pages of a typical static web site over a 2 minute period. The 10 pages included 141 URLs (CSS, images, etc) as well as 2 relatively large pages (335kB and 1.1MB). The total size of the testcase was just under 2MB (including HTTP overhead).

We tested under real-world network conditions: the target server is on the west coast of the USA while the load was generated from the east coast. Three tests under each browser configuration (total of six tests) were performed back-to-back to minimize the effects from daily fluctuations of Internet traffic patterns. The most average result for each browser configuration was selected. Some differences can be seen in each of the results, but were well within our tolerances for a rule-of-thumb conclusion.

Several sizing tests were used to estimate the capacity prior to the six tests. The test was then configured to increase users during the test in a stepped ramp configuration. Each user simulated 5Mbps of download bandwidth and randomly varied the between-page think times from 50% to 200% of the recorded think times. The test results quoted in this report ramped quickly to a load well below the performance limits and then ramped slowly (10 users over 2 minutes), then holding one minute for steady-state data analysis.

The capacity of the site is measured by the number of simultaneous users serviced while meeting the performance goals of the site. Each page has a 4-second performance goal, except for the largest, which has a 6-second goal. In addition, the test has a page failure rate goal of 1.0% (which was never exceeded during the tests). At each user level, the collected metrics are analyzed against those goals - in this case the average page duration and average wait time must not exceed the duration goal for each page. The site is said to meet the goals of the site only if each performance goal is met at that user level.

Server Configuration

Apache 2.2.3 on CentOS 5.0 (pre-fork MPM) on a 2x1.86GHz Core 2 Duo with 1G RAM.

The prefork MPM configuration settings were left to the default settings:

StartServers       8
MinSpareServers 5
MaxSpareServers 20
ServerLimit 256
MaxClients 256
MaxRequestsPerChild 4000

Load Generator Configuration

Load Tester version 4.0.7621 was used to generate load against the server, gather metrics, analyze results and generate the report (including the charts shown below). The load engine was running on a 2x1.86GHz Core 2 Duo with 1G RAM.

Results

The estimated user capacity, as calculated by the Load Tester software, is 230 users when the clients use 2 connections but drops to 90 when the client uses 6 connections - a decrease of ~60%.

User capacity with 2 connectionsUser capacity with 6 connections

In each case, the user capacity was limited by one or more pages failing the configured duration goals - which means that the average duration recorded in a 10-second sample period exceeded the goal configured for the page. The following charts show the detailed page durations for each of the two tests. The tests exhibited a very similar pattern of degradation. Note that only the 4-second goal is marked on the chart - which was the duration goal configured for 10 of the 11 pages (the last page was much larger and had a 6-second duration goal).

In the chart below (2 connections/client) it is easy to see where the configured connection limitations are encountered - indicated by sharp peaks in the page durations. This behavior is evident at 200-220 users, well before the system failed to meet the performance goals.

scatter plot of page durations - 6 connections / client

In the 6 connections/client test (below), two distinct differences can be seen. First, the page durations are much lower - because the browser can use more connections to fetch resources - thus reducing the effects of network latency. This is the reason that the browser manufacturers have increased the connection limits. Second, the effects of the connection limitation are less notable prior to the failure of the performance goals.

scatter plot of page durations - 6 connections / client

Conclusions

In our evaluation, the capacity of an Apache server, measured by compliance with performance goals under load, was reduced 60% when visited exclusively by the latest generation of browsers (for example, IE 8.0 and Firefox 3.5). When browsers used 6 connections per host, compared to 2 connections per host in previous generations of browsers, the default connection limits were exceeded at much lower user levels. Administrators of Apache web servers (or any other server that has a relatively low fixed connection limit) should evaluate how close their servers are to those connection limits today and increase those limits as the adoption of new browsers raises the connection demand on the servers.

References

Version History

v1.0 - 1st public release (October 8, 2009)

Resources

Copyright © 2024 Web Performance, Inc.

A Durham web design company

×

(1) 919-845-7601 9AM-5PM EST

Just complete this form and we will get back to you as soon as possible with a quote. Please note: Technical support questions should be posted to our online support system.

About You
How Many Concurrent Users