Menu

Impact of APC Module on PHP Performance

Load Testing SugarCRM and the APC Module

Christopher L Merrill
©2007 Web Performance, Inc.
v1.1

Read and post comments

Summary

The performance of our reference PHP application under load (a default SugarCRM installation) showed a 180% increase, measured by total system capacity, after installation of the APC module.

Overview

This article measures the performance impact of the APC module (a PHP code cache) on a real-world processor-bound PHP application (SugarCRM) under load.

While researching our previous PHP Optimization report, we were referred to the APC module for PHP. On the module web site, it is described this way: "APC is a free, open, and robust framework for caching and optimizing PHP intermediate code."

We tested the impact of the APC module using SugarCRM as our reference application. According to the SugarCRM web site, SugarCRM is "the market-leading commercial open source CRM application". SugarCRM is written in PHP. Based on previous testing, SugarCRM becomes quickly processor-bound on a typical installation. Measurement showed that the majority of the CPU load (~75%) was consumed in the Apache httpd processes, which makes it a perfect choice for testing the impact of a PHP cache/optimizer.

Our primary measure of performance is user capacity. We define that as the number of simultaneous users that the system can support while meeting the specified performance criteria. The performance criteria for this test require that all pages load within 6 seconds and no errors are encountered in the application.

Caveats

This report is concerned with the performance of the system under load. It does not make any statements about the performance of an idle system. The measurements may be quite different when a single user accesses the system.

The purpose of the report is not to estimate the capacity of a SugarCRM system or make any predictions about expected performance. Each SugarCRM deployment will be different.

Methodology

The analysis is based on the results of a series of load tests performed in our test lab.

To skip the methodology, you can jump straight to the analysis.

Software

For the optimized test, the APC module version 3.0.14 was installed vi PECL. The default options were used. The only change to the php.ini file was the addition of "extension=apc.so" to activate the module.

Both tests were run using PHP 5.1.6.

The server is running version 4.5.1e of SugarCRM installed via the standard installer (SugarOS-4.5.1e.zip).

The output from phpinfo() was captured for the baseline and optimized test configurations.

The load-testing software is Web Performance Load Tester 3.5.5447. The Web Performance Server Agent (same version) was installed on the server.

Hardware and OS

The SugarCRM server is running on a Dell Poweredge SC1420 with 2 x 2.8 GHz Xeon processors and 2GB of RAM. It has an 800MHz bus and 1MB of L2 Cache. All factory BIOS settings were restored prior to testing. This includes the hyperthreading support which is enabled. The floppy drive support in the BIOS disabled (no floppy drive is installed). The server contains a single Western Digital 80G (WD800JD) SATA disk drive and uses the on-board 1G network adapter. The operating system is the standard server install of CentOS 5 with unneeded services deactivated.

The load-generating engines (3) are each running on a Dell Poweredge 300 with 2 x 800MHz Pentium III processors with 1G of RAM. Each engine is running the Web Performance Load Engine 3.5 Boot Disk version 3.5.5275. For these tests, one load engine would have been sufficient.

The engines and server are networked via a Dell 2324 Power connect switch. The server is connected to a 1Gb port and the load engines are each connected to 100Mb ports.

The load test controller (the Load Tester GUI) is running on Windows XP SP2 on an IBM Thinkpad Z61m. It is connected via a second switch on a 100Mb port. This has no impact on the results, since the controller bandwidth requirements are small, but is mentioned here for completeness.

Test cases

The test cases selected for this load test mirror those selected for the SugarCRM Load Testing tutorial.

Please note that the user mix, think times, etc may or may not reflect your SugarCRM implementation, so the rated user capacity of the system is not likely to reflect any specific real-world deployment. To a large extent, these choices are irrelevant to this test, since the goal is not to evaluate the performance of SugarCRM, but rather determine the impact of the optimizer on before-and-after results.

The test data repository file is available for examination - the demo version of Load Tester can view the test cases, load configurations and detailed raw metrics. It cannot view the test reports, which is why they are linked from this page (see references).

Load Configuration

The Load Configuration was created with these parameters:

The test parameters were determined after preliminary tests were used to approximate system capacity. The tests were terminated shortly after the system had reached maximum capacity.

Database Management

The database was restored to an identical state prior to each test using the procedures outlined in the SugarCRM Load Testing tutorial.

Test Procedure

Each test run followed these steps:
  1. Restore database to pre-configured state with 100 users and 250 accounts, 1000 contacts and 3000 notes.
  2. Restart the server
  3. Start the server monitoring agent
  4. Run warm-up test (5 users running the View Note test case for 3 minutes)
  5. Run the load test
For the before (unoptimized) and after (optimized) tests, 3 runs were completed under each configuration. The best run of the 3 was selected as representative of the configuration. However, the results of the 3 runs showed minimal differences. For completeness, they are included in the test results file.

Analysis

User Capacity

Our analysis shows the capacity of the baseline system was 30 users. After installing the APC module, the capacity increased to 86 - an increase of 180% over the baseline capacity. This analysis, along with a variety of charts is produced automatically in Load Tester's Load Test Report. Consult the User Capacity section of the test reports for more details on this analysis (see references). The charts shown below are extracted from that report section. The red line on the chart depicts the maximum page duration at each user level measured during the test. The location where that line crosses the duration threshold indicates the point where performance no longer met the criteria. Had any errors occurred during the test, the yellow line would depict the error rate and indicate when it crossed the error threshold. Note that the baseline test encountered some page durations that were higher than the APC test, resulting in a different scale on the two charts.

Baseline User Capacity analysis

User Capacity Chart

APC User Capacity analysis

User Capacity Chart

Page Duration

To confirm the findings from the User Capacity analysis, we next look to the Page Duration charts. These show the minimum, maximum and average page durations during the test. These charts are from the Summary section of the load test reports. Note that the baseline test encountered some page durations that significantly higher than the optimized test, resulting in a different scale on the two charts.

Notice that on the baseline test, the average page durations had roughly doubled (from 1 second to 2) when the system had reached its maximum capacity. When the maximum capacity was reached running with APC, the average page durations had not degraded substantially.

Baseline page durations

Page Durations Chart

APC page durations

Page Durations Chart

Server processor utilization

As expected, the limiting factor in the application performance is processor utilization. The charts below (extracted from the Servers section of the load test reports) show several processor-related measurements that were collected during the test using the Advanced Server Analysis module. The light-blue plot depicts the CPU utilization (%). In both tests, the CPU is nearing 100% utilization when the page duration thresholds fail.

Baseline processor measurements

Processor Measurements Chart

APC processor measurements

Processor Measurements Chart

Conclusion

Under the conditions tested, the reference PHP application (SugarCRM) showed a 180% increase in total user capacity when the APC module was used. In other words, the APC optimized system had 2.8 times the capacity of the baseline (unoptimized) system.

Reference Materials

  1. phpinfo() output for baseline test
  2. complete load test report for baseline test
  3. phpinfo() output for APC configuration
  4. complete load test report for the APC test
  5. Load Tester repository file (to be used with Web Performance Load Tester for further data analysis)

Feedback & Comments

Comments about this report may be posted at the company blog post.

Version History

v1.0 - 1st public release (2 Jan 2008)
v1.1 - email cleanup (23 Jan 2009)

Resources

Copyright © 2018 Web Performance, Inc.

A Durham web design company

×

(1) 919-845-7601 9AM-5PM EST

Just complete this form and we will get back to you as soon as possible with a quote. Please note: Technical support questions should be posted to our online support system.

About You
What you Need