Load Testing with 5,000,000 Concurrent Users

Ever run a test with more than a million users?  We’ve been getting requests for larger and larger tests recently with the upcoming elections, not to mention the trend for more online testing.  Although we’ve been running large scale tests for over a decade, to make sure the test results are valid we are constantly doing internal testing. This ground work has already come in handy preparing for the 2016 election primaries, and we’re currently working to make sure state-level election websites are ready to go in November.

Web Performance Consulting

Doesn’t a CDN Solve all of My Problems?

Many of our customers are now using CDNs, reasoning that it will greatly increase their site’s capacity without a lot of trouble, and while that’s true, it also doesn’t happen automatically.  Time and time again we’ve tested a newly CDN-enabled site only to discover that the site as configured was showing only a marginal performance increase, and certainly not enough to justify the price of a CDN.  Luckily, testing showed what was slowing down the sites, and enabled configuration changes to make sure the CDN was delivering all of the performance gains it was capable of.

Configuring a Website to handle 5,000,000 Users

The first problem is testing millions of concurrent users is fairly expensive, but we’ve invested the time and money to insure large tests go smoothly and give accurate results, creating thousands of computers in the cloud (over and over again) to generate up to 5,000,000 concurrent users at last count.  The second problem is in order to test such a large load you of course need to host sites big enough to withstand the onslaught, but its necessary to make sure we’re ready to go when you need large amounts of traffic. NGINX has been invaluable in this regard, which is incredibly fast for the static websites we use for this type of large-scale testing.

The first level of testing we do for very large scale tests is can the requested load be generated accurately?  The following chart shows that the 1,200 load generators were able to smoothly create and sustain users at every level up to 5,000,000.  Note that the load generators are monitored for CPU and memory usage, and thus won’t add users unless there are enough system resources.


Screen Shot 2016-07-14 at 3.56.01 PM



What does 5,000,000 Concurrent Users Really Mean?

One of the big questions in capacity measurement is how to translate between hard measurements like bandwidth and hits/sec into concurrent users.  After all, 100 people clicking on a new page every 10 seconds creates twice as much traffic as 100 people clicking on a new page every 20 seconds.  For new sites this is just a guess, but for existing sites, a careful analysis of the analytics greatly increases test accuracy.  For example, Google Analytics keeps track on how long people spend on each page, and for the average time is around two minutes:

Screen Shot 2016-07-14 at 4.26.25 PM

The overall “think time” for each page in a test case would then be set for two minutes, while the software varies this by +/- 25-50%.  For important pages, such as forms and the home page, you can custom-set the think times if they vary much by the site average.  In the test I just ran this week to try out 5,000,000 users, I wanted to try and push the think times to the lowest practical value that might be seen in the field, which translated into a maximum of 1,300,000 pages per second or roughly 4 seconds spent on each page, which is really fast compared to live commercial sites.

Screen Shot 2016-07-14 at 4.19.44 PM

The website being tested, in this case based on NGINX, is pretty darn fast with static content, which gave incredibly fast sub-second response times for testing purposes.

Screen Shot 2016-07-14 at 4.35.00 PM

The other factor to look for is bandwidth, and each load generator had 100Gbps available, giving us a smooth bandwidth usage profile out of the web servers.

Screen Shot 2016-07-14 at 4.37.57 PM

Currently any number of concurrent users higher than 500,000 is handled by our services department, and thanks to our experience is very competitively priced.  Let us know if you need any help preparing your massively scalable web-based application.


Add Your Comment

You must be logged in to post a comment.


Copyright © 2022 Web Performance, Inc.

A Durham web design company


(1) 919-845-7601 9AM-5PM EST

Just complete this form and we will get back to you as soon as possible with a quote. Please note: Technical support questions should be posted to our online support system.

About You
What you Need