Load Testing for the 2017 Eclipse
It has been 99 years since a solar eclipse has crossed the entire continental US, coast-to-coast. 14 states will be treated to 2½ minutes of total darkness by the August 2017 eclipse. I remember watching the partial solar eclipse of February 1979 in my school playground. I am (obviously) a little bit of a science geek, so when I got the assignment to load test the Eclipse Live 2017 site, I was excited. Besides the prestige, it’s really fun to be associated with a project that will be seen by millions, even in a minor role.
NASA hosts much of their site on Amazon AWS, fronted by the Limelight CDN, with much of their video streaming for this project provided by UStream (a.k.a. IBM Cloud Video). So our expectations were very high. The implementation did not disappoint.
We have worked with Mobomo (our customer and NASA’s contracted agency for the project) before, so the project started smoothly. When they asked to test the system for 1.5 million concurrent users, we did take a deep breath – this was close to the largest test we have ever been asked to run. They asked if we could go even higher, so in the background we ratcheted up some ongoing infrastructure improvements to increase our capacity.
Based on data from previous big events, the user model predicted that the average user will be be active on the Eclipse Live page for 30 minutes, watching various video streams. Some users would visit other pages on the NASA site, particularly those linked directly from the Eclipse Live page. Eventually, Mobomo asked us to bump up the load test to 2 million concurrent users. Based on this model, 2 million concurrent users would generate ~2500 page views per second. Over the expected 3 hour window of highest activity, this comes to ~27 million page views (on top of the daily NASA traffic).
In our test, the site performed exceptionally well — the end-user experience actually improved as we cranked up the load. In the hour-long test, we hit the site for 6.8 million page views and 82.7 million HTTP transactions. Five HTTP-level errors were detected (yes, just 5) for a failure rate of 0.000006% – fantastic by any standard. The data indicates that the Eclipse Live 2017 page is well prepared for the event. As always, there are caveats. Our tests included nasa.gov resources, but the associated sites that provide services to the pages from other domains were outside the scope of our assignment. This excluded the video streaming from our tests (it is my understanding that UStream and the IBM Cloud Video team handled preparations on that end). The last mile is critical and cannot be tested economically…certainly not at this scale. If this event is as popular as predicted, some users will no doubt experience slow performance due to bottlenecks in their ISP, particularly those streaming video on mobile networks that have limited excess capacity. But for those of us lucky enough to have fiber to the door, the experience should be flawless.
As our customers have ratcheted up their need for large-scale tests in recent years, we have made numerous infrastructure improvements allowing us to easily generate the load needed for this project. Our own monitoring data indicates we can load test at levels 10x the amount of traffic generated for this test – probably much more than that. We are here to help you prepare for your eclipse event.
Happy viewing…and don’t forget your solar glasses!
Chris Merrill, Chief Engineer, Web Performance
When his dad brought home a Commodore PET computer, Chris was drawn into computers. 7 years later, after finishing his degree in Computer and Electrical Engineering at Purdue University, he found himself writing software for industrial control systems. His first foray into testing software resulted in an innovative control system for testing lubricants in automotive engines. The Internet grabbed his attention and he became one of the first Sun Certified Java Developers. His focus then locked on performance testing of websites. As Chief Engineer for Web Performance since 2001, Chris now spends his time turning real-world testing challenges into new features for the Load Tester product.