Load Testing Blog
Analyze measurements by hostname
The Periodic Measurement Collector plugin, part of the Measurements extension for MuseIDE, now has the ability to add the hostname to each measurement it collects. This will allow analysis tools to analyze performance based on the source of the measurement. In the context of load testing, this would allow identification of regions that are experiencing slower page load times.
Here is an example of the durations of a homepage under load, identified by source hostname:
To enable this feature, turn on the Add source hostname parameter in the plugin configuration. Note that it applies to all measurements collected by the plugin.
This … Continue reading »
Announcing the Graphite extension for MuseIDE
To complement our Measurements and Parallel extensions, the Graphite extension sends measurements to a Graphite-compatible listener (Carbon, InfluxDB, etc). This allows, for example, viewing load test measurements live in a compatible UI, such as Graphite, Grafana or Chronograf.
After installing the extension (via the Extensions… button), it must be configured with the hostname and port that Graphite will be listening to.
You will also need a Periodic Measurement Collector plugin running, which was added in the Measurement extension 0.2 release. The periodic collector gathers the measurements and sends them to the Graphite plugin. It has a … Continue reading »
Bootstrapping an Alpine EC2 instance for Ansible and Docker
One of the challenges in running a large load test is the orchestration of a large number of machines to generate load. This involves a series of steps:
Creating the instances
Install the load testing software
Sending the test configuration
Run the test
Collect test results
Shutdown the instances
Load Tester does that pretty effortlessly in EC2 – our customers as well as our own test engineers love not having to worry about those steps. It just works. As I evolve our next generation of testing tools, I am revisiting this problem and looking at solutions from a different angle. Last week, I decided to investigate … Continue reading »
Introducing the OkHttp extension for MuseIDE
The Muse Test Framework is designed around extensibility – and specifically around the ease of adding new types of steps and value sources. The intent was to make it easy for 3rd parties to add new abilities to the framework and have those abilities seamlessly integrated into the MuseIDE without much effort. This made it a natural choice for us when looking to develop new load testing capabilities beyond our current tools such as Load Tester.
This first version of the OkHttp extension is very simple – almost embarassingly so. It provides just a handful of new capabilties:
Create … Continue reading »
Introducing the Parallel extension for MuseIDE
Executing a load test implies running many tests at the same time…making this ability a crucial part of my work to add load testing capabilities to the Muse Test Framework and MuseIDE. This extension provides just that feature – running multiple tests in a test suite in parallel. After all, a load test is really just a test suite executed with a high level of parallelism and additional (performance) goals.
Measurements extension 0.2 released
The Measurements extension adds performance measurement features to tests developed with the MuseIDE. The 0.2 release adds several new plugins that are crucial to our goal of using Muse for load testing: Periodic Measurement Collector, Store Measurements to Local Filesystem, Store Measurements to CSV and the Step Measurements Producer.
What is the Fastest Web Browser in 2018?
Even though interoperability has been conquered, browsers still want to compete on performance, so picking the fastest one is a reasonable question. But what does browser performance mean in a world in which many people now have 1 Gb/s network connections in their home, and a 4G LTE phone connection can go up to 173Mb/s?
Legion: Status, and Future Plans
Over the past two years, my employer, Web Performance, Inc, has supported my
work on the Legion Load Testing Framework when my other duties permit. The result has been a very flexible suite of software that has given us the ability to take on work that would otherwise be beyond the scope of our usual tools.
That being said. Legion remains experimental. Legion may be most appropriate for the unusual edge cases: proprietary, eccentric, or unusual protocols, or project requirements that other tools can’t handle, or if you want to write your load test using your own Node.js client APIs.
Legion absolutely … Continue reading »
Introducing the Measurements extension for MuseIDE
The Measurements extension is a free (and open-source) project extension for MuseIDE and the Muse Test Framework that adds evaluation of performance criteria to a test. This initial release adds two new capabilities:
Collect and store the durations of steps in the test, for later analysis.
Compare the duration of steps to performance goals and record a test failure when the goal is exceeded.
The extension is available for installation directly within the MuseIDE: after opening your project, go to Extensions and switch to the Available tab. The Measurement extension can be installed into your project with the click of a button:
Each … Continue reading »
The Case for Legion
What is Legion?
Legion is a distributed, protocol-agnostic load testing tool. You can learn more or get started with Legion on the legion-starter-pack GitHub page.
Why does it exist?
I wanted to create a tool that would solve the most challenging problems I’ve
encountered in my seven years of helping clients improve the performance,
capacity, and risk exposure of their network-enabled applications.
Those problems include:
Complexity of the use case or test design,
Unusual or proprietary protocols,
Requirements to scale beyond one million concurrent users,
Difficulty understanding or trusting the results of a test, and,
Difficulties with training new people on a given tool.
My hope is that Legion will eventually … Continue reading »