How Many Accounts do you need for Load Testing?
Eventually in every planning meeting for web load testing the question comes up: can we test with just one user account?
The short answer is “no”.
The longer answer is “heck no”.
The best practice is always to simulate real users as closely as practical, which in this case means concurrent simulated users should login on separate accounts, just like real users. If the test cases are idempotent, i.e. running them doesn’t affect the state of the account, and user accounts aren’t cached, then the answer is easy: at a minimum you need as many test accounts as the maximum number of concurrent users. If your max test level is 1,000 concurrent users, you need at least that many test accounts. If you want to be really accurate, you’d pre-propulate the test system with as many customer accounts as you’ll expect to have in production in the next couple of years.
Where it gets tricky is when a user account can only be used once, either because of account caching or because of the nature of the workflow. Take, for example, the scenario of a student taking an exam. Each student can only submit the test once, as the system will reject any attempts to submit a second time.
Here are some more examples:
* Registration forms that can only be submitted once per account
* Checkout processes where orders can only be completed once
* Applications where a user can only apply for a service once
Here’s a simple example calculation: There are 500 maximum concurrent users, and the test starts at 100 concurrent users, then 200, then 300, etc, until the 500 concurrent user max. To make it easier, the test will run for 5 minutes at each load level, and the single test case lasts 5 minutes. The minimum number of test accounts needed is:
100+200+300+400+500 = 1,500
We do this enough at Web Performance that it made sense to write a calculator so that customers can fill in the details on their load testing plans.
The calculator requires four key inputs:
1. Peak Concurrent Users: The maximum number of simultaneous virtual users that will be active at the peak of your test.
2. Increment Per Load Level: How many additional users do you add at each load level as your test ramps up? For example, if you start with 100 users and add 100 more at each level, your increment is 100.
3. Duration at Each Load Level (minutes): How long each load level runs before increasing to the next level.
4. Test Case Duration (minutes): How long a single test case takes to execute from start to finish.
When to Use Different Scenarios
The calculator handles three different scenarios:
Scenario 1: Test Case Duration = Level Duration
When these two values are equal, each account is used exactly once per level. This is the simplest scenario – each user needs one account per level.
Scenario 2: Level Duration > Test Case Duration
When levels last longer than test cases, each user will execute multiple test cases per level. This means each user will need multiple accounts for that level.
For example, if your level duration is 10 minutes and your test case takes 2 minutes, each user will run 5 test cases during each level, requiring 5 accounts per user.
Scenario 3: Test Case Duration > Level Duration
When test cases span multiple load levels, the calculator typically uses the peak concurrent users as the account count. This is because users must maintain their session while the test level increases.
Another Example
Let’s say you’re testing a purchase workflow where:
– Peak load will be 5000 concurrent users
– You’ll increase by 1000 users every 15 minutes
– Each test case (full purchase) takes 5 minutes
In this example, each test case will be executed an average of 3 times per load level, so the total number required logins is:
3000+6000+9000+12000+15000=45,000 concurrent users
Best Practices
1. Always add buffer accounts: Create 10-15% more accounts than calculated to handle any unexpected issues
2. Verify account states: Ensure all accounts are in the proper state before testing
3. Consider data dependencies: If test cases require unique data beyond accounts, calculate those needs separately
4. Update calculations when changing test plans: Recalculate when modifying duration, user count, or increments