Best Practices for Successful Load Testing
Whether using Neustar Professional Services Full Service Load Testing or going it alone, load testing requires a good deal of planning and forethought. Many items need to be taken into consideration to maximize effectiveness. To that end, the testing process should occur over the course of several weeks to allow for careful planning, test execution and a thorough examination of testing results.
The Five Main Components of Load Testing
There are five main components that make up the load testing process, including: (1.) planning, (2.) configuration, (3.) scripting, (4.) execution and (5.) analysis. The Neustar Professional Services team has discussed each of these topics, in-depth, in their load testing blog series, and you can find links to each blog post in the series at the end of the article. (If you have worked with the Neustar Professional Services team previously, you might recognize this methodology. Our Professional Services Engineers follow these steps to ensure a successful engagement.)
Successful load testing is about asking the right questions at each phase of testing – not just in the initial planning period. For example, it is important to understand the purpose of the test, stress test versus load test, and know how many tests to run. Neustar recommends planning for three tests: the first to provide a benchmark; the second to validate changes made after the first test; and the third test as final validation that the application is ready.
There are several common pitfalls customers encounter when testing that can be easily avoided. The most common is vastly overestimating the number of users a site can handle. The Concurrent User Calculator is a useful tool in this regard to aid in determining the number of users to test. We also commonly see fast ramping of users in a test, but generally discouraged this practice. Instead, we recommend ramping up users gradually. As a best practice, Neustar Professional Services recommends ramping users in over the course of 10 six-minute intervals, allocating 10 percent of the load at each interval.
This method of ramping provides a snapshot of site performance at each interval. Ramping in users quickly generally leads to a failed test and determining the point of failure can be difficult. In general, tests will be more informative and yield better results with a well-structured ramp schedule.
Scripting can be one of the more challenging portions to load testing, especially when the test scenario is long or complex. Users attempting the script process should include verbose comments to the code for easy reference later, enabling others to follow the code. If time or expertise is in short supply, consider contracting the Neustar Professional Services team for the script development portion of the load test. The Professional Services team can quickly develop a script, reducing the time needed for the testing process.
The Final Phases
Test execution and analysis are the final phases of the testing process before starting the process once more. In sum, each iteration of a test is an opportunity to understand site performance and isolate bottlenecks. Once the test has completed review the results and if you’re working with a Professional Services Engineer, ask questions about any results that are not fully understood. After a thorough analysis, make the necessary configuration changes and start again at the planning process. Many aspects of a second or third test will be similar to the first – but going through the process multiple times ensures better tests and results.
If you are looking to learn more about load testing – or if you are just getting started – the five-part load testing blog series by the Neustar Professional Services Team covers the entire load testing process from beginning to end. Reading the articles in this series is strongly recommended to anyone unfamiliar with load testing – or simply anyone who wishes to learn more. If you are without the necessary resources to tackle a load testing project or need additional expertise, the Neustar Professional Services team is here to meet your needs.
Part 1: In the Driver’s Seat: Load Testing Best Practices – Planning
Part 2: In The Driver’s Seat: Load Testing Best Practices – Configuration
Part 3: In The Driver’s Seat: Load Testing Best Practices – Scripting
Part 4: In The Driver’s Seat: Load Testing Best Practices – Execution
Part 5: In The Driver’s Seat: Load Testing Best Practices – Analysis
Series Summary: In The Driver’s Seat: Load Testing Best Practices