Unified communications systems today carry a far greater level of complexity that even a few years ago. The level of complexity can vary, depending on the organization and industry. UC environments are also prone to regular change, and in today's workplace, not only are there ongoing upgrades, additions and improvements, but remote working has added a whole other level of intricacy.
So how do you know when these changes affect the overall experience for users and customers, and ensure peak performance from all your systems? The answer is performance testing.
Performance testing will help ensure your software meets the expected levels of service and provide a positive user experience. Performance testing will highlight improvements you should make to your applications relative to speed, stability, and scalability before they go into production. If you release applications without testing, you 'll almost certainly encounter a variety of different types of problems that could lead to a damaged brand reputation. The adoption, success, and productivity of applications depends directly on properly implementing performance testing.
In this guide, we'll look at four key types of performance testing: Load testing, stress testing, capacity testing, and soak testing. We'll describe why performance testing is crucial, focusing on two performance testing subsets in particular: Load testing and stress testing.
Find out how you can test your entire technology ecosystem and prevent your business from lagging behind with our guide.
Load testing vs stress testing
While load and stress testing are different, they're not independent from each other. Let's examine the differences.
What is load testing?
A load test ensures that a network system can handle an expected volume of traffic and how it behaves when bombarded with specific levels of simultaneous requests. This is unlike stress testing which assesses the robustness of a system with requests beyond the upper limit.
The goal of load testing is to prove that a system is capable of handling its load limit, with minimal to acceptable performance degradation. Before carrying out a load test, the threshold of acceptable performance degradation needs to be pre-defined by the testers.
The example in the graph below, shows a load of 20 users, testing to see that the page time does not exceed 3.5 seconds.
Image source: Radview
When should you perform load testing?
Load testing should be done when you want to test how many users your system can actually handle. You can configure a load test to simulate various user scenarios focusing on different parts of your system. You can also test how the load behaves when coming from different geo-locations, or how the load might build up, then level out. A load test should be performed regularly to ensure your system is always spot on.
What type of things can you test for with load test?
- Load testing allows you to measure response times, throughput rates, and resource usage levels, and to identify your application’s breaking point.
- Load testing assumes that the breaking point occurs below the peak load condition - or falls within established parameters, (whereas a stress test by definition tests a system beyond extreme loads).
- Load testing helps identify bottlenecks, page-load issues, system lag, and anything else that might affect the system when multiple users access an application or flood a system with sudden traffic.
Load Testing for Service Level Agreements (SLAs)
Load testing is best performed in a production environment to understand average response times under expected user load. These average response times become the baseline for an acceptable Service Level Agreement (SLA). From here, testers can determine additional load thresholds that are considered unacceptable in terms of expected performance for optimal customer experience.
Read our blog on cloud load testing vs on-premises load testing
What is stress testing?
Stress tests are designed to increase the number of simultaneous requests on a system beyond the upper limit of resource usage, and where performance is degraded - even to the point of complete failure. Where a load test will peak out in the number of simultaneous users, a stress test will continue to increase load on the system capacity until the resources are overloaded. This type of performance testing pushes the system to a state of potential failure, to see how the system behaves, and whether it recovers successfully.
Why should you perform stress testing?
While load testing ensures that a given function, program, or system can simply handle its expected load, stress testing is about overloading a system past its upper limits, until it breaks. Both stress testing and load testing play important roles in determining exactly how well a given piece of front-end software, such as a website, or a back-end system can deal with their actual load.
Stress testing deliberately induces a failure scenario under extreme load, so that you can analyze the risk involved at the breaking points. This helps with capacity planning, for example, by creating the opportunity to tweak programs to make them break more gracefully. Stress testing prepares for the unexpected, fine tuning response time, and through spike testing, determines exactly how far a given system can be pushed. This method of performance testing measures the outer limits of performance capacity.
What can you measure with a stress test?
Depending on the application, software, or technology being used in your environment/system, what's measured during a stress test can vary, but some of the metrics include overall performance issues, unexpected traffic load spikes, memory leaks, bottlenecks and more:
- Response times. Stress testing can indicate the amount of time it takes to receive a response after a request is sent.
- Hardware constraints. This measures CPU usage, RAM, disk I/O. If response times are delayed or slow, these hardware components could be potentially to blame.
- Throughput. How much data is being sent or received during the stress test based on bandwidth levels.
- Database reads and writes. If your application utilizes multiple systems, stress tests can indicate which system, or unit, is causing a bottleneck.
- Open database connections. Large databases can severely impact performance, slowing response times.
- Third-party content. Web pages and applications rely on many third-party components. Stress testing will show you which ones may impacting your page or application’s performance.
The example stress test below shows that up to a load size of 41 users, the system functions, despite the increase in page time. But when the load size reaches 42 users, unexpected traffic spikes cause a deterioration, with page time reaching 15-17 seconds.
Image source: Radview
Why is stress testing important?
Stress testing enables IT teams to interpret the performance of system software during its failures, allowing them to identify and analyze the behavior of the software. Stress testing enables test teams to check whether data has been saved by the software before crashing and verify whether any lost data can be restored.
Stress testing also enables test teams to understand the behavior of software before crashing. There are many case scenarios in eCommerce where performance testing proves invaluable. For example holidays or promotions like Black Friday, or Cyber Monday, when a website might witness a sudden influx of visitors, causing an increased load, and severe performance issues, or even a total collapse. The inability to handle such massive spikes could lead to loss of revenue and reputation.
Below is a screenshot of a website crash message from Walmart in November 2020, due to heavy load volumes during a buying frenzy on Playstation 5.
Types of stress testing
Let's look at 5 basic types of stress testing techniques:
Image source: Try QA
1. Distributed Stress Testing
In this type of performance testing, all the clients linked with the website servers are tested. Distribution of a group of stress tests to each one of the clients as well as a follow up on their status, is the server’s role and responsibility.
2. Transactional Stress Testing
A transactional stress test is used to test the transactions that occur between the applications. The purpose of this method of performance testing is to fine tune and optimize the system.
3. Application Stress Testing
Stress testing of applications is typically used to identify bottlenecks in performance, network issues, data blockages and locks .
4. Exploratory Stress Testing
In exploratory stress testing, performance testing is carried out under abnormal conditions which will not likely to occur in a real time scenario:
- Extremely large numbers of concurrent users try to log into the application
- A database linked to the website shuts down when the website tries to reach it from the front end
- An abnormally large volume of data added into a database
5. Systematic Stress Testing
Systematic stress testing is used to test many systems that run on website servers. It enables the testing of defects where one software interferes with another.
Stress Testing Memory Leaks
Memory leak is a type of resource leak that happens when a software incorrectly manages memory allocation. Memory leak can also happen when an object is stored in memory but cannot be accessed by the running code. A dynamic analysis tool that can track memory leaks generally monitors both the allocation and de-allocation of memory.
What is capacity Testing
Capacity testing is sometimes called scalability testing, and helps identify the maximum capacity of users the system can support, while not exceeding a maximum defined page time.
The main objective is to identify the ‘safety zone’ for peak performance of your system, and to what extent can you extend it, without hurting end user experience.
In the results graph below, for a page time of 3.5 seconds, the system supports 28 users. But when the load size reaches 29, page time starts exceeding the threshold of 3.5 seconds.
Why is capacity testing important?
The data gleaned from capacity testing is crucial to a business. It's a fact that 40% of users will abandon a site if it takes more than 3 seconds to load. Just a one second delay can cause small to medium sized eCommerce websites to bleed over $250,000 annually. As per Marketing Bulldog, 79% of customers who report dissatisfaction with website performance are less likely to return.
Image source: Radview
What is soak testing?
The purpose of a soak test is to reveal performance problems that surface only over a long period of time. Soak testing highlights issues such as:
- A constant degradation in response time when the system is run over time.
- Deteriorations in system resources that aren't evident during short runs, but will surface when a test is run for a longer time. For example, memory consumption and performance, free disk space, operating capacity, etc.
- Any periodical process that may affect the performance of the system, but which can only be detected during a prolonged system run. For example, a backup process that runs once a day, exporting data into a 3rd party system, etc.
Soak testing looks for trends and changes in system behavior. In the use case below, there is a constant load size, but then, after 50 minutes of running the test, there’s an increase in memory, indicated by the red line, which hurts throughput. This is due to a periodical, background process that affects the system when it starts running. A typical soak test can last hours, days or even weeks.
Image source: Radview
Performance test metrics
Metrics are the key performance indicators (KPIs) that determine the success of each performance test. The most commonly used metrics include:
Average transaction response time - the average time taken to perform transactions during each second of the scenario.
Total transactions per second - the total number of transactions that passed, the total number of transactions that failed and the total number of transactions that stopped during the performance test run.
Transactions per second - for each transaction, the number of times it passed, failed or stopped during each second of the performance test.
Transaction response time (under load) - transaction times relative to the number of users at any given point during performance tests.
Errors per second - Average number of errors that occur during each second of performance tests.
Hits per second - the number of HTTP requests made by users to the web server during each second of the performance test.
Contact center performance testing
In today's business world, Interactive Voice Response (IVR) systems are an important part of an organization's customer service and communication infrastructure. IVRs are used by almost all industries and their respective applications like banking, insurance, utilities, telecom, travel, retail etc. to process a customer's phone call, rather than connecting to an actual human agent.
Performance testing tracks metrics that allow you to confirm your entire environment works as expected, meaning that CPU & memory consumption don't go out of range, response times stay within acceptable ranges and network occupancy levels stay in the safe zone.
Performance testing an organization's IVR system is crucial to maintain a positive customer experience. IR's customer experience testing solutions provide accurate analytics and valuable IVR system information.
Read more about the importance of IVR performance testing and how it can ensure that your customer experience is the best it can be.
