A modern performance test tool must provide excellent solutions to a number of performance test requirements to be a cost-effective and productive investment.
The following topics describe a selection of challenges for performance test tools and our solutions with MBC.
Test tool productivity solutions
With productivity solutions we mean a test tool's capability of working efficiently and thus producing more test results.
An example of more test results can be less time needed to execute a test case, which enables more tests executed per time period.
Other examples are better quality on measurement results, less or no time needed for manual actions, and efficient storage of performance measurement data.
Test automation solutions
Contrary to function tests that can be done manually or with test tools, system performance tests are always done with performance test tools.
With performance test automation solutions we mean not only a performance test tool's capability to run a performance test, but to do it at any time without manual interventions.
This also includes automated handling of all actions related to starting, running, and finishing a performance test case, i.e. all actions shall be scripted in a test case.
Test tool scalability solutions
A test tool's ability to generate load can increase dramatically during development due to customer requirements.
With performance test tool scalability solutions we mean a performance test tool's capability to extend the load capacity to any desired limit with only minor changes.
Test tool flexibility solutions
A test tool's ability easily to create new test cases based on old is important.
With performance test tool flexibility solutions we mean a performance test tool's capability to reuse or expand test cases for other performance tests.
Complex load scenario solutions
A test tool's ability to generate load may increase dramatically during development of a product due to customer requirements.
With complex load scenario solutions we mean a performance test tool's capability to manage complex load with different types of load control.
Extreme load condition solutions
Some performance tests require extreme load patterns to measure a tested system's behaviour.
With extreme load condition solutions we mean a performance test tool's capability to generate extreme load patterns.
Complex test scenario solutions
Some performance tests require complex mixes of user requests to simulate a tested system's behaviour.
With complex test scenario solutions we mean a performance test tool's capability to handle complex service scenarios such as IMS-AKA.
Other cases are complex mixes of service requests reflecting a statistical distribution of of real traffic.