March 2020 marks the beginning of the COVID-19 crisis in Belgium. At a moment’s notice, teleworking became the norm. However, many questions came with it. How do I organize my time? How do I collaborate? And how do I interact socially? How do I organize myself?
Awaere aims to tackle some of these questions. Developed by AE, Awaere offers an integrated light-weight solution for operational and wellbeing challenges, by providing a digital layer to facilitate collaboration and the possibility to indicate when help is required.
Being a new product on the market, the Awaere team realized the importance of performance testing:
- Having a load time of up to 5 seconds increases the chance that customers will not continue with 90%
- A 1 second delay in page loading time decreases customer satisfaction with 16%
- 79% of customers who have trouble with site performance say they won’t return
The combination of the ‘need for speedy insights’ and the lack of performance testing knowledge within their team resulted in a partnership with Quality@Speed.
At the start of the project, it became apparent many obstacles would need to be tackled.
A first challenge was found in the lack of (structural) performance testing. No automated performance tests were executed and as such the team was not aware of the performance or application limits. Additionally, they did not undertake any benchmarking or did not monitor performance.
The set-up of performance tooling was a second obstacle to be overcome. At the time, tests could only be run locally from 1 Jmeter instance. Tests were thus not scalable and a maximum of 500 users could be simulated. In addition, no continuous integration existed, meaning test scripts needed to be manually adapted before each test execution, consuming more of the team’s valuable time.
A final difficulty that needed to be addressed was the lack of performance testing knowledge within the team, leading to numerous manual interactions which make the performance testing set-up vulnerable for human error.
To tackle the challenges above, it was decided to use ACCELLR, since it provides an easy and speedy answer for all mentioned issues.
- Firstly, an obvious one, ACCELLR provides a performance testing suite, which includes an automated data provision service, addressing the difficulty of the numerous manual changes of tests before every test execution.
- Secondly, ACCELLR also offers the possibility to parametrize test scrips & test set-up. Important to notice, all existing Jmeter tests are re-used, with only minor changes made.
- Thirdly, the team is now able to run multiple Jmeter instances for the same database.
- Fourthly, ACCELLR also provides an answer to the visibility issues, by logging and storing all performance test results and metrics. This enables the team to benchmark their results.
A critical success factor to make this story come to happy ending was coaching of the Awaere team to use and understand ACCELLR. This will allow them to undertake certain actions themselves.
Overall, it was clear that speedy performance insights were on top of the Awaere team’s agenda. Since ACCELLR can provide insights from day 1, ACCELLR was an obvious choice. This decision saved the Awaere team at least 10 man-days in set-up costs. Due to having several Jmeter instances running, they gained the ability to autoscale.
The Awaere team can now opt for an infinite number of users simulated instead of their previous limitation of 500. Another achieved benefit is the set-up of the data provisioning. They will never run out of test data & won't have to go through the trouble again of creating their own. Finally, the automated reports saved them plenty of time in analyzing the results.
All these actions combined gave the Awaere team the largest benefit of all: visibility on performance. The on-demand automated test execution and comparing of performance resulted in a 10 times faster detection of non-performant code.