How we cut down our testing time to a third of the time
What sounds like an idea to tune a car is actually a normal question at our work for the kartenmacherei, an e-commerce company for personalized cards and gifts. Customers who personalize their Xmas cards or family photo books want an easy, smooth user experience. (Remember the last time you tried to assemble one of those? Exactly.)
Part of our job on the tech team is to deploy more features, so the customer has more possibilities to create exactly what they want. Happy customer, happy life!
The challenge we faced here was testing those features. At first, we clicked manually through the application. This is thorough and it works, but … “Can you make this thing go faster?”
In order to not have this manual effort, we decided to create a regression test suite which automatically tests our website. This opens multiple doors: On the one hand, you can execute these tests at any time, because you don’t need a person who is clicking manually. On the other hand, you can work on other topics instead of manual checks.
For the automation, we decided to use Cypress. Cypress is a common framework for automated end-to-end testing. One Cypress slogan is: “Fast, easy and reliable testing for anything that runs in a browser.”
The configuration and creation of tests is simple to do, and the execution time is acceptable.
With around 150 tests, we made sure the website brings the best experience to our customers.
Did you know that UI testing is the most time-consuming activity of testing? But it is also the most safe testing method for the end user. So for us, it was important to find a good balance between coverage of functionalities and execution time.
On the Unit test layer, only written code is tested. Integration tests look at the communication between your own services and other services. But in UI testing, also called end-to-end tests, the application is tested from the end user perspective. This was important for us in order to make sure that all tested functionalities are working for our customers.
In the beginning, we created the most important tests and executed them one by one. This led to an execution time of 90 minutes. This is already quite good! But also: Time is money, so … “Can you make this thing go faster?”
Imagine one of our developers is working on adding some new features in one of our services. They code, write their unit tests and push it to stage. This push triggers our UI tests, which run on a pipeline and give the developer indication if something is wrong. Another very important thing is the urgency of deployments. As an example, we need to fix a very critical bug in a short time.
The developer knows what to do and can deploy in a really short time. Now our tests are running 90 minutes which means we have at least 1.5 hour between identifying the issue, fixing the bug and executing tests.
Sometimes the fix is not working with the first commit, so it will take another one and a half hour.
Wouldn’t it be cool to reduce the test execution time to a minimum?
Cypress combined with Github supports parallel test execution. With this feature, we are able to execute 20 tests instead of 1 test at the same time.
Github offers a function called matrix in github actions. Here we can define all our test classes which we want to execute. Automatically, Github workflows create multiple job runs based on the combination of variables.
By being able to do parallel test execution, we actually “made the thing go faster” — without missing a mistake. We reduced the time from 90 minutes to 30. This means we can test our platform 3 times faster.
As a possible next step, we will split the test classes into smaller ones, so the parallelization can grow from 20 to any specific number of machines (depending on the infrastructure).
All in all, we can say that the parallelization is allowing us to deliver more features in a better tested way and in a shorter period of time.
Ich habe nie die Schlussfolgerungen für meine Dissertationen oder Hausarbeiten selbst geschrieben. Ich habe vor langer Zeit Spezialisten entdeckt und habe meinen Freunden empfohlen, diese Website zu besuchen seminararbeit schreiben lassen . Dieser Ansatz hat mir viel Zeit gespart, besonders weil das Schreiben einer Dissertation eine herausfordernde Aufgabe sein kann. Diese Fachleute übertreffen mich immer, deshalb verlasse ich mich weiterhin auf ihre Expertise.