Test automation is standard in many IT projects today, especially where speed of change delivery and software quality matter. Frameworks such as Selenium, Playwright and Cypress enable testers and developers to create automated tests that speed up error detection and reduce the time needed for manual testing.
Teams are increasingly trying to automate testing, but while this usually brings many benefits, it is not always a cost-effective solution. There are situations in which the costs of creating, maintaining and updating automated tests can outweigh the benefits of implementing them.
In this article, we’ll take a look at when test automation might not be the best idea, and why sometimes it’s better to rely on a traditional testing approach.
When not to automate tests?
1. Short term projects
Let’s imagine that we are testing a website for a promotional campaign that will only run for a month. Is it worth it to automate testing in such a project? Probably not.
Automation requires time to prepare the framework and write the scripts. In short projects, the cost of implementing such tests can be higher than the benefits of running and maintaining them. Such tests simply “won’t pay off.”
In such cases, manual testing can be a more effective solution, allowing faster coverage of key areas of the application.
2. Application under dynamic development
That is, in other words, the application, where often changing requirements. This could be, for example, a small MVP app, where descriptions of key functionality change every sprint. In such cases, automated tests may require frequent updates and their maintenance cost may be too high to be a worthwhile investment.
As in the example above, here manual tests should suffice.
3. Lack of a stable environment
Unfortunately, it happens. Testing requires an environment where test results can be repeatable and predictable. If the test environment is unstable or its configuration changes frequently (or simply doesn’t work), implementing automation misses the point.
Of course, you can try to deal with this situation, for example, you can use mocks that emulate the operation of external systems or APIs. Alternatively, you can simply wait to test until the environment is more stable.
4. Complex visual tests
Visual tests are not a very common type of test, but they can be challenging.
Visual tests, such as assessing interface aesthetics or UX, are difficult to fully automate. Visual testing tools can assist, but their configuration can be complex.
