Let’s do Automated Frontend Tests! This sounds quite easy – but usually turns out to be a real pain in the ass of both testers and project managers. Usually, frontend tests provide a nice way to visually present the quality of a software artefact, and demonstrate the value of test automation, to non-tech stakeholders. These people tend to not really care about stuff like unit tests, because they simply don’t understand it. With frontend tests, they get a nice visual insight into the daily work of the software developers and testers they usually pay for (very stereotype I know – but still pretty accurate for most projects ;D). In this article, I’ll give you my three top reasons why this usually don’t work out in practice.
1You need to rely on a stable visual appearance
One of the most annoying tasks when you create your automated frontend tests is identifying the paths of the UI elements that you need to interact with. If you are not building your UI in a testable way, every time you move a button, it might get a new id, destroying all frontend tests that use this single button. Building UI in a testable way is difficult, as developers are usually not testers. And even if they do it, most changes still lead to a hell of problems (changing the hierarchical structure of the UI also destroys your deliberately handcrafted xpath queries). This problem also exists in standard unit testing – but at a different scale, as unit tests are easier to fix (no need to run the UI several times to find out what has changed and query the new paths).
As a result, even if you have developed automated frontend tests, they don’t execute most of the time, anyways. Every change in the code usually destroys the frontend tests, leading to a whole bunch of work required to make them run again. So why create them in the first place?
2They take forever to execute
Your average unit test finishes with the glimpse of an eye (like literally, milliseconds). In this same time, in your frontend test, your frontend did not even start yet. Each action usually takes up to one second. And that includes entering data in some field, pressing a button, scrolling, all the things you do if you would manually execute the test case. Then you also have to add some freezes, to account for potentially slow loading pages (particularly relevant if the internet is included), which add another few seconds to your test. So in the end, you and up with at least 10-20 seconds for a reasonably meaningful frontend test of a reasonable complex UI – with up to 1 minute if you have some networking issues, heavy UI elements that need long loading times, or complex workflows that require to enter a lot of data and load many pages. Again, that’s just one single test.
And that’s not it – if you run the tests on your PC, your monitor is blocked for opening and closing screens that need to be played by your favorite test driver. So you might get that extra cup of coffe every time you click run on your frontend tests – if you are unlucky though, you get back to your PC just to see that there was some timeout due to a change in the frontend (see reason 1 above ;D). You can run them in you nightly build of course. But what’s the purpose of having shiny frontend tests for your non-tech stakeholders if you can’t actually show them? And also, setting up the infrastructure for this nightly build can be a whole bunch of work. First of all, you need a dedicated machine (as the frontend tests cannot be interrupted by any other running process). Secondly, you need to install the frontend with all its dependencies on this machine. Well, that’s what docker is made for. Fair enough – but still tricky once you start dealing with stuff like security though – but that’s a topic for another day. And even if you manage to execute them in your nightly build – most mornings you will just start your day annoyed by seeing that your tests froze or crashed because of a change in the UI or a network issue…and you will have to re-execute them on your PC, anyways (time to get that cup of coffe now ;D).
So to sum up: running your automated frontend tests will take a whole lot of time (and effort, if you opt for a NB).
3You cannot debug them
The purpose of testing is not just finding that there is an issue in your code, but also identifying the source of this issue via debugging. However, with automated frontend tests, developers usually have a very hard time doing that. Why? First of all, a failing frontend test does not imply a bug most of the time, but rather one of the issues already mentioned in this article. But even if there’s an actual bug behind the failure,
(i) running the test over and over again, until you’ve reached the point of the failure, can be pretty cumbersome,
(ii) identifying the point at which the test fails can already be tricky, and
(iii) debugging through all the layers of your software that might lead to this error is the next cumbersome, sometimes even impossible task (even though the error appears in the frontend, the source can obviously be hidden in any part of your software that is used by the frontend – so ANY part of your software).
As as a conclusion: better not do automated frontend tests 😉
So, this is my take on automated frontend tests. Do you agree? What’s your take? Let me know in the comments section below.
When you visit any web site, it may store or retrieve information on your browser, mostly in the form of cookies. Control your personal Cookie Services here.