Don’t unit-test GUI
I’m currently rereading parts of the book Test-Driven Development: A Practical Guide
The author tries to convince me that developing my user interfaces using a test-driven approach is a good thing to do. I disagree.
I love TDD, it’s one of the most powerful development techniques I know, but it’s not without limitations. For one thing, code with unit-tests attached is more difficult to change. This is often an acceptable price to pay since the benefit of producing and having the unit-tests is usually greater.
But the return of the investment isn’t always bigger than the price, and sometimes the cost of change exceeds the benefit of protection. That’s why most developers won’t use TDD for experimental code, and that’s why I’m not using it to test my user interfaces.
I prefer to develop GUIs in a RAD environment, visually dragging and dropping components, moving them around, exchanging them for others if better ones are to be found. In that process unit-testing just gets in my way. After all, the GUI is supposed to be a thin layer, without business logic, so there is only so much to test.
One could theoretically test that the form contains certain key components, that they are visible, have a certain position or layout, stuff like that – but I find that kind of testing too specific for my taste.
In my opinion, unit-testing should test functionality, not usability. It shouldn’t dictate whether I decide to show a set of items in a plain list or in a combo-box. What it should do, is test that the business logic of my code produce the correct set of items, and leave the graphical worries to the testers.
This brings us to something that is sometimes misunderstood: Unit-testing can never replace conventional testing. Some things are just better left to humans. Like testing user interfaces.
Cheers!
For GUIs, since you have to look at the results a lot, I always try to work with programming idioms where either the computer says the code is defective (compile-time or run-time) or the problem is visually manifested. It is not that hard to accomplish, if for instance you always make sure that any ‘thing’ sent to the interface is ‘always’ presented in the screens.
I’ve also found over the years, that my gold-standard for interfaces comes not from having the code generated for me, but by my using a small number of ‘formatters’ that apply presentation information directly to the underlying models. Most systems only have a small number of ‘different’ screens, which if all of the presentation information is packed together means a small number of data formatting methods (proportionally). If all 200 screens only use 12 basic formatting routines, then ‘consistency’ is automatically enforced by the architecture. Slower in the beginning, but way way faster over the long run, and the results are guaranteed to be consistent (and less testing too, because you only need to look at 12 screens to find bugs!).
Paul.
Interesting, what type of systems do you usually build?
Generally, medium to large systems. Most often from scratch, sometimes for commercial, sometimes for in-house or ASP development. I’ve used C, Perl and Java the most, and I tend towards the UNIX platforms, but PCs keep pulling me back. Not that it is that interesting, but my resume is on-line.
My best example of the above second paragraph was a large commercial product that I wrote over five years. I kept the code base extremely tight, and the iterative cycles were usually short, often not much longer than a month between releases. You might think that the time is too short to do any significant work, but I had built the code up to be a framework and a toolkit. In less than a day I could whack out fully functioning screens, in a week a stand-alone app. But it took five years to get there…
It’s why often in my writing I keep trying to convince developers to leverage their code, instead of just ponding it out as fast as possible. If you know that you are going to be working on the same project for five years, you can use that ‘fact’ to your advantage. There are lots of techniques, for abstracting or packing the code. To me it comes under the “work smarter, not harder” category. Seems to be a hot topic right now as Steve Yegge, Jeff Atword and Reginald Braithwaite are all writing about it. Me too 🙂
(eek, it’s hard to get so many links into the comment box, hopefully they all worked 🙂
Paul.
I used to agree with this sentiment, but I’ve recently changed my mind somewhat.
In the book “wxPython in Action” by Noel Rappin and Robin Dunn, one chapter (chapter 5?) has an excellent section on unit testing GUI apps. The focus is not so much on appearance of the app as behavior. So you aren’t necesarily checking that particular elements of your form are displayed, but more for “if I click Button A, will the right data show up in my list control” type of stuff. I think this type of testing is perfectly reasonable.
Kevin
You made me curious on the wxPython in Action book, I should get myself a copy. I might even end up in learning Python. I love learning new languages.
But the example you give doesn’t convince me. If I press Button A, the code attached to that event has typically invoke business logic, which I already unit-test, so I should mock that. And with your example, it might also contain a call to populate the list control with the result.
With the reasons I gave in the post, I prefer to skip test-driving my GUI since it brings so little, the business logic is already covered, and the gui-specific unit-tests ties the interface to a specific solution, and compared to my programming interfaces, my guis tend to change relatively often.
I will check the Python-book though, in case I missed a point.
[ and compared to my programming interfaces, my guis tend to change relatively often. ]
How do you make your changes? Compile, redeploy, and then go to that screen? Unit tests could save you this time consuming way of developing by running a small test right there.
And if it breaks often, good 🙂 It might save you from shooting yourself in the foot.
What about testing presentation logic? E.g. when I select this value in dropdown A, the list of options in drop-down B should be filtered. Another example would be highlighting rows in a grid in different colours depending on the values of certain columns.
(Offtopic: in this comment form, what’s the difference between “subscribe to comments via email” and the “notify me of followup comments via e-mail” checkboxes? The label on the second one toggles the check of the first one, by the way. I’ll just check both.)
Good points but this is a trade-off I’m prepared to make. And, the heart of such presentation logic could often be built into the business logic layer, and tested accordingly. What then remains is to test that the GUI invokes the correct business logic method, and that the GUI responds to the events fired. And again, that would be too specific testing for my taste.
About the checkboxes, I guess they both mean the same. Strange though, I don’t have the “notify me…” checkbox. Maybe because I’m the author. I should investigate this.
I disagree heartily 🙂
I’d regard some aspects of ui behaviour as functional – “if I hit the apply button, certain views or editors should update” for example – this is more integration testing or system testing, so maybe you were using unit-testing as precise terminology.
Testing the back-end (or business logic) is still the priority, and gui testing still suffers for lack of good tooling, but I still lash together tests for the important parts.
I don’t see that unit testing for aesthetics or usability could be achieved. Testing a ui conforms to a particular visual template also seems of little value.
Thank you for your comment.
You are absolutely right. Some aspects of ui is indeed functional. But I believe one could push most of the functionality into the business logic layer, and keep the GUI as thin as possible with events. That way, you can exchange most “if I hit apply button, certain views or editors should update”, for “if I invoke business logic method A, events B and C should be fired with correct data”, and test the most important parts without having to test the actual GUI (in a kind of integration testing as you pointed out).
Business applications have more GUI bugs than business logic bugs. Those GUI bugs are caused by custom controls (date, money, etc.) and fancy presentation logic (enable/disable of fields, interrelated dropdowns, etc.). Now, I know unit testing a GUI can be difficult, but in some MVC GUIs you can mock out just the display. You can then test your custom controls and fancy presentation logic. But, nonetheless, GUI unit tests should be carefully selected and kept to the bare minimum for ease of maintenance and likelihood that they won’t be abandoned.
You’re definitely right about business applications and GUI bugs, that’s my experience too. But then again, a lot of the presentation logic, like enable/disable of fields and interrelated dropdowns as you mention, could be built into the business logic layer (maybe application logic layer is a better name). The GUI would then be a mere displayer of state through an observer pattern, and an invoker of change. That’s how I like to build my user interfaces.
“For one thing, code with unit-tests attached is more difficult to change.”
I became a GUI unit tester when reading the book several years ago.
The book establishes the view/editor pattern. The view usually often changes and makes it hard to test with Jemmy/FEST or other tools.
The editors are editors are changing much slower and are easily tested. Together the view and the editor represent the GUI layer.
Peace
-stephan
Peace brother! 🙂
I might also become a GUI tester one day, but as today I’m not convinced it’s to my benefit.
Check out Michael Feather’s article on the Humble Dialog box. When I used to do C++ GUI development I had no idea how to test a GUI – because of the problems you discuss. The solution is to make the GUI so dumb that the drag and drop and design you are talking about, the kind of thing you wouldn’t want to test, doesn’t require test because the only logic it has are gets and sets from the controller. It’s just another type of MVC.
Been TDD for years and I’d never recommend writing scripts to click buttons or what-not. A very rough guideline I use is “if it has an if statement – it needs to be tested.”
http://www.objectmentor.com/resources/articles/TheHumbleDialogBox.pdf
That’s exactly my point of view too. Make the GUI dumb. Maybe one should think of the GUI as composed of a View layer (dumb components, no need for unit-testing) and an Application Logic layer (which can be tested).
Unit testing provides the benefit of finding bugs during refactoring. In an environment where you are making many changes to your code, having unit tests will save you more time than you’ll spend doing it in the first place.
The larger a software project becomes, the longer that regression testing is going to take. Unless your test team continues to grow with the size of your software, or your development cycle gets longer with each iteration, without unit tests you are just not testing everything.
It seems to me that because unit testing GUI’s is difficult, programmers try to find ways to avoid doing it. But you are simply leaving your software full of holes this way.
I will continue to write unit tests for my GUI’s, it helps me sleep better at night knowing that the bugs I fixed last week aren’t going to creep back in next week.
I think you are right, but I suggest a different solution. I believe a lot of the GUI-related problems could be addressed by changing the design, pulling the presentation logic out of the GUI layer.
By building the application logic and then the GUI you may focused on the behaviour of the application, not on the implementation.
You could then test the logic, and the view right after. Since you would provide interfaces between logic and view.
I believe a view in not important as is, it should be a state representation of the application.
Amen to that!
GUI is unit-testable (automatically tested)! I test my Swing code with UISpec4J and I am satisfied with it. I admit testing GUI still has some limitations, but the limitations are decreasing due to tools’ support.