When is Testing Silly?

Gary King compares test-driven development with literate programming, saying that writing the tests often ends up being \”silly\”, since he doesn\’t yet know where he\’s headed and the tests get thrown out. I agree that tests are sometimes unnecessary, but I think part of Gary\’s problem must be that he\’s developing tests the wrong way.

The appropriateness of tests depends on why you\’re writing the code. Do you have some use for it in mind, or is it pure speculation and experimentation? If it\’s the former, writing tests is paramount and should be easy (or, at least it gets easy once you get into the habit); the closer you get to experimentation, however, tests become harder and less appropriate. I think that if you have any inkling of how you\’d use the library, you should write some tests for it. It just gives you a bit of direction, and a way to see what you still need to accomplish. This can mean just grabbing part of your REPL session and throwing it in a file.

How is it possible that testing can be easy? Well, if you\’re writing a library that you have a planned use for, the trick is to use it before you write it. You don\’t sit down and try to come up with an API or some spec, you just pretend that your library already exists — perfectly coded — and use it. Even before TDD (now better explained as Behavior-Driven Development) this was an approach I used frequently. Write examples and other usages of your new library so you have some target to reach.

These are your tests. These are the things that need to work, and they are your effective spec. Of course, new use cases are going to come along and some of your assumptions are going to change, but they should almost entirely come from your (and others) usage of the library. The hard part always seems getting a test framework in place so you can slice up your usage and make tests out of them.

I think BDD is a significant step up from TDD because of its shift in mindset — it encourages you to think of what should happen as opposed to thinking that you need to test all functionality. Only test what you use, and only implement what you test.

It\’s hard when you start, but with a bit of practice it becomes natural.

Note: I know testing end-user applications and Web sites is difficult. All I can offer is to make that layer as thin as possible, and do more things as libraries. This is the natural way of Lisp … you stop thinking in terms of applications at all, because you can play with everything interactively. Any user interface is there to simplify the API for less sophisticated users.

2 Responses to “When is Testing Silly?”

  1. Peter Arrenbrecht Says:

    I fully agree. I would go even further to say that you should also write a first draft of the documentation. This goes hand in hand with use-case tests, as I show in greater detail in my article “Source Citing: Making Examples Work”:

    http://www.arrenbrecht.ch/articles/SourceCiting.htm

    In fact, I have just now taken a break from doing just that, writing test and doc for a new aspect of my system – before writing a single line of code. It sure helps to think from the user’s perspective.

  2. Matt MacKenzie Says:

    I’ve apparently fallen behind in my reading, as I was completely ignorant to the fact that what I’ve been doing for a while has a name (BDD). I’ve seen may APIs from co-workers that would have been much more intuitive had they derived it from samples and trying to use the API as if it already existed.

    I tend to use this technique for XML vocabularies as well…