The Limitations of TDD

During the last 12-18 months, TDD has broken into the mainstream, it seems. And now, we're starting to see some backlash, as its limitations become better understood. Here is a sample discussion from Artima.com. Cédric Beust, who wrote the commentary, is not some unknown guy with a weird name. He wrote the TestNG unit testing framework, which is second only to JUnit in popularity. He also wrote the book, Next Generation Java Testing, which is probably the best book on pragmatic software testing that I've read in a long time. Here goes...

> That's an interesting point. Are you, in effect, saying
> that unit testing is overly emphasized, and at the expense
> of other forms of testing?

This has also been my experience, although to be honest, I see this problem more in agile/XP literature than in the real world.

This is the reason why I claim that:

- TDD encourages micro-design over macro-design
- TDD generates code churn

If you obsessively do TDD, you write tests for code that you are pretty much guaranteed to throw away. And when you do that, you will have to refactor your tests or rewrite them completely. Whether this refactoring can be done automatically or not is beside the point: you are in effect creating more work for yourself.

When I start solving a problem, I like to iterate two or three times on my code before I'm comfortable enough to write a test.

Another important point is that unit tests are a convenience for *you*, the developer, while functional tests are important for your *users*. When I have limited time, I always give priority to writing functional tests. Your duty is to your users, not to your test coverage tools.

You also bring up another interesting point: overtesting can lead to paralysis. I can imagine reaching a point where you don't want to modify your code because you will have too many tests to update (especially in dynamically typed languages, where you can't use tools that will automate this refactoring for you). The lesson here is to do your best so that your tests don't overlap.

--Cedric Beust