Before getting into my concerns about whether unit testing's popularity has peaked, let me state that I think unit testing is the most important benefit wrought by the agile revolution. I agree that you can write perfectly good programs without unit tests (we did put man on the moon in 1969, after all), but for most programs of any size, you're likely to be far better off using unit tests than not.
The problem is that only a small subset of developers understand that. And recent data points suggests that the number of programmers who use unit tests is not exactly growing quickly. I'll list some of the data points below that I've been developing for my column in SD Times.
1) Commercial products on the wane. Agitar was a company whose entire fate was tied to the popularity of unit testing. Despite very good products, a free service to auto-generate unit tests for your code, and some terrific exponents (especially Alberto Savoia and Jeff Frederick) to tell their story, the company closed a down a few weeks ago, essentially having come to the conclusion that it could never be sold at a price that could repay investors. So rather than ask for more funding, it closed down. If unit testing were gaining popularity robustly, Agitar surely would have come to a different conclusion.
2) Few OSS products. Except for the xUnit frameworks themselves, few FOSS tools for unit testing have been adopted. The innovative Jester project, which built a tool that looked for untested or poorly tested logic, essentially stopped development a long time ago because to quote the founder, Ivan Moore, in a comment to me "so few sites are into unit testing enough to care about perfecting their tests."
3) Major Java instructors aren't teaching it. Consider this interview with Cay Horstmann, co-author of the excellent Core Java books. (He asks, "If so many experienced developers don't write unit tests, what does that say?" In speculating on an answer, he implies that good developers don't need unit tests. Ugh!)
4) Unit testing books are few and far between. I am seeing about one new one a year. And as yet, not a single book on JUnit 4, which has been out for nearly three years(!).
5) Alternative unit-testing frameworks, such as the excellent TestNG, are essentially completely invisible. I was at a session on scripting this spring at SD West and in a class of 30 or so, two people had heard of TestNG (the teacher and I).
I could speculate on causes, but I have no clear culprit to point to. Certainly, unit testing needs to be evangelized more. And evangelized correctly. The folks who insist on 100% code coverage are making a useful tool unpalatable to serious programmers (as discussed here by Howard Lewis Ship, the inventor of Tapestry). But, I think the cause has to be something deeper than this. I would love to hear thoughts from readers in real-world situations where unit testing has been abandoned, cut back, or simply rejected--and why.
It would be a shame to have unit testing disappear and its current users viewed as aging, pining developers hankering for a technology the world has largely passed by. That would return programmers to the tried-and-true practice of glassy-eyed staring at a debugger for hours--something I have not missed at all.
Thursday, May 22, 2008
Is the popularity of unit tests waning?
Subscribe to:
Post Comments (Atom)
45 comments:
On the other hand, unit testing has become well enough established to have triggered a backlash. Personally, I think this is one for the long haul, there's so much clean up to do in our industry.
To respond to some of your points:
1) Agitar was cool technology with some very good people but, from what I saw, it wasn't well sold. I don't know if they changed their pitch later but the early ones I saw didn't make the case.
3) I think there's still a lot of catch up in our industry. TDD is a huge shift for a lot of people, especially those who've been good at what they do.
4) Does jUnit4 need a book? It needs some documentation, but it's pretty simple and not that much different from jUnit3.
5) Just because one of the frameworks has low visibility doesn't kill the discipline. I suspect the problem with TestNG is that jUnit is the obvious choice and, anyway, it seems to be pitched at a higher level than units. Anyway Cedric doesn't appear to believe in unit testing :)
Good question.
The fact that most developers don't write unit tests does not necessarily imply they aren't all they're made out to be. However, it does raise an interesting question as to why they aren't more widely adopted.
In my experience unit tests have been most widely adopted in product developments where all concerned know the life of the product will exist across many iterations and changes. In building a product you are willing to invest the time and energy to automate testing because you know that in the long run it will make a difference.
In service based development the desire to invest in unit testing (and yes, I do believe there is an upfront investment) is often outweighed by the reality that the solutions you build will not evolve (rather they'll be replace in a year or so), need to be delivered in very small time frames (often weeks not months).
There is also a fundamental challenge in that the cost of investing in unit tests is measurable and the benefits are more often more difficult to gauge and are often only realised in the future.
I think there might be a difference depending on what language community you're in. Unit testing is still very fad in the Ruby community. That is not fad in a negative sense, but simply that it's still popular and cool to write unit tests. Actually the cool thing is TDD, which is probably more rare than plain unit testing.
At the last RubyConf, David Chelimsky asked how many people use Test::Unit and how many people use RSpec. It was basically split down the middle. Then he asked how many don't write unit tests at all. There were four, I think.
Granted, the developers who go to conferences are at least a step above the average developer. Still, to have a 99% adoption rate even in that group is pretty encouraging.
As far as books go, I know of three books about RSpec alone being published this year.
I particularly use Unit Test.
Development in RoR and Java and I do not see developing in Ror without RSpec.
Indeed even feel that lack in other languages.
But you are correct, the problems decrease if unit tests were made regularly.
Excellent post.
Fabio Nascimento
What makes you think that the engineers at NASA didn't use unit testing (along with many other forms of testing)?
It's probably because developers are generally filled with hubris, not humility.
Time is always the issue for unit tests. Many developers and managers mistakenly think that by cutting testing, they well get feature "X" out faster. What they don't realize is that you are robbing Peter to pay Paul. Feature "X" may be faster, but now "Y" & "Z" are slower because you must manually test "X" after each checkin.
Unit testing will make a comeback. All truly good ideas do. It took me a long time to appreciate that unit testing takes you farther, faster. The up front cost of writing tests, pays you back in the long run with interest. It frees you to experiment (major architectural experiments) without worry of breaking things. So you can refactor to your hearts content and keep the good stuff.
This is one good thing about the Rails community. All the leaders openly endorse unit testing ... with RSpec being the flavour-du-jour. Kent Beck will be speaking at Railsconf this year.
I think you're arguing that profitable (or OSS) unit testing software is going out of style, rather than unit testing itself.
Jumble (http://jumble.sourceforge.net/) is an alternative to Jester that seems to work very well. We use it a little bit, though unfortunately it's too slow to run as part of continuous integration.
Maybe 1 2 and 5 have the same cause - the impression that the free xUnit frameworks are "good enough". I would add though that in my experience too many developers just do not understand what a unit test, and they just write what are essentially integration tests, so even when they find there is a problem they *still* have to stare glassy eyed at the debugger, just as they would if doing their testing from the application's UI.
As long as people use or rely on it, it wont disappear. But a declining popularity per se isnt so bad.
I picture that in a perfect world we would not need unit testing.
Sounds not realistic? But I like this goal, so I work for it as well.
I think ultimately unit tests just ensure that the apps work more reliable. But there are other ways to create reliable and working software.
The language that is used for this is important as well. Different languages make things differently easy (or difficult).
For my part I try to actively avoid unit testing. The only testing I will accept is my own eyes and brain, or some kind of bot that does user-testing input in random ways.
Such a bot would be better too, I think, because you could simulate users and you can reuse it without ANY extra code on a specific project
I think it's just that most agile java developers gone to ruby & rails community, where by the way unit/behavior testing blossoms everywhere.
Anyone not writing tests for their ruby code is considered to be lame, and I have not saw any Rails project without some kind of unit tests (some tests are not of great quality, but they are there).
Hey Andrew,
Interesting comments on Agitar and unit testing.
Unit testing is really valuable. I consider it similar to documenting your code - also really valuable. But until the semantic equivalent of unit testing is actually built into the programming language itself, it won't really take off. That's fundamentally the problem in terms of why it's not being adopted as widely as some would like. Any time you have a second system outside the language that tries to mirror what's actually in the language, it doesn't persist well past initial development. A few cases in point:
Program documentation itself - often out of date with the actual code unless you have a public API or interface that requires documentation.
UML modeling - often a great start to a project, but often abandonded once things get under way.
Aspect oriented programming: Gregor Kiczales at Xerox Parc actually invented a new language, AspectJ, and notice that's the most popular AOP system out there.
So I predict someone like Anders Hejlsberg or someone in the Java world will eventually come up with some way to express what Unit testing does, as actual formal syntax elements of one of the existing popular languages out there.
You've get C# attributes and Java annotations as a generic mechanism for extending the metadata associated with language elements, to essentially allow those outside the actual language developers themselves to extend the language somewhat. But I that's kidn of klunky and ultimately not powerful enough. I also think this is a really hard problem to tackle - really hard, and probably won't be solved for a number of years.
A little about my background: I've been pretty close to Agitar over the years, and have also seriously investigated a number of other unit testing tools including JTest, of course JUnit, NUnit, and a few others. I've worked as either an architect or senior developer on countless Agile projects, and have even consulted to large companies to help them make the transition to Agile. I've been programming professionally a little over 30 years, and have specialized in development tools for many years (managed Delphi at Borland) so I guess I'm just saying all that to set up my perspective.
I think what you're seeing isn't that unit testing is less popular, it is that it's growth in popularity has essentially stopped. Essentially, the subset of developers who actively work to continually improve (ie, those that are most likely to adopt unit testing) have all done so, so the initial rush of projects, books, tools, etc, is over. The much larger set, who for the most part, don't read or write blogs, don't buy books or tools (unless forced by management), and don't expend much than the minimal amount of effort are either 1) being dragged unwillingly into unit testing environments, or 2) succesfully killing attempts to start unit testing in their environments.
Maybe it's different in your corner of the developer world, but on my end it's very different.
Unit testing is extremely common in the open source Python world. Two different unit test libraries are in the standard library (unitest, which is like xUnit, and doctest, which is nothing like it). There are a number of open source unit test runner frameworks. Popular ones are Nose, py.test and zope.testrunner. These include powerful features such as code coverage analysis. Test runner features are also included in Python packaging frameworks such a setuptools.
There are also frameworks such as zope.testbrowser, which emulate a web browser to do integration testing in either classic unit tests or doctests. I also know of various mock-object libraries.
Perhaps it's true there is relatively little innovation with new approaches in this field. I'd argue though that both doctest and zope.testbrowser are innovations that still need to spread far and wide.
I haven't seen a waning of popularity of testing in the Python world. You don't see a lot of hype and buzz about it, but that's more like it's a standard, familiar feature of the developer toolbox. You don't see a lot of buzz about object oriented programming either these days, but that's not because it's unpopular, just because it's familiar.
Just in my own experience I wouldn’t say its popularity is waning I would say that in the past it was talked about a lot and practiced very little. Now the talk has tapered off a bit. When I first tried out unit testing I was working in a large financial company in Boston – I felt that unit testing my code helped me produce and gave me the feeling of having a safety net. At the same time I never saw any other developers do unit testing and tests were not integrated into our builds nor would one find test code in source control. It wasn’t on anyone’s radar screen. Later I went to a startup and the value that people brought to the project was not evaluated by their contribution to the testing of code. When I had people working for me the tests were not something people were interested in writing and certainly not interested in maintaining - (esp if written by someone else). And for the most part when people are not interested in doing something there is always the old “if I have to do this (extra stuff) then I wont be able to meet the deadline…” And that works with management. Now I work for a large multinational Silicon Valley technology company. The developers on my current project are considered to be so elite, the product is considered to be so pivotal to the companie’s future strategy. There are no tests; I have NUnit on my dev machine but these elite developers don’t know what that is; and the given the design and code style they couldn’t use it to test much anyways. There is test talk but it’s more of the “we need you guys to write bug free, real high quality stuff so be extra careful” type of uselessness.
So what can I say? At the best, writing unit tests is likely to help developers develop better coding style, help them to make code modifications with more confidence, it may help get code working faster, and as a side effect it may produce more robust code. At the worst, it’s a wash – some extra effort, some extra time savings in return.
And then the reality is today my manager is going to come to me with a specification change that for that day is going to be the most important thing ever – the crisis point in this project – that has to be done – special and now. The incentives driving my manager are to get the “done” box on her spreadsheet filled in by whatever methods she thinks will drive me to that end faster. The idea of asking for unit tests will not occur to her. She may oppose the idea. I’m guessing young developers will see learning about unit testing as taking away from the time they could be using to learn about LINQ or AJAX or whatever the latest coolest is that they think will help them go to the next engineering level. And of course some developers want to be in management – so engineering methods aren’t the interest.
Ultimately the difficulty has to be the incentives. The people who judge our contributions don’t factor in details about testing and wouldn’t what unit testing is anyway. Coworkers may see unit testing as just extra work on their way towards some other goal. Is a job well done enough motivation for everyone? And then there is the big picture – how cheap – how quickly done – these ‘metrics’ are the focus of my organization; the other things developers might talk about like how correct, how well architected – quality in general cant be measured by business people so testing is going to be a tough sell.
In the Ruby community, there has been a lot of innovation in the realm of testing. Check out Rspec, which is very popular. I do think it's sad that universities aren't teaching empirical software design as part of their CS core curriculum.
Maybe because it's essentially useless? It is too much work for really a insignificant benefit for someone trying to get software out the door. It is not always essential to get things right the first time and it is okay to spin and respin code. Everyone needs to accept this. Money is made this way. People are employed this way. And it is insane to expect people to write more code to test their code (maybe if someone else writing the unit test was encouraged!!!) -- having to think through to a software solution and then having to be think some more and make more and more decisions to write up unit tests is simply too detail oriented for most humans.
The problem with unit testing is that it increases the cost of development before you know whether what you developed is what you needed.
Afterwards when you have a working solution, there's little interest in writing unit-tests as what you have works, and your management wants you to move on.
So if the act-of-god bug scenario is easier to justify to management, requires less discipline, and appears to make you more flexible the rest of the time, why are you surprised that unit-tests don't take off?
Unit testing has been around for decades and is here to stay. AUTOMATED, Programatic unit testing using units like jUnit is more recent and less accepted. Truely isolated automated tests have had a bit of a backlash, yes. I think we're finding out that there is simply more than one way to skin a cat.
If these are the best you've got, unit testing sounds like it's in great shape. As Linus once said, Linux will have made it when it's no longer in the news (i.e., it's assumed). I think unit testing is here to stay *because* nobody talks about it any more -- it's assumed.
1. All kinds of commercial development environments are on the wane. How many companies do you know selling development tools of any sort today?
2. How many F/OSS unit testing tools do I need? I've never heard of Jester, but I've heard of those techniques. Jester seems to be for Java, so if you're using one of the bazillion languages which does not start with a "J", it's easier to write one yourself, in-house. Tools that operate on source code aren't hard to write; that's just a myth the Java and IDE people made up. Seibel wrote a unit test framework in a couple dozen lines, in PCL.
3. I don't know what makes a Java instructor "major", nor do I know why I would care what they teach. Java is a language; unit tests are a kind of code. Do your "MJIs" teach how to write every possible kind of code? I have a CS degree but none of the code I've written for work has been the same as the code I wrote in school.
4. I have only 2 compiler books; does that mean compilers are "on the wane", or simply that I don't need 17 books on the same subject? Is there anything magic in JUnit 4 that requires a book? Why can't I figure it out from the unit testing principles I learned years ago and the manual, like I do for every other topic?
5. Things reasonably labelled "alternative" tend to have low visibility. That's why you call them "alternative". See also: there are no "alternative" albums in the top-20, and "alternative" fuel cars are a tiny minority.
Unit test are useful, and maybe xUnit popularity has reached it's plateau, but please, please, please, no more evangelism!! :)
I feel they're waning because once again, it's discovered that it's not a silver bullet.
I'm not convinced that it ever was popular. :-(
I liked your post enough to respond to it over at my blog.
It seems that currently the unit test innovations that drive adoption are those that make tests more accessible to users.
Rspec is an obvious case study where an elegant api helps to spur excitement and innovation.
I think that there are some interesting technologies that actively encourage programmer productivity.
For example, look at the upcoming Javascript testing frameworks. There is a good amount of innovation here.
Javascript's weak typing also makes testing more valuable that other languages.
Some examples are Screw Unit (http://github.com/nkallen/screw-unit), Screw Unit Server (http://github.com/pivotal/screw-unit-server/tree/master), Js Spec (http://code.google.com/p/js-spec/), JS Unit (http://www.jsunit.net/), Js Unit Test (http://github.com/drnic/jsunittest), etc.
Unit tests are important, but I believe they are used too much.
The majority of unit testing is testing the contract of a method. The contract should be embedded in the code (e.g. like javadoc) and tested by the infrastructure.
Writing unit tests instead of explicitly stating the contract obfuscates what the contract really is, is more brittle and takes up lots of time writing code that the computer could write for you.
I don't believe those who say that the unit tests document how to use the code - there are always more unhappy paths than happy paths, so by definition the majority of unit test code shows how NOT to use the code, and has all sorts of other crazy stuff going on too.
Of course, there is still room form unit testing code even code that has been Designed By Contract, but unit tests are absolutely the wrong place and way to document and test contract.
1) I think agitar had a great product, but everyone knew that automatically generated unit tests were never going to be any good. Combine that with the cost of the tool and it became a poor proposition.
2) All the best developers I've worked with write good unit tests. Yet I think you are right, the grass roots support is pretty low.
It also inspired a more detailed entry on my part
http://www.jroller.com/BrightCandle/entry/why_evangelising_unit_testing_needs
I work for an Aerospace company.
For all flight critical code, we have to unit test(The FAA requires it DO-178B). Unit testing has caught many of errors. We mostly have new software people and sub contractors doing the unit testing. We don't use an Agile approach so our unit tests are tested against our design and compared with our code using a in house simulation test harness. When the more experienced testers are given a chance to code on the hardware they are much better because they have done unit testing.
Unit tests are important, but I believe they are used too much.
I think that its more of Unit Tests are reported to serve too many purposes.
Writing unit tests instead of explicitly stating the contract obfuscates what the contract really is
Excellent point. Part of the problem is the lack of tools for the xUnit software to follow the logic of the method being tested. Rspec introduces nested describes, which makes it easier to follow the use cases for each method.
On my current project, we have been successful in utilizing nested describes for us to get a sense of the use cases for our methods.
That being said, its not necessarily the end solution. It seems like there is a start of a convergence of test code + documentation, at least in the ruby community.
Unit test utility is more obvious in dynamic languages than in static ones because they guard against errors that the compiler would normally catch.
However they are still valuable in any modern language because all of the runtime scenarios cannot be verified with a compile time checker.
---
Of course, I'm still waiting for the language that reads my mind, is more intelligent that I am, and creates perfect programs. Maybe when we get to that point, we no longer need to verify our software. :)
Unit test are useful, and maybe xUnit popularity has reached it's plateau, but please, please, please, no more evangelism!! :)
This is exactly one of the problems! only around 1% of developers go to conferences and they seem to be the same people whose management understand that they should be going. These guys have heard enough about Unit testing so the topic does not have a lot of traction with the organizers which may lead to new people coming to the conference to think the practice is not that important - it's a tough one!!!
As for teachers not teaching these practices - that is shocking! For those 'in the know' (i.e. the people above) most are of the opinion that Unit Testing (in whatever language) is a good thing and helps stabilize your app etc, especially with continuous integration or even automated Unit testing performed at arbitrary times. It's not a language feature but it is good practice (when possible. I understand there are times you can't do this or it is hard - IDE plug-ins for example). When I learned to drive, I didn't just learn how to operate a car - I learned the rules of the road to keep me safe!
Unit testing evangelism is one of the worst things to come out of the agile movement, imo.
I write and insist on the correct level of unit testing and modular software construction, but the obsession with pre-band-aiding little bits of code which we should (frankly) be expected to produce properly gives us a false sense of security. Just because we show 100% code coverage does not mean that we have effectively tested our code -- and achieving that 100% comes at a resource cost which could have often been better spent achieving a higher level of quality verification with a different kind of test.
I know this and I suspect you know this, but the evangelism, by focusing on one little (very low-level) type of test, causes a large portion of people to miss the forrest for the trees.
And consider this: it takes a lot more than unit testing to send a man to the moon. By exclusively evangelizing unit testing, the proponents have marginalized the overall discipline of software quality and testing, which is much bigger in scope.
I would not ride to the moon in a vehicle where the primary automated mechanism of software quality assurance involved was the unit test.
Let me tell you a story; I work in a quite big project where quite a lot of developer are involved.
Most of them don't think Unit test is necessary, I do.
Everytime I ask one of the other to refactor something it usually takes a lot of time and breaks something, because they have no way to test other than launching the application and clik here and there and see if something goes "boom"...
Some time ago they asked me to do some massive refactoring of a module I wrote; it took me half a day and made no error. Why? Because I'm a greatly better developer than they? Because I never make mistakes? No, just because I write unit tests to cover my "back" in case of change!
I hope that unit test will live strong until something really better comes up - and I don't think this will happen anywhere soon...
It happened once that I got 0 points for my program in a university because the teachers got confused by the unit tests. They did not understand what is a unit test. This was a C++ class, so it is not only a java-specific problem.
We can connect the following : JUnit to Java to J2EE to enterprises.
Now let us apply the following facts:
1. Java is a bloated language.
2. Most (not all) entrprise programmers are average programmers.
3. Average programmers treat programming as a job, so effectively they will not care about their creations.
In addition, since Java is a bloated languages, a tired average programmer will veer away from writing even more Java code.
So I think Junit was adopted only by those few who care about their work. It will never peak nor be on the wane.
Replace Java with a powerful language and let us wait for the results :-)
Good question
@Deepak> Replace Java with a powerful language and let us wait for the results :-)
I cringe at the idea that one's professional standards could be somehow related to the language he uses, no matter how bloated it is.
Well from my experience I have been using unit tests for a long time and I personally feel that they provide additional benefits than just testing your code
1. It documents your code.
2. It opens up opportunities to better design your system
3. It provides a way to prove your theories by just adding couple unit tests
There are numerous books written on unit tests. However, the reason you don't see too many coming out every year is because unit testing is such a simple and easy concept which doesn't need too much explanation. Just because some experienced developers are confident of their code that doesn't mean they don't have to write unit tests. I feel the term testing is somewhat ego sensitive for developers :-)
One factor here is that managers usually don't see the benefit of unit tests except in some theoretical sense. The folks who really grok unit tests are the good developers, and their managers frequently push them to spend their time writing "real code" instead of tests which never get shipped to customers.
Managers (particlarly the pointy haired kind) like statistics and trends. If the unit testing tools gave more quantitative metrics and trending, more managers would go for it. Metrics such as some summary "confidence" number; something to express test failures and remediation in terms of dollars (cost of leaving the defect undetected vs. cost to write the test); and something to correlate tests to defects (more tests/assertions should mean fewer defects).
The low phase that has set into the IT sector is no longer news. The fall in US economy was a major blow to the IT world all over the world. In countries like India where you can find a software development company on every nook and corner are rapidly closing down because of lack of work. What is to be seen is the strategy formulated by the big fishes such as Infosys and Tata. The current situation is of uncertainty and fear as companies are sacking employees, something that was unthought of a few years back!
As a seasoned Java and ex C++ developer, I'm of the firm opinion that unit testing in Java is mostly a waste of time. There is no point in testing ctors, no point in testing accessors, little point in testing basic methods. Java is so predictable as a language that it is only the interactions between classes that are sufficiently complex to test. However it is exactly these interactions that are nearly impossible to write unit tests for.
I work on a pretty progressive dev team and we are required to have automated unit tests for all of our new code. Devs and managers all see the value and everyone is onboard. But ultimately, the cost/benefit ratio seems low and the benefits not all that tangible. The initial cost of unit testing is too high for most teams to swallow.
Anything other than a trivial unit test usually requires stubbing out database calls, config file access and other dependencies. We use a nice library called RhinoMocks to do this, but it is yet another thing you have to learn in order to get started.
I find developers fall into two camps: those that find writing unit tests an interesting challenge, and those that think they're a waste of time. Both viewpoints are valid.
A lot of unit tests are a waste of time. Specifically, the trivial ones that do input validation, making sure function Foo throws an exception when parameter x is null. These kinds of tests should be autogenerated or handled automatically by the test framework.
The more interesting unit tests can be a challenge to write and make you think about how testable your code is, but not many programmers find this kind of work very satisfying.
Updating and maintaining an existing codebase with unit tests is a mixed bag. The good news is that most bug fixes are a breeze. Running the unit tests immediately tells you if you screwed something up or affected the system in some unexpected way. This is probably the best case for unit tests and really does build a lot of confidence.
The bad news is that if you need to make more substantial changes to the application, you will find that your changes break most of the unit tests. Ugh! Now you have to decide whether to fix all of the tests or rewrite them.
Also how the unit tests are written has a big impact. Too few low quality tests, gives a false sense of security. However, at the other extreme, I've worked on a project where changing any line of code at all was guaranteed to break at least a dozen unit tests. Maintaining the unit tests took five times longer than the original code change.
All that said, I think automated unit testing is here to stay in one form or another, and really is needed in order to validate program correctness and improve maintainability. Commercial software development really needs to adopt more of an engineering mindset.
Is the popularity of unit test waning? Probably. Somehow the notion of writing and maintaining MORE code in order to make maintaining your code easier is a hard sell. And I'm afraid with existing tools and the limited breath of knowledge out there, most people will be turned OFF of unit testing pretty quickly.
"The best tool is still the best tool". I take this to mean that whatever you use to test, it should be whatever you can best assemble for that occasion. Re-usability is important, but it takes a second priority to getting the job done.
Rolling your own test framework is always going to be better than buying one (sorry to Agitar). Unit tests are fast to create and fast to run. Integration unit tests (that work between classes) are slower to create and slower to run. So automate writing them with more code. More code you have to test? Yes, but refactoring tests is even easier and faster than refactoring tested production code. Every bit of abstraction away from the complexity makes your job easier.
There are a lot of books on JUnit, TestNG, xUnit, CI, and TDD. I think there will be more. As well as more magazine articles, blog posts, and everything else...
There's much simpler reason why most developer do not use unit test : we don't have enough time.
No matter what you say, unit test IS a luxury. It double development time. I'd rather sleep 6 hours/day without writing any unit test than only sleep 2 hours/day but writing proper unit test.
So, the only one to blame is not the developer, but those bastard project manager who set such very tight schedule.
I know I'm a little bit late on this, but the blog gives a few mis-impressions. Agitator was an automatic test data generator, completely different from JUnit or TestNG. JUnit gives us a box, but an empty box. Agitator put things in that box. Agitator also gave very good tests, not the random garbage that most test generators give. It used very sophisticated algorithms to generate tests that try to satisfy CoC.
What they found is that companies do a lot of unit testing, but few were willing to pay money to generate high quality tests. That is, it wasn't unit testing that companies refused to buy into, but criteria-based test design.
I know I'm a little bit late on this, but the blog gives a few mis-impressions. Agitator was an automatic test data generator, completely different from JUnit or TestNG. JUnit gives us a box, but an empty box. Agitator put things in that box. Agitator also gave very good tests, not the random garbage that most test generators give. It used very sophisticated algorithms to generate tests that try to satisfy CoC.
What they found is that companies do a lot of unit testing, but few were willing to pay money to generate high quality tests. That is, it wasn't unit testing that companies refused to buy into, but criteria-based test design.
Post a Comment