tag:blogger.com,1999:blog-9620948.post7642737437896590940..comments2023-10-20T12:24:17.734-07:00Comments on Binstock on Software: Is the popularity of unit tests waning?Andrew Binstockhttp://www.blogger.com/profile/16321156191558412680noreply@blogger.comBlogger45125tag:blogger.com,1999:blog-9620948.post-84092253939467938442011-02-18T11:36:15.647-08:002011-02-18T11:36:15.647-08:00I know I'm a little bit late on this, but the ...I know I'm a little bit late on this, but the blog gives a few mis-impressions. Agitator was an automatic test data generator, completely different from JUnit or TestNG. JUnit gives us a box, but an empty box. Agitator put things in that box. Agitator also gave <b>very</b> good tests, not the random garbage that most test generators give. It used very sophisticated algorithms to generate tests that try to satisfy CoC.<br /><br />What they found is that companies do a lot of unit testing, but few were willing to pay money to generate high quality tests. That is, <b>it wasn't unit testing that companies refused to buy into, but criteria-based test design</b>.Unknownhttps://www.blogger.com/profile/13693029389703689482noreply@blogger.comtag:blogger.com,1999:blog-9620948.post-30105662748906325882011-02-18T11:35:39.851-08:002011-02-18T11:35:39.851-08:00I know I'm a little bit late on this, but the ...I know I'm a little bit late on this, but the blog gives a few mis-impressions. Agitator was an automatic test data generator, completely different from JUnit or TestNG. JUnit gives us a box, but an empty box. Agitator put things in that box. Agitator also gave <b>very</b> good tests, not the random garbage that most test generators give. It used very sophisticated algorithms to generate tests that try to satisfy CoC.<br /><br />What they found is that companies do a lot of unit testing, but few were willing to pay money to generate high quality tests. That is, <b>it wasn't unit testing that companies refused to buy into, but criteria-based test design</b>.Unknownhttps://www.blogger.com/profile/13693029389703689482noreply@blogger.comtag:blogger.com,1999:blog-9620948.post-57792734328850689712008-09-28T21:09:00.000-07:002008-09-28T21:09:00.000-07:00There's much simpler reason why most developer do ...There's much simpler reason why most developer do not use unit test : we don't have enough time.<BR/><BR/>No matter what you say, unit test IS a luxury. It double development time. I'd rather sleep 6 hours/day without writing any unit test than only sleep 2 hours/day but writing proper unit test.<BR/><BR/>So, the only one to blame is not the developer, but those bastard project manager who set such very tight schedule.Peter Harjantohttps://www.blogger.com/profile/03864491558753619209noreply@blogger.comtag:blogger.com,1999:blog-9620948.post-66584988713612431202008-08-15T19:03:00.000-07:002008-08-15T19:03:00.000-07:00"The best tool is still the best tool". I take th..."The best tool is still the best tool". I take this to mean that whatever you use to test, it should be whatever you can best assemble for that occasion. Re-usability is important, but it takes a second priority to getting the job done.<BR/><BR/>Rolling your own test framework is always going to be better than buying one (sorry to Agitar). Unit tests are fast to create and fast to run. Integration unit tests (that work between classes) are slower to create and slower to run. So automate writing them with more code. More code you have to test? Yes, but refactoring tests is even easier and faster than refactoring tested production code. Every bit of abstraction away from the complexity makes your job easier.<BR/><BR/>There are a lot of books on JUnit, TestNG, xUnit, CI, and TDD. I think there will be more. As well as more magazine articles, blog posts, and everything else...drehttps://www.blogger.com/profile/17414510788948258195noreply@blogger.comtag:blogger.com,1999:blog-9620948.post-68385726604360394862008-08-13T13:24:00.000-07:002008-08-13T13:24:00.000-07:00I work on a pretty progressive dev team and we are...I work on a pretty progressive dev team and we are required to have automated unit tests for all of our new code. Devs and managers all see the value and everyone is onboard. But ultimately, the cost/benefit ratio seems low and the benefits not all that tangible. The initial cost of unit testing is too high for most teams to swallow.<BR/><BR/>Anything other than a trivial unit test usually requires stubbing out database calls, config file access and other dependencies. We use a nice library called RhinoMocks to do this, but it is yet another thing you have to learn in order to get started.<BR/><BR/>I find developers fall into two camps: those that find writing unit tests an interesting challenge, and those that think they're a waste of time. Both viewpoints are valid.<BR/><BR/>A lot of unit tests are a waste of time. Specifically, the trivial ones that do input validation, making sure function Foo throws an exception when parameter x is null. These kinds of tests should be autogenerated or handled automatically by the test framework.<BR/><BR/>The more interesting unit tests can be a challenge to write and make you think about how testable your code is, but not many programmers find this kind of work very satisfying.<BR/><BR/>Updating and maintaining an existing codebase with unit tests is a mixed bag. The good news is that most bug fixes are a breeze. Running the unit tests immediately tells you if you screwed something up or affected the system in some unexpected way. This is probably the best case for unit tests and really does build a lot of confidence.<BR/><BR/>The bad news is that if you need to make more substantial changes to the application, you will find that your changes break most of the unit tests. Ugh! Now you have to decide whether to fix all of the tests or rewrite them.<BR/><BR/>Also how the unit tests are written has a big impact. Too few low quality tests, gives a false sense of security. However, at the other extreme, I've worked on a project where changing any line of code at all was guaranteed to break at least a dozen unit tests. Maintaining the unit tests took five times longer than the original code change.<BR/><BR/>All that said, I think automated unit testing is here to stay in one form or another, and really is needed in order to validate program correctness and improve maintainability. Commercial software development really needs to adopt more of an engineering mindset.<BR/><BR/>Is the popularity of unit test waning? Probably. Somehow the notion of writing and maintaining MORE code in order to make maintaining your code easier is a hard sell. And I'm afraid with existing tools and the limited breath of knowledge out there, most people will be turned OFF of unit testing pretty quickly.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-9620948.post-58741033864737220302008-08-13T13:22:00.000-07:002008-08-13T13:22:00.000-07:00As a seasoned Java and ex C++ developer, I'm of th...As a seasoned Java and ex C++ developer, I'm of the firm opinion that unit testing in Java is mostly a waste of time. There is no point in testing ctors, no point in testing accessors, little point in testing basic methods. Java is so predictable as a language that it is only the interactions between classes that are sufficiently complex to test. However it is exactly these interactions that are nearly impossible to write unit tests for.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-9620948.post-27059971603904999002008-06-29T23:42:00.000-07:002008-06-29T23:42:00.000-07:00The low phase that has set into the IT sector is n...The low phase that has set into the IT sector is no longer news. The fall in US economy was a major blow to the IT world all over the world. In countries like India where you can find a <A HREF="http://www.infysolutions.com" REL="nofollow">software development company</A> on every nook and corner are rapidly closing down because of lack of work. What is to be seen is the strategy formulated by the big fishes such as Infosys and Tata. The current situation is of uncertainty and fear as companies are sacking employees, something that was unthought of a few years back!Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-9620948.post-42002715224514034552008-06-15T08:31:00.000-07:002008-06-15T08:31:00.000-07:00One factor here is that managers usually don't see...One factor here is that managers usually don't see the benefit of unit tests except in some theoretical sense. The folks who really grok unit tests are the good developers, and their managers frequently push them to spend their time writing "real code" instead of tests which never get shipped to customers.<BR/><BR/>Managers (particlarly the pointy haired kind) like statistics and trends. If the unit testing tools gave more quantitative metrics and trending, more managers would go for it. Metrics such as some summary "confidence" number; something to express test failures and remediation in terms of dollars (cost of leaving the defect undetected vs. cost to write the test); and something to correlate tests to defects (more tests/assertions should mean fewer defects).Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-9620948.post-59271813613135309172008-06-06T11:48:00.000-07:002008-06-06T11:48:00.000-07:00Well from my experience I have been using unit tes...Well from my experience I have been using unit tests for a long time and I personally feel that they provide additional benefits than just testing your code<BR/><BR/>1. It documents your code.<BR/>2. It opens up opportunities to better design your system<BR/>3. It provides a way to prove your theories by just adding couple unit tests<BR/><BR/>There are numerous books written on unit tests. However, the reason you don't see too many coming out every year is because unit testing is such a simple and easy concept which doesn't need too much explanation. Just because some experienced developers are confident of their code that doesn't mean they don't have to write unit tests. I feel the term testing is somewhat ego sensitive for developers :-)Raghu Kashyaphttps://www.blogger.com/profile/16627252842238807340noreply@blogger.comtag:blogger.com,1999:blog-9620948.post-466131415745894522008-06-06T09:06:00.000-07:002008-06-06T09:06:00.000-07:00@Deepak> Replace Java with a powerful language and...@Deepak> Replace Java with a powerful language and let us wait for the results :-)<BR/><BR/>I cringe at the idea that one's professional standards could be somehow related to the language he uses, no matter how bloated it is.Anonymoushttps://www.blogger.com/profile/13987512594987806769noreply@blogger.comtag:blogger.com,1999:blog-9620948.post-21706753377274929232008-06-05T07:18:00.000-07:002008-06-05T07:18:00.000-07:00Good questionGood questionAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-9620948.post-56681343025879714262008-06-04T06:01:00.000-07:002008-06-04T06:01:00.000-07:00We can connect the following : JUnit to Java to J2...We can connect the following : JUnit to Java to J2EE to enterprises.<BR/><BR/>Now let us apply the following facts:<BR/><BR/>1. Java is a bloated language.<BR/>2. Most (not all) entrprise programmers are average programmers.<BR/>3. Average programmers treat programming as a job, so effectively they will not care about their creations.<BR/><BR/>In addition, since Java is a bloated languages, a tired average programmer will veer away from writing even more Java code.<BR/><BR/>So I think Junit was adopted only by those few who care about their work. It will never peak nor be on the wane.<BR/><BR/>Replace Java with a powerful language and let us wait for the results :-)Deepak Surtihttps://www.blogger.com/profile/17621415181212724238noreply@blogger.comtag:blogger.com,1999:blog-9620948.post-16422307318427326952008-06-02T05:52:00.000-07:002008-06-02T05:52:00.000-07:00It happened once that I got 0 points for my progra...It happened once that I got 0 points for my program in a university because the teachers got confused by the unit tests. They did not understand what is a unit test. This was a C++ class, so it is not only a java-specific problem.Kockahttps://www.blogger.com/profile/10505773298750628833noreply@blogger.comtag:blogger.com,1999:blog-9620948.post-47387142857565545962008-06-01T02:24:00.000-07:002008-06-01T02:24:00.000-07:00Let me tell you a story; I work in a quite big pro...Let me tell you a story; I work in a quite big project where quite a lot of developer are involved. <BR/>Most of them don't think Unit test is necessary, I do.<BR/>Everytime I ask one of the other to refactor something it usually takes a lot of time and breaks something, because they have no way to test other than launching the application and clik here and there and see if something goes "boom"...<BR/>Some time ago they asked me to do some massive refactoring of a module I wrote; it took me half a day and made no error. Why? Because I'm a greatly better developer than they? Because I never make mistakes? No, just because I write unit tests to cover my "back" in case of change!<BR/>I hope that unit test will live strong until something really better comes up - and I don't think this will happen anywhere soon...Riccardo Cossuhttps://www.blogger.com/profile/13543323003322472907noreply@blogger.comtag:blogger.com,1999:blog-9620948.post-40260203879746875422008-05-29T07:12:00.000-07:002008-05-29T07:12:00.000-07:00Unit testing evangelism is one of the worst things...Unit testing evangelism is one of the worst things to come out of the agile movement, imo.<BR/><BR/>I write and insist on the correct level of unit testing and modular software construction, but the obsession with pre-band-aiding little bits of code which we should (frankly) be expected to produce properly gives us a false sense of security. Just because we show 100% code coverage does not mean that we have effectively tested our code -- and achieving that 100% comes at a resource cost which could have often been better spent achieving a higher level of quality verification with a different kind of test.<BR/><BR/>I know this and I suspect you know this, but the evangelism, by focusing on one little (very low-level) type of test, causes a large portion of people to miss the forrest for the trees.<BR/><BR/>And consider this: it takes a lot more than unit testing to send a man to the moon. By exclusively evangelizing unit testing, the proponents have marginalized the overall discipline of software quality and testing, which is much bigger in scope.<BR/><BR/>I would not ride to the moon in a vehicle where the primary automated mechanism of software quality assurance involved was the unit test.Unknownhttps://www.blogger.com/profile/06183383479472751819noreply@blogger.comtag:blogger.com,1999:blog-9620948.post-61021372773557304482008-05-27T07:35:00.000-07:002008-05-27T07:35:00.000-07:00Unit test are useful, and maybe xUnit popularity h...<I>Unit test are useful, and maybe xUnit popularity has reached it's plateau, but please, please, please, no more evangelism!! :)</I><BR/><BR/>This is exactly one of the problems! only around 1% of developers go to conferences and they seem to be the same people whose management understand that they should be going. These guys have heard enough about Unit testing so the topic does not have a lot of traction with the organizers which may lead to new people coming to the conference to think the practice is not that important - it's a tough one!!!<BR/><BR/>As for teachers not teaching these practices - that is shocking! For those 'in the know' (i.e. the people above) most are of the opinion that Unit Testing (in whatever language) is a good thing and helps stabilize your app etc, especially with continuous integration or even automated Unit testing performed at arbitrary times. It's not a language feature but it is good practice (when possible. I understand there are times you can't do this or it is hard - IDE plug-ins for example). When I learned to drive, I didn't just learn how to operate a car - I learned the rules of the road to keep me safe!Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-9620948.post-19405562290262563442008-05-24T11:39:00.000-07:002008-05-24T11:39:00.000-07:00Unit tests are important, but I believe they are u...<I>Unit tests are important, but I believe they are used too much.</I><BR/><BR/>I think that its more of Unit Tests are reported to serve too many purposes.<BR/><BR/><I>Writing unit tests instead of explicitly stating the contract obfuscates what the contract really is</I><BR/><BR/>Excellent point. Part of the problem is the lack of tools for the xUnit software to follow the logic of the method being tested. Rspec introduces nested describes, which makes it easier to follow the use cases for each method.<BR/><BR/>On my current project, we have been successful in utilizing nested describes for us to get a sense of the use cases for our methods.<BR/><BR/>That being said, its not necessarily the end solution. It seems like there is a start of a convergence of test code + documentation, at least in the ruby community.<BR/><BR/>Unit test utility is more obvious in dynamic languages than in static ones because they guard against errors that the compiler would normally catch.<BR/><BR/>However they are still valuable in any modern language because all of the runtime scenarios cannot be verified with a compile time checker.<BR/><BR/>---<BR/><BR/>Of course, I'm still waiting for the language that reads my mind, is more intelligent that I am, and creates perfect programs. Maybe when we get to that point, we no longer need to verify our software. :)Brian Takitahttps://www.blogger.com/profile/05145263135844768540noreply@blogger.comtag:blogger.com,1999:blog-9620948.post-66834332475474132012008-05-24T08:49:00.000-07:002008-05-24T08:49:00.000-07:00I work for an Aerospace company.For all flight cri...I work for an Aerospace company.<BR/>For all flight critical code, we have to unit test(The FAA requires it DO-178B). Unit testing has caught many of errors. We mostly have new software people and sub contractors doing the unit testing. We don't use an Agile approach so our unit tests are tested against our design and compared with our code using a in house simulation test harness. When the more experienced testers are given a chance to code on the hardware they are much better because they have done unit testing.Unknownhttps://www.blogger.com/profile/02240680267767099833noreply@blogger.comtag:blogger.com,1999:blog-9620948.post-67979705296894569342008-05-24T02:12:00.000-07:002008-05-24T02:12:00.000-07:001) I think agitar had a great product, but everyon...1) I think agitar had a great product, but everyone knew that automatically generated unit tests were never going to be any good. Combine that with the cost of the tool and it became a poor proposition.<BR/><BR/>2) All the best developers I've worked with write good unit tests. Yet I think you are right, the grass roots support is pretty low.<BR/><BR/>It also inspired a more detailed entry on my part<BR/><BR/>http://www.jroller.com/BrightCandle/entry/why_evangelising_unit_testing_needsPaul Keeblehttps://www.blogger.com/profile/01672124418792339749noreply@blogger.comtag:blogger.com,1999:blog-9620948.post-22304812996511927242008-05-24T01:32:00.000-07:002008-05-24T01:32:00.000-07:00Unit tests are important, but I believe they are u...Unit tests are important, but I believe they are used too much.<BR/><BR/>The majority of unit testing is testing the contract of a method. The contract should be embedded in the code (e.g. like javadoc) and tested by the infrastructure.<BR/><BR/>Writing unit tests instead of explicitly stating the contract obfuscates what the contract really is, is more brittle and takes up lots of time writing code that the computer could write for you.<BR/><BR/>I don't believe those who say that the unit tests document how to use the code - there are always more unhappy paths than happy paths, so by definition the majority of unit test code shows how NOT to use the code, and has all sorts of other crazy stuff going on too.<BR/><BR/>Of course, there is still room form unit testing code even code that has been Designed By Contract, but unit tests are absolutely the wrong place and way to document and test contract.kybhttps://www.blogger.com/profile/17766996004612885478noreply@blogger.comtag:blogger.com,1999:blog-9620948.post-20297683110189475832008-05-23T21:10:00.000-07:002008-05-23T21:10:00.000-07:00It seems that currently the unit test innovations ...It seems that currently the unit test innovations that drive adoption are those that make tests more accessible to users.<BR/><BR/>Rspec is an obvious case study where an elegant api helps to spur excitement and innovation.<BR/><BR/>I think that there are some interesting technologies that actively encourage programmer productivity.<BR/><BR/>For example, look at the upcoming Javascript testing frameworks. There is a good amount of innovation here.<BR/>Javascript's weak typing also makes testing more valuable that other languages.<BR/><BR/>Some examples are Screw Unit (http://github.com/nkallen/screw-unit), Screw Unit Server (http://github.com/pivotal/screw-unit-server/tree/master), Js Spec (http://code.google.com/p/js-spec/), JS Unit (http://www.jsunit.net/), Js Unit Test (http://github.com/drnic/jsunittest), etc.Brian Takitahttps://www.blogger.com/profile/05145263135844768540noreply@blogger.comtag:blogger.com,1999:blog-9620948.post-19068657725087227902008-05-23T18:35:00.000-07:002008-05-23T18:35:00.000-07:00I'm not convinced that it ever was popular. :-(I l...I'm not convinced that it ever was popular. :-(<BR/><BR/>I liked your post enough to <A HREF="http://pastorgeek.blogspot.com/2008/05/unit-testing.html" REL="nofollow">respond </A>to it over at my blog.Pastor Geekhttps://www.blogger.com/profile/11551535352537367324noreply@blogger.comtag:blogger.com,1999:blog-9620948.post-9628771602147982592008-05-23T13:15:00.000-07:002008-05-23T13:15:00.000-07:00I feel they're waning because once again, it's dis...I feel they're waning because once again, it's discovered that it's not a silver bullet.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-9620948.post-84458581433476862292008-05-23T12:06:00.000-07:002008-05-23T12:06:00.000-07:00Unit test are useful, and maybe xUnit popularity ...Unit test are useful, and maybe xUnit popularity has reached it's plateau, but please, please, please, no more evangelism!! :)Anonymoushttps://www.blogger.com/profile/17525961934302794120noreply@blogger.comtag:blogger.com,1999:blog-9620948.post-8293692792126617442008-05-23T10:28:00.000-07:002008-05-23T10:28:00.000-07:00If these are the best you've got, unit testing sou...If these are the best you've got, unit testing sounds like it's in great shape. As Linus once said, Linux will have made it when it's no longer in the news (i.e., it's assumed). I think unit testing is here to stay *because* nobody talks about it any more -- it's assumed.<BR/><BR/><BR/>1. All kinds of commercial development environments are on the wane. How many companies do you know selling development tools of any sort today?<BR/><BR/>2. How many F/OSS unit testing tools do I need? I've never heard of Jester, but I've heard of those techniques. Jester seems to be for Java, so if you're using one of the bazillion languages which does not start with a "J", it's easier to write one yourself, in-house. Tools that operate on source code aren't hard to write; that's just a myth the Java and IDE people made up. Seibel wrote a unit test framework in a couple dozen lines, in PCL.<BR/><BR/>3. I don't know what makes a Java instructor "major", nor do I know why I would care what they teach. Java is a language; unit tests are a kind of code. Do your "MJIs" teach how to write every possible kind of code? I have a CS degree but none of the code I've written for work has been the same as the code I wrote in school.<BR/><BR/>4. I have only 2 compiler books; does that mean compilers are "on the wane", or simply that I don't need 17 books on the same subject? Is there anything magic in JUnit 4 that requires a book? Why can't I figure it out from the unit testing principles I learned years ago and the manual, like I do for every other topic?<BR/><BR/>5. Things reasonably labelled "alternative" tend to have low visibility. That's why you call them "alternative". See also: there are no "alternative" albums in the top-20, and "alternative" fuel cars are a tiny minority.Anonymousnoreply@blogger.com