Thursday, March 22, 2007

Characterization Tests

In my current column in SD Times, I discuss characterization tests, which are an as-yet little discussed form of testing. They are unit tests whose purpose is not to validate the correctness of code but to capture in tests the behavior of existing code. The idea, first popularized in Michael Feathers' excellent volume Working Effectively with Legacy Code, is that the tests can reveal the scope of any changes you make to a codebase.

You write comprehensive tests of the codebase before you touch a line. Then, you make your changes to the code, and rerun the tests. The failing tests reveal the dependencies on the code you modified. You clean up the failing tests and now they reflect the modified code base. Then, rinse, lather, repeat.

The problem historically with this approach is that writing the tests is a colossal pain, especially on large code bases. Moreover, large changes in the code break so many tests that no one wants to go back and redo 150 tests, say, just to capture the scope of changes. And so frequently, the good idea falls by the way side--with unfortunate effects when it comes time for functionally testing the revised code.To the rescue comes Agitar with a free service called JunitFactory. Get a free license and send them your Java code they will send you back the characterization tests for it. Cool idea. Change your code, run the tests, verify that nothing unexpected broke. And then have JUnitFactory re-create a complete set of characterization tests. How cool is that?

Actually, I use Agitar's service not only for characterization but for my program as I am developing it. The tests always show me unit tests I never thought of writing. Try it out. You'll like it.

2 comments:

  1. Andrew,

    I have been greatly enjoying your posts on testing. I have become somewhat obsessed with code testing in the Oracle environment over the last five years or so. Back at the turn of the millenium, I built utPLSQL, the Junit equivalent for PL/SQL. More recently, I have been deeply involved in a new tool from Quest Software called Quest Code Tester for Oracle. Like Agitar, Code Tester generates automatically many lines of test code, rather than requiring developers to write the code themselves (a losing proposition if I've ever experienced one).

    I encourage you to check it out at http://www.quest.com/code-tester-for-oracle/.

    I was intrigued by Agitar's comments regarding lines of code. I had thought that it was commonly accepted in the world of testing that if you have a program of, say, 10 lines, you should expect to have to write 100 or more lines of code. That certainly matches my experience in both utPLSQL and Code Tester.

    Of course, when you are generating test code, this ratio becomes almost irrelevant; the question becomes how many test cases do you have defined for how many units? I assume that Agitar, whose products automatically generate many, many boundary condition tests, would have the same basic viewpoint.

    I am not sure if Agitar engineers were simply trying to put a positive spin on whatever test code construction people do so as not to discourage (as in "If we can only get them to write 1000 lines of test code for 1000 lines of program code, we (the software world) will be doing great!), or whether they have a different understanding from me and from what I have read in unit testing resources.

    Regardless, the more software professionals talk about testing, seek out automated testing tools, and actually start testing their code more thoroughly, the better off we all (by this I mean humanity as a whole) will be!

    Warm regards,
    Steven Feuerstein
    PL/SQL Evangelist, Quest Software
    www.quest.com
    http://www.quest.com/code-tester-for-oracle/

    ReplyDelete
  2. Great site buddy . Really I like ur site I think u r also a Java Developer like me any ways I have some reference books of java which are readable .... Following are the books

    Programmer's Guide to Java
    Certification A Comp. Primer SE
    By Khalid A. Mughal, Rolf W. Rasmussen Publisher : Addison Wesley
    Pub Date : August 04, 2003

    Java™ Development on PDAs
    Building Applications PocketPC
    By Daryl Wilding-McBride Publisher : Addison Wesley
    Pub Date : June 05, 2003


    Learning Java™, 2nd Edition
    Publisher : O'Reilly Pub Date : July 2002
    Pub Date : June 05, 2003

    Jython for Java Programmers
    By Robert W. Bill Publisher : New Riders Publishing
    Pub Date : December 21, 2001

    Enterprise JavaBeans, 3rd Edition
    By Richard Monson-Haefel Publisher : O'Reilly
    Pub Date : September 2001

    Java 1.5 Tiger: A Developer's Notebook
    By David Flanagan, Brett McLaughlin Publisher : O'Reilly
    Pub Date : June 2004


    Java Tutorials Index 1
    Java Data Objects
    By David Jordan, Craig Russell Publisher : O'Reilly
    Pub Date : April 2003

    Java™ Extreme Programming Cookbook
    By Eric M. Burke, Brian M. Coyner Publisher : O'Reilly
    Pub Date : March 2003


    Java™ Performance Tuning, 2nd Edition
    By Jack Shirazi Publisher : O'Reilly
    Pub Date : January 2003

    Java™ Performance Tuning, 2nd Edition
    By Jack Shirazi Publisher : O'Reilly
    Pub Date : January 2003

    JavaScript & DHTML Cookbook
    By Danny Goodman Publisher : O'Reilly
    Pub Date : April 2003

    Java Servlet & JSP Cookbook
    By Bruce W. Perry Publisher : O'Reilly
    Pub Date : January 2004
    Java Tutorials Index 2

    ReplyDelete