Thursday, March 22, 2007

Characterization Tests

In my current column in SD Times, I discuss characterization tests, which are an as-yet little discussed form of testing. They are unit tests whose purpose is not to validate the correctness of code but to capture in tests the behavior of existing code. The idea, first popularized in Michael Feathers' excellent volume Working Effectively with Legacy Code, is that the tests can reveal the scope of any changes you make to a codebase.

You write comprehensive tests of the codebase before you touch a line. Then, you make your changes to the code, and rerun the tests. The failing tests reveal the dependencies on the code you modified. You clean up the failing tests and now they reflect the modified code base. Then, rinse, lather, repeat.

The problem historically with this approach is that writing the tests is a colossal pain, especially on large code bases. Moreover, large changes in the code break so many tests that no one wants to go back and redo 150 tests, say, just to capture the scope of changes. And so frequently, the good idea falls by the way side--with unfortunate effects when it comes time for functionally testing the revised code.To the rescue comes Agitar with a free service called JunitFactory. Get a free license and send them your Java code they will send you back the characterization tests for it. Cool idea. Change your code, run the tests, verify that nothing unexpected broke. And then have JUnitFactory re-create a complete set of characterization tests. How cool is that?

Actually, I use Agitar's service not only for characterization but for my program as I am developing it. The tests always show me unit tests I never thought of writing. Try it out. You'll like it.

2 comments:

Steven Feuerstein said...

Andrew,

I have been greatly enjoying your posts on testing. I have become somewhat obsessed with code testing in the Oracle environment over the last five years or so. Back at the turn of the millenium, I built utPLSQL, the Junit equivalent for PL/SQL. More recently, I have been deeply involved in a new tool from Quest Software called Quest Code Tester for Oracle. Like Agitar, Code Tester generates automatically many lines of test code, rather than requiring developers to write the code themselves (a losing proposition if I've ever experienced one).

I encourage you to check it out at http://www.quest.com/code-tester-for-oracle/.

I was intrigued by Agitar's comments regarding lines of code. I had thought that it was commonly accepted in the world of testing that if you have a program of, say, 10 lines, you should expect to have to write 100 or more lines of code. That certainly matches my experience in both utPLSQL and Code Tester.

Of course, when you are generating test code, this ratio becomes almost irrelevant; the question becomes how many test cases do you have defined for how many units? I assume that Agitar, whose products automatically generate many, many boundary condition tests, would have the same basic viewpoint.

I am not sure if Agitar engineers were simply trying to put a positive spin on whatever test code construction people do so as not to discourage (as in "If we can only get them to write 1000 lines of test code for 1000 lines of program code, we (the software world) will be doing great!), or whether they have a different understanding from me and from what I have read in unit testing resources.

Regardless, the more software professionals talk about testing, seek out automated testing tools, and actually start testing their code more thoroughly, the better off we all (by this I mean humanity as a whole) will be!

Warm regards,
Steven Feuerstein
PL/SQL Evangelist, Quest Software
www.quest.com
http://www.quest.com/code-tester-for-oracle/

Mantasha said...

Great site buddy . Really I like ur site I think u r also a Java Developer like me any ways I have some reference books of java which are readable .... Following are the books

Programmer's Guide to Java
Certification A Comp. Primer SE
By Khalid A. Mughal, Rolf W. Rasmussen Publisher : Addison Wesley
Pub Date : August 04, 2003

Java™ Development on PDAs
Building Applications PocketPC
By Daryl Wilding-McBride Publisher : Addison Wesley
Pub Date : June 05, 2003


Learning Java™, 2nd Edition
Publisher : O'Reilly Pub Date : July 2002
Pub Date : June 05, 2003

Jython for Java Programmers
By Robert W. Bill Publisher : New Riders Publishing
Pub Date : December 21, 2001

Enterprise JavaBeans, 3rd Edition
By Richard Monson-Haefel Publisher : O'Reilly
Pub Date : September 2001

Java 1.5 Tiger: A Developer's Notebook
By David Flanagan, Brett McLaughlin Publisher : O'Reilly
Pub Date : June 2004


Java Tutorials Index 1
Java Data Objects
By David Jordan, Craig Russell Publisher : O'Reilly
Pub Date : April 2003

Java™ Extreme Programming Cookbook
By Eric M. Burke, Brian M. Coyner Publisher : O'Reilly
Pub Date : March 2003


Java™ Performance Tuning, 2nd Edition
By Jack Shirazi Publisher : O'Reilly
Pub Date : January 2003

Java™ Performance Tuning, 2nd Edition
By Jack Shirazi Publisher : O'Reilly
Pub Date : January 2003

JavaScript & DHTML Cookbook
By Danny Goodman Publisher : O'Reilly
Pub Date : April 2003

Java Servlet & JSP Cookbook
By Bruce W. Perry Publisher : O'Reilly
Pub Date : January 2004
Java Tutorials Index 2