Learning Java via TDD: An impressive approach

I have never been a fan of test-driven development. I think the concept is a curiosity that makes you write lots of throw-away code, while keeping lots of code that really doesn't test anything useful. But my most fundamental disagreement with TDD is that it is counter to the way people think. And part of the great joy of programming is building solutions, rather than building problems for the solutions to solve. For this reason, I think that many people who like TDD in the abstract eventually move back to a model of writing the code and immediately writing unit tests to exercise it. (This happens to be my model, but I didn't come to it via TDD. Rather, I adopted it because I became profoundly and deeply convinced of the value of lots of unit tests written immediately after coding.)

I've expressed this view before and would have held on to it unmodified had I not come across a terrific and unique Java tutorial: Agile Java by Jeff Langr. Forget the "agile" attribute. This is a book that teaches Java via TDD. And, it turns out, this is a very interesting way of doing things. How many times have we seen Java tomes start with "Hello World"? This one starts by teaching JUnit. And through JUnit, it teaches TDD, and then Java.

This approach, which is brilliantly original, has very distinct benefits: Firstly, readers are much more in tune with Java's verbose but unrevealing error messages. Rather than looking at a stack dump in disbelief, they're used to debugging and jumping in. Secondly, of course, they're used to testing and to writing code that is testable. The third and for me most important benefit is that this book cannot present Java in the usual way. (Such as: here's an inner class. Here's why you need it. Here's a code snippet. OK, now off to statics....) Rather, each chapter requires writing a mini-app that exercises the topics the author wants to present. So, you get to think about OO and Java, rather than merely learning language syntax.

The book also forces the reader to think about challenging testing issues. And I do mean challenging. It is the only text I've ever read anywhere that shows how to run a unit test on a log record written to the console. How would you solve that problem?

The only drawback is that the same topics are discussed in many different places throughout the book (each discussion amplifying earlier points), so you really can't use it as a reference after the fact. This is a small gripe, as there are plenty of Java references on line and in print.

I recently wrote a review of Java tutorials (before I knew about this book). Despite the many great titles I discuss in that article, if I were teaching a Java class today, this is definitely the book I would use. Bar none. And I don't even like TDD.

One activity that is inherently productive: unit testing

In a post today in his blog, Larry O'Brien states: "...let's be clear that of all the things we know about software development, there are only two things that we know to be inherently highly productive: Well-treated talented programmers and iterative development incorporating client feedback"

I find it hard not to add unit testing to this list.

Of all the things that have changed in how I program during the last six or seven years, nothing comes close to unit testing in terms of making me more productive. In addition, it has made me a better programmer, because as I write code I am thinking about how to test it. And the changes that result are almost always improvements.

Multicores not as productive as you expected?

For a while, I have been intrigued by how small the performance pop is from multicore processor designs. I've written about this and, finally, I think I can begin to quantify it. I'll use AMD's processors for the sole reason that the company has for years posted a performance measure for each of its processors. (Originally, this data was a move to counter Intel's now-abandoned fascination with high clock speeds.)

This chart shows the performance as published by AMD and the corresponding clock speeds for most of its recent processors. I have broken the figures out for the three branches in the Athlon processor family (which is AMD's desktop chip).

There are several interesting aspects to this chart, but the one I want to focus on is the performance of the rightmost entry. The dual-core Athon 64 X2 with a rating of 5200 has a clock speed of 2600MHz. Now, notice the Athlon XP with a rating of 2600 (10th entry from the left): it has a clock speed of 2133 MHz.

In theory, since the AMD ratings are linear, a dual-core processor should give you near but not quite the performance of two single-core chips. So two 2600-rated chips should give you roughly the performance of a 5200-rated dual core chip. Using the chart, we would expect two 2.133GHz cores to give us the 5200 performance figure. In reality, though, it takes two 2.6GHz cores to do this--far more than we would expect. It's actually an even wider gap that that, because the dual-core chips have faster buses and larger caches than the Athlon processors we're comparing it to, so it can make far better use of the processor on each clock cycle.

So, why does it take so much more than twice the clock speed to deliver dual-core performance? The orignial 2600-rated Athlon XP had a memory manager built into the chip. On the X2 chip, however, the two cores do not have dedicated memory managers--instead, they share a single on-chip memory controller. This adds overhead. The cores also share interfaces to the rest of the system and so again must work through resource contention to get the attention they need.

Don't be fooled into thinking this is an AMD-specific issue. (As I said earlier, I used AMD only because they are kind enough to publish clock speed and performance data for their chips.) This is not an AMD-only problem. Intel is in exactly the same boat--what is shared between cores is plenty expensive. Expect, as time passes, to see chip vendors trying to limit these shared resources.

Enter The Komodo Dragon

Komodo 4.0 from Active State is out of beta, and was released today. Komodo has long been viewed as the premier IDE for scripting languages (Python, Perl, and Tcl especially). It continues this tradition with this release and adds Ruby and RoR support. It's the first high-end IDE with intelligent editing and debugging for Ruby and RoR that I'm aware of.

Komodo also provides numerous tools for Ajax development including Javascript debugging, specific editors for XML, HTML, and CSS. Plus an HTTP viewer and a DOM editor.

If you work in any of the languages Komodo supports, you owe it to yourself to examine it (for free). If you work in any two of them, you probably should just buy it. At $245, it's a steal.

C PDF Library

In my current column in SD Times, I discuss the open-source iText library for creating PDF files. iText enables developers to create reports in PDF, HTML, and RTF from any Java application--including servlets. I mentioned in the article that I previously had looked for a PDF library in C and could not find one that was free and open source.

Eagle-eyed reader Reid Thompson kindly sent me a note pointing out the Libharu project. In my quick scan, it's not quite as feature-packed as iText, but it does cover pretty much all the functionality needed for business reports. And it has one advantage over iText that will prove compelling to some developers: Ruby bindings.

Want to be a Jolt Award Finalist? Three Steps

I have been a Jolt judge for all 17 years that the award has existed. This position has been a great privilege because during that time span, the award has become the equivalent of the Academy Awards for software-development tools. One reason for this rise to prominence is that vendors sense that the judges put a lot of work and deliberation into their choices. This somewhat understates the work we do, as I'll explain shortly. This post, however, focuses on a common query from vendors whose products did not advance to the final round: what more could we have done to advance? The answer frequently is: plenty.

To give context to the pointers below, it's important to understand a few things about the Jolt judging process:

• For judges, the Jolt season represents a period of intense activity. I expect that in any given Jolt season I will spend more than 100 hours on product selection. That's a lot of time, and it's all volunteer. We receive no payment for our time.
  
  
• Judge deliberations are secret. We use several mechanisms for sharing our perspectives. Their contents are sealed at the end of the judging cycle. Only discussions relating to procedural matters are retained year to year. So, asking for the judges' rationale for a certain decision will not (or, at least, should not) result in useful information.
  
  
• Judges vote in secret. No one but the person tabulating the results at Doctor Dobb's knows which judge voted for what. Frequently, the results are mystifying to me. I don't understand why product X was left out, while the clearly superior product Y was included. There is only one obvious answer: other people evaluate the product differently than I do.
  
  
• Judges recuse themselves from any category of products in which they can derive financial benefit or where they work for one of the vendors whose product is nominated.

So, how can a vendor influence a product's fate?

1. Have a good product. This more than any other factor will improve your prospects. If your product is nominated year after year, make sure that you have something new to say. We frequently kick out products that are the same as last year's save for a few tweaks. Remember this is an annual award, so greatness must have occurred during the coverage year (generally November to November).
   
   
2. Be able to articulate why your product is better than others. Judges who have never heard of your product need a reason to vote for it. Give it to them. Many vendors set up portals specifically for Jolt judges. They include movie clips of the product (10-15 minutes), screen shots, and generated reports. This is a superb idea. Judges can go to the site and in 20 minutes figure out whether they see any magic there. If you choose to do this, emphasize how your product is different or better than others. Don't try to demo every feature. The judges just want to know why they should vote for you. If you want to create a competitive grid with feature comparisons, this helps too.
   
   
3. Follow up with the judges. Two categories I voted in this weekend (when we voted for finalists) had more than 30 products. After looking at a large number of websites, the products tend to blur. Even though I take notes, when I go back and re-read them, it's hard to remember my exact perspective. If I don't know a product beforehand, it's likely to fall in this blurry region unless it has some incredibly good (or bad) feature. You have a PR agency, right? Put them to work. Have them contact me. Send me a press kit in the mail. Some companies used to send 'swag'--an industry terms for those inexpensive promotional chotchkas vendors give out. Better yet, send out a boxed copy of your software. This helps. In a sea of choices, having a name to remember and with which I can associate specific features is a big plus.
   
   Depending on how I split my votes I can vote for anywhere from 4 to 10 products per category. I almost never vote for as few as four. A problem I have is that once I've voted for the top products, I might have only a few votes left for the remaining 20-30 products. At this point, I need some reason to vote for your product. Just because it's good is not sufficient. Make its good points memorable and you're likely to get one of those last few votes.
   

The coming stage--choosing the winner from the finalists--is completely different. Now judges will want to download software. A common error vendors commit at this point is to make the licensing difficult. The more difficult you make the licensing, the less time I will have to look at your product. This seems obvious, but many vendors have a fear that somehow judges will do something terrible with their software--something like using it after the judging period is over. All the judges know that they can't turn over their license key to their employers. At worst, they will keep using the product for themselves. This is desirable. You want judges of the product to tell other developers, "I use the Acme debugger and it runs circles around product X." Almost invariably, the judges are influencers in major communities. Celebrate their use of your product. Make licensing easy.

I hope this post answers some preliminary questions.

Correction Re Java Tutorials

Grrr. I intensely dislike being wrong in reviews and recommendations. I put lots of effort into my published writings so that readers can rely with confidence on what I recommend. So, it's quite painful--actually, it's a sense of shame and depression coupled with a lot of pain--to realize that I bollixed a review in print.

Such is the case with my quick guide to Java tutorials that appears in the current column in SD Times. In it, I write the following (after discussing the major books in this market):

Finally, for those who want something more serious but don’t require the omnibus tomes, there’s “The Java Tutorial” 4th Edition, by Zakhour et al. (Addison-Wesley Professional). In 600 pages, it presents all of the language proper, with well-chosen code examples, plus the basics of the major API sets. It’s put out by the same team that developed Sun’s outstanding online Java tutorials, which might be the best tutorials ever developed for any language. Get this book to start with, unless one of the others has a particular feature you feel is critical. Either way, you’ll be treated well.

The problem is this book is really not very good and I have to retract my recommendation. It contains some unpardonable errors (its list of Java keywords is incorrect; it refers readers to other books for topics it introduces and never brings to a close--but it only gives you the other book's title, not a chapter or page number, nor even a link to a website that covers the same topic; it makes reference to material that has not yet been presented; and, finally, it makes points that it flags as important, with no explanation as to why.). Don't consider this book, instead go to Volume 1 of Core Java to get the basics of the language in fewer than 800 pages.

My erroneous recommendation comes from an aspect that I failed to consider. The book does have good parts, especially those that refer to the latest items added to Java (enums, for example). I looked at these and was impressed. I then made the error of thinking that the rest of the book was as good as the sections I'd read. The trouble is this book is a gang project and clearly there is considerable fluctuation in the work quality of various authors. I happened to hit the few highlights in my review pass.

Curiously, a few years ago, this problem would not have occurred. Books were primarily written by one or two authors, so quality was consistent throughout--be it good or bad. Any given pair of chapters pretty much reflected the book's overall quality. With the gang books, this is much more rarely the case, as I was badly reminded.

Outsource Spam Filtering to Google

A friend of mine was bending my ear the other day about the amount of spam he has to deal with. Said he, "I have three mail accounts, two of which are widely available on the Web and I am averaging more than 800 pieces of spam a day. My ISV's spam filter is OK, but I still have to check it periodically for any false positives."

He added, "I am thinking of subscribing to one of those spam services like IBM offers where I can route my mail to them. They clear out the spam, and leave the rest in my inbox."

Why pay for what you can get for free? I suggested he create an account at Google Mail, which has excellent spam detection capabilities and has never had a false negative in my experience. Then route your accounts to your Gmail account, and set up your mail agent to poll the one Gmail account. Given the 2.8GB of space Google gives you, even huge amounts of spam are not going to overflow your mailbox.

Sure enough, my friend tried it and he says in an email full of gratitude, he is free from spam.

Additional note: This solution could be done with other 'free mail' services, such as Yahoo, but I recommend doing it with Google, because:

• I find Google's spam filtering is superior;

• because Google resisted subpoenas to reveal user data. This last point makes me feel comfy routing my mail to them;
• and because of Google's superior webmail interface, which facilitates checking mail from the road.

Give it a try.

Porting C/C++ Development from Visual Studio to Eclipse: Does that make any sense?

This link is a long step-by-step tutorial on IBM Developer Works re porting C/C++ projects from Visual Studio .NET to Eclipse. You might wonder, as I did, why anyone who was using Visual Studio .NET would be tempted to port their development to a Java hosted environment to compile .NET code. I believe there is no good reason.

The article actually confirms my view , pointing out on several occasions that the Eclipse platform is indeed the wrong place for Windows development: "Neither Eclipse nor GDB understand the debugging information generated by Microsoft compilers. As a result, it is a challenge to select CDT as a full-time development environment for Windows development. However, you can use Debugging Tools for Windows for debugging side by side with Eclipse as a development environment." Ugh!

Eclipse CDT, as the article also points out, knows nothing about resources.

In other words, you'd be unlikely to ever migrate from VS.NET to Eclipse.

The article misses the opportunity to explain what Windows coding you would use the CDT for: porting code from other platforms to Windows when you don't have access to Visual Studio. That makes sense.

The new features of Java 7

With Java 6 set to be released, here is a peek into Java 7. It's a PDF of a slide deck by Danny Coward, the Java SE platform lead.

Among the interesting features are: BigDecimal overloads of arithmetic operands, strings in case/switch statements, better XML and Xpath support, and the invokedynamic instruction, which facilitates the use of dynamic languages on the JVM. (Note: in my September 1 post, I mistakenly described this instruction as a feature of Java 6. It is actually in Java 7, as a commentator thoughtfully pointed out in response to that post, and revisited by this presentation.)

How much of a nerd are you, really?

Some of my fans might be distressed to see that I'm a mere 59 on a scale of 100. Jeez, I barely missed a passing grade. So, what's your rating?

Convert file formats on-line

This site makes it simple to convert file formats if you don't have the converter software at your easy reach--provided you're willing to send your file to a service.

Live Data Visualization at its Best

This live visualization enables you to see the daily flights of FedEx planes throughout the US. Very cool and, I believe, increasingly the way data will be presented the future now that broadband is so widely available in the first world.

Mozilla Foundation Finds its Mojo

Holy Toledo! The Mozilla guys are on fire! First, they released a good upgrade to the Firefox browser (despite being unable to recognize it as an upgrade from within the browser itself). Then, in the course of two days, Mozilla becomes the recipient of two major product codebases: Qualcomm's beloved Eudora mail agent and Adobe's ECMAscript execution engine. This latter project, called Tamarin, consists of the virtual machine that runs Flash Player 9.

Cool stuff! Mozilla used to be the boneyard for failed Netscape projects, but now it is suddenly at the heart of the open-source world--a pinnacle until recently occupied by the Apache Software Foundation.

As for my beloved Eudora, I need to see what happens with the project. I have been a paying customer for years. If Mozilla keeps it going, I'll stick with it. If not, I'll probably switch to Thunderbird.

Isn't Firefox 2 an Upgrade?

So, isn't Firefox 2.0 an upgrade to 1.x releases? Apparently not, according to this notice when I tried to get Firefox to download the new version.

Ralph Griswold dies

Ralph Griswold died earlier this month of cancer. Griswold was an important, but unsung , pioneer of programming language design. He was the inventor of SNOBOL and Icon (now Unicon), two languages distinguished by unsurpassed string processing capabilities. And I do mean unsurpassed. No language since has had such extensive native support for string operations.

The other outstanding characteristics of SNOBOL and Icon were that they were very high-level languages. This color picker was written in 31 lines of easily readable code. It might be a simple tool, but I doubt many languages could come close to that level of brevity for this same functionality (much less maintain such readability). Languages today could use a bit more of Griswold's vision, I believe.

Useful links:

Icon (which is still actively supported.)

Unicon (Icon with objects, advanced network and file functions, plus ODBC)

Griswold obituary

Solving the problem of embedded comments

One problem language designers face is handling embedded comments. The basic problem is that if you comment out a large block, your intent might be thwarted by comments within the block. For example in C, the /* symbol marks the beginning of a block and */ the end. To comment out a block of code it's not sufficient to place these markers at the beginning or end of the block, because if a */ occurs inside the block, it will close the comment. So, most languages forbid the use of embedded comments. (As a result, various work-arounds are used. In C, for example, the #if 0 / #endif combo is used.)

Lua, which is a nifty dynamic language, is the first language I've seen to come up with a solution to this problem. Here goes:

In Lua, block comments start with: --[[ and end with --]] If you want to comment out a chunk that might contain block comments, you can add one or more = signs between the brackets: Open with --[=[ and close with ]=]. You can use any number of = signs between the brackets and Lua will comment out everything until it finds a pair of closing brackets with a matching number of equal signs. So, --[==[ and ]==] or --[===[ and ]===] are valid comment block markers. You can use as many equal signs as you want, as long as beginning and ending counts match. So, you can always block out your code regardless of how many embedded comments it might contain.

I think that's pretty clever. Anyone know of another language with a similar mechanism?

Interesting Uses Of Virtualization

Last week, I presented two sessions at InfoWorld's Virtualization Executive Forum 2006. One was a panel on best practices for virtual test-lab automation, with representatives from VMware/Akimbi and Surgient.

The other was a solo presentation on interesting uses of virtualization--that is, compelling use cases outside of server consolidation. While hardware consolidation is still the leading driver for adoption of virtualization by IT, it will be overtaken by a slew of very interesing applications that are as yet little known. To see what I mean, have a look at the slides from my presentation. I have kept the deck short and to the point.

Amdahl's Law Revised for Hyper-Threading

In the book that Rich Gerber and I wrote on Intel's Hyper-Threading Technology, we present a revised version of Amdahl's Law that reflects the constraints of Hyper-Threading Technology. There was an error in that equation that was recently reported by Gang Chen of Shanghai Jiao Tong University. The corrected formula is:

Speedup = 1 / (S + (1-S)/(0.67n) + Hn)

where S = sequential time, n = number of logical processors, H = overhead.

Clay Breshears of Intel was kind enough to double-check the revision. Ignoring overhead, if a program is 99% parallelized on a Hyper-Threading chip with two logical processors, the speedup is:

Speedup = 1 / (.01 + .99 / (.67*2)) = 1 / 0.749 = 1.34

Intel has long suggested that a 30% overall speedup was the most improvement that could realistically be achieved. The 34% improvement shown here is more theoretical than real because no overhead is included and the code is 99% parallelized. So, Intel's projections are consistent with this formula.

Thanks to Clay and to Gang Chen.

A Really Useful Ruby Book

In my latest column in SD Times, I try to point out that not everything is super-cool about Ruby. One of the things I point to is the lack of good books. Sure the pick-axe book is a good place to start, but you quickly encounter the need for a practical reference that illustrates the idioms for doing standard things.

O'Reilly and Associates has just released such a book, Ruby Cookbook, (ISBN 0-596-52369-6), which contains more than 800 pages of helpful routines accompanied by thoughtful explanations. It shows how things are normally coded in Ruby. It's an impressive work especially because of its tremendous sweep: whether you need string handling, file I/O, database programming, network programming, multitasking, or accessing BitTorrent, it's all covered in this book. At a $33 street price, the book is a steal, and most Ruby developers are likely to keep a copy beside their workstation.

Other Ruby books are on the way. Most specialize on Rails, but some focus on the language as such. (Surely, there will be the bumper crop of "Ruby in 7 Days" kind of titles and other popularizing volumes, as well). So, my complaint about the lack of good books might be resolved by year end. We'll see. Meanwhile, this volume is a huge step forward.