Saturday, August 19, 2006

Dijkstra's Three Rules for Project Selection

Edsger Dijkstra promulgated these rules in the context of scientific research, but I think they work well for selection of programming projects or, really, most intellectual endeavors that might break new ground.

The Three Golden Rules for Successful Scientific Research.

This note is devoted to three rules, the following of which is necessary if you want to be successful in scientific research. (If you manage to follow them, they will prove close to sufficient, but that is another story.) They are recorded for the benefit of those who would like to be successful in their scientific research, but fail to be so because, being unaware of these rules, they violate them. In order to avoid any misunderstanding I would like to stress, right in its first paragraph, that this note is purely pragmatic: no moral judgments are implied, and it is completely up to you to decide whether you wish to regard trying to be successful in scientific research as a noble goal in life or not. I even leave you the option of not making that decision at all.

The first rule is an "internal" one: it has nothing to do with your relation with others, it concerns you yourself in isolation. It is as follows:

"Raise your quality standards as high as you can live with, avoid wasting your time on routine problems, and always try to work as closely as possible at the boundary of your abilities. Do this, because it is the only way of discovering how that boundary should be moved forward."

This rule tells us that the obviously possible should be shunned as well as the obviously impossible: the first would not be instructive, the second would be hopeless, and both in their own way are barren.

The second rule is an "external" one: it deals with the relation between "the scientific world" and "the real world". It is as follows:

"We all like our work to be socially relevant and scientifically sound. If we can find a topic satisfying both desires, we are lucky; if the two targets are in conflict with each other, let the requirement of scientific soundness prevail."

The reason for this rule is obvious. If you do a piece of "perfect" work in which no one is interested, no harm is done, on the contrary: at least something "perfect"—be it irrelevant—has been added to our culture. If, however, you offer a shaky, would-be solution to an urgent problem, you do indeed harm to the world which, in view of the urgency of the problem, will only be too willing to apply your ineffective remedy. It is no wonder that charlatanry always flourishes in connection with incurable diseases. (Our second rule is traditionally violated by social sciences to such an extent that one can now question if they deserve the name "sciences" at all.)

The third rule is on the scale "internal/external" somewhere in between: it deals with the relation between you and your scientific colleagues. it is as follows:

"Never tackle a problem of which you can be pretty sure that (now or in the near future) it will be tackled by others who are, in relation to that problem, at least as competent and well-equipped as you."

Again the reason is obvious. If others will come up with as good a solution as you could obtain, the world doesn't loose a thing if you leave the problem alone. A corollary of the third rule is that one should never compete with one's colleagues. If you are pretty sure that in a certain area you will do a better job than anyone else, please do it in complete devotion, but when in doubt, abstain. The third rule ensures that your contributions --if any!-- will be unique.

***

I have checked the Three Golden Rules with a number of my colleagues from very different parts of the world, living and working under very different circumstances. They all agreed. And were not shocked either. The rules may strike you as a bit cruel... If so, they should, for the sooner you have discovered that the scientific world is not a soft place but--like most other worlds, for that matter--a fairly ruthless one, the better. My blessings are with you.

4 comments:

Anonymous said...

Where does this come from? Was this published by Dijkstra in a journal, book, or other forum? Just curious about the original material and context.

Thanks.

Andrew Binstock said...

It is part of the Dijkstra archive at the University of Texas. The specific text is at: www.cs.utexas.edu/users/EWD/transcriptions/EWD06xx/EWD637.html
while the archive itself is found at: http://www.cs.utexas.edu/users/EWD/welcome.html

Unknown said...

I would like to take issue with this rule #2, particularly with the all too provocative remark on the Social Sciences. I believe that this statement, exemplarily for much of the debate and resentment between natural scientists and social scientists reflects a couple of misconceptions. First of all, anyone who has given some attention to current empirical research in the Social Sciences will agree that most contributors take the requirements of the scientific method very serious - to the degree that some discussions are, and rightfully so, dominated by methodological considerations.
But aside this apparent misrepresentation, there remain fundamental differences between the natural sciences and the social sciences, the latter appearing - at first sight - as less rigorous in its approach and less generalizable in its findings. I would argue that this difference stems from the subject under study, and not, as is implied in this critique, from inadequacy of the social sciences as an academic discipline. The study of human behavior and cognition, the societies, cultures and institutions they give rise to differs from the subject of the natural sciences in a number of ways. Most easily argued, the realm of the social sciences is characterized by a degree of complexity and opacity unknown to any other discipline. The analysis of any given social phenomena typically reveals and hints at plethora of possible explanatory factors, which are, in turn, often interrelated necessarily rendering a single comprehensive explanation impossible, at least given today's technology and capacity for human abstraction. It is because of this overload that social scientists resort to the vagueness of theory and the limits of partial and contextual explanations - to bring order to the chaos, and to hint at preliminary results where other are unavailable or incomprehensible. A second, related source of problems for social scientists is that of measurement. Leaving aside for the moment the question of whether a purely positivist approach is adequate for the study of humans and desirable for any agenda of change, it must be apparent to everyone who has ever read a paper on measurement in the social sciences, that their subject really is a moving target. In the social sciences, there are few, if any, measurements which could not arguably influence their vary response by imposing its a priori operationalization. In a way, every subject of social scientific investigation is characterized by heisenbergian uncertainty - not just the electrons. Moreover, in contrast to the electrons, for social subjects, there is no formal language readily available to describe them, but the language of their own reality: a social phenomenon can never fully be described mathematically, it's last instance of abstraction always needs to retain meaning, which in turn, varies from subject to subject.
Moreover, as long as we uphold the notion of a free human will and of the reality of human consciousness, the study of the social world is structurally underdetermined. In contrast to the electron, the human has agency and can and will at any time defy whichever probabilistic or deterministic law s/he has previously given rise to.

To my knowledge, the "world formula", unifying the macro and the micro has not been found yet, even in the natural sciences. But even without such perfection, researchers continue their work, improve our understanding, and sometimes, our world.
The same holds true for the social sciences, only that their job is so much harder and, their tools and theories so much less adequate. Social scientific research, for the foreseeable future will continue to be probabilistic, contextual, disintegrated and, occasionally, disconfirmed. It is this very discoursive process which drives progress in the social sciences.
Does all of this mean, the social sciences should abide by rule # 2 and close shop? Should we, in the absence of a unifying theory of the social, refrain from thinking about limited but possible progressions and setbacks? I believe that such a position is profoundly reactionary and anti-humanist, considering the improvements to human lifes that even the smallest increments of social scientific understanding have enabled. It may be so that social scientific understanding is, incidentally, an evolutionary process.

But even if we could, one day, find the unifying and universal social formula - should we seek it? This disenchanted social world, governed by a set of complex but limited and comprehensive deterministic laws, will it still be a habitable place for humans to live in?
I rest my case.

Just A Scientist said...

Further advice on doing research from prominent scientists (the Nobel prize winners and the like) can be found here.