Chapter 3 - How Can We Study Programming?

This is a continuation of the review of Gerald M. Weinberg's book, The Psychology of Computer Programming. The review begins here for Chapter 1 - Reading Programs. The post on Chapter 2 can be found here.

What chance have we to understand the human factor? How do you study programming when the important majority of it takes place between one's ears?

Perhaps programming is too complex a behavior to be studied and must remain largely a mysterious process.

INTROSPECTION

Although some modern students of human behavior tend to discredit it as nonscientific, introspection has always been the first foundation stone of their science.

The author tells a story about a student having difficulty with a particular PL/I programming problem:

ANGLES(I)=2*ATAND(SQRT((S-A(IND(I,1)))*(S-A(IND(I,2))))/(S*(S-A(IND(I,3))))));

When it was finished the author asked the student to recall the thought processes -- the difficulties -- he had experienced with this statement, here is that list.

1. He had trouble getting the parentheses matched, because there were so many of them.
2. The original ANGLES(i) was a structure, not an arry, so the compiler complained of an illegal subscript list, but he had not been able to find the problem because there were so many subscript lists.
3. When the program finally got into execution, there were these further difficulties:
1. IND was a matrix on another page of the listing, and he had trouble finding it.
2. Even though the parentheses were now matched, one pair was in the wrong place and was most difficult to find.
3. The last difficulty turned out to be a problem of precision caused by capricious declaration of different data types, by the large expression, and by the division--which would not have been necessary if the two-argument form of ATAND had been used.

The student provided a number of insights, from the proper size of statements, the choice of data structures, to the design of compiler and execution-time diagnostics.

None of these insights could be obtained without introspection of this sort, yet thousands of programmers [millions now days - ed.] each day have theses same problems and simply grumble that they are having "bugs." Without the introspection, they will simply continue to have "bugs" until, in the end, the bugs will have them.

From this single instance of introspection, we could come up with "Laws of Programming"

1. The mind cannot cope with more than five levels of parentheses.
2. Compiler diagnostics should be more explicit.
3. PL/I precision rules are too complicated to use.

Although each of these statements might turn out to be a "law" in the psychology of programming, writing it on the basis of a single introspective example hardly qualifies it as a law. To be a "law," a principle must be explored so as to set limits to its application, because all laws have limits. Indeed, the limits are Usually more important to know than the laws themselves, and it is only by investigating a multitude of cases that limits can be set. Thus, without introspections, investigation would be sterile, but without investigation, introspection will be of questionable value in application.

OBSERVATION

One way to follow up introspection is by observation of what people actually do, as opposed to what they think they do.

Like the study of parentheses and how many levels of nesting programmers use.

One of the problems with observation, however, is that it tells us what people do do, but not necessarily what they can do.

If your census showed that programmers never nested parentheses more than 5 levels deep, does that mean they are unable to?

A second problem with observation is deciding just what it is we are observing.

All you know about the programmers who limit themselves to 5 levels of parentheses is a simple fact, to know why they don't use 6 or more would of course require introspection on the programmers part. If you were designing a language, would you limit the size of arrays to three dimensions simply because you've never seen anyone use more?

A third problem with observation is the problem of interference of the observer with the subject being observed.

This is called the "Hawthorne Effect". The act of observing workers actually affected their performance, a kind of uncertainty principle.

EXPERIMENT

Here is a very interesting section where the author describes the costs of conducting experiments on programmers using typical experimental psychology methods. How do you collect data without influencing the results. To a degree it is somewhat like the problem of testing the performance of a program. The modifications necessary to observe the programs use of memory or CPU time actually uses memory and CPU time so how can you know your results haven't been skewed?

You could do experiments using programming majors in school but would the results apply to professionals? You could study professionals but that could be prohibitively expensive. The author describes doing an experiment using one-quarter time of nine experienced programmers for three months. The average salary was $14,000 [I'm assuming annually - ed.], add in overhead (workspace, machine time, etc.) came to $20,000 apiece, or $1250 apiece over the course of the study. Add to that the experimenters salaries and so forth and the bill comes to $30,000 for nine subjects. [In 1970 dollars! - ed.] Using the Inflation Calculator ($30,000 in 1970 compared to 2009) this comes out to a cost of $163,254.90. Serious money, just to study programming.

... we must deal with one other problem, which is important because of a seldom-questioned view of programming. That view is that programming is an individual activity, not a social activity.

Not that the individual level is unimportant, but we might start by asking why, if the average programmer spends two-thirds of his time working with other people rather than working alone (yes, it's true!), that 99 percent of the studies have been on individual programmers.

The answer is that it is expensive to study programmers in groups. Plus, you can't put a number of trainees together and call them a 'team'.

Putting a bunch of people to work on the same problem doesn't make them a team--as the sloppy performance in all-star games should teach us.

The neglect of social effects also casts doubt upon all individual studies, since these studies force the individual to separate himself from his normal working environment. In one of our studies, one of the programmers came to me when he had finished coding and asked who could check his work. In his home group, this was standard practice at this point in development, but it was not allowed during the study -- lest it should "invalidate" the results.

Looking back, it seems that forcing programmers to work in isolation is like trying to study the swimming behavior of frogs by cutting off their legs and putting those legs in the water to watch them swim. After all, everyone knows that the frog swims with its legs, so why complicate the experiments with all the rest of the frog?

PSYCHOLOGICAL MEASUREMENT

For the present, most of the work in the psychology of programming is going to have to be "looking for questions."

I conjecture that we will move beyond looking for questions only when we start believing and acting on the questions that we've found over the almost 40 years since this book was published. This is definitely not a programming problem. When the software managers are able to dictate practices based on a whim or the local political winds, "We don't have time to do code reviews! Why can't you just make it work? I don't understand why it's constantly breaking!", you might start looking for answers in the abnormal psychology section of the library.

USING BEHAVIORAL SCIENCE DATA

Because of the nature of programming, there is not much hope that actual results can be transferred directly from other fields to ours. Our use of results has to be for insights, not for answers.

So maybe all those comparisons with engineering and construction are invalid on the face of it. We might look at Paul Graham's book Hackers and Painters for insight.

Finally, as we pass successfully through all this, we may find ourselves in a sufficiently objective frame of mind to examine some of the myths of programming -- the articles of programming faith. Faith, as Bertrand Russell pointed out, is the belief in something for which there is no evidence; and myths, as Ambrose Bierce once defined them, are the sacred beliefs of other people. Perhaps we shall simply carry away our old beliefs unchanged, but if just one unfounded programming myth should die as a result, this book will have been worth the effort.

SUMMARY

Common pitfalls:

1. Using introspection without validation.
2. Use of observation on an overly narrow base.
3. Observing the wrong variables, or failing to observe the right ones.
4. Interfering with the observed phenomenon.
5. Generating too much data and not enough information.
6. Overly constraining experiments.
7. Depending excessively on trainees as subjects.
8. Failing to study group effects and group behaviour.
9. Measuring what is simple to measure.
10. Using unjustified precision.

1. 
   

Several of these items have had some attention paid to them over the last decade. Extreme Programming was one of the first movements focused on groups of developers to get enough traction to garner a following. There are several similar movements, SCRUM, Agile, etc. Instead of assuming the problem was in the process that humans were following, they finally addressed the human factor as a first-order success factor (thanks to Alistair Cockburn)

The question is, when will we learn that we are the problem?


This ends Part 1 - Programming as Human Performance. Next up is Part 2 Chapter 4 - Programming as a Social Activity