What Makes A Good Program?

This is a continuation of the review of "The Psychology of Programming". You can find the post on Chapter 1 here.

Chapter 2 - What Makes A Good Program?

If we plan to study programming as a human activity, we are going to have to develop some measures of programming performance. That is, we are going to have some idea of what we mean when we say that one programmer is better than another, or one program is better than another. Although we all have opinions on these questions, we shall find that the answers are not as simple as we might wish. For programming is not just human behavior; it is complex human behavior.

What do we mean by complex human behavior? This article at betterexplained.com does a good job highlighting the false dichotomy between simplicity and powerful. Another explanation my good friend Scott sent me used simple, complicated, and complex. I'll let Dave Nicolette speak for himself.

The spectrum goes from easy to understand and predict to extremely hard to understand and predict (but possible with enough effort), to impossible to understand and predict regardless of the level of effort.

In our studies of the psychology of programming, we shall be hampered by our inability to measure the goodness of programs on an absolute scale.

Plenty of effort has been wasted trying to discover that mythical absolute scale; lines-of-code, lines per day, runtime, memory usage, the list goes on and on. The only thing that is agreed it would be that there is no single metric which can be applied to all programs.

Looking honestly at the situation, we are never looking for the best program, seldom looking for a good one, but always looking for one that meets the requirements.

This might be the hidden reason for why so much lip-service is paid to software quality. Crappy code that ready right now always beats higher-quality, however defined, that is late. Unless there is a business reason which drives a requirement connected to quality (like uptime, throughput, etc.) where time-to-market is trumped by proper outcomes, managers will revert to turning a blind-eye to quality. Risk deferred is risk ignored.


Specifications

Of all the requirements that we might place on a program, first and foremost is that it be correct. In other words, it should give the correct outputs for each possible input. This is what we mean when we say that a program "works," and it is often and truly said that "any program that works is better than any program that doesn't."

Here follows a good story about a system for a car maker to handle all the options which prospective car purchasers could order. When the programmer arrives on the scene of a disaster in the making, the long-settled approach used was not only overly complicated but didn't produce the correct results. When the programmer exclaimed that the Emporer had no clothes, he was condemned for being uncooperative. While on the way home, he can't stop thinking about the problem. In a fit of insight, he realizes that a workable approach was within reach. Upon returning to the customer site, he describes his solution. Naturally the audience gave him a cool reception but they listened without questions - until the original create of the old system raised his hand and asked, "And how long does _your_ program take?" to which the response was, "That varies with the input but on average, about ten seconds per card" (an indication of how old this story is). "Aha," was the triumphant reply. "But _my_ program takes only one second per card." This seemed to settle the issue with the audience until our protagonist observes,

"But your program doesn't work. If the program doesn't have to work, I can write one that takes one millisecond per card - and that's faster than our card reader."

Here is where the sarcastic could say "We just redefine what 'done' means", or "that isn't a bug, it's an undocumented feature"

In effect, then, there is a difference between a program written for one user and a piece of "software." When there are multiple users, there are multiple specifications. When there are multiple specifications, there are multiple definitions of when the program is working. In our discussions of programming practices, we are going to have to take into account the difference between programs developed for one user and programs developed for many. They will be evaluated differently, and they should be produced by different methods.

Imagine a scene from "I Love Lucy" where Ricky Ricardo and Fred Mertz are trying to rearrange furniture at the behest of Lucy and Ethel. "Move that painting to the other wall", Lucy commands. When she then turns her attention to having the men move the couch "a teensy bit" closer to the wall, Fred moves his side an inch while Ricky moves his side a foot; after all, how many "teensies" are in a foot? Ethel, meanwhile, sneaks over to the picture while everyone else is busy with the couch and moves the painting back to its original location, "It looked better that anyway", she thinks to herself.

However comical this might be, when this happens in software it can be many, many times worse. With multiple stakeholders, multiple upstream and downstream dependencies, and a large group of developers trying to write code, it should be easy to realize how requirements (and their management) can become a nightmare.

Schedule

One of the recurring problems in programming is meeting schedules, and a program that is late is often worthless.

Most people prefer to wait a fixed ten minutes for the bus each morning than to wait one minute on four days and tewenty-six minutes once a week. Even though the average wait is six minutes in the second case, the derangement caused by one long and unexpected delay more than compensates for this. This is where a psychological study would be rewarding. Project management is not a science and only partially is controlled by mathematics. Knowing the psychological landscape in which the project functions can make a big difference in its success.

Adaptability

Few programmers of any experience would contradict the assertion that most programs are modified in their lifetime. Why, then, when we are forced to modify programs do we find it such a Herculean task that w4e often decide to throw them away and start over? Reading programs gives us some insight, for we rarely find a program that contains any evidence of having been written with an eye to subsequent modification. But this is only a symptom, not a disease. Why, we should ask, do programmers, who know full well that programs will inevitably be modified, write programs with no thought to such modifications? Why, indeed, their programs sometimes look as if they had been devilishly contrived to resist modification - protected like the Pharaohs' tomb against all intruders?

This really goes to the heart of the quality software movement. If it's shown time and again that the majority of software cost is in maintenance, why do we fail so miserably to factor in maintainability while writing software?

Adaptability is not free. Sometimes, to be sure, we get a program that happens to be adaptable as well as satisfactory in all other ways, but we generally pay for what we get - and often fail to get what we pay for.

It's as if the manager figures that maintenance doesn't come out of his budget and she's only being measured by whether the project comes in on time even if the resulting code will cost twice as much to maintain.

Fisher's Fundamental Theorem states - in terms appropriate to the present context - that the better adapted a system is to a particular environment, the less adaptable it is to new environments.

This has large effects on requirements. If one requirement is to have a database that runs on a specific system, a hidden cost is now attached where if that vendor goes out of business, the code can be next to impossible to move to another platform. You satisfied the requirement but paid a huge price. "An ounce of prevention is worth a pound of cure."

However, the same managers who scream for efficiency are the ones who tear their hair when told the cost of modifications. Conversely, the managers who ask for generalized and easily modified programs are wont to complain when they find out how slow and spacious there programs turn out to be. We must be adult about such matters: neither psychology nor magic is going to help us to achieve contradictory goals simultaneously. Asking for efficiency and adaptability in the same program is like asking for a beautiful and modest wife. Although beauty and modesty have been known to occur in the same woman, we'll probably have to settle for one or the other. At least that's better than neither.

Be carefule what you wish for, you just might get it...

Efficiency

Moreover, with the cost per unit of computation decreasing every year and cost per unit of programming increasing, we have long since passed the point where the typical installation spends more money on programming than it does on production work. [like the cost of machine time, ed.] This imbalance is even more striking when all the work improperly classified as "production" is put under the proper heading of "debugging."

Even 30 years ago, the cost of development was the largest cost in the field of programming. If you don't have a manager who understands the challenges and complexities of software development, you are facing a two-front war, 1) the challenge of writing software that works in the time required, using the resources allotted and, 2) a struggle to keep a clueless, however well-meaning, boss from making things worse through ignorance much less incompetence.

Summary

Questions to ponder:
1. Does the program meet specifications? Or, rather, how well does it met specifications?
2. Is it produced on schedule, and what is the variability in the schedule that we can expect from particular approaches?
3. Will it be possible to change the program when conditions change? How much will it cost to make the change?
4. How efficient is the program, and what do we mean by efficiency? Are we trading efficiency in one area for inefficiency in another?


I can here the pointy-haired-boss now, "Why are you asking so many (implied -stupid-) questions? I don't understand why you're making it so hard, just go write it already!"

This is a good place to point out that repeating the same behavior and expecting a different outcome is one of the definitions of insanity.

Next is Chapter 3 - How Can We Study Programming

Review - The Psychology of Computer Programming

I got a "new" book in the mail a few weeks ago and finally had time to crack it open, "The Psychology of Computer Programming" by Gerald M. Weinberg published in 1971. There is a Silver Anniversary edition that recently came out but I wanted a cheap copy in hardback so mine came from an online used book site. Why read a computer programming book that is almost as old as myself? Because it addresses the core issue of producing computer programs, the people who read, write, edit, maintain, or otherwise cuss at (other people's of course), code. People are the primary factor in all programs; before hardware, language, specifications, anything. If you don't understand how humans go about creating these abstract entities called programs, you will constantly be mystified by missed deadlines, frustrated by missing functionality, and stymied by endless scope creep. Enough of my pontificating, let's talk about the book.

Table of Contents

1. Programming as Human Performance




1. Reading Programs


2. What Makes a Good Program?


3. How Can We Study Programming?




2. Programming as a Social Activity




4. The Programming Group


5. The Programming Team


6. The Programming Project




3. Programming as an Individual Activity




7. Variations in the Programming Task


8. Personality Factors


9. Intelligence, or Problem-Solving Ability


10. 1Motivation, Training, and Experience




4. Programming Tools




11. Programming Languages


12. Some Principles for Programming Language Design


13. Other Programming Tools




5. Epilogue

"Computer programming is a human activity." A pretty bold thesis from the intro to Part I. Is there a mystique to programming? "Either you can program or you cannot. Some have it; some don't." Both quotes give you a good idea as to what is in this book, tackled expertly by Mr. Weinberg.

Chapter 1 - Reading Programs

Some years ago, when COBOL was the great white programming hope, one heard much talk of the possibility of executives being able to read programs. Weith the perspective of time, we can see that this claim was merely inteded to attract the funds of exectutives who hoped to free themselves from bondage to their programmers. Nobody can seriously have believed that executives could read programs. Why should they? Even programmers do not read programs.

I hear a similar story from when assemblers were introduced - as in, "With the development of assemblers, we won't need programmers anymore!" I believe similar statements have been propagated by hordes of 4th generation language and CASE salesmen.

"Programming is, among other things, a kind of writing."

This is not a very mainstream view in the programming world even though we work with constructs which are literally "languages".

"We shall need a method of approach for reading programs, for, unlike novels, the best way to read them is not always from beginning to end. They are not even like mysteries, where we can turn to the penultimate page for the best part -- or like sexy books, which we can let fall open to the most creased pages in order to find the warmest passages. No, the good parts of a program are not found in any necessary places -- although we will later see how we can discover crucial sections for such tasks as debugging and optimization. Instead, we might base our reading on a conceptual framework consisting of the origin of each part. In other words, as we look at each peice of code, we ask ourselves the questions, 'Why is this piece here?'"

The author begins examining a section of PL/I code showing how certain machine, language, and human limitations influence a program. Machine limitations like lack of memory to hold the entire problem set in memory at the same time causing the use of two loops instead of one. Language limitations like the lack of an end-of-file indicator which forces the operators to include a special character (or card in the 1970's punch-card centric world) and causes the programmer to account for the special character in code. Programmer limitations like not really understanding array notation of the language, in this instance PL/I. As a program is modified over time, machines change, languages are updated, and programmers come and go.

"And so, some years later, a novice programmer who is given the job of modifying this program will congratulate himself for knowing more about PL/I than the person who originally wrote this program. Since that person is probably his supervisor, an unhealthy attitude may develop -- which, incidentally, is another psychological reality of programming life which we shall have to face eventually."

Some programs have inscrutable logic like the use of special characters which are ordinarily invalid; "magic numbers" used for interim states for some long-forgotten problem.

"Once the patch was made, it worked so well that everyone forgot about it -- more psychology -- and there it sat until unearthed many years later by two archeologist programmers."

To different versions of the same PL/I program are compared, the first includes many limitations of which we've already spoken, the second much improved by removal of the previous limitations. Of the comparison,

"When we look at the difference between Figures 1-1 and 1-4, we might begin to believe that very little of the coding that is done in the world has much to do with the problems we are trying to solve"

Would we be any better off if we could use the latest code (Figure 1-4) as our spec?

"Specifications evolve together with programs and programmers. Writing a program is a process of learning -- both for the programmer and the person who commissions the program. Moreover, this learning takes place in the context of a particular machine, a particular programming language, a particular programmer or programming team in a particular working environment, and a particular set of historical events that determine not just the form of the code but also what the code does!

In a way, the most important reason for stufying the process by which programs are written by people is not to make the programs more efficient, more compact, cheaper, or more easily understood. Instead, the most important gain is the prospect of getting from our programs what we really want -- rather than just whatever we can manage to produce in our fumbling, bumbling way"

Hopefully the reason to study a 30+ year-old book is apparent; those who fail to learn from history are bound to repeat it.

Let's study historical mistakes so we can make our own new mistakes instead of repeating mistakes of programmers long-retired.

The rest of the book is just as rich in lessons we can still take to heart. By approaching the book chapter by chapter, I'm hoping to increase my chances of getting all the way through without writing a 50-page article all at once which would never get finished.

Here is the link for the next chapter: Chapter 2 - What Makes a Good Program

Or Else What?

So, I'm about to do some development on a project that will be using IBM's RAD7. In prepration, the developers are going to a RAD7 class. During class the instructor is dutifully showing us all the bells and whistles and he does this:

if ( somecondition)
{
return success;
}
else
{
return failure;
}

I thought I'd exhert some independence and remove the else block:

if ( somecondition )
{
return success;
}

return failure;

My code is equivalent to the previous example but I began to wonder how do you know when the 'else' clause is unnecessary? In simple cases like this one, it can be easy to show that it doesn't matter. Is this a case where we can make things less clear by being more explicit? Consider the example that uses the 'else' block; if another programmer were to come along and wanted to add more code, how easy would it be for them to decide whether to put the new line in the 'else' block or after it?
Could that question indicate that the 'else' statement is part of a code smell? I'm not sure what the smell would be called, I'm not entirely convinced that it is even a smell. What keeps bugging (ha!) me is how similiar it is, at least as a symptom, to why one of our code standards is to require braces everywhere even if they are optional. I might look this up in Scott Meyer's C++ books.

NON-STANDARD:

if ( chaporones_are_gone) drink;
cleanup;

STANDARD:

if ( chaporones_are_gone)
{
drink;
}
cleanup;

The braces are the safety-net to keep the programmer from inadvertantly making changes to code that isn't affected by the 'if'. The compiler will never be a replacement for a thinking human and since humans are fallible, it makes sense to develop habits that keep us from burning ourselves. How can we define a rule to keep us from overusing the 'else' clause? My initial attempt was something like "only use an else if the else block contains another if" but that doesn't really do it. Maybe when the condition includes something like 'Do x unless a=1' but

Let's add an alternative activity, dancing. If you converted the condition into English syntax, you could say "The teenagers will dance or drink depending on whether the chaperones are gone; in any event they'll have to clean up." This says we clean up always and isn't really affected by the condition. There are two exclusive activities mentioned. We never do both. What I notice is that the sentence has two verbs and one object. I'm having flashbacks to sentence diagramming from junior high, not pleasant.

This is related to the discussion about using negative logic in conditions. Yes, the order of the 'then' and 'else' blocks won't matter to the compiler but one way makes it easier for humans to read. Eliezer Yudkowsky discusses this in a Bayesian logic article using the example of why story problems can be so hard for students to understand.

In all the efforts to get programmers to use more powerful tools, maybe we can spare some time to hone our logic skills. However much we'd like expect our tools to do more work for us, there are some things that will always be left to the individual. Computers won't gather requirements for you. Humans can parse the customers requests and know what to take literally or when to use context to help with the interpretation. A sound foundation in the English language can be a powerful tool in the developers toolkit.

Just Flailing Away

You're back? I'd better be careful. If more people start showing up to these group therapy sessions, they might start charging admission. That would clear out this place and I'd be able to go back to tilting at windmills. Tempting...

I was just thinking (I know I know, *slaps hand*, stop that!) about training someone to be a good developer. The accepted mantra is always, "Send them to training". But even when training is available, it always seems to be too early or too late. Have I ever told you about my friend Jack? I have? I'll act like I didn't hear that. Jack is a developer at a large company. They embrace new technology the way someone lost in a desert for days embraces a cactus; so thirsty it is easy to rationalize, "It won't hurt that much and I'm really thirsty and want a drink right now!" They can quench their thirst but its a prickly proposition. Anyway, Jack was telling me about getting training at work. Ok, he was ranting to beat the band but I don't want you to think he's crazy, that's reserved for me. Jack gets put on this cutting-edge project using new technology on a new platform developed with new tools. So Jack gets sent to training and then doesn't get to use it for months.

[From the back row]:*boo!*, *hiss!*
What are you griping about, you get what you paid for. If you want good comedy, go somewhere that'll demand some lucre. I'm just here to hear myself talk. The jokes are free, just like admission.

So back to Jack. Jack's training showed him all the bells and whistles. You know the drill. How to save a file: "If you want to save a file, hit the [Save] button!" or how to color your text, "This button will turn all your text blue!" More 'how's than a tribe of Native Americans in a marathon of bad spaghetti westerns. I've always preferred my learning to be more about 'why' (Pardon my French) than 'how'. A book or on-line tutorial and show me 'how'. My search for ones that can tell me 'why' (Sorry, about that) sometimes makes me feel like I'm panning for gold and only finding fools-gold. Finding such a rare breed "Look, she can clearly explain why I would want my text blue!" (Excuse me! I must have a rude tic or something.) can feel like finding that one gold nugget that makes it all worthwhile.

Even when Jack came back from training, whenever he'd try to apply what he learned, if it slowed anyone down or required them to change how they worked, he'd be met with "We don't have time to learn how to use that fancy new tool! Just do it the way we've always done it."
Hearing that story, I couldn't help but imagine how that would work in real-life. Imagine your hometown city pool (Hey, we're imagining here! I don't care if your city was too poor to have a pool.) where you went to high school, complete with life-guards, deep end, diving board and lots of squealing kids among other things. You get a job there as a life-guard even though you can't swim. They tell you they'll provide the training. When you show up to work, no training. Your boss tells you not to worry, no one every drowns anyway. You take him at his word until the day the Senior Class President "Johnnie" shows his true colors and throws the class geek's kid brother, who also can't swim, into the pool. Jane, the head cheerleader, screams, "Johnnie! Don't you care that he can't swim?!?" To which Johnnie replies with a shrug and a disaffected scowl, "Nah! That's what life-guards are for." Ignoring his bad grammar, you realize that he means *you*! Jumping in, you flail your arms to keep afloat and manage to make your way over to the little geek where he grabs onto you with a death-grip. You keep from getting water-logged long enough to get back to the edge of the pool where everybody's safe and all is right with the world; until you run into your boss later. "What do you mean you want to go to swimming class?" with a tone that implies that it's your fault people need lifeguards that can swim. "We don't have time for you to learn how to swim." Blithely deflecting your eloquent recounting of the days incident, "It was a fluke, a million-to-one. Go back to work. What are you worried about anyway, everything worked out ok, right?" That night, when you go to bed your dreams are filled with flailing arms and water being flung everywhere and you wake up in a cold sweat. You feel thirsty but don't know why.

Wake up out there! The snoring is so loud someone might come and wonder what the commotion is all about. And no, the concession stand did not bribe me to tell that story.

So, what does that leave? The next time you want training (or want to apply training to real-work), remember Jack and his developer cohorts, struggling to keep their heads above water, their cubes like mini-pools, and they're all just flailing away.