Thinking Backwards or Forwards

I've heard references to a new style for documenting requirements. Greg Young talks a little about it in his post on business requirements at CodeBetter.com



I think he is onto something, as it is usually fairly obvious that business customers can handle some pretty complicated procedures and process flows, so why do they struggle with a machine, or state, based approach?  The difference I suspect is that they focus on the action and not the state. The specs he describes are centered around a state, lists an action and only then describe the behavior that results from said action; let's write it thusly:




State ->Action->Behavior.



I'd be willing to bet that the users are really thinking like this, "I want it to do X (Behavior) so I have to push the yellow button (Action) but it only works if the red light is on (State), or



Behavior<-Action<-State





The style of documenting has enough in common that they understand what is being asked for even if, to them, it is listed backwards. As an aside, another major difference is I've never had a business user ever want to have a meta-discussion on how they think, so we tend to have to take it into account when designing the requirement gathering process but that will have to wait for its own post.

Martin Fowler, Alistair Cockburn, and Optimism

The effort to make Software Engineering actually live up to the 'Engineering' part has been greatly helped by Martin Fowler's book on refactoring. Martin recently posted about his decision to not participate in an effort by others in the industry, not lacking in optimism, to define a common process for software development.

Martin Fowler declined to get involved in a similar effort and for the same reason.  The key quote is,

“Why this is so was primarily crystallized for me by Alistair Cockburn who explained that since people are the central element in software development, and people are inherently non-linear and unpredictable - such an effort is fundamentally doomed.”

While my previous post about Alistair's paper covers much of the same ground I felt it was worthy of an update.  There will always be souls who crave consistency just as there will always be salesmen willing to offer the illusion of it.  Facing our own nature is one of life's hardest lessons. Some of those who ignore the human factor are optimists seeking to improve their chances of consistency, which has driven many towards the latest and greatest geegaw, a new language, a new GUI, a new design tool. Others are simply deluding themselves in the hopes that if only people would sit up and fly right (read this to mean 'do it their way') all our problems would be solved. 

Studying the process of computer programming has been around for decades.   Even so, we still have trouble internalizing human foibles - people aren't predictable and follow "standards" about as well as a random-walk.  So the most successful way to get people to be consistent is to lead by example, blaze a trail ahead and show them the way.  In this manner Extreme Programming was its own worst enemy and its own best cheerleader. By showing how a set of practices work in situ, it created a much-needed focus. The "Waterfall" model was the favorite whipping-boy, essentially dead, and the iterative approach was in its infancy.  So XP served as a lightning-rod for discussions on the craft of software development.  It provided an example of practices that could be emulated and provided an easy target for the doubters, disbelievers, and denialists; no doubt the negativity coming from the critics was channeling  their innate understanding of the darkside of human behavior.

While Extreme Programming hasn't become the standard development model, that doesn't mean it failed.  When the history of Software Development is written, XP will be given credit for re-introducing the most important factor; not tools nor process, but people.

Programmings Best Kept Secret

Software development is often compared to engineering or construction which has a hidden and debatable assumption that the source code is the product.  One camp has declared that source code is the design .   Other camps declare that programming as a branch of applied mathematics; see Dijkstra's "uncomfortable truth" quotes. Extreme Programming as a movement may have failed, it at least served as the flashpoint for a new examination of the nature of programming. Instead of arguing about engineering we can brag about how Agile we are or that we are Software Craftsmen.

Programmers are not engineers, nor are they artists.  Even when they are compared to craftsmen,  the comparison is always with woodworkers or blacksmiths. Programmers are nothing of the sort, programmers are writers.  Danny Thorpe discusses how you might tell what sort of literature your app may be.

Danny is on the right track but when it comes to software, there is an elephant in the room that needs to be discussed in the open.  Developers do not produce software, they write code.  This means that sometimes they need the logic skills of an engineer but just as often they need the communication skills of a poet.  Reading and writing, thinking about how to achieve a desired affect through the written word, those are the qualities of a writer.  It just so happens that the one of the chosen audiences is a single-minded overly-literal pendantic polyglot idiot-savant, otherwise known as a computer.  The other audience is other humans that may or may not have to actually follow the instructions, they do have to understand them well enough to tell if the instructions are correct.


It would be useful to compare programming to other fields and see how the comparison holds up.  It is not a coincidence that the games industry has adopted the filmmaking model for the production of games. Good choices would be publishing a book, producing a movie, or creating a recipe.


Programming as publishing


Programmers are the authors.  The compiler is the printing press.  Publishers are the companies that agree to fund the process in hopes of turning a profit.  Self publishing would be like creating training manuals for internal use; no profit is involved but the medium is appropriate for other reasons.  The book still needs to be edited, proofread, typeset, marketed, etc. I've covered this ground before but there is still much to contemplate.


Programming as filmmaking

Filmmaking is in interesting comparison because it deals with translating the written word into another medium. Programmers are the writers of the scripts and screenplays.  The producer is the stakeholder who desires the software. The director is the project lead.  Instead of actors and crew  we have the compiler which takes our instructions and executes whatever script of commands we provide; call it a screenplay or call it an application, the actions and actors my be different but it's still a set of commands given to actors to be performed.



Programming as cooking

Programmers create a recipe for a given dish.  The recipe is not the final product, the result of following the recipe is.  You don't buy a cookbook for the fabulous chocolate cake recipe, you buy it because you want to eat the cake.  In this case the compiler would be the tools available to the chef; pots and pans, utensils, and the stove/oven.  Having the right tools really helps but even the best oven won't correct a recipe that calls for 2 cups of vinegar for the sugar cookies.  Eventually software may survive the patent wars and finally be considered on par with a recipe; instructions to a cook in order to produce the desired product.

Summary

Software is  actually a mix of disciplines, all trying to contribute towards a common goal. The game industry may have it right, evidenced by them being to busy producing games and not endlessing debating whether their process is broken or seeking the latest silver bullet.  They use directors, producers, art and sound specialists, etc.  The compiler is their camera, used to capture the intent of the contributors; the CPU the silver screen for which to project the final product for consumption by the audience.

The Myth of Code Construction

Ever work with a developer who avoided letting anyone see their code, refused to program in pairs or participate in a code review? They say "This is my code" as if they have sole right of refusal on the final product regardless of who else may be paying the bill.

Great writers still have editors. No matter how great a writer you are, you still have to submit your work to an editor before it will be published. Let's examine the idea of putting code into production as a form of publication.

Before we jump into a new analogy, we should look at the current state of affairs. If engineering were a family construction would be the successful older brother leaving software as the immature younger brother, never quite living up to the accolades of the elder sibling yet endlessly compared nonetheless.

In general, the phases in software are: analysis, design, construction, test, deployment, and training. When you want to build a building of any size, let's use a downtown skyscraper as an example. If we continue the comparison we'd start with analysis which includes many factors like the target location or how many square feet are required. The location can impose limitations to the footprint which then dictates how many floors would be needed to achieve the required square footage. Such trade-offs are examined and decided upon during the design phase. Construction is all about execution with limited opportunity for design changes. How do you test a building, what does that even mean? You might might test the subsystems, like elevators, fire alarms, etc. but how do you test the entire building? The closest we can come in scale might be the opening and closing the retractable roof of an almost complete stadium like the new Cowboy's stadium (this being written in May of 2009). How do you deploy a building? You don't, you open it for business. There is no pull of the lever where what did not exist suddenly does. As for iterations, the closest construction can come is building a new wing on an existing building, adding a room to the house at hand. The cost and effort of remodeling can end up being a significant ratio to the original price of the house.

Now let's discuss a comparison of software development with writing a book. A publisher decides the market is ready for a new book on ant farming. They either have recent submissions by authors interested in writing such a book or they solicit existing authors who have experience in the subject matter. In any event, they request an outline of the book with maybe a sample of one chapter to gauge the material matches what the publish wants. The author or authors work at creating the manuscript all the while knowing that editors will read the book and suggest changes. Editors who are not experts in the content but are experts in the publishing of books. Even the highest-paid authors - Mary Higgins-Clark, Stephen King, or J. K. Rowlings to name a few - will all submit their work to the editors pen.

When the manuscript is complete and the editors happy, there are still other contributors to the final product, researchers, artists, designers, and the like who do things like create illustrations, populate an index, check facts, or plan the visual layout of the chapters. All these activities are coordinated by the publisher who has a target date for when the book will be printed, planning a marketing blitz to coincide with the publish date. Getting the manuscript done is important but still is a small part of the whole effort and if an author cannot produce on time, a lot of investment by the publisher goes to waste; a good way to dissuade the publishing company to find an author who can deliver. The key is that it doesn't have to be perfect but does have to be complete. If there is enough demand, it is easier to publish a new edition rather than hold off on the initial printing. Testing is as easy as a small test run where the final product can be validated by all contributors before scaling up into production-sized runs; images replaced, a new cover chosen, typos fixed. Printing becomes like running the build with a target of production. This means that "build" isn't a phase, it's an activity; unlike any in construction. Deployment becomes a straightforward, if not simple matter, of distributing copies of the book to various retail book-sellers or their distributors.

My suggestion is for the software developers of the world realize that allowing our craft to be constantly compared to the construction industry is to remain the little boy, wanting to live up to the expectations of those who came before, never earning the respect or acceptance they so long for. Look instead to the world of publishing, where writing is still an art and yet the businesspeople have found a way to make money without rejecting the fundamental nature of writing, be that a novel, the latest tech book, why not the latest edition of your favorite word processor?

p.s. Kudos to the wife for the idea.