Thinking Backwards or Forwards

I've heard references to a new style for documenting requirements. Greg Young talks a little about it in his post on business requirements at CodeBetter.com



I think he is onto something, as it is usually fairly obvious that business customers can handle some pretty complicated procedures and process flows, so why do they struggle with a machine, or state, based approach?  The difference I suspect is that they focus on the action and not the state. The specs he describes are centered around a state, lists an action and only then describe the behavior that results from said action; let's write it thusly:




State ->Action->Behavior.



I'd be willing to bet that the users are really thinking like this, "I want it to do X (Behavior) so I have to push the yellow button (Action) but it only works if the red light is on (State), or



Behavior<-Action<-State





The style of documenting has enough in common that they understand what is being asked for even if, to them, it is listed backwards. As an aside, another major difference is I've never had a business user ever want to have a meta-discussion on how they think, so we tend to have to take it into account when designing the requirement gathering process but that will have to wait for its own post.

Programming as an Individual Activity

I've decided to come back to Weinberg's classic 'The Psychology of Computer Programming' since it is at the heart of why I took up writing about programming in the first place.  At my first job oh so long ago, it was obvious that we were nowhere close to a solution to programming-in-the-large.  With applications small enough to be written by a single developer we could manage to deliver software that works, maybe it was inefficient or clunky but it worked well enough.  Sure large systems were built using the waterfall approach but they would invariably overrun both their budget and their schedule.  The industry was by no means young, even though the PC revolution was in full swing, the underpinnings had already been discussed ad nauseum. You could dismiss the waterfall method but would still be left with the same questions, 'what do they want?', 'how do we build it?', 'how long will it take?', and so forth.  The Agile or Lean movements have been able to change the discussion to ask 'what is the smallest amount of work that we need to address at one time' and go from there.

I believe that we have been avoiding the most important questions - which are uncomfortable to contemplate - like, "what is the largest single aspect that controls a software schedule?" or "How do we plan a project when the members of the team are not pluggable resources?"

This brings me back to one of the biggest unresolved question of our field, "Why can't we engineer software like we engineer buildings or spacecraft?"  I ask this not for serious comparison since the differences are legion, but rather to point at the single most important advancement in the field of engineering since the advent of the first simple tools (the lever, the wheel, etc), calculus. 

Why calculus you ask? because it gives us a way to plan for infinite variability and still have confidence in the result.  Planning a trip to the moon would be impossible without being able to calculate how much fuel is needed since how much fuel is needed depends upon the weight of the rocket which creates a recursive dependency.

Software will have no equivalent for calculus because the variability comes from the people participating in the project.  Until we accept that people are a first-order component of any software project we will continue to look to silver-bullets, the next development process fad, or the latest new-fangled technology for relief.

XSLT - The Modern Day Snake Oil

Technology seems to attract all kinds, no surprise these days when everybody and their dog are high-tech consumers. But those of us who earn their living in that fast-paced world every day can't help but get a bit jaded seeing the same mistakes repeated time and time again.  From the desperate or possibly Machiavellian managers who want to ignore common sense and decades of history and continually chase the proverbial silver bullet, to the greedy vendors selling the latest 'enterprisey' tool with promises which sound more like a late-nite comedy skit, "Not only is it a floor wax, it's a tasty dessert topping!".  In software it is, "Without writing a single line of code". Why do people fall for these pitches? You can blame ignorance, malice, or greed; in any event, it is up to each of us to keep our wits about us when evaluting claims for the latest-and-greatest doohickey the next salesdroid offers.

There is one phrase that I've seen many times over the last twenty years and whenever I hear it, I immediately think of snake oil.  Whenever you hear a consultant say anything to the effect of, "easy to use by non-programmers!", smile and quickly look for the exit.  How trite is it? Well, early in my college days I remember hearing stories from the early mainframe days when assemblers were first coming out, "It works so well you won't need programmers anymore!"  At my first job at a large company, the 4GL vendors would use the same line on the managers, distracting their mark from the large price tag with promises of savings from lower staffing.  The reality was nobody but programmers could figure out how to actually use the immense packages, assuming they could get them installed correctly in time to use on a project. Even modern tools like Requisite Pro fall, which I really liked, prey to this. They are supposed to improve productivity but the learning curve is so large that the only way to properly use it on a large project is to have someone who already knows it configure it for you.

Recently I ran accross the claim in an old book, "Professional Java XML Programming with Servlets and JSP" by Alexander Nakhimovsky and Tom Myers published in 1999 (page 526 to be exact) which attempts to summarize the ream of paper constituting the previous chapters.  In a section appropriately but belatedly (considering where it appears) titled "What's XSLT for?" the authors explain,

We are rapidly approaching the summary and conclusions to this final chapter of the book In the meantime, the main conclusion about XSLT that we hope you will draw from this chapter is this: XSLT is best described as a general-purpose mini-language, intended to make many common operations on XML documents easy to express and accessible to non-programmers.

Is the book old and out-dated? Of course. The age of the book only matters though if the authors would express a different attitude given a chance to update their book. Granted, right after the offending paragraph they attach some conditions,

Allows non-programmers to do some transformations:
- without tools, only simple things
- with tools, more complex things

When I read the above, I couldn't help thinking, "Have you guys _looked_ at what you've written lately? If it takes a book 500 pages long to explain it, you're never going to get non-programmers to use it"  Why is programming hard, I don't know and do not want to venture a guess.  What I do know is when competent programmers who are at least trained - if not adept - at imperative object-oriented languages face a significant learning curve when working with a declarative language like SQL much less a functional one like Erlang.  XSLT is like a combination of web templates with an embeded declarative language which also include functional aspects, although the authers probably disagree saying, "As of today [1999], JSP is a tool for a prgrammer while XSLT is a tool for a Competent User who is NOT a programmer."

Am I trying to call out XSLT or XML? No, they are useful tools (like automatically formatting your log4j config file) and at this point we couldn't get rid of them if we tried. The issue is quite old and better men than myself have already weighed in.  What I do want is for people who should know better, like said authors, to refrain from propagating the myth that if only we had better tools or languages, non-programmers would rush in to join the computerized fun.
...

Pattern Ignorance

I've been in graduate school for a few years now and keep asking myself how we could engender good habits in students - like any good Codewright - during their education, like usage of version control or refactoring.

My database class provides a good example of the potential and the obstacles. The first project involves creating a schema where the tables have contraints that prevent the tables to be created all at once and those same constraints make it impossible to load the sample data all at once. You don't need the actual schema, I'm just describing a bit of complexity where it influences the assignment.

So the data is provided in text files in a comma seperated form and the approach I decided upon is simply parsing the values and then generate SQL statements in the right order.  First off, I thought about using an ORM framework, like Hibernate, but have no experience with one.  I did find a CSV library to parse the data which cut down the code I had to write but will complicate the grading because the TA will need to download and extract the library in order to run what I submit so I'll have to include those details in a readme file.

So, processing a single file is fairly straightforward; parse the file into a list of lines, loop over the lines, generate the appropriate SQL statement for the data from that file. I soon found myself repeating code for looping over the lines since the files for each table needed slightly different SQL statements.  In rejecting the ORM approach, avoiding the definition of domain objects as simple containers for data -- I also didn't really need domain objects for each file because there was not going to be any behaviour, not to mention the lack of a data framework -- I continued the straight text approach.  Looping over the lines in the first file (Employee), it was easy to directly call a routine to generate SQL rows.  This caused a problem when I wanted to reuse the looping structure for the next file (Departments).  I could leave the loop and add an argument but that would require defining an interface and implementing a distinct class for each file. Avoiding data objects puts me on the slippery slope of avoiding other patterns too; what I keep telling myself is it's only for this one project, it's due in two days, and the TA isn't going to give bonus points for using nifty patterns assuming he's ever been exposed to them.

In short, there is no incentive to produce well-written code.  The assignment is graded by evaluating the results without regard for the methods used to produce those results. 

Testing and the Parable of the Painter

One of the many stories from the real world is having a coworker who doesn't understand the idea of how to test an application.  When asked to write a test program, they think that means writing a second program with the same logic - which is redundant of course - and any other approach makes no sense.

Here is a story to tell these folks.

There were two painters who kept busy painting houses around town. The painters were brothers named Jerry and Dale. They'd come by the house to be painted and the owner would tell them what color to use, the people in town didn't have eclectic tastes which made it easy for the men, the choices were: white, blue, red, or yellow. They had both been painting for many years and made a comfortable living this way.

A retired sea captain moved to town and bought an older house that was in dire need of a paint job.  He hired Dale to paint his house with instructions to paint the house ocean blue. The captain had bent Dale's ear about how the sea was his first love and she showed many different faces and that you could encounter waters of midnight blue during a storm or nearly turquoise near a shallow lagoon on a sunny afternoon.

Dale goes back to his shop, picks up his blue paint, telling himself that blue is blue, and heads off to do the job. When he's done the captain comes around, takes one look at his house and exclaims, "I've never seen the wild ocean look like that! I said ocean blue, not sky blue!".  The captain dismissed Dale and refuses to pay.

Dale returns home and grumbles to his brother Jerry about the whole affair, "If he wanted a specific shade of blue he should have said so before I painted the whole house!"

The next day, Jerry is surprised to find the very same captain at his door asking to have his house painted a, "proper shade of ocean blue."  Jerry, with the hindsight of his brothers experience, takes the captain to his paint shed and quickly mixes several different shades of blue until the captain smiles and says, "That one reminds me of calm seas off the coast on a sunny day, I'll take it.".  Jerry splashes the chosen color onto a thin peice of wood and the two men agree on a price. The captain directs Jerry to get started the next day while he is away at the docks helping a friend repair a fishing boat.

The work goes quickly and when the house is done Jerry goes to fetch the captain to ask him to come see the finished product. On the way to the docks the sky darkens and starts to fill with ominous clouds, a storm is headed this way. Jerry finally reaches the docs and finds the captain in a sour mood, the captain snaps, "It had better be the right color, I'm not going to live in a house that doesn't remind me of the fickle sea."  Upon seeing his house the captain throws a fit, "That is not the color of the ocean! What kind of painter are you? You're not going to get one cent out of me for such a lousy job". He continues his verbal abuse until Jerry pulls out the peice of wood with paint splattered over it and holds it up saying, "You recognize this?  This is the color you agreed to before we started.  When I hold it next to your house, you can clearly see that they are the same color.  You might have changed your mind but I've held up my end of the bargin and expect to be paid fair-and-square.  If you want it painted some other color, I'll be happy to add as many coats as you have money to buy, but you'll have to pay for this one first."   

The moral of the story? Agreeing on the outcome beforehand is the only way to prevent any arguments over what 'done' means at the end.