XSLT - The Modern Day Snake Oil

Technology seems to attract all kinds, no surprise these days when everybody and their dog are high-tech consumers. But those of us who earn their living in that fast-paced world every day can't help but get a bit jaded seeing the same mistakes repeated time and time again.  From the desperate or possibly Machiavellian managers who want to ignore common sense and decades of history and continually chase the proverbial silver bullet, to the greedy vendors selling the latest 'enterprisey' tool with promises which sound more like a late-nite comedy skit, "Not only is it a floor wax, it's a tasty dessert topping!".  In software it is, "Without writing a single line of code". Why do people fall for these pitches? You can blame ignorance, malice, or greed; in any event, it is up to each of us to keep our wits about us when evaluting claims for the latest-and-greatest doohickey the next salesdroid offers.

There is one phrase that I've seen many times over the last twenty years and whenever I hear it, I immediately think of snake oil.  Whenever you hear a consultant say anything to the effect of, "easy to use by non-programmers!", smile and quickly look for the exit.  How trite is it? Well, early in my college days I remember hearing stories from the early mainframe days when assemblers were first coming out, "It works so well you won't need programmers anymore!"  At my first job at a large company, the 4GL vendors would use the same line on the managers, distracting their mark from the large price tag with promises of savings from lower staffing.  The reality was nobody but programmers could figure out how to actually use the immense packages, assuming they could get them installed correctly in time to use on a project. Even modern tools like Requisite Pro fall, which I really liked, prey to this. They are supposed to improve productivity but the learning curve is so large that the only way to properly use it on a large project is to have someone who already knows it configure it for you.

Recently I ran accross the claim in an old book, "Professional Java XML Programming with Servlets and JSP" by Alexander Nakhimovsky and Tom Myers published in 1999 (page 526 to be exact) which attempts to summarize the ream of paper constituting the previous chapters.  In a section appropriately but belatedly (considering where it appears) titled "What's XSLT for?" the authors explain,

We are rapidly approaching the summary and conclusions to this final chapter of the book In the meantime, the main conclusion about XSLT that we hope you will draw from this chapter is this: XSLT is best described as a general-purpose mini-language, intended to make many common operations on XML documents easy to express and accessible to non-programmers.

Is the book old and out-dated? Of course. The age of the book only matters though if the authors would express a different attitude given a chance to update their book. Granted, right after the offending paragraph they attach some conditions,

Allows non-programmers to do some transformations:
- without tools, only simple things
- with tools, more complex things

When I read the above, I couldn't help thinking, "Have you guys _looked_ at what you've written lately? If it takes a book 500 pages long to explain it, you're never going to get non-programmers to use it"  Why is programming hard, I don't know and do not want to venture a guess.  What I do know is when competent programmers who are at least trained - if not adept - at imperative object-oriented languages face a significant learning curve when working with a declarative language like SQL much less a functional one like Erlang.  XSLT is like a combination of web templates with an embeded declarative language which also include functional aspects, although the authers probably disagree saying, "As of today [1999], JSP is a tool for a prgrammer while XSLT is a tool for a Competent User who is NOT a programmer."

Am I trying to call out XSLT or XML? No, they are useful tools (like automatically formatting your log4j config file) and at this point we couldn't get rid of them if we tried. The issue is quite old and better men than myself have already weighed in.  What I do want is for people who should know better, like said authors, to refrain from propagating the myth that if only we had better tools or languages, non-programmers would rush in to join the computerized fun.
...

Pattern Ignorance

I've been in graduate school for a few years now and keep asking myself how we could engender good habits in students - like any good Codewright - during their education, like usage of version control or refactoring.

My database class provides a good example of the potential and the obstacles. The first project involves creating a schema where the tables have contraints that prevent the tables to be created all at once and those same constraints make it impossible to load the sample data all at once. You don't need the actual schema, I'm just describing a bit of complexity where it influences the assignment.

So the data is provided in text files in a comma seperated form and the approach I decided upon is simply parsing the values and then generate SQL statements in the right order.  First off, I thought about using an ORM framework, like Hibernate, but have no experience with one.  I did find a CSV library to parse the data which cut down the code I had to write but will complicate the grading because the TA will need to download and extract the library in order to run what I submit so I'll have to include those details in a readme file.

So, processing a single file is fairly straightforward; parse the file into a list of lines, loop over the lines, generate the appropriate SQL statement for the data from that file. I soon found myself repeating code for looping over the lines since the files for each table needed slightly different SQL statements.  In rejecting the ORM approach, avoiding the definition of domain objects as simple containers for data -- I also didn't really need domain objects for each file because there was not going to be any behaviour, not to mention the lack of a data framework -- I continued the straight text approach.  Looping over the lines in the first file (Employee), it was easy to directly call a routine to generate SQL rows.  This caused a problem when I wanted to reuse the looping structure for the next file (Departments).  I could leave the loop and add an argument but that would require defining an interface and implementing a distinct class for each file. Avoiding data objects puts me on the slippery slope of avoiding other patterns too; what I keep telling myself is it's only for this one project, it's due in two days, and the TA isn't going to give bonus points for using nifty patterns assuming he's ever been exposed to them.

In short, there is no incentive to produce well-written code.  The assignment is graded by evaluating the results without regard for the methods used to produce those results. 

Computer Science or Software Engineering - An Elemental Problem

What is the difference between Computer Science and Software Engineering?  That depends on the nature of the problem you are trying to solve -  is your problem one of Silicon or one of Carbon?

In my 20-year career in working with software, the single biggest issue has also been an elephant in the room: the abysmal success rate of large software systems.  Plenty of books have been written on the subject with lots of great, if unheeded, advice.  Scope creep, ambiguous or missing requirements, poor communications, bad risk management; lots of reasons have been documented and just as many if not more solutions have been suggested.  Brooks gave us "The Mythical Man-Month", Yourdon writes of "The Death March", Tom DeMarco has "Peopleware". All of them attempting to answer the question, "Why is it so hard to write software?"

If you take the all the books on writing software and put their suggestions in one big list, the first question would be, "If we have all the answers, why don't we use them?"  That might be nice for getting it off your chest but isn't has helpful in pointing us in the right direction. At its core, this question does have a useful kernel, it assumes that 'we' are the problem.  If only we used this process, if only we implemented that practice, if only we hired the right people, if only we kept our teams motivated.  As an aside, most of the reasons for failure are management reasons, not technical ones.  It should be no surprise that many find the case study on the sinking of the Vasa in 1628 to be very instructive. The blame is as much, if not more so, on the King's unrealistic demands as on the builder's efforts to failure to meet the royal requirements. In our endless efforts to build bigger, more complex and complicated software solutions in the face of almost certain failure, should question our sanity; one definition of insanity being, "doing the same thing and expecting different results." Biting off more than we can chew, choking, and then doing it all again time after time only shows how our hubris can get the better of us, allowing us to continue on our Sisyphean task.

This brings me to an interesting observation.  There is a long running debate over whether computer programming is/should be considered a form of artistic expression or of engineering prowess.  Computers have such a wide scope that many sub-fields have crept up, hardware and software being the most obvious. Hardware is typically the realm of the engineer, designing circuit boards or embedded sensors.  This changes as the hardware gets larger in size or scope.  CPUs have microcode, missiles have on-board navigation systems with programmable targets.

Most universities that offer degrees related to programming computers name the department Computer Science.  Now, being considered a 'science' may be just a way of expressing the notion that computers are deterministic and worthy of comparison with mathematics or engineering, having likely grown as an offshoot of the field of electrical engineering.  Whether it be data mining, security, networking, or pure algorithms, most of the areas seem to ignore the question of why software is so hard. They assume the problem is in the silicon computer chips.  Curiously, these fields don't even attempt to address this most difficult of questions, assuming improvements of our silicon constructs or their usage will somehow mitigate our human foibles. If only we had better tools, or faster computers, etc. then our carbon-based brains could somehow ascend above the project management miasma and become a beacon to those who struggle and toil without end. Considering the gargantuan number of man-hours and humongous amounts of money wasted, such an accomplishment would be of heroic proportions, mythic even; a modern-day retelling of Prometheus or Sisyphus.

The truth is that Computer Science may bring improvements in areas where the problem is in the silicon, it is mute when asked to provide answers to problems of a carbon nature.  Software Engineering attempts to develop solutions that include how humans work together.  Software Engineering does not assume that software development is as cut-and-dried as putting the right circuits on a board to achieve the desire product, or breaking down a complex problem into constituent algorithms to make the problem more manageable. There is no magic bullet, no single development process, no single answer.  To understand why humans have a hard time developing software, we must admit that humans are part of the equation. Doing is brings with it a cold truth about humans - they are non-deterministic by nature.  Alistair Cockburn describes it as people being non-linear, first order components in software development.   To answer the question of what is software development, we might have to take detours through Psychology, and may be surprised to end up working with Philosophy even.

Student, Teach Thyself - A Theoretical Computer Science QnA site

I've been a fan of Stackoverflow for a while.  Jeff Atwood and Joel Spolsky have helped the Internet turn a corner when it comes to helping people find answers to their questions.  Google may do an incredible job indexing pages but it would be to no avail if the pages themselves were all junk.

I don't have a ton of time to be on the new site-proposal system called area51.stachexchange.com. During one of my CS grad school classes, I asked a question regarding my schoolwork on the main Stackoverflow site and was underwhelmed by the responses.  Since the main site is more for working programmers who typically expect a code snippet as an answer, I figured the proposal for a site on theoretical computer science deserved a shout out.

Now if they would only restart their podcast.

What I learned about Cryptography

In preparing a presentation on cryptography for a graduate class, I was surprised by what I learned.

Cryptography has been used from ancient times through the Industrial Age to the modern day. It's been used for military, diplomatic, criminal, and commercial purposes.

In the simplest and oldest form, Greeks used strips of paper wrapped around sticks of a certain size called a scytale - rhymes with Italy.

Julius Caesar is one of the first known users of the simple letter substitution method. The modern day cousin is ROT13.

If he had anything confidential to say, he wrote it in cipher, that is, by so changing the order of the letters of the alphabet, that not a word could be made out. If anyone wishes to decipher these, and get at their meaning, he must substitute the fourth letter of the alphabet, namely D, for A, and so with the others.
—Suetonius, Life of Julius Caesar

Hiding messages (steganography) is as old as the ancients. An example would be a tattoo on a shaved head. The hair would grow back hiding the tattoo. A modern example is hiding text in the pixels of a digital image.

Charles Babbage is a name taught in every into to computer science class as a pioneer in computing machines. I never knew that he helped the English during the Crimean War by breaking cryptological codes.

Mary "Queen of Scots" was caught when one of her secret codes were broken.

The Voynich Manuscript may have been an elaborate hoax to defraud a king, who had a interest in mysteries like coded messages, out of a large amount of gold.

A mob boss used encrypted messages to give out orders to his lieutenants and was arrested when the police cracked his amateur code.

Cryptography is thousands of years old and is used all around us everyday.

In Search of Wisdom

The other day I was reading StackOverflow, a forum for developers to ask and answer technical questions, and ran across a post from a discouraged student. The responses had one thing in common: programmers know that they must keep learning. Tomorrow's hot new technology is just as soon to be yesterday's lost and forgotten relic. The best programmers are not the experts in a given niche but the ones who are comfortable with learning as a way of life. They know that they do not know and accept it. This makes them resistant to dictated dogma; a good example being Joel Spolsky's "Emperor's New Clothes" treatment of C-style strings. They are then freed from the ups and downs of the technological tide and are not battered by the constant waves of the newest, latest fad, nor do they feel deserted when some favored tech is washed away into the sea or wrecked upon the shoals.

Socrates is said to have walked the streets of Athens looking for a wise man. Wisdom is usually associated with knowledge but since it is impossible to know everything, true Wisdom is the awareness that you don't know. You can know a little about some things or a lot about a few. Those who believe they do know are blinded by their own ego and are thus willing to dismiss all doubts and set a course powered by a self-fulfilling sense of certainty. This describes how many a tribe of men met their tragic fate. Technology is not immune from the siren's song of certainty hence the never-ending search for the always-elusive "Silver Bullet".

When it comes to Wisdom, what sets apart the Codewright from the Code Monkey? A healthy dose of self-doubt. Doubt enough to realize that risks should not be dismissed solely because they are improbable. Doubt enough to allow the most skillful to help do the work, the most far-sighted to help guard the course, the most cynical to help figure the schedule. Doubt enough to keep unjustified self-confidence from enabling acts of hubris. This is just as true for teams as well. The best teams are the ones who understand both their strengths and their weaknesses. A group of individual superstars who can't learn to play together isn't always guaranteed success while a bunch of unknown amateurs who work has a cohesive unit can perform miracles.

Software development as a field naturally attracts those who are willing to accept the reality that there will always be more to learn. It is a good sign, one that indicates our chosen field will never stagnate. There will always be something to discover, a constantly flowing river of new challenges to overcome and puzzles to solve. The search for Wisdom is not for a desired destination, but of a fulfilling journey.