Computer Science or Software Engineering - An Elemental Problem

What is the difference between Computer Science and Software Engineering?  That depends on the nature of the problem you are trying to solve -  is your problem one of Silicon or one of Carbon?

In my 20-year career in working with software, the single biggest issue has also been an elephant in the room: the abysmal success rate of large software systems.  Plenty of books have been written on the subject with lots of great, if unheeded, advice.  Scope creep, ambiguous or missing requirements, poor communications, bad risk management; lots of reasons have been documented and just as many if not more solutions have been suggested.  Brooks gave us "The Mythical Man-Month", Yourdon writes of "The Death March", Tom DeMarco has "Peopleware". All of them attempting to answer the question, "Why is it so hard to write software?"

If you take the all the books on writing software and put their suggestions in one big list, the first question would be, "If we have all the answers, why don't we use them?"  That might be nice for getting it off your chest but isn't has helpful in pointing us in the right direction. At its core, this question does have a useful kernel, it assumes that 'we' are the problem.  If only we used this process, if only we implemented that practice, if only we hired the right people, if only we kept our teams motivated.  As an aside, most of the reasons for failure are management reasons, not technical ones.  It should be no surprise that many find the case study on the sinking of the Vasa in 1628 to be very instructive. The blame is as much, if not more so, on the King's unrealistic demands as on the builder's efforts to failure to meet the royal requirements. In our endless efforts to build bigger, more complex and complicated software solutions in the face of almost certain failure, should question our sanity; one definition of insanity being, "doing the same thing and expecting different results." Biting off more than we can chew, choking, and then doing it all again time after time only shows how our hubris can get the better of us, allowing us to continue on our Sisyphean task.

This brings me to an interesting observation.  There is a long running debate over whether computer programming is/should be considered a form of artistic expression or of engineering prowess.  Computers have such a wide scope that many sub-fields have crept up, hardware and software being the most obvious. Hardware is typically the realm of the engineer, designing circuit boards or embedded sensors.  This changes as the hardware gets larger in size or scope.  CPUs have microcode, missiles have on-board navigation systems with programmable targets.

Most universities that offer degrees related to programming computers name the department Computer Science.  Now, being considered a 'science' may be just a way of expressing the notion that computers are deterministic and worthy of comparison with mathematics or engineering, having likely grown as an offshoot of the field of electrical engineering.  Whether it be data mining, security, networking, or pure algorithms, most of the areas seem to ignore the question of why software is so hard. They assume the problem is in the silicon computer chips.  Curiously, these fields don't even attempt to address this most difficult of questions, assuming improvements of our silicon constructs or their usage will somehow mitigate our human foibles. If only we had better tools, or faster computers, etc. then our carbon-based brains could somehow ascend above the project management miasma and become a beacon to those who struggle and toil without end. Considering the gargantuan number of man-hours and humongous amounts of money wasted, such an accomplishment would be of heroic proportions, mythic even; a modern-day retelling of Prometheus or Sisyphus.

The truth is that Computer Science may bring improvements in areas where the problem is in the silicon, it is mute when asked to provide answers to problems of a carbon nature.  Software Engineering attempts to develop solutions that include how humans work together.  Software Engineering does not assume that software development is as cut-and-dried as putting the right circuits on a board to achieve the desire product, or breaking down a complex problem into constituent algorithms to make the problem more manageable. There is no magic bullet, no single development process, no single answer.  To understand why humans have a hard time developing software, we must admit that humans are part of the equation. Doing is brings with it a cold truth about humans - they are non-deterministic by nature.  Alistair Cockburn describes it as people being non-linear, first order components in software development.   To answer the question of what is software development, we might have to take detours through Psychology, and may be surprised to end up working with Philosophy even.