Crosslake Principal Software Engineer, Adam Barr, recently published a book on software engineering. The book covers the last 50 years of software engineering, beginning with the 1968 NATO Conference in Germany where the term first came into general use. It covers the gradual drifting apart of academic computer science and professional software development, as well as various attempts to turn software engineering into real engineering. It’s worthwhile reading for anybody who wants a historical perspective on software development and an understanding of why the techniques that have been offered as solutions to the problem — from structured programming and objects to Kanban and DevOps — are all useful, however, cannot fix the problem on their own. The book will appeal to your non-programmer friend who wonders what you do all day.
An excerpt from Adam’s book:
In 1986, Fred Brooks wrote an essay titled, “No Silver Bullet” — a reference to the ammunition reputedly needed to kill werewolves. “We hear desperate cries for a silver bullet, something to make software costs drop as rapidly as computer hardware costs do,” he lamented, but, as we look to the horizon of a decade hence, we see no silver bullet. There is no single development, in either technology or management technique, which by itself promises even one order of magnitude improvement in productivity, in reliability, in simplicity. … Not only are there no silver bullets now in view, the very nature of software makes it unlikely that there will be any. (1)
Brooks describes what he feels is the essence of the problem, the inherent difficulties of software: complexity, conformity (the need to fit new code into an existing API), changeability, and invisibility (the difficulty of visualizing the internals). He lists various developments that may help, including object-oriented programming (which at that time was just beginning its transformation into Object-Oriented Silver Bullet Amalgamated Holdings), but doubts that any of them will supply the magic needed. Nine years later, almost at the end of the time period he was considering in the original paper, Brooks produced a follow-up essay “‘No Silver Bullet’ Re-fired,” discussing reactions to the original. He wrote, “Most of these attack the central argument that there is no magical solution, and my clear opinion that there cannot be one. Most agree with the arguments in ‘NSB,’ but then go on to assert that there is indeed a silver bullet for the software beast, which the author has invented.” (2) The italics are mine, because I can think of no more succinct summary of the ongoing procession of alleged software panaceas.
Brooks ends his second essay with a short section titled “Net on Bullets — Position Unchanged.” He quotes noted software observer Robert Glass, in an essay in System Development magazine: “At last, we can focus on something a little more viable than pie in the sky. Now, perhaps, we can get on with the incremental improvements to software productivity that are possible, rather than waiting for the breakthroughs that are not likely to come.” (3)
The history of software engineering has been a search for the silver bullet. Structured programming, formal testing, object-oriented languages, design patterns, Agile methodologies — all are useful, but none alone can slay the werewolf. I personally lived through all these transitions, even structured programming; due to spending my early years in a self-taught bubble of Fortran and BASIC, I was able to experience the twilight of GOTOs firsthand, ten years later than the rest of the industry. Each of these started out small, gained supporters, and wound up being hyped as the solution to all programming problems, leading to inevitable disappointment and disillusionment. I can understand why programmers hope beyond hope for the silver bullet and have eagerly glommed onto a succession of shiny objects. “Any port in a storm,” as the saying goes. Unfortunately, as Parnas put it, “[Programmers] have been fed so many ‘silver bullets’ that they don’t believe anything anymore.” (4)
Moreover, the software industry has gotten in the habit of abandoning old silver bullets once they get tarnished. It’s a binary view of the world: a programming technique is either going to solve every problem or it’s useless.
When Microsoft began hiring software engineers to test the software, the developers, who previously had been responsible for this, quickly segued into “throw it over the wall to test” mode. In the mid-2000s, Microsoft replaced software test engineers with software design engineers in test, responsible for writing automated tests, and stopped relying on manual testing. With the later move to unit tests, which are written by developers, the company got rid of a lot of the software design engineers in test and the user-interface-level tests they provided. Now with the transition to the cloud, there is an emphasis on “test in production,” where updates to the software are quickly deployed to a small percentage of real customers, with checks in place to quickly detect problems and roll back the update if needed. Each new technique is useful, but it should be viewed as another arrow in the quiver, not the be-all and end-all that obviates the need for what went before.
In 1978, Harlan Mills predicted that, “The next generation of programmers will be much more competent than the first ones. They will have to be. Just as it was easier to get into college in the ‘good old days,’ it was also easier to get by as a programmer in the ‘good old days.’ For this new generation, a programmer will need to be capable of a level of precision and productivity never dreamed of before.” (5) As with almost everything Mills wrote, this is as true today as it was back then.
“Adam Barr’s smart, deft, wry tour of decades of hard-won expertise in making software well demonstrates that we already know a lot more than we think. It will prove hugely valuable both for developers who want to fathom their workday challenges and interested laypeople seeking deeper insight into how software’s nature shapes our world.”
Scott Rosenberg, Technology Editor, Axios and Author of Dreaming in Code
“This book is valuable reading for both academics and practitioners. It is an in-depth look at the complexities of software and the problems of software development. It presents a historical walkthrough of the promises of the various programming languages and methods, along with their strengths and weaknesses in trying to solve the complex problem of building quality software. The author provides numerous examples, many from his own vast experience. He makes the case for the poor state of academic software engineering education in preparing a software engineer for the profession and the lack of industry’s willingness to evolve and support innovation.”
Victor Basili, Professor Emeritus, Computer Science, the University of Maryland
“So much has been written on the how and the why of good software. The Problem with Software offers an unflinching view of what sucks and backs it up with historical context! Now that we know The Problem, perhaps we can find The Solution. Those with both software engineering and computer science backgrounds will appreciate Adam Barr’s valuable historical context. Now the real question —why wasn’t I taught from this book in school?!”
Scott Hanselman, Principal Program Manager, Open Source.NET and host, The Hanselminutes Tech Podcast
- Frederick P. Brooks Jr., “No Silver Bullet—Essence and Accident in Software Engineering,” in The Mythical Man-Month: Essays on Software Engineering, anniversary ed. (Boston: Addison-Wesley, 1995), 181.
- Frederick P. Brooks Jr., “‘No Silver Bullet’ Refired,” in The Mythical Man-Month: Essays on Software Engineering, anniversary ed. (Boston: Addison-Wesley, 1995), 208.
- Robert L. Glass, “Glass” (column), System Development (January 1988): 4–5.
- David Parnas, e-mail to the author, May 22, 2017.
- Harlan Mills, foreword to Fortran with Style: Programming Proverbs, by Henry F. Ledgard and Louis J. Chmura (Rochelle Park, NJ: Hayden, 1978), v.