This Atlantic piece has a pretty alarming title — The Coming Software Apocalypse — but, well, it’s not entirely wrong.
Thirty years ago, we wrote (mostly) to the bare metal. The whole system was plausibly knowable. Now, software is built on software that’s built on software; it’s turtles all the way down, and it’s impossible to understand the entirety of ANY modern effort — because even if you have perfect knowledge of YOUR code (or your organization’s code), you’re dependent on libraries and systems running below you that are opaque.
If all this was just about controlling your VCR or your favorite Office app, it might not matter as much, but we are insanely cavalier about software quality in places where lives are at stake — in 911 systems, in cars, and especially in avionics. But think also about power plants, or other critical areas of infrastructure. Software quality (avoidance of bugs, from the benign to the catastrophic) and software security (keeping others from exploiting the code) are quite often afterthoughts, if they’re thought of AT ALL.
(Incidentally, this is why most software people stay far, far away from “internet of things” gadgets controlled via apps and the cloud. They’re AWFUL from a security POV. And so is your car, most likely. And so is your so-called SmartTV. At our house, the Samsung isn’t even on the network — we use it as a dumb display panel, because we do not, and should not, trust Samsung’s code.)
The piece goes into some ways we might be able to ameliorate this in the future, and some of the steps are very technical and some honestly involve a bit of magical thinking. But a key aspect is taking these things seriously from the getgo, and not being cavalier about any of them (as, say, Jeep and Toyota have been).
