Physicality and Comprehensibility
Lately, I’ve been mulling the problem of comprehensibilty: given some mechanism, do people really understand what it does or how it works?
This isn’t an idle question. The programmers who created some system may understand it in detail ("may"? I’ll skip the details in this post, but in machine learning systems the reason for some behavior may remain obscure), but the public, both ordinary citizens who may be affected by it and the legislators and government officials who may wish to regulate it, probably do not. There are a lot of reasons why they don’t, but it’s not just lack of a programming background. Quite simply, what’s happening has no intuitive meaning—most people can’t sense what’s happening.
Contrast that physical objects: you can see how they move and interact. Certainly, there can be subtlety in a mechanical design, but you can poke it, watch it, often even decompose it.
A centrifugal speed governor is an interesting case in point.
Contrast that with the software equivalent. There will be convoluted code to read the speed sensor, complete with such unintuitive things as debouncing and a moving average of the last several samples, etc. Then there’s another complex section of code to move the actuator that controls the fuel valve; this code needs to know how fast to move the valve, perhaps more code to read the current position, etc. On top of all of that, there probably needs to be some hysteresis—at a rate that is itself software-defined—to prevent rapid changes in the speed as it "hunts" for the exact setting.
The greater comprehensibility of physical mechanisms matters even more for children—and we want them to understand the mechanisms around them, since understanding is a prerequisite to tinkering. Tinkering, of course, is how we all learn.
Now, understanding software has its own charms. Back when I was a working programmer, rather than a researcher, my strength was reading code rather than writing it—and I really loved doing it, even the code following this classic comment in 6th Edition Unix.
So what does all this mean? I’m certainly not suggesting that we give up software, or only use software with an obvious physical analog. I am suggesting that we need to pay more attention to making sure that everyone can understand how our software works.
