Usenet, Authentication, and Engineering (or: Early Design Decisions for Usenet)
A Twitter thread on trolls brought up mention of trolls on Usenet. The reason they were so hard to deal with, even then, has some lessons for today; besides, the history is interesting. (Aside: this is, I think, the first longish thing I’ve ever written about any of the early design decisions for Usenet. I should note that this is entirely my writing, and memory can play many tricks across nearly 40 years.)
A complete tutorial on Usenet would take far too long; let it suffice for now to say that in the beginning, it was a peer-to-peer network of multiuser time-sharing systems, primarily interconnected by dial-up 300 bps and 1200 bps modems. (Yes, I really meant THREE HUNDRED BITS PER SECOND. And some day, I’ll have the energy to describe our home-built autodialers—I think that the statute of limitations has expired…) Messages were distributed via a flooding algorithm. Because these time-sharing systems were relatively big and expensive and because there were essentially no consumer-oriented dial-up services then (even modems and dumb terminals were very expensive), if you were on Usenet it was via your school or employer. If there was abuse, pressure could be applied that way—but it wasn’t always easy to tell where a message had originated—and that’s where this blog post really begins: why didn’t Usenet authenticate requests?
We did understand the need for authentication. Without it, there was no way to perform control functions, such as deleting articles. We needed site authentication; as will be seen later, we needed user authentication as well. But how could this be done?
The obvious solution was something involving public key cryptography, which we (the original developers of the protocol: Tom Truscott, the late Jim Ellis, and myself) knew about: all good geeks at the time had seen Martin Gardner’s "Mathematical Games" column in the August 1977 issue of Scientific American (paywall), which explained both the concept of public key cryptography and the RSA algorithm. For that matter, Rivest, Shamir, and Adleman’s technical paper had already appeared; we’d seen that, too. In fact, we had code available: the xsend command for public key encryption and decryption, which we could have built upon, was part of 7th Edition Unix, and that’s what is what Usenet ran on.
What we did not know was how to authenticate a site’s public key. Today, we’d use certificate issued by a certificate authority. Certificates had been invented by then, but we didn’t know about them, and of course there were no search engines to come to our aid. (Manual finding aids? Sure—but apart from the question of whether or not any accessible to us would have indexed bachelor’s theses, we’d have had to know enough to even look. The RSA paper gave us no hints; it simply spoke of a "public file" or something like a phone book. It did speak of signed messages from a "computer network"—scare quotes in the original!—but we didn’t have one of those except for Usenet itself. And a signed message is not a certificate.) Even if we did know, there were no certificate authorities, and we certainly couldn’t create one along with creating Usenet.
Going beyond that, we did not know the correct parameters: how long a key to use (the estimates in the early papers were too low), what was secure (the xsend command used an algorithm that was broken a few years later), etc. Maybe some people could have made good guesses. We did not know and knew that we did not know.
The next thing we considered was neighbor authentication: each site could, at least in principle, know and authenticate its neighbors, due to the way the flooding algorithm worked. That idea didn’t work, either. For one thing, it was trivial to impersonate a site that appeared to be further away. Every Usenet message contains a Path: line; someone trying to spoof a message would simply have to claim to be a few hops away. (This is how the famous kremvax prank worked.)
But there’s a more subtle issue. Usenet messages were transmitted via a generic remote execution facility. The Usenet program on a given computer executed the Unix command
uux neighborsite!rnewswhere neighborsite is the name of the next-hop computer on which the rnews command would be executed. (Before you ask: yes, the list of allowable remotely requested commands was very small; no, the security was not perfect. But that’s not the issue I’m discussing here.) The trouble is that any knowledgeable user on a site could issue the uux command; it wasn’t and couldn’t easily be restricted to authorized users. Anyone could have generated their own fake control messages, without regard to authentication and sanity built in to the Usenet interface. (Could uux have been secured? This is itself a complex question that I don’t want to go into now; please take it on faith and don’t try to argue about setgid(), wrapper programs, and the like. It was our judgment then—and my judgment now—that such solutions would not be adopted. The minor configuration change needed to make rnews an acceptable command for remote execution was a sufficiently high hurdle that we provided alternate mechanisms for sites that wouldn’t do it.)
That left us with no good choices. The infrastructure for a cryptographic solution was lacking. The uux command rendered illusory any attempts at security via the Usenet programs themselves. We chose to do nothing. That is, we did not implement fake security that would give people the illusion of protection but not the reality.
This was the right choice.
But the story is more complex than that. It was the right choice in 1979 but not necessarily right later, for several reasons. The most important is that the online world in 1979 was very different than it is now. For one thing, since only a very few people had access to Usenet, mostly CS students and tech-literate employees of large, sophisticated companies—the norms were to some extent self-enforcing: if someone went too far astray, their school or employer could come down on them. For another, our projections of participation and volume were very low. In my most famous error, I projected that Usenet would grow to 50-100 sites, and 1-2 articles a day, ever. The latest figures, per Wikipedia, puts traffic at about 74 million posts per day, totaling more than 37 terabytes. (I suppose it’s an honor to be off by seven orders of magnitude—not many people help create a system that’s successful enough to have a chance at such a lack of foresight!) On the one hand, a large network has much more need for management, including ways to deal with people and traffic that violates the norms. On the other, simply as a matter of statistics a large network will have at the least proportionately more malefactors. Furthermore, the increasing democratization of access meant that there were people who were not susceptible to school or employer pressure.
Traffic volume was the immediate driver for change. B-news came along in 1981, only a year or so after the original A-news software was released. B-news did have control messages. They were necessary, useful—and abused. Spam messages were often countered by cancelbots, but of course cancelbots were not available only to the righteous. And online norms are not always what everyone wants them to be. The community was willing to act technically against the first large-scale spam outbreak, but other issues—a genuine neo-Nazi, posts to the misc.kids newsgroup by a member of NAMBLA, trolls on the soc.motss newsgroup, and more were dealt with by social pressure.
There are several lessons here. One, of course, is that technical honesty is important. A second, though, is that the balance between security and functionality is not fixed—environments and hence needs change over time. B-news was around for a long time before cancel messages were used or abused on a large scale, and this mass good behavior was not because the insecurity wasn’t recognized: when I had a job interview at Bell Labs in 1982, the first thing Dennis Ritchie said to me was "[B-news] is a tool of the devil!" A third lesson is that norms can matter, but that the community as a whole has to decide how to enforce them.
There’s an amusing postscript to the public key cryptography issue. In 1979-1981, when the Usenet software was being written, there were no patents on public key cryptography nor had anyone heard about export licenses for cryptographic technology. If we’d been a bit more knowledgeable or a bit smarter, we’d have shipped software with such functionality. The code would have been very widespread before any patents were issued, making enforcement very difficult. On the other hand, Tom, Jim, Steve Daniel (who wrote the first released version of the software—my code, originally a Bourne shell script that I later rewrote in C—was never distributed beyond UNC and Duke) and I might have had some very unpleasant conversations with the FBI. But the world of online cryptography would almost certainly have been very different. It’s interesting to speculate on how things would have transpired if cryptography was widely used in the early 1980s.
