SMBlog -- 26 December 2019

The Early History of Usenet, Part IX: The Great Renaming

26 December 2019

The Great Renaming was a significant event in Usenet history, since it involved issues of technology, money, and governance. From a personal perspective—and remember that this series of blog posts is purely my recollections—it also marked the end of my “official” involvement in “running” Usenet. I put “running” in quotation marks in the previous sentence because of the difficulty of actually controlling a non-hierarchical, distributed system with no built-in, authenticated control mechanisms.

As with so many other major changes in Usenet, the underlying problem was volume. Here, it wasn’t so much the volume that individuals could consume as it was volume for sites to send, receive, and store. There was simply too much traffic. The problem was exacerbated by the newsgroup naming structure: it was too flat, and the hierarchy that did exist—net, fa (for “from ARPA”, ARPANET mailing lists that were gatewayed into Usenet newsgroups), and mod, for moderated newsgroups—wasn’t very helpful for managing load. The hierarchy was not semantic, it was based on how content could appear: posted by anyone (net), relayed from a mailing list (fa), or controlled by a moderator (mod). Clearly, something had to be done to aid manageability. But who had both the authority and the power to make such decisions?

Although in theory, all Usenet nodes were equal, in practice some were more equal than others. In technical terms, though Usenet connectivity is considered a graph, in practice it was more like a set of star networks: a very few nodes had disproportionately high connectivity. These few nodes fed many end-sites, but they also talked to each other. In effect, those latter links were the de facto network backbone of Usenet, and the administrators of these major nodes wielded great power. They, together with a few Usenet old-timers, including me and Gene Spafford, constituted what became known as the “Backbone Cabal”. The Backbone Cabal had no power de jure; in practice, though, any newsgroups excluded by the entire cabal would have seen very little distribution outside of the originating region.

The problem had been recognized for quite a while before anything was actually done, see, e.g., this post by Chuq von Rospach, which is arguably the first detailed proposal. The essence of it and the scheme that was finally adopted were the same: organize groups into hierarchies that reflected both subject matter and signal-to-noise ratio. The latter was a significant problem; the volume of shouting in some newsgroups compares unfavorably to the “Comments” section of many web pages. The result was the same, though: sites could easily select what they wanted to receive, via broad categories rather than a long, long list of desired or undesired groups.

Contrary to what some, e.g., the Electronic Frontier Foundation have said, the issue was not censorship, even censorship designed to ensure that Usenet never created the kind of scandal that would lead to public outcry that would threaten the project. And the backbone sites never had to hide from immediate management; as I have indicated, management was very well aware of Usenet and—for backbone sites—was willing to absorb the phone bills. (“Companies so big that their Usenet-related long distance charges were lost in the dictionary-sized bills the company generated every month”—sorry, it doesn’t work that way in any organization I’ve ever been associated with. Every sub-organization had its own budget and had to cover its own phone bills.) There were budget issues and there were worries about scandal, but to the best of my recollection these were more on some non-backbone sites. But the backbone sites had to administer their feeds, and that demanded hierarchy.

To be sure, the top-level hierarchies into which some newsgroups were put was political. It couldn’t help being political, because everyone knew that moving something to the talk hierarchy would sharply curtail its distribution. And yes, members of the Cabal (including, of course, me) had their own particular interests. But that notwithstanding, trying to impose a hierarchical classification system on knowledge is hard—ask any librarian. (Thought experiment: how would you classify Apollo 11? Under “rocketry”? The “space race”? The “Cold War”? What about Werner von Braun’s contribution to the project? Is he a subcategory of the Apollo Project? Or of the history of rocketry, or of World War II?) There was not and could not be a perfect solution.

(The Wikipedia article on the Great Renaming says that the two immediate drivers were the complexity of listing which groups which sites would receive, and/or the cost of the overseas links from seismo to Europe. That my very well be; I simply do not remember specific issues other than load writ broadly.)

The ultimate renaming scheme was the subject of a lot of discussion, and changes were made to the original proposals. Ultimately, it was adopted—and there was rapid counter-action. The alt hierarchy was created as a set of newsgroups explicitly outside the control the Backbone Cabal. And it succeeded, because technology had changed. For one thing, the cost of phone calls was dropping. For another, the spread of the Internet to many sites meant that Usenet didn’t have to flow via phone calls billed by the minute: RFC 977, which prposed a standard for transmitting Usenet over the Internet, came out in early 1986. In other words, the notional control of the Backbone Cabal over content and distribution was just that: notional. The success of the alt hierarchy showed that Usenet had passed a critical point, where the disappearance of a very few nodes could have killed the whole idea of Usenet. At least partially in reaction to this, the Backbone Cabal disappeared—but it left unanswered the question of governance: who could or should control the net?

Newsgroup creation was one early topic. Creation was approved by voting: rough, imperfect voting, which gave rise to proposals for change. There was also the issue of unwanted or improper content, the creation of cancelbots, and more. People worried about liability, jurisdiction, copyright, and more, very early on. These issues are still largely unresolved. Fundamentally, the debate then was between a purely hands-off approach and some form of control; the latter, though, required both consensus on who should have the right to exercise authority and also the creation of appropriate technical mechanisms. Both of these issues are still with us today. I’ll have more to say on them in the next (and final substantive) installment of this series.

Here is the table of contents, actual and projected, for this series.