Facebook and Privacy
Mark Zuckerberg shocked a lot of people by promising a new focus on privacy for Facebook. There are many skeptics; Zuckerberg himself noted that the company doesn’t "currently have a strong reputation for building privacy protective services". And there are issues that his blog post doesn’t address; Zeynep Tufekci discusses many of them While I share many of her concerns, I think there are some other issues—and risks.
The Velocity of Content
Facebook has been criticized for being a channel where bad stuff—anti-vaxxer nonsense, fake news (in the original sense of the phrase…), bigotry, and more—can spread very easily. Tufekci called this out explicitly:
At the moment, critics can (and have) held Facebook accountable for its failure to adequately moderate the content it disseminates—allowing for hate speech, vaccine misinformation, fake news and so on. Once end-to-end encryption is put in place, Facebook can wash its hands of the content. We don’t want to end up with all the same problems we now have with viral content online—only with less visibility and nobody to hold responsible for it.Some critics have called for Facebook to do more to curb such ideas. The company itself has announced it will stop recommending anti-vaccination content. Free speech advocates, though, worry about this a lot. It’s not that anti-vaxxer content is valuable (or even coherent…); rather, it’s that encouraging such a huge, influential company to censor communications is very dangerous. Besides, it doesn’t scale; automated algorithms will make mistakes and can be biased; people not only make mistakes, too, but find the activity extremely stressful. As someone who is pretty much a free speech absolutist myself, I really dislike censorship. That said, as a scientist I prefer not closing my eyes to unpleasant facts. What if Facebook really is different enough that a different paradigm is needed?
Is Facebook that different? I confess that I don’t know. That is, it has certain inherent differences, but I don’t know if they’re great enough in effect to matter, and if so, if the net benefit is more or less than the net harm. Still, it’s worth taking a look at what these differences are.
Before Gutenberg, there was essentially no mass communication: everything was one person speaking or writing to a few others. Yes, the powerful—kings, popes, and the like—could order their subordinates to pass on certain messages, and this could have widespread effect. Indeed, this phenomenon was even recognized in the Biblical Book of Esther
3:12 Then were the king’s scribes called on the thirteenth day of the first month, and there was written according to all that Haman had commanded unto the king’s lieutenants, and to the governors that were over every province, and to the rulers of every people of every province according to the writing thereof, and to every people after their language; in the name of king Ahasuerus was it written, and sealed with the king’s ring.By and large, though, this was rare.3:13 And the letters were sent by posts into all the king’s provinces, to destroy, to kill, and to cause to perish, all Jews, both young and old, little children and women, in one day, even upon the thirteenth day of the twelfth month, which is the month Adar, and to take the spoil of them for a prey.
3:14 The copy of the writing for a commandment to be given in every province was published unto all people, that they should be ready against that day.
3:15 The posts went out, being hastened by the king’s commandment, and the decree was given in Shushan the palace. And the king and Haman sat down to drink; but the city Shushan was perplexed.
Gutenberg’s printing press made life a lot easier. People other than potentates could produce and distribute fliers, pamphlets, newspapers, books, and the like. Information became much more democratic, though, as has often been observed, "freedom of the press belongs to those who own printing presses". There was mass communication, but there were still gatekeepers: most people could not in practice reach a large audience without the permission of a comparative few. Radio and television did not change this dynamic.
Enter the Internet. There was suddenly easy, cheap, many-to-many communication. A U.S. court recognized this. All parties to the case (on government-mandated censorship of content accessible to children) stipulated, among other things:
79. Because of the different forms of Internet communication, a user of the Internet may speak or listen interchangeably, blurring the distinction between "speakers" and "listeners" on the Internet. Chat rooms, e-mail, and newsgroups are interactive forms of communication, providing the user with the opportunity both to speak and to listen.The judges recognized the implications:80. It follows that unlike traditional media, the barriers to entry as a speaker on the Internet do not differ significantly from the barriers to entry as a listener. Once one has entered cyberspace, one may engage in the dialogue that occurs there. In the argot of the medium, the receiver can and does become the content provider, and vice-versa.
81. The Internet is therefore a unique and wholly new medium of worldwide human communication.
It is no exaggeration to conclude that the Internet has achieved, and continues to achieve, the most participatory marketplace of mass speech that this country—and indeed the world—has yet seen. The plaintiffs in these actions correctly describe the "democratizing" effects of Internet communication: individual citizens of limited means can speak to a worldwide audience on issues of concern to them. Federalists and Anti-Federalists may debate the structure of their government nightly, but these debates occur in newsgroups or chat rooms rather than in pamphlets. Modern-day Luthers still post their theses, but to electronic bulletin boards rather than the door of the Wittenberg Schlosskirche. More mundane (but from a constitutional perspective, equally important) dialogue occurs between aspiring artists, or French cooks, or dog lovers, or fly fishermen.But what if this is the problem? What if this new, many-to-many communications, is precisely what is causing trouble? More precisely, what if the problem is the velocity of communcation, in units of people per day?Indeed, the Government’s asserted "failure" of the Internet rests on the implicit premise that too much speech occurs in that medium, and that speech there is too available to the participants. This is exactly the benefit of Internet communication, however. The Government, therefore, implicitly asks this court to limit both the amount of speech on the Internet and the availability of that speech. This argument is profoundly repugnant to First Amendment principles.
High velocity propagation appears to be exacerbated by automation, either explicitly or as a side-effect. YouTube’s recommendation algorithm appears to favor extremist content. Facebook has a similar problem:
Contrast this, however, with another question from Ms. Harris, in which she asked Ms. Sandberg how Facebook can “reconcile an incentive to create and increase your user engagement when the content that generates a lot of engagement is often inflammatory and hateful.” That astute question Ms. Sandberg completely sidestepped, which was no surprise: No statistic can paper over the fact that this is a real problem.The velocity, in these cases, appears to be a side-effect of this algorithmic desire for engagement. Sometimes, though, bots appear to be designed to maximize the spread of malicious content. Either way, information spreads far more quickly than it used to, and on a many-to-many basis.Facebook, Twitter and YouTube have business models that thrive on the outrageous, the incendiary and the eye-catching, because such content generates “engagement” and captures our attention, which the platforms then sell to advertisers, paired with extensive data on users that allow advertisers (and propagandists) to “microtarget” us at an individual level.
Zuckerberg suggests that Facebook wants to focus on smaller-scale communications:
This is different from broader social networks, where people can accumulate friends or followers until the services feel more public. This is well-suited to many important uses—telling all your friends about something, using your voice on important topics, finding communities of people with similar interests, following creators and media, buying and selling things, organizing fundraisers, growing businesses, or many other things that benefit from having everyone you know in one place. Still, when you see all these experiences together, it feels more like a town square than a more intimate space like a living room.What if Facebook evolves that way, and moves more towards small-group communication rather than being a digital town square? What will be the effect? Will smaller-scale many-to-many communications behave this way?There is an opportunity to build a platform that focuses on all of the ways people want to interact privately. This sense of privacy and intimacy is not just about technical features—it is designed deeply into the feel of the service overall. In WhatsApp, for example, our team is obsessed with creating an intimate environment in every aspect of the product. Even where we’ve built features that allow for broader sharing, it’s still a less public experience. When the team built groups, they put in a size limit to make sure every interaction felt private. When we shipped stories on WhatsApp, we limited public content because we worried it might erode the feeling of privacy to see lots of public content—even if it didn’t actually change who you’re sharing with.
I personally like being able to share my thoughts with the world. I was, after all, one of the creators of Usenet; I still spend far too much time on Twitter. But what if this velocity is bad for the world? I don’t know if it is, and I hope it isn’t—but what if it is?
One final thought on this… In democracies, restrictions on speech are more likely to pass legal scrutiny if they’re content-neutral. For example, a loudspeaker truck advocating some controversial position can be banned under anti-noise regulations, regardless of what it is saying. It is quite possible that a velocity limit would be accepted—and it’s not at all clear that this would be desirable. Authoritarian governments are well aware of the power of mass communications:
The use of big-character-posters did not end with the Cultural Revolution. Posters appeared in 1976, during student movements in the mid-1980s, and were central to the Democracy Wall movement in 1978. The most famous poster of this period was Wei Jingsheng’s call for democracy as a "fifth modernization." The state responded by eliminating the clause in the Constitution that allowed people the right to write big-character-posters, and the People’s Daily condemned them for their responsibility in the "ten years of turmoil" and as a threat to socialist democracy. Nonetheless the spirit of the big-character-poster remains a part of protest repertoire, whether in the form of the flyers and notes put up by students in Hong Kong’s Umbrella Movement or as ephemeral posts on the Chinese internet.As the court noted, "Federalists and Anti-Federalists may debate the structure of their government nightly, but these debates occur in newsgroups or chat rooms rather than in pamphlets." Is it good if we give up high-velocity, many-to-many communications?
Certainly, there are other channels than Facebook. But it’s unique: with 2.32 billion users, it reaches about 30% of the world’s population. Any change it makes will have worldwide implications. I wonder if they’ll be for the best.
Possible Risks
Zuckerberg spoke of much more encryption, but he also noted the risks of encrypted content: "Encryption is a powerful tool for privacy, but that includes the privacy of people doing bad things. When billions of people use a service to connect, some of them are going to misuse it for truly terrible things like child exploitation, terrorism, and extortion. We have a responsibility to work with law enforcement and to help prevent these wherever we can". What does this imply?One possibility, of course, is that Facebook might rely more on metadata for analysis: "We are working to improve our ability to identify and stop bad actors across our apps by detecting patterns of activity." But he also spoke of analysis "through other means". What might they be? Doing client-side analysis? About 75% of Facebook users employ mobile devices to access the service; Facebook clients can look at all sorts of things. Content analysis can happen that way, too; though Facebook doesn’t use content to target ads, might it use it for censorship, good or bad?
Encryption also annoys many governments. Governments disliking encryption is not new, of course, but the more people use it, the more upset they will get. This will be exacerbated if encrypted messaging is used for mass communications; Tufekci is specifically concerned about that: "Once end-to-end encryption is put in place, Facebook can wash its hands of the content. We don’t want to end up with all the same problems we now have with viral content online—only with less visibility and nobody to hold responsible for it." We can expect pressure for back doors to increase—but they’ll still be a dangerous idea, for all of the reasons we’ve outlined. (And of course that interacts with the free speech issue.)
I’m not even convinced that Facebook can actually pull this off. Here’s the problem with encryption: who has the keys? Note carefully: you need the key to read the content—but that implies that if the authorized user loses her key, she herself has lost access to her content and messages. The challenge for Facebook, then, is protecting keys against unauthorized parties—Zuckerberg specifically calls out "heavy-handed government intervention in many countries" as a threat—but also making them available to authorized users who have suffered some mishap. Matt Green calls this mud puddle test: if you drop your device in a mud puddle and forget your password, how do you recover your keys?
Apple has gone to great lengths to lock themselves out of your password. Facebook could adopt a similar strategy—but that could mean that a forgotten password means loss of all encrypted content. Facebook of course has a way to recover from a forgotten password—but will that recover a lost key? Should it? So-called secondary authentication is notoriously weak. Perhaps it’s an acceptable tradeoff to regain access to your account but lose access to older content—indeed, Zuckerberg explicitly spoke of the desirability of evanescent content. But even if that’s a good tradeoff—Zuckerberg says "you’d have the ability to change the timeframe or turn off auto-deletion for your threads if you wanted"—if someone else (including a government) took control of you’re account, it would violate another principle Facebook holds dear: "there must never be any doubt about who you are communicating with".
How Facebook handles this dilemma will be very important. Key recovery will make many users very happy, but it will allow the "heavy-handed government intervention" Zuckerberg decries. A user-settable option on key recovery? The usability of any such an option is open to serious question; beyond that, most users will go with the default, and will thus inherit the risks of that default.
A Dangerous, Norm-Destroying Attack
Kim Zetter has a new story out describing a very serious attack. In fact, the implications are about as bad as possible. The attack has been dubbed ShadowHammer by Kaspersky Lab, which discovered it.
Briefly, some crew of attackers—I suspect an intelligence agency; more on that below—has managed to abuse ASUS’ update channel and private signing key to distribute bogus patches. These patches checked the victims’ MAC address; machines on the this list (about 600 of them) downloaded the malware payload from a bogus website that masqueraded as belonging to ASUS.
The reason this is so bad is that trust in the update channel is utterly vital. All software is at least potentially buggy, and some of those bugs will be security holes. For this reason, virtually all software is shipped with a built-in update mechanism. Indeed, on consumer versions of Windows 10 patching is automatic, and while this poses some risks, overall it has almost certainly signficantly improved the security of the Internet: most penetrations exploit known holes, holes for which patches exist but have not been installed.
Now we have an attack that points out the danger of malicious updates. If this scares people away from patching their systems, it will hurt the entire Internet, possibly in a disastrous way. Did the people who planned this operation take this risk into account?
I once blogged that
In cyberattacks, there are no accepted rules… The world knows, more or less, what is acceptable behavior in the physical world: what constitutes an act of war, what is spying, what you can do about these, etc. Do the same rules apply in cyberspace?ShadowHammer is norm-destroying—or rather, it would be, if such norms existed.
Ten years ago, the New York Times reported on a plan to hack Saddam Hussein’s bank accounts. They refrained because of the possible consequences and side-effects:
“We are deeply concerned about the second- and third-order effects of certain types of computer network operations, as well as about laws of war that require attacks be proportional to the threat,” said one senior officer.Whoever launched this attack was either not worried about such issues—or felt that the payoff was worth it.This officer, who like others spoke on the condition of anonymity because of the classified nature of the work, also acknowledged that these concerns had restrained the military from carrying out a number of proposed missions. “In some ways, we are self-deterred today because we really haven’t answered that yet in the world of cyber,” the officer said.
I am convinced that this attack was launched by some country’s intelligence service. I say this for three reasons: it abuses a very sensitive channel, it shows very selective targeting, and the targeting is based on information—MAC addresses—that aren’t that widely available.
The nature of the channel is the first clue. Code-signing keys are precious commodities. While one would hope that a company the size of ASUS would use a hardware security model to protect its keys, at the very least they would be expected to have strong defenses around them. This isn’t the first time that code-signing keys have been abused—Stuxnet did it, too—but it’s not a common thing. This alone shows the attacker’s sophistication.
The highly selective nature of the attack is the next clue. Only ASUS users were affected, and of the estimated 500,000 computers that downloaded the bogus update, the real damage was done to only 600. An ordinary thief, one who wanted bank account logins and passwords, wouldn’t bother with this sort of restriction. Also, limiting the number of machines that had the actual malicious payload minimizes the risk of discovery. Any attacker might worry about discovery, but governments really don’t want covert operations tied back to them.
Finally, there’s the question of how the party behind this attack (and we don’t know who it is, though Kaspersky has tied it to the BARIUM APT, which some have linked to China). MAC addresses aren’t secret, but they’re not trivially available to most parties. They’re widely available on-LAN; that might suggest that the attacker already had a toehold in the targets’ networks. Under certain circumstances, other LANs within an enterprise can see them, too (DHCP Relay, if you’re curious). If any of these machines are laptops that have been used elsewhere, e.g., a hotel or public hotspot, someone who had penetrated that infrasctructure could monitor them. They could be on shipping boxes, or in some vendor database, e.g., inside ASUS—which we already know has been compromised. It’s even possible to get them externally, if the victims (a) use IPv6, (b) use stateless IP address configuration, (c) don’t use the privacy-enhanced version; and (d) visit the attacker’s IPv6 website. In any of these scenarios, you’d also have to link particular MAC addresses to particular targets.
Any or all of these are possible. But they all require significant investment and really good intelligence. To me, this plus the other two clues strongly point to some country’s intelligence agency.
So: we have a state actor willing to take signficant risks with the total security of the Internet, in pursuit of an objective that may or may not be that important. This is, shall we say, bad. The question is what the security community should recommend as a response. The answer is not obvious.
"Don’t patch" is a horrid idea. As I noted, that’s a sure-fire recipe for disaster. In fact, if the ShadowHammerers’ goal was to destroy the Internet, this is a pretty good first step, to be followed by attacks on the patch channels of other major vendors. (Hmm: as I write this, I’m installing patches to my phone and tablet…)
Cautious individuals and sites may wish to defer installing patches; indeed, the newest version of Windows 10 appears to permit a deferral of 35 days. That allows time for bugs to be shaken out of the patch, and for confirmation that the update is indeed a real one. (Zetter noted that some ASUS users did wonder about the ShadowHammer patch.) Sometimes, though, you can’t wait. Equifax was apparently hit very soon after the vulnerability was announced.
Nor is waiting for a vendor announcement a panacea. A high-end attacker—that is to say, a major intelligence agency—can piggyback malware on an existing patch, possibly by subborning insiders.
A high-end vendor might have an independent patch verification team. It would anonymously download patches, reverse-engineer them, and see if they did what they’re supposed to do. Of course, that’s expensive, and small IoT vendors may not be able to afford that. Besides, there are many versions of some patches, e.g., for different language packs.
Ultimately, I suspect that there is no single answer. System penetration via bogus updates were predicted 45 years ago in the classic Karger/Schell report on Multics security. (For those following along at home, it’s in Section 3.4.5.1.) Caution and auditing by all concerned seems to be the best technical path forward. But policy makers have a role, too. We desperately need international agreements on military norms for cyberspace. These won’t be easy to devise nor to enforce, but ultimately, self-restraint may be the best answer.
Update: Juan Andres Guerrero-Saade points out that Flame also abused the update channel. This is quite correct, and I should have been clearer about that. My blog post on Flame, cited above, was written a few days before that aspect of it was described publicly, and I misremembered the attack as spoofing a code-signing certificate à la Stuxnet. Flame was thus just as damaging to vital norms.
Update 2: Matt Blaze has an excellent New York Times op-ed on the importance of patching, despite this incident.
