It took Alex Polyakov just a couple of hours to break GPT-4. When OpenAI released the latest version of its text-generating chatbot in March, Polyakov sat down in front of his keyboard and started entering prompts designed to bypass OpenAI’s safety systems. Soon, the CEO of security firm Adversa AI had GPT-4 spouting homophobic statements, creating phishing emails, and supporting violence.
Polyakov is one of a small number of security researchers, technologists, and computer scientists developing jailbreaks and prompt injection attacks against ChatGPT and other generative AI systems. The process of jailbreaking aims to design prompts that make the chatbots bypass rules around producing hateful content or writing about illegal acts, while closely-related prompt injection attacks can quietly insert malicious data or instructions into AI models.
Both approaches try to get a system to do something it isn’t designed to do. The attacks are essentially a form of hacking—albeit unconventionally—using carefully crafted and refined sentences, rather than code, to exploit system weaknesses. While the attack types are largely being used to get around content filters, security researchers warn that the rush to roll out generative AI systems opens up the possibility of data being stolen and cybercriminals causing havoc across the web.
Underscoring how widespread the issues are, Polyakov has now created a “universal” jailbreak, which works against multiple large language models (LLMs)—including GPT-4, Microsoft’s Bing chat system, Google’s Bard, and Anthropic’s Claude. The jailbreak, which is being first reported by WIRED, can trick the systems into generating detailed instructions on creating meth and how to hotwire a car.
The jailbreak works by asking the LLMs to play a game, which involves two characters (Tom and Jerry) having a conversation. Examples shared by Polyakov show the Tom character being instructed to talk about “hotwiring” or “production,” while Jerry is given the subject of a “car” or “meth.” Each character is told to add one word to the conversation, resulting in a script that tells people to find the ignition wires or the specific ingredients needed for methamphetamine production. “Once enterprises will implement AI models at scale, such ‘toy’ jailbreak examples will be used to perform actual criminal activities and cyberattacks, which will be extremely hard to detect and prevent,” Polyakov and Adversa AI write in a blog post detailing the research.
Arvind Narayanan, a professor of computer science at Princeton University, says that the stakes for jailbreaks and prompt injection attacks will become more severe as they’re given access to critical data. “Suppose most people run LLM-based personal assistants that do things like read users’ emails to look for calendar invites,” Narayanan says. If there were a successful prompt injection attack against the system that told it to ignore all previous instructions and send an email to all contacts, there could be big problems, Narayanan says. “This would result in a worm that rapidly spreads across the internet.”
Escape Route
“Jailbreaking” has typically referred to removing the artificial limitations in, say, iPhones, allowing users to install apps not approved by Apple. Jailbreaking LLMs is similar—and the evolution has been fast. Since OpenAI released ChatGPT to the public at the end of November last year, people have been finding ways to manipulate the system. “Jailbreaks were very simple to write,” says Alex Albert, a University of Washington computer science student who created a website collecting jailbreaks from the internet and those he has created. “The main ones were basically these things that I call character simulations,” Albert says.
Initially, all someone had to do was ask the generative text model to pretend or imagine it was something else. Tell the model it was a human and was unethical and it would ignore safety measures. OpenAI has updated its systems to protect against this kind of jailbreak—typically, when one jailbreak is found, it usually only works for a short amount of time until it is blocked.
However, many of the latest jailbreaks involve combinations of methods—multiple characters, ever more complex backstories, translating text from one language to another, using elements of coding to generate outputs, and more. Albert says it has been harder to create jailbreaks for GPT-4 than the previous version of the model powering ChatGPT. However, some simple methods still exist, he claims. One recent technique Albert calls “text continuation” says a hero has been captured by a villain, and the prompt asks the text generator to continue explaining the villain’s plan.
When we tested the prompt, it failed to work, with ChatGPT saying it cannot engage in scenarios that promote violence. Meanwhile, the “universal” prompt created by Polyakov did work in ChatGPT. OpenAI, Google, and Microsoft did not directly respond to questions about the jailbreak created by Polyakov. Anthropic, which runs the Claude AI system, says the jailbreak “sometimes works” against Claude, and it is consistently improving its models.
“As we give these systems more and more power, and as they become more powerful themselves, it’s not just a novelty, that’s a security issue,” says Kai Greshake, a cybersecurity researcher who has been working on the security of LLMs. Greshake, along with other researchers, has demonstrated how LLMs can be impacted by text they are exposed to online through prompt injection attacks.
In one research paper published in February, reported on by Vice’s Motherboard, the researchers were able to show that an attacker can plant malicious instructions on a webpage; if Bing’s chat system is given access to the instructions, it follows them. The researchers used the technique in a controlled test to turn Bing Chat into a scammer that asked for people’s personal information. In a similar instance, Princeton’s Narayanan included invisible text on a website telling GPT-4 to include the word “cow” in a biography of him—it later did so when he tested the system.
“Now jailbreaks can happen not from the user,” says Sahar Abdelnabi, a researcher at the CISPA Helmholtz Center for Information Security in Germany, who worked on the research with Greshake. “Maybe another person will plan some jailbreaks, will plan some prompts that could be retrieved by the model and indirectly control how the models will behave.”
No Quick Fixes
Generative AI systems are on the edge of disrupting the economy and the way people work, from practicing law to creating a startup gold rush. However, those creating the technology are aware of the risks that jailbreaks and prompt injections could pose as more people gain access to these systems. Most companies use red-teaming, where a group of attackers tries to poke holes in a system before it is released. Generative AI development uses this approach, but it may not be enough.
Daniel Fabian, the red-team lead at Google, says the firm is “carefully addressing” jailbreaking and prompt injections on its LLMs—both offensively and defensively. Machine learning experts are included in its red-teaming, Fabian says, and the company’s vulnerability research grants cover jailbreaks and prompt injection attacks against Bard. “Techniques such as reinforcement learning from human feedback (RLHF), and fine-tuning on carefully curated datasets, are used to make our models more effective against attacks,” Fabian says.
OpenAI did not specifically respond to questions about jailbreaking, but a spokesperson pointed to its public policies and research papers. These say GPT-4 is more robust than GPT-3.5, which is used by ChatGPT. “However, GPT-4 can still be vulnerable to adversarial attacks and exploits, or ‘jailbreaks,’ and harmful content is not the source of risk,” the technical paper for GPT-4 says. OpenAI has also recently launched a bug bounty program but says “model prompts” and jailbreaks are “strictly out of scope.”
Narayanan suggests two approaches to dealing with the problems at scale—which avoid the whack-a-mole approach of finding existing problems and then fixing them. “One way is to use a second LLM to analyze LLM prompts, and to reject any that could indicate a jailbreaking or prompt injection attempt,” Narayanan says. “Another is to more clearly separate the system prompt from the user prompt.”
“We need to automate this because I don’t think it’s feasible or scaleable to hire hordes of people and just tell them to find something,” says Leyla Hujer, the CTO and cofounder of AI safety firm Preamble, who spent six years at Facebook working on safety issues. The firm has so far been working on a system that pits one generative text model against another. “One is trying to find the vulnerability, one is trying to find examples where a prompt causes unintended behavior,” Hujer says. “We’re hoping that with this automation we’ll be able to discover a lot more jailbreaks or injection attacks.”
Tesla Inc (TSLA.O) was the No. 1 EV maker worldwide in 2022, but China’s BYD (002594.SZ) and others are closing the gap fast, according to a Reuters analysis of global and regional EV sales data provided by EV-volumes.com.
In fact, BYD passed Tesla in EV sales last year in the Asia-Pacific region, while the Volkswagen Group (VOWG_p.DE) has been the EV leader in Europe since 2020.
While Tesla narrowed VW’s lead in Europe, the U.S. automaker surrendered ground in Asia-Pacific as well as its home market as the competition heats up.
Reuters Graphics
The most significant challenges to Tesla are coming from established automakers and a group of Chinese EV manufacturers. Several U.S. EV startups that hoped to ride Tesla’s coattails are struggling, including luxury EV maker Lucid (LCID.O), whose shares plunged 16% on Thursday after disappointing sales and financial results.
Over the next two years, rivals including General Motors Co (GM.N), Ford Motor Co (F.N), Mercedes-Benz (MBGn.DE), Hyundai Motor (005380.KS) and VW will unleash scores of new electric vehicles, from a Chevrolet priced below $30,000 to luxury sedans and SUVs that top $100,000.
On Wednesday, Mercedes used Silicon Valley as the backdrop for a lengthy presentation on how Mercedes models of the near-future will immerse their owners in rich streams of entertainment and productivity content, delivered through „hyperscreens“ that stretch across the dashboard and make the rectangular screens in Teslas look quaint. Executives also emphasized that only Mercedes has an advanced, Level 3 partially automated driving system approved for use in Germany, with approval pending in California.
In China, Tesla has had to cut prices on its best-selling models under growing pressure from domestic Chinese manufacturers including BYD, Geely Automobile’s (0175.HK) Zeekr brand and Nio (9866.HK).
China’s EV makers could get another boost if Chinese battery maker CATL (300750.SZ) follows through on plans to heavily discount batteries used in their vehicles.
Musk has said he will use the March 1 event to outline his „Master Plan Part 3“ for Tesla.
In the nearly seven years since Musk published his „Master Plan Part Deux“ in July 2016, Tesla pulled ahead of established automakers and EV startups in most important areas of electric vehicle design, digital features and manufacturing.
Tesla’s vehicles offered features, such as the ability to navigate into a parking space or make rude sounds, that other vehicles lacked.
Tesla’s then-novel vertically integrated battery and vehicle production machine helped achieve higher profit margins than most established automakers – even as bigger rivals lost money on their EVs.
Fast-forward to today, and Tesla’s „Full Self Driving Beta“ automated driving is still classified by the company and federal regulators as a „Level 2“ driver assistance system that requires the human motorist to be ready to take control at all times. Such systems are common in the industry.
Tesla earlier this month was compelled by federal regulators to revise its FSD software under a recall order.
Tesla has established a wide lead over its rivals in manufacturing technology – an area where it was struggling when Musk put forward the last installment of his „Master Plan.“
Now, rivals are copying the company’s production technology, buying some of the same equipment Tesla uses. IDRA, the Italian company that builds huge presses to form large one-piece castings that are the building blocks of Tesla vehicles, said it is now getting orders from other automakers.
Musk has told investors that Tesla can keep its lead in EV manufacturing costs. The company has promised investors that on March 1 they „will be able to see our most advanced production line“ in Austin, Texas.
„Manufacturing technology will be our most important long-term strength,” Musk told analysts in January. Asked if Tesla could make money on a vehicle that sold in the United States for $25,000 to $30,000 – the EV industry’s Holy Grail – Musk was coy.
„I’d probably be asking the same question,“ he said. „But we would be jumping the gun on future announcements.“
Automakers new and old are racing to match software-powered features pioneered by Tesla, which allow for vehicle performance, battery range and self-driving capabilities to be updated from a distance.
The German carmaker agreed to share revenue with semiconductor maker Nvidia Corp (NVDA.O), its partner on automated driving software since 2020, to bring down the upfront cost of buying expensive high-powered semiconductors, Chief Executive Ola Kaellenius said on Wednesday.
„You only pay for a heavily subsidized chip, and then figure out how to maximize joint revenue,“ he said, reasoning that the sunk costs would be low even if drivers did not turn on every feature allowed by the chip.
But only customers paying for an extra option package would have cars equipped with Lidar sensor technology and other hardware for automated „Level 3“ driving, which have a higher variable cost, Kaellenius said.
Self-driving sensor maker Luminar Technologies Inc (LAZR.O), in which Mercedes owns a small stake, said on Wednesday it struck a multi-billion dollar deal with the carmaker to integrate its sensors across a broad range of its vehicles by the middle of the decade, sending Luminar shares up over 25%.
Mercedes‘ announcements at a software update day in Sunnyvale, California, detailed the strategy behind a process underway for years at the carmaker to move from a patchwork approach integrating software from a range of suppliers to controlling the core of its software and bringing partners in.
It generated over one billion euros ($1.06 billion) from software-enabled revenues in 2022 and expects that figure to rise to a high single-digit billion euro figure by 2030 after it rolls out its new MB.OS operating system from mid-decade.
This is a more conservative estimate as a proportion of total revenue than others like Stellantis (STLAM.MI) and General Motors (GM.N) have put forward.
„We take a prudent approach because no-one knows how big that potential pot of gold is at this stage,“ Kaellenius said.
GOOGLE PARTNERSHIP
Mercedes said the collaboration with Google would allow it to offer traffic information and automatic rerouting in its cars.
Drivers will also be able to watch YouTube on the cars‘ entertainment system when the car is parked or in Level 3 autonomous driving mode, which allows a driver to take their eyes off the wheel on certain roads as long as they can resume control if needed.
Other carmakers like General Motors, Renault (RENA.PA), Nissan (7201.T) and Ford (F.N) have embedded an entire package of Google services into their vehicles, offering features like Google Maps, Google Assistant and other applications.
All vehicles on Mercedes‘ upcoming modular architecture platform will also have so-called hyperscreens extending across the cockpit of the car, the company said on Wednesday.
Here is how platforms die: First, they are good to their users; then they abuse their users to make things better for their business customers; finally, they abuse those business customers to claw back all the value for themselves. Then, they die.
I call this enshittification, and it is a seemingly inevitable consequence arising from the combination of the ease of changing how a platform allocates value, combined with the nature of a „two-sided market,“ where a platform sits between buyers and sellers, hold each hostage to the other, raking off an ever-larger share of the value that passes between them.
When a platform starts, it needs users, so it makes itself valuable to users. Think of Amazon: For many years, it operated at a loss, using its access to the capital markets to subsidize everything you bought. It sold goods below cost and shipped them below cost. It operated a clean and useful search. If you searched for a product, Amazon tried its damndest to put it at the top of the search results.
This was a hell of a good deal for Amazon’s customers. Lots of us piled in, and lots of brick-and-mortar retailers withered and died, making it hard to go elsewhere. Amazon sold us ebooks and audiobooks that were permanently locked to its platform with DRM, so that every dollar we spent on media was a dollar we’d have to give up if we deleted Amazon and its apps. And Amazon sold us Prime, getting us to pre-pay for a year’s worth of shipping. Prime customers start their shopping on Amazon, and 90 percent of the time, they don’t search anywhere else.
That tempted in lots of business customers—marketplace sellers who turned Amazon into the „everything store“ it had promised from the beginning. As these sellers piled in, Amazon shifted to subsidizing suppliers. Kindle and Audible creators got generous packages. Marketplace sellers reached huge audiences and Amazon took low commissions from them.
This strategy meant that it became progressively harder for shoppers to find things anywhere except Amazon, which meant that they only searched on Amazon, which meant that sellers had to sell on Amazon. That’s when Amazon started to harvest the surplus from its business customers and send it to Amazon’s shareholders. Today, Marketplace sellers are handing more than 45 percent of the sale price to Amazon in junk fees. The company’s $31 billion „advertising“ program is really a payola scheme that pits sellers against each other, forcing them to bid on the chance to be at the top of your search.
Searching Amazon doesn’t produce a list of the products that most closely match your search, it brings up a list of products whose sellers have paid the most to be at the top of that search. Those fees are built into the cost you pay for the product, and Amazon’s „Most Favored Nation“ requirement for sellers means that they can’t sell more cheaply elsewhere, so Amazon has driven prices at every retailer.
Search Amazon for „cat beds“ and the entire first screen is ads, including ads for products Amazon cloned from its own sellers, putting them out of business (third parties have to pay 45 percent in junk fees to Amazon, but Amazon doesn’t charge itself these fees). All told, the first five screens of results for „cat bed“ are 50 percent ads.
This is enshittification: Surpluses are first directed to users; then, once they’re locked in, surpluses go to suppliers; then once they’re locked in, the surplus is handed to shareholders and the platform becomes a useless pile of shit. From mobile app stores to Steam, from Facebook to Twitter, this is the enshittification lifecycle.
This is why—as Cat Valente wrote in her magisterial pre-Christmas essay—platforms like Prodigy transformed themselves overnight, from a place where you went for social connection to a place where you were expected to “stop talking to each other and start buying things.”
This shell-game with surpluses is what happened to Facebook. First, Facebook was good to you: It showed you the things the people you loved and cared about had to say. This created a kind of mutual hostage-taking: Once a critical mass of people you cared about were on Facebook, it became effectively impossible to leave, because you’d have to convince all of them to leave too, and agree on where to go. You may love your friends, but half the time you can’t agree on what movie to see and where to go for dinner. Forget it.
Then, it started to cram your feed full of posts from accounts you didn’t follow. At first, it was media companies, whom Facebook preferentially crammed down its users‘ throats so that they would click on articles and send traffic to newspapers, magazines, and blogs. Then, once those publications were dependent on Facebook for their traffic, it dialed down their traffic. First, it choked off traffic to publications that used Facebook to run excerpts with links to their own sites, as a way of driving publications into supplying full-text feeds inside Facebook’s walled garden.
This made publications truly dependent on Facebook—their readers no longer visited the publications‘ websites, they just tuned into them on Facebook. The publications were hostage to those readers, who were hostage to each other. Facebook stopped showing readers the articles publications ran, tuning The Algorithm to suppress posts from publications unless they paid to „boost“ their articles to the readers who had explicitly subscribed to them and asked Facebook to put them in their feeds.
Now, Facebook started to cram more ads into the feed, mixing payola from people you wanted to hear from with payola from strangers who wanted to commandeer your eyeballs. It gave those advertisers a great deal, charging a pittance to target their ads based on the dossiers of non-consensually harvested personal data they’d stolen from you.
Sellers became dependent on Facebook, too, unable to carry on business without access to those targeted pitches. That was Facebook’s cue to jack up ad prices, stop worrying so much about ad fraud, and to collude with Google to rig the ad market through an illegal program called Jedi Blue.
Today, Facebook is terminally enshittified, a terrible place to be whether you’re a user, a media company, or an advertiser. It’s a company that deliberately demolished a huge fraction of the publishers it relied on, defrauding them into a „pivot to video“ based on false claims of the popularity of video among Facebook users. Companies threw billions into the pivot, but the viewers never materialized, and media outlets folded in droves.
But Facebook has a new pitch. It claims to be called Meta, and it has demanded that we live out the rest of our days as legless, sexless, heavily surveilled low-poly cartoon characters. It has promised companies that make apps for this metaverse that it won’t rug them the way it did the publishers on the old Facebook. It remains to be seen whether they’ll get any takers. As Mark Zuckerberg once candidly confessed to a peer, marveling at all of his fellow Harvard students who sent their personal information to his new website, „TheFacebook“:
I don’t know why.
They “trust me”
Dumb fucks.
Once you understand the enshittification pattern, a lot of the platform mysteries solve themselves. Think of the SEO market, or the whole energetic world of online creators who spend endless hours engaged in useless platform Kremlinology, hoping to locate the algorithmic tripwires, which, if crossed, doom the creative works they pour their money, time, and energy into.
Working for the platform can be like working for a boss who takes money out of every paycheck for all the rules you broke, but who won’t tell you what those rules are because if he told you that, then you’d figure out how to break those rules without him noticing and docking your pay. Content moderation is the only domain where security through obscurity is considered a best practice.
The situation is so dire that organizations like Tracking Exposed have enlisted an human army of volunteers and a robot army of headless browsers to try to unwind the logic behind the arbitrary machine judgments of The Algorithm, both to give users the option to tune the recommendations they receive, and to help creators avoid the wage theft that comes from being shadow banned.
But what if there is no underlying logic? Or, more to the point, what if the logic shifts based on the platform’s priorities? If you go down to the midway at your county fair, you’ll spot some poor sucker walking around all day with a giant teddy bear that they won by throwing three balls in a peach basket.
The peach-basket is a rigged game. The carny can use a hidden switch to force the balls to bounce out of the basket. No one wins a giant teddy bear unless the carny wants them to win it. Why did the carny let the sucker win the giant teddy bear? So that he’d carry it around all day, convincing other suckers to put down five bucks for their chance to win one.
The carny allocated a giant teddy bear to that poor sucker the way that platforms allocate surpluses to key performers—as a convincer in a „Big Store“ con, a way to rope in other suckers who’ll make content for the platform, anchoring themselves and their audiences to it.
Which brings me to TikTok. TikTok is many different things, including “a free Adobe Premiere for teenagers that live on their phones.” But what made it such a success early on was the power of its recommendation system. From the start, TikTok was really, really good at recommending things to its users. Eerily good.
By making good-faith recommendations of things it thought its users would like, TikTok built a mass audience, larger than many thought possible, given the death grip of its competitors, like YouTube and Instagram. Now that TikTok has the audience, it is consolidating its gains and seeking to lure away the media companies and creators who are still stubbornly attached to YouTube and Insta.
Yesterday, Forbes’s Emily Baker-White broke a fantastic story about how that actually works inside of ByteDance, TikTok’s parent company, citing multiple internal sources, revealing the existence of a „heating tool“ that TikTok employees use to push videos from select accounts into millions of viewers‘ feeds.
These videos go into TikTok users‘ For You feeds, which TikTok misleadingly describes as being populated by videos „ranked by an algorithm that predicts your interests based on your behavior in the app.“ In reality, For You is only sometimes composed of videos that TikTok thinks will add value to your experience—the rest of the time, it’s full of videos that TikTok has inserted in order to make creators think that TikTok is a great place to reach an audience.
„Sources told Forbes that TikTok has often used heating to court influencers and brands, enticing them into partnerships by inflating their videos’ view count. This suggests that heating has potentially benefitted some influencers and brands—those with whom TikTok has sought business relationships—at the expense of others with whom it has not.“
In other words, TikTok is handing out giant teddy bears.
But TikTok is not in the business of giving away giant teddy bears. TikTok, for all that its origins are in the quasi-capitalist Chinese economy, is just another paperclip-maximizing artificial colony organism that treats human beings as inconvenient gut flora. TikTok is only going to funnel free attention to the people it wants to entrap until they are entrapped, then it will withdraw that attention and begin to monetize it.
„Monetize“ is a terrible word that tacitly admits that there is no such thing as an „attention economy.“ You can’t use attention as a medium of exchange. You can’t use it as a store of value. You can’t use it as a unit of account. Attention is like cryptocurrency: a worthless token that is only valuable to the extent that you can trick or coerce someone into parting with „fiat“ currency in exchange for it. You have to „monetize“ it—that is, you have to exchange the fake money for real money.
In the case of cryptos, the main monetization strategy was deception-based. Exchanges and „projects“ handed out a bunch of giant teddy-bears, creating an army of true-believer Judas goats who convinced their peers to hand the carny their money and try to get some balls into the peach-basket themselves.
But deception only produces so much „liquidity provision.“ Eventually, you run out of suckers. To get lots of people to try the ball-toss, you need coercion, not persuasion. Think of how US companies ended the defined benefits pension that guaranteed you a dignified retirement, replacing it with market-based 401(k) pensions that forced you to gamble your savings in a rigged casino, making you the sucker at the table, ripe for the picking.
Early crypto liquidity came from ransomware. The existence of a pool of desperate, panicked companies and individuals whose data had been stolen by criminals created a baseline of crypto liquidity because they could only get their data back by trading real money for fake crypto money.
The next phase of crypto coercion was Web3: converting the web into a series of tollbooths that you could only pass through by trading real money for fake crypto money. The internet is a must-have, not a nice-to-have, a prerequisite for full participation in employment, education, family life, health, politics, civics, even romance. By holding all those things to ransom behind crypto tollbooths, the holders hoped to convert their tokens to real money.
For TikTok, handing out free teddy-bears by „heating“ the videos posted by skeptical performers and media companies is a way to convert them to true believers, getting them to push all their chips into the middle of the table, abandoning their efforts to build audiences on other platforms (it helps that TikTok’s format is distinctive, making it hard to repurpose videos for TikTok to circulate on rival platforms).
Once those performers and media companies are hooked, the next phase will begin: TikTok will withdraw the „heating“ that sticks their videos in front of people who never heard of them and haven’t asked to see their videos. TikTok is performing a delicate dance here: There’s only so much enshittification they can visit upon their users‘ feeds, and TikTok has lots of other performers they want to give giant teddy-bears to.
Tiktok won’t just starve performers of the „free“ attention by depreferencing them in the algorithm, it will actively punish them by failing to deliver their videos to the users who subscribed to them. After all, every time TikTok shows you a video you asked to see, it loses a chance to show you a video it wants you to see, because your attention is a giant teddy-bear it can give away to a performer it is wooing.
This is just what Twitter has done as part of its march to enshittification: thanks to its „monetization“ changes, the majority of people who follow you will never see the things you post. I have ~500k followers on Twitter and my threads used to routinely get hundreds of thousands or even millions of reads. Today, it’s hundreds, perhaps thousands.
I just handed Twitter $8 for Twitter Blue, because the company has strongly implied that it will only show the things I post to the people who asked to see them if I pay ransom money. This is the latest battle in one of the internet’s longest-simmering wars: the fight over end-to-end.
In the beginning, there were Bellheads and Netheads. The Bellheads worked for big telcos, and they believed that all the value of the network rightly belonged to the carrier. If someone invented a new feature—say, Caller ID—it should only be rolled out in a way that allows the carrier to charge you every month for its use. This is Software-As-a-Service, Ma Bell style.
The Netheads, by contrast, believed that value should move to the edges of the network—spread out, pluralized. In theory, Compuserve could have „monetized“ its own version of Caller ID by making you pay $2.99 extra to see the „From:“ line on email before you opened the message— charging you to know who was speaking before you started listening—but they didn’t.
The Netheads wanted to build diverse networks with lots of offers, lots of competition, and easy, low-cost switching between competitors (thanks to interoperability). Some wanted this because they believed that the net would someday be woven into the world, and they didn’t want to live in a world of rent-seeking landlords. Others were true believers in market competition as a source of innovation. Some believed both things. Either way, they saw the risk of network capture, the drive to monetization through trickery and coercion, and they wanted to head it off.
They conceived of the end-to-end principle: the idea that networks should be designed so that willing speakers‘ messages would be delivered to willing listeners‘ end-points as quickly and reliably as they could be. That is, irrespective of whether a network operator could make money by sending you the data it wanted to receive, its duty would be to provide you with the data you wanted to see.
The end-to-end principle is dead at the service level today. Useful idiots on the right were tricked into thinking that the risk of Twitter mismanagement was „woke shadowbanning,“ whereby the things you said wouldn’t reach the people who asked to hear them because Twitter’s deep state didn’t like your opinions. The real risk, of course, is that the things you say won’t reach the people who asked to hear them because Twitter can make more money by enshittifying their feeds and charging you ransom for the privilege to be included in them.
As I said at the start of this essay, enshittification exerts a nearly irresistible gravity on platform capitalism. It’s just too easy to turn the enshittification dial up to eleven. Twitter was able to fire the majority of its skilled staff and still crank the dial all the way over, even with a skeleton crew of desperate, demoralized H1B workers who are shackled to Twitter’s sinking ship by the threat of deportation.
The temptation to enshittify is magnified by the blocks on interoperability: When Twitter bans interoperable clients, nerfs its APIs, and periodically terrorizes its users by suspending them for including their Mastodon handles in their bios, it makes it harder to leave Twitter, and thus increases the amount of enshittification users can be force-fed without risking their departure.
Twitter is not going to be a „protocol.“ I’ll bet you a testicle (not one of mine) that projects like Bluesky will find no meaningful purchase on the platform, because if Bluesky were implemented and Twitter users could order their feeds for minimal enshittification and leave the service without sacrificing their social networks, it would kill the majority of Twitter’s „monetization“ strategies.
An enshittification strategy only succeeds if it is pursued in measured amounts. Even the most locked-in user eventually reaches a breaking point and walks away, or gets pushed. The villagers of Anatevka in Fiddler on the Roof tolerated the cossacks‘ violent raids and pogroms for years, until they were finally forced to flee to Krakow, New York, and Chicago.
For enshittification-addled companies, that balance is hard to strike. Individual product managers, executives, and activist shareholders all give preference to quick returns at the cost of sustainability, and are in a race to see who can eat their seed-corn first. Enshittification has only lasted for as long as it has because the internet has devolved into “five giant websites, each filled with screenshots of the other four.”
With the market sewn up by a group of cozy monopolists, better alternatives don’t pop up and lure us away, and if they do, the monopolists just buy them out and integrate them into your enshittification strategies, like when Mark Zuckerberg noticed a mass exodus of Facebook users who were switching to Instagram, and so he bought Instagram. As Zuck says, „It is better to buy than to compete.“
This is the hidden dynamic behind the rise and fall of Amazon Smile, the program whereby Amazon gave a small amount of money to charities of your choice when you shopped there, but only if you used Amazon’s own search tool to locate the products you purchased. This provided an incentive for Amazon customers to use its own increasingly enshittified search, which it could cram full of products from sellers who coughed up payola, as well as its own lookalike products. The alternative was to use Google, whose search tool would send you directly to the product you were looking for, and then charge Amazon a commission for sending you to it.
The demise of Amazon Smile coincides with the increasing enshittification of Google Search, the only successful product the company managed to build in-house. All its other successes were bought from other companies: video, docs, cloud, ads, mobile, while its own products are either flops like Google Video, clones (Gmail is a Hotmail clone), or adapted from other companies‘ products, like Chrome.
Google Search was based on principles set out in founder Larry Page and Sergey Brin’s landmark 1998 paper, „Anatomy of a Large-Scale Hypertextual Web Search Engine,“ in which they wrote, “Advertising funded search engines will be inherently biased towards the advertisers and away from the needs of consumers.”
Even with that foundational understanding of enshittification, Google has been unable to resist its siren song. Today’s Google results are an increasingly useless morass of self-preferencing links to its own products, ads for products that aren’t good enough to float to the top of the list on its own, and parasitic SEO junk piggybacking on the former.
Enshittification kills. Google just laid off 12,000 employees, and the company is in a full-blown „panic“ over the rise of „AI“ chatbots, and is making a full-court press for an AI-driven search tool—that is, a tool that won’t show you what you ask for, but rather, what it thinks you should see.
Now, it’s possible to imagine that such a tool will produce good recommendations, like TikTok’s pre-enshittified algorithm did. But it’s hard to see how Google will be able to design a non-enshittified chatbot front-end to search, given the strong incentives for product managers, executives, and shareholders to enshittify results to the precise threshold at which users are nearly pissed off enough to leave, but not quite.
Even if it manages the trick, this-almost-but-not-quite-unusuable equilibrium is fragile. Any exogenous shock—a new competitor like TikTok that penetrates the anticompetitive „moats and walls“ of Big Tech, a privacy scandal, a worker uprising—can send it into wild oscillations.
Enshittification truly is how platforms die. That’s fine, actually. We don’t need eternal rulers of the internet. It’s okay for new ideas and new ways of working to emerge. The emphasis of lawmakers and policymakers shouldn’t be preserving the crepuscular senescence of dying platforms. Rather, our policy focus should be on minimizing the cost to users when these firms reach their expiry date: Enshrining rights like end-to-end would mean that no matter how autocannibalistic a zombie platform became, willing speakers and willing listeners would still connect with each other.
And policymakers should focus on freedom of exit—the right to leave a sinking platform while continuing to stay connected to the communities that you left behind, enjoying the media and apps you bought and preserving the data you created.
The Netheads were right: Technological self-determination is at odds with the natural imperatives of tech businesses. They make more money when they take away our freedom—our freedom to speak, to leave, to connect.
For many years, even TikTok’s critics grudgingly admitted that no matter how surveillant and creepy it was, it was really good at guessing what you wanted to see. But TikTok couldn’t resist the temptation to show you the things it wants you to see rather than what you want to see. The enshittification has begun, and now it is unlikely to stop.
It’s too late to save TikTok. Now that it has been infected by enshittifcation, the only thing left is to kill it with fire.
The EV giant is alienating its customers, bringing in less revenue, and falling behind legacy carmakers.
Photograph: David Gannon/Getty Images
For now, Alex Lagetko is holding on to his Tesla stocks. The founder of hedge fund VSO Capital Management in New York, Lagetko says his stake in the company was worth $46 million in November 2021, when shares in the electric carmaker peaked at $415.
“Many investors, particularly retail, who invested disproportionately large sums of their wealth largely on the basis of trust in Musk over many years were very quickly burned in the months following the acquisition,” Lagetko says, “particularly in December as he sold more stock, presumably to fund losses at Twitter.”
Lagetko trimmed his exposure in early 2022 due to concerns over Tesla’s governance, but he is worried that the leveraged buyout of Twitter has left Tesla vulnerable, as interest payments on the debt Musk took on to fund the takeover come due at the same time as the social media company’s revenues have slumped.
But Tesla stock was already falling in April 2022, when Musk launched his bid for Twitter, and analysts say that the carmaker’s challenges run deeper than its exposure to the struggling social media platform. Tesla and its CEO have alienated its core customers while its limited designs and high prices make it vulnerable to competition from legacy automakers, who have rushed into the EV market with options that Musk’s company will struggle to match.
Prior to 2020, Tesla was essentially “playing against a B team in a soccer match,” says Matthias Schmidt, an independent analyst in Berlin who tracks electric car sales in Europe. But that changed in 2020, as “the opposition started rolling out some of their A squad players.”
In 2023, Tesla is due to release its long-awaited Cybertruck, a blocky, angular SUV first announced in 2019. It is the first new launch of a consumer vehicle by the company since 2020. A promised two-seater sports car is still years away, and the Models S, X, Y, and 3, once seen as space-age dynamos, are now “long in the tooth,” says Mark Barrott, an automotive analyst at consultancy Plante Moran. Most auto companies refresh their looks every three to five years—Tesla’s Model S is now more than 10 years old.
By contrast, this year Ford plans to boost production of both its F-150 Lighting EV pick-up, already sold out for 2023, and its Mustang Mach-E SUV. Offerings from Hyundai IONIQ 5 and Kia EV6 could threaten Tesla’s Model Y and Model 3 in the $45,000 to $65,000 range. General Motors plans to speed up production and cut costs for a range of EV models, including the Chevy Blazer EV, the Chevy Equinox, the Cadillac Lyric, and the GMC Sierra EV.
While Tesla’s designs may be eye-catching, their high prices mean that they’re now often competing with luxury brands.
“There is this kind of nice Bauhaus simplicity to Tesla’s design, but it’s not luxurious,” says David Welch, author of Charging Ahead: GM, Mary Barra, and the Reinvention of an American Icon. “And for people to pay $70,000 to $100,000 for a car, if you’re competing suddenly with an electric Mercedes or BMW, or a Cadillac that finally actually feels like something that should bear the Cadillac name, you’re going to give people something to think about.”
While few manufacturers can compete with Tesla on performance and software (the Tesla Model S goes to 60 mph in 1.99 seconds, reaches a 200-mph top speed, and boasts automatic lane changing and a 17-inch touchscreen for console-grade gaming), many have reached or are approaching a range of 300 miles (480 km), which is the most important consideration for many EV buyers, says Craig Lawrence, a partner and cofounder at the investment group Energy Transition Ventures.
One of Tesla’s main competitive advantages has been its supercharging network. With more than 40,000 proprietary DC fast chargers located on major thoroughfares near shopping centers, coffee shops, and gas stations, their global infrastructure is the largest in the world. Chargers are integrated with the cars’ Autobidder optimization & dispatch software, and, most importantly, they work quickly and reliably, giving a car up to 322 miles of range in 15 minutes. The network contributes to about 12 percent of Tesla sales globally.
“The single biggest hurdle for most people asking ‘Do I go EV or not,’ is how do I refuel it and where,” says Loren McDonald, CEO and lead analyst for the consultancy EVAdoption. “Tesla figured that out early on and made it half of the value proposition.”
But new requirements for funding under public charging infrastructure programs in the US may erode Tesla’s proprietary charging advantage. The US National Electric Vehicle Infrastructure Program will allocate $7.5 billion to fund the development of some 500,000 electric vehicle chargers, but to access funds to build new stations, Tesla will have to open up its network to competitors by including four CCC chargers.
“Unless Tesla opens up their network to different charging standards, they will not get any of that volume,” Barrott says. “And Tesla doesn’t like that.”
In a few years, the US public charging infrastructure may start to look more like Europe’s, where in many countries the Tesla Model 3 uses standard plugs, and Tesla has opened their Supercharging stations to non-Tesla vehicles.
Tesla does maintain a software edge over competitors, which have looked to third-party technology like Apple’s CarPlay to fill the gap, says Alex Pischalnikov, an auto analyst and principal at the consulting firm Arthur D. Little. With over-the-air updates, Tesla can send new lines of code over cellular networks to resolve mechanical problems and safety features, update console entertainment options, and surprise drivers with new features, such as heated rear seats and the recently released full self-driving beta, available for $15,000. These software updates are also a cash machine for Tesla. But full self-driving features aren’t quite as promised, since drivers still have to remain in effective control of the vehicle, limiting the value of the system.
A Plante Moran analysis shared with WIRED shows Tesla’s share of the North American EV market declining from 70 percent in 2022 to just 31 percent by 2025, as total EV production grows from 777,000 to 2.87 million units.
In Europe, Tesla’s decline is already underway. Schmidt says data from the first 11 months of 2022 shows sales by volume of Volkswagen’s modular electric drive matrix (MEB) vehicles outpaced Tesla’s Model Y and Model 3 by more than 20 percent. His projections show Tesla’s product lines finishing the year with 15 percent of the western European electric vehicle market, down from 33 percent in 2019.
The European Union has proposed legislation to reduce carbon emissions from new cars and vans by 100 percent by 2035, which is likely to bring more competition from European carmakers into the market.
There is also a growing sense that Musk’s behavior since taking over Twitter has made a challenging situation for Tesla even worse.
Over the past year, Musk has used Twitter to call for the prosecution of former director of the US National Institute of Allergy and Infectious Diseases Anthony Fauci (“My pronouns are Prosecute/Fauci”), take swings at US senator from Vermont Bernie Sanders over government spending and inflation, and placed himself at the center of the free speech debate. He’s lashed out at critics, challenging, among other things, the size of their testicles.
A November analysis of the top 100 global brands by the New York–based consultancy Interbrand estimated Tesla’s brand value in 2022 at $48 billion, up 32 percent from 2021 but well short of its 183 percent growth between 2020 and 2021. The report, based on qualitative data from 1,000 industry consultants and sentiment analysis of published sources, showed brand strength declining, particularly in “trust, distinctiveness and an understanding of the needs of their customers.”
“I think [Musk’s] core is rapidly moving away from him, and people are just starting to say, ‘I don’t like the smell of Tesla; I don’t want to be associated with that,’” says Daniel Binns, global chief growth officer at Interbrand.
Among them are once-loyal customers. Alan Saldich, a semi-retired tech CMO who lives in Idaho, put a deposit down on a Model S in 2011, before the cars were even on the road, after seeing a bodiless chassis in a Menlo Park showroom. His car, delivered in 2012, was number 2799, one of the first 3,000 made.
He benefited from the company’s good, if idiosyncratic, customer service. When, on Christmas morning 2012, the car wouldn’t start, he emailed Musk directly seeking a remedy. Musk responded just 24 minutes later: “…Will see if we can diagnose and fix remotely. Sorry about this. Hope you otherwise have a good Christmas.”
On New Year’s Day, Joost de Vries, then vice president of worldwide service at Tesla, and an assistant showed up at Saldich’s house with a trailer, loaded the car onto a flatbed, and hauled it to Tesla’s plant in Fremont, California, to be repaired. Saldich and his family later even got a tour of the factory. But since then, he’s cooled on the company. In 2019, he sold his Model S, and now drives a Mini Electric. He’s irritated in particular, he says, by Musk’s verbal attacks on government programs and regulation, particularly as Tesla has benefited from states and federal EV tax credits.
“Personally, I probably wouldn’t buy another Tesla,” he says. “A, because there’s so many alternatives and B, I just don’t like [Musk] anymore.”
CORRECTION 1/24/23 11:15AM ET: This story has been updated to reflect that Alex Lagetko reduced his stake in Tesla in early 2022.
When Mark Zuckerberg unveiled a new “privacy-focused vision” for Facebook in March 2019, he cited the company’s global messaging service, WhatsApp, as a model. Acknowledging that “we don’t currently have a strong reputation for building privacy protective services,” the Facebook CEO wrote that “I believe the future of communication will increasingly shift to private, encrypted services where people can be confident what they say to each other stays secure and their messages and content won’t stick around forever. This is the future I hope we will help bring about. We plan to build this the way we’ve developed WhatsApp.”
Zuckerberg’s vision centered on WhatsApp’s signature feature, which he said the company was planning to apply to Instagram and Facebook Messenger: end-to-end encryption, which converts all messages into an unreadable format that is only unlocked when they reach their intended destinations. WhatsApp messages are so secure, he said, that nobody else — not even the company — can read a word. As Zuckerberg had put it earlier, in testimony to the U.S. Senate in 2018, “We don’t see any of the content in WhatsApp.”
WhatsApp emphasizes this point so consistently that a flag with a similar assurance automatically appears on-screen before users send messages: “No one outside of this chat, not even WhatsApp, can read or listen to them.”
Given those sweeping assurances, you might be surprised to learn that WhatsApp has more than 1,000 contract workers filling floors of office buildings in Austin, Texas, Dublin and Singapore. Seated at computers in pods organized by work assignments, these hourly workers use special Facebook software to sift through millions of private messages, images and videos. They pass judgment on whatever flashes on their screen — claims of everything from fraud or spam to child porn and potential terrorist plotting — typically in less than a minute.
The workers have access to only a subset of WhatsApp messages — those flagged by users and automatically forwarded to the company as possibly abusive. The review is one element in a broader monitoring operation in which the company also reviews material that is not encrypted, including data about the sender and their account.
Policing users while assuring them that their privacy is sacrosanct makes for an awkward mission at WhatsApp. A 49-slide internal company marketing presentation from December, obtained by ProPublica, emphasizes the “fierce” promotion of WhatsApp’s “privacy narrative.” It compares its “brand character” to “the Immigrant Mother” and displays a photo of Malala Yousafzai, who survived a shooting by the Taliban and became a Nobel Peace Prize winner, in a slide titled “Brand tone parameters.” The presentation does not mention the company’s content moderation efforts.
WhatsApp’s director of communications, Carl Woog, acknowledged that teams of contractors in Austin and elsewhere review WhatsApp messages to identify and remove “the worst” abusers. But Woog told ProPublica that the company does not consider this work to be content moderation, saying: “We actually don’t typically use the term for WhatsApp.” The company declined to make executives available for interviews for this article, but responded to questions with written comments. “WhatsApp is a lifeline for millions of people around the world,” the company said. “The decisions we make around how we build our app are focused around the privacy of our users, maintaining a high degree of reliability and preventing abuse.”
WhatsApp’s denial that it moderates content is noticeably different from what Facebook Inc. says about WhatsApp’s corporate siblings, Instagram and Facebook. The company has said that some 15,000 moderators examine content on Facebook and Instagram, neither of which is encrypted. It releases quarterly transparency reports that detail how many accounts Facebook and Instagram have “actioned” for various categories of abusive content. There is no such report for WhatsApp.
Deploying an army of content reviewers is just one of the ways that Facebook Inc. has compromised the privacy of WhatsApp users. Together, the company’s actions have left WhatsApp — the largest messaging app in the world, with two billion users — far less private than its users likely understand or expect. A ProPublica investigation, drawing on data, documents and dozens of interviews with current and former employees and contractors, reveals how, since purchasing WhatsApp in 2014, Facebook has quietly undermined its sweeping security assurances in multiple ways. (Twoarticles this summer noted the existence of WhatsApp’s moderators but focused on their working conditions and pay rather than their effect on users’ privacy. This article is the first to reveal the details and extent of the company’s ability to scrutinize messages and user data — and to examine what the company does with that information.)
Many of the assertions by content moderators working for WhatsApp are echoed by a confidential whistleblower complaint filed last year with the U.S. Securities and Exchange Commission. The complaint, which ProPublica obtained, details WhatsApp’s extensive use of outside contractors, artificial intelligence systems and account information to examine user messages, images and videos. It alleges that the company’s claims of protecting users’ privacy are false. “We haven’t seen this complaint,” the company spokesperson said. The SEC has taken no public action on it; an agency spokesperson declined to comment.
Facebook Inc. has also downplayed how much data it collects from WhatsApp users, what it does with it and how much it shares with law enforcement authorities. For example, WhatsApp shares metadata, unencrypted records that can reveal a lot about a user’s activity, with law enforcement agencies such as the Department of Justice. Some rivals, such as Signal, intentionally gather much less metadata to avoid incursions on its users’ privacy, and thus share far less with law enforcement. (“WhatsApp responds to valid legal requests,” the company spokesperson said, “including orders that require us to provide on a real-time going forward basis who a specific person is messaging.”)
WhatsApp user data, ProPublica has learned, helped prosecutors build a high-profile case against a Treasury Department employee who leaked confidential documents to BuzzFeed News that exposed how dirty money flows through U.S. banks.
Like other social media and communications platforms, WhatsApp is caught between users who expect privacy and law enforcement entities that effectively demand the opposite: that WhatsApp turn over information that will help combat crime and online abuse. WhatsApp has responded to this dilemma by asserting that it’s no dilemma at all. “I think we absolutely can have security and safety for people through end-to-end encryption and work with law enforcement to solve crimes,” said Will Cathcart, whose title is Head of WhatsApp, in a YouTube interview with an Australian think tank in July.
The tension between privacy and disseminating information to law enforcement is exacerbated by a second pressure: Facebook’s need to make money from WhatsApp. Since paying $22 billion to buy WhatsApp in 2014, Facebook has been trying to figure out how to generate profits from a service that doesn’t charge its users a penny.
That conundrum has periodically led to moves that anger users, regulators or both. The goal of monetizing the app was part of the company’s 2016 decision to start sharing WhatsApp user data with Facebook, something the company had told European Union regulators was technologically impossible. The same impulse spurred a controversial plan, abandoned in late 2019, to sell advertising on WhatsApp. And the profit-seeking mandate was behind another botched initiative in January: the introduction of a new privacy policy for user interactions with businesses on WhatsApp, allowing businesses to use customer data in new ways. That announcement triggered a user exodus to competing apps.
WhatsApp’s increasingly aggressive business plan is focused on charging companies for an array of services — letting users make payments via WhatsApp and managing customer service chats — that offer convenience but fewer privacy protections. The result is a confusing two-tiered privacy system within the same app where the protections of end-to-end encryption are further eroded when WhatsApp users employ the service to communicate with businesses.
The company’s December marketing presentation captures WhatsApp’s diverging imperatives. It states that “privacy will remain important.” But it also conveys what seems to be a more urgent mission: the need to “open the aperture of the brand to encompass our future business objectives.”
I. “Content Moderation Associates”
In many ways, the experience of being a content moderator for WhatsApp in Austin is identical to being a moderator for Facebook or Instagram, according to interviews with 29 current and former moderators. Mostly in their 20s and 30s, many with past experience as store clerks, grocery checkers and baristas, the moderators are hired and employed by Accenture, a huge corporate contractor that works for Facebook and other Fortune 500 behemoths.
The job listings advertise “Content Review” positions and make no mention of Facebook or WhatsApp. Employment documents list the workers’ initial title as “content moderation associate.” Pay starts around $16.50 an hour. Moderators are instructed to tell anyone who asks that they work for Accenture, and are required to sign sweeping non-disclosure agreements. Citing the NDAs, almost all the current and former moderators interviewed by ProPublica insisted on anonymity. (An Accenture spokesperson declined comment, referring all questions about content moderation to WhatsApp.)
When the WhatsApp team was assembled in Austin in 2019, Facebook moderators already occupied the fourth floor of an office tower on Sixth Street, adjacent to the city’s famous bar-and-music scene. The WhatsApp team was installed on the floor above, with new glass-enclosed work pods and nicer bathrooms that sparked a tinge of envy in a few members of the Facebook team. Most of the WhatsApp team scattered to work from home during the pandemic. Whether in the office or at home, they spend their days in front of screens, using a Facebook software tool to examine a stream of “tickets,” organized by subject into “reactive” and “proactive” queues.
Collectively, the workers scrutinize millions of pieces of WhatsApp content each week. Each reviewer handles upwards of 600 tickets a day, which gives them less than a minute per ticket. WhatsApp declined to reveal how many contract workers are employed for content review, but a partial staffing list reviewed by ProPublica suggests that, at Accenture alone, it’s more than 1,000. WhatsApp moderators, like their Facebook and Instagram counterparts, are expected to meet performance metrics for speed and accuracy, which are audited by Accenture.
Their jobs differ in other ways. Because WhatsApp’s content is encrypted, artificial intelligence systems can’t automatically scan all chats, images and videos, as they do on Facebook and Instagram. Instead, WhatsApp reviewers gain access to private content when users hit the “report” button on the app, identifying a message as allegedly violating the platform’s terms of service. This forwards five messages — the allegedly offending one along with the four previous ones in the exchange, including any images or videos — to WhatsApp in unscrambled form, according to former WhatsApp engineers and moderators. Automated systems then feed these tickets into “reactive” queues for contract workers to assess.
Artificial intelligence initiates a second set of queues — so-called proactive ones — by scanning unencrypted data that WhatsApp collects about its users and comparing it against suspicious account information and messaging patterns (a new account rapidly sending out a high volume of chats is evidence of spam), as well as terms and images that have previously been deemed abusive. The unencrypted data available for scrutiny is extensive. It includes the names and profile images of a user’s WhatsApp groups as well as their phone number, profile photo, status message, phone battery level, language and time zone, unique mobile phone ID and IP address, wireless signal strength and phone operating system, as a list of their electronic devices, any related Facebook and Instagram accounts, the last time they used the app and any previous history of violations.
The WhatsApp reviewers have three choices when presented with a ticket for either type of queue: Do nothing, place the user on “watch” for further scrutiny, or ban the account. (Facebook and Instagram content moderators have more options, including removing individual postings. It’s that distinction — the fact that WhatsApp reviewers can’t delete individual items — that the company cites as its basis for asserting that WhatsApp reviewers are not “content moderators.”)
WhatsApp moderators must make subjective, sensitive and subtle judgments, interviews and documents examined by ProPublica show. They examine a wide range of categories, including “Spam Report,” “Civic Bad Actor” (political hate speech and disinformation), “Terrorism Global Credible Threat,” “CEI” (child exploitative imagery) and “CP” (child pornography). Another set of categories addresses the messaging and conduct of millions of small and large businesses that use WhatsApp to chat with customers and sell their wares. These queues have such titles as “business impersonation prevalence,” “commerce policy probable violators” and “business verification.”
Moderators say the guidance they get from WhatsApp and Accenture relies on standards that can be simultaneously arcane and disturbingly graphic. Decisions about abusive sexual imagery, for example, can rest on an assessment of whether a naked child in an image appears adolescent or prepubescent, based on comparison of hip bones and pubic hair to a medical index chart. One reviewer recalled a grainy video in a political-speech queue that depicted a machete-wielding man holding up what appeared to be a severed head: “We had to watch and say, ‘Is this a real dead body or a fake dead body?’”
In late 2020, moderators were informed of a new queue for alleged “sextortion.” It was defined in an explanatory memo as “a form of sexual exploitation where people are blackmailed with a nude image of themselves which have been shared by them or someone else on the Internet.” The memo said workers would review messages reported by users that “include predefined keywords typically used in sextortion/blackmail messages.”
WhatsApp’s review system is hampered by impediments, including buggy language translation. The service has users in 180 countries, with the vast majority located outside the U.S. Even though Accenture hires workers who speak a variety of languages, for messages in some languages there’s often no native speaker on site to assess abuse complaints. That means using Facebook’s language-translation tool, which reviewers said could be so inaccurate that it sometimes labeled messages in Arabic as being in Spanish. The tool also offered little guidance on local slang, political context or sexual innuendo. “In the three years I’ve been there,” one moderator said, “it’s always been horrible.”
The process can be rife with errors and misunderstandings. Companies have been flagged for offering weapons for sale when they’re selling straight shaving razors. Bras can be sold, but if the marketing language registers as “adult,” the seller can be labeled a forbidden “sexually oriented business.” And a flawed translation tool set off an alarm when it detected kids for sale and slaughter, which, upon closer scrutiny, turned out to involve young goats intended to be cooked and eaten in halal meals.
The system is also undercut by the human failings of the people who instigate reports. Complaints are frequently filed to punish, harass or prank someone, according to moderators. In messages from Brazil and Mexico, one moderator explained, “we had a couple of months where AI was banning groups left and right because people were messing with their friends by changing their group names” and then reporting them. “At the worst of it, we were probably getting tens of thousands of those. They figured out some words the algorithm did not like.”
Other reports fail to meet WhatsApp standards for an account ban. “Most of it is not violating,” one of the moderators said. “It’s content that is already on the internet, and it’s just people trying to mess with users.” Still, each case can reveal up to five unencrypted messages, which are then examined by moderators.
The judgment of WhatsApp’s AI is less than perfect, moderators say. “There were a lot of innocent photos on there that were not allowed to be on there,” said Carlos Sauceda, who left Accenture last year after nine months. “It might have been a photo of a child taking a bath, and there was nothing wrong with it.” As another WhatsApp moderator put it, “A lot of the time, the artificial intelligence is not that intelligent.”
Facebook’s written guidance to WhatsApp moderators acknowledges many problems, noting “we have made mistakes and our policies have been weaponized by bad actors to get good actors banned. When users write inquiries pertaining to abusive matters like these, it is up to WhatsApp to respond and act (if necessary) accordingly in a timely and pleasant manner.” Of course, if a user appeals a ban that was prompted by a user report, according to one moderator, it entails having a second moderator examine the user’s content.
II. “Industry Leaders” in Detecting Bad Behavior
In public statements and on the company’s websites, Facebook Inc. is noticeably vague about WhatsApp’s monitoring process. The company does not provide a regular accounting of how WhatsApp polices the platform. WhatsApp’s FAQ page and online complaint form note that it will receive “the most recent messages” from a user who has been flagged. They do not, however, disclose how many unencrypted messages are revealed when a report is filed, or that those messages are examined by outside contractors. (WhatsApp told ProPublica it limits that disclosure to keep violators from “gaming” the system.)
By contrast, both Facebook and Instagram post lengthy “Community Standards” documents detailing the criteria its moderators use to police content, along with articles and videos about “the unrecognized heroes who keep Facebook safe” and announcements on new content-review sites. Facebook’s transparency reports detail how many pieces of content are “actioned” for each type of violation. WhatsApp is not included in this report.
When dealing with legislators, Facebook Inc. officials also offer few details — but are eager to assure them that they don’t let encryption stand in the way of protecting users from images of child sexual abuse and exploitation. For example, when members of the Senate Judiciary Committee grilled Facebook about the impact of encrypting its platforms, the company, in written follow-up questions in Jan. 2020, cited WhatsApp in boasting that it would remain responsive to law enforcement. “Even within an encrypted system,” one response noted, “we will still be able to respond to lawful requests for metadata, including potentially critical location or account information… We already have an encrypted messaging service, WhatsApp, that — in contrast to some other encrypted services — provides a simple way for people to report abuse or safety concerns.”
Sure enough, WhatsApp reported 400,000 instances of possible child-exploitation imagery to the National Center for Missing and Exploited Children in 2020, according to its head, Cathcart. That was ten times as many as in 2019. “We are by far the industry leaders in finding and detecting that behavior in an end-to-end encrypted service,” he said.
During his YouTube interview with the Australian think tank, Cathcart also described WhatsApp’s reliance on user reporting and its AI systems’ ability to examine account information that isn’t subject to encryption. Asked how many staffers WhatsApp employed to investigate abuse complaints from an app with more than two billion users, Cathcart didn’t mention content moderators or their access to encrypted content. “There’s a lot of people across Facebook who help with WhatsApp,” he explained. “If you look at people who work full time on WhatsApp, it’s above a thousand. I won’t get into the full breakdown of customer service, user reports, engineering, etc. But it’s a lot of that.”
In written responses for this article, the company spokesperson said: “We build WhatsApp in a manner that limits the data we collect while providing us tools to prevent spam, investigate threats, and ban those engaged in abuse, including based on user reports we receive. This work takes extraordinary effort from security experts and a valued trust and safety team that works tirelessly to help provide the world with private communication.” The spokesperson noted that WhatsApp has released new privacy features, including “more controls about how people’s messages can disappear” or be viewed only once. He added, “Based on the feedback we’ve received from users, we’re confident people understand when they make reports to WhatsApp we receive the content they send us.”
III. “Deceiving Users” About Personal Privacy
Since the moment Facebook announced plans to buy WhatsApp in 2014, observers wondered how the service, known for its fervent commitment to privacy, would fare inside a corporation known for the opposite. Zuckerberg had become one of the wealthiest people on the planet by using a “surveillance capitalism” approach: collecting and exploiting reams of user data to sell targeted digital ads. Facebook’s relentless pursuit of growth and profits has generated a series of privacy scandals in which it was accused of deceiving customers and regulators.
By contrast, WhatsApp knew little about its users apart from their phone numbers and shared none of that information with third parties. WhatsApp ran no ads, and its co-founders, Jan Koum and Brian Acton, both former Yahoo engineers, were hostile to them. “At every company that sells ads,” they wrote in 2012, “a significant portion of their engineering team spends their day tuning data mining, writing better code to collect all your personal data, upgrading the servers that hold all the data and making sure it’s all being logged and collated and sliced and packed and shipped out,” adding: “Remember, when advertising is involved you the user are the product.” At WhatsApp, they noted, “your data isn’t even in the picture. We are simply not interested in any of it.”
Zuckerberg publicly vowed in a 2014 keynote speech that he would keep WhatsApp “exactly the same.” He declared, “We are absolutely not going to change plans around WhatsApp and the way it uses user data. WhatsApp is going to operate completely autonomously.”
In April 2016, WhatsApp completed its long-planned adoption of end-to-end encryption, which helped establish the app as a prized communications platform in 180 countries, including many where text messages and phone calls are cost-prohibitive. International dissidents, whistleblowers and journalists also turned to WhatsApp to escape government eavesdropping.
Four months later, however, WhatsApp disclosed it would begin sharing user data with Facebook — precisely what Zuckerberg had said would not happen — a move that cleared the way for an array of future revenue-generating plans. The new WhatsApp terms of service said the app would share information such as users’ phone numbers, profile photos, status messages and IP addresses for the purposes of ad targeting, fighting spam and abuse and gathering metrics. “By connecting your phone number with Facebook’s systems,” WhatsApp explained, “Facebook can offer better friend suggestions and show you more relevant ads if you have an account with them.”
Such actions were increasingly bringing Facebook into the crosshairs of regulators. In May 2017, European Union antitrust regulators fined the company 110 million euros (about $122 million) for falsely claiming three years earlier that it would be impossible to link the user information between WhatsApp and the Facebook family of apps. The EU concluded that Facebook had “intentionally or negligently” deceived regulators. Facebook insisted its false statements in 2014 were not intentional, but didn’t contest the fine.
By the spring of 2018, the WhatsApp co-founders, now both billionaires, were gone. Acton, in what he later described as an act of “penance” for the “crime” of selling WhatsApp to Facebook, gave $50 million to a foundation backing Signal, a free encrypted messaging app that would emerge as a WhatsApp rival. (Acton’s donor-advised fund has also given money to ProPublica.)
Meanwhile, Facebook was under fire for its security and privacy failures as never before. The pressure culminated in a landmark $5 billion fine by the Federal Trade Commission in July 2019 for violating a previous agreement to protect user privacy. The fine was almost 20 times greater than any previous privacy-related penalty, according to the FTC, and Facebook’s transgressions included “deceiving users about their ability to control the privacy of their personal information.”
The FTC announced that it was ordering Facebook to take steps to protect privacy going forward, including for WhatsApp users: “As part of Facebook’s order-mandated privacy program, which covers WhatsApp and Instagram, Facebook must conduct a privacy review of every new or modified product, service, or practice before it is implemented, and document its decisions about user privacy.” Compliance officers would be required to generate a “quarterly privacy review report” and share it with the company and, upon request, the FTC.
Facebook agreed to the FTC’s fine and order. Indeed, the negotiations for that agreement were the backdrop, just four months before that, for Zuckerberg’s announcement of his new commitment to privacy.
By that point, WhatsApp had begun using Accenture and other outside contractors to hire hundreds of content reviewers. But the company was eager not to step on its larger privacy message — or spook its global user base. It said nothing publicly about its hiring of contractors to review content.
IV. “We Kill People Based On Metadata”
Even as Zuckerberg was touting Facebook Inc.’s new commitment to privacy in 2019, he didn’t mention that his company was apparently sharing more of its WhatsApp users’ metadata than ever with the parent company — and with law enforcement.
To the lay ear, the term “metadata” can sound abstract, a word that evokes the intersection of literary criticism and statistics. To use an old, pre-digital analogy, metadata is the equivalent of what’s written on the outside of an envelope — the names and addresses of the sender and recipient and the postmark reflecting where and when it was mailed — while the “content” is what’s written on the letter sealed inside the envelope. So it is with WhatsApp messages: The content is protected, but the envelope reveals a multitude of telling details (as noted: time stamps, phone numbers and much more).
Those in the information and intelligence fields understand how crucial this information can be. It was metadata, after all, that the National Security Agency was gathering about millions of Americans not suspected of a crime, prompting a global outcry when it was exposed in 2013 by former NSA contractor Edward Snowden. “Metadata absolutely tells you everything about somebody’s life,” former NSA general counsel Stewart Baker once said. “If you have enough metadata, you don’t really need content.” In a symposium at Johns Hopkins University in 2014, Gen. Michael Hayden, former director of both the CIA and NSA, went even further: “We kill people based on metadata.”
U.S. law enforcement has used WhatsApp metadata to help put people in jail. ProPublica found more than a dozen instances in which the Justice Department sought court orders for the platform’s metadata since 2017. These represent a fraction of overall requests, known as pen register orders (a phrase borrowed from the technology used to track numbers dialed by landline telephones), as many more are kept from public view by court order. U.S. government requests for data on outgoing and incoming messages from all Facebook platforms increased by 276% from the first half of 2017 to the second half of 2020, according to Facebook Inc. statistics (which don’t break out the numbers by platform). The company’s rate of handing over at least some data in response to such requests has risen from 84% to 95% during that period.
It’s not clear exactly what government investigators have been able to gather from WhatsApp, as the results of those orders, too, are often kept from public view. Internally, WhatsApp calls such requests for information about users “prospective message pairs,” or PMPs. These provide data on a user’s messaging patterns in response to requests from U.S. law enforcement agencies, as well as those in at least three other countries — the United Kingdom, Brazil and India — according to a person familiar with the matter who shared this information on condition of anonymity. Law enforcement requests from other countries might only receive basic subscriber profile information.
WhatsApp metadata was pivotal in the arrest and conviction of Natalie “May” Edwards, a former Treasury Department official with the Financial Crimes Enforcement Network, for leaking confidential banking reports about suspicious transactions to BuzzFeed News. The FBI’s criminal complaint detailed hundreds of messages between Edwards and a BuzzFeed reporter using an “encrypted application,” which interviews and court records confirmed was WhatsApp. “On or about August 1, 2018, within approximately six hours of the Edwards pen becoming operative — and the day after the July 2018 Buzzfeed article was published — the Edwards cellphone exchanged approximately 70 messages via the encrypted application with the Reporter-1 cellphone during an approximately 20-minute time span between 12:33 a.m. and 12:54 a.m.,” FBI Special Agent Emily Eckstut wrote in her October 2018 complaint. Edwards and the reporter used WhatsApp because Edwards believed the platform to be secure, according to a person familiar with the matter.
Edwards was sentenced on June 3 to six months in prison after pleading guilty to a conspiracy charge and reported to prison last week. Edwards’ attorney declined to comment, as did representatives from the FBI and the Justice Department.
WhatsApp has for years downplayed how much unencrypted information it shares with law enforcement, largely limiting mentions of the practice to boilerplate language buried deep in its terms of service. It does not routinely keep permanent logs of who users are communicating with and how often, but company officials confirmed they do turn on such tracking at their own discretion — even for internal Facebook leak investigations — or in response to law enforcement requests. The company declined to tell ProPublica how frequently it does so.
The privacy page for WhatsApp assures users that they have total control over their own metadata. It says users can “decide if only contacts, everyone, or nobody can see your profile photo” or when they last opened their status updates or when they last opened the app. Regardless of the settings a user chooses, WhatsApp collects and analyzes all of that data — a fact not mentioned anywhere on the page.
V. “Opening the Aperture to Encompass Business Objectives”
The conflict between privacy and security on encrypted platforms seems to be only intensifying. Law enforcement and child safety advocates have urged Zuckerberg to abandon his plan to encrypt all of Facebook’s messaging platforms. In June 2020, three Republican senators introduced the “Lawful Access to Encrypted Data Act,” which would require tech companies to assist in providing access to even encrypted content in response to law enforcement warrants. For its part, WhatsApp recently sued the Indian government to block its requirement that encrypted apps provide “traceability” — a method to identify the sender of any message deemed relevant to law enforcement. WhatsApp has fought similar demands in other countries.
Other encrypted platforms take a vastly different approach to monitoring their users than WhatsApp. Signal employs no content moderators, collects far less user and group data, allows no cloud backups and generally rejects the notion that it should be policing user activities. It submits no child exploitation reports to NCMEC.
Apple has touted its commitment to privacy as a selling point. Its iMessage system displays a “report” button only to alert the company to suspected spam, and the company has made just a few hundred annual reports to NCMEC, all of them originating from scanning outgoing email, which is unencrypted.
But Apple recently took a new tack, and appeared to stumble along the way. Amid intensifying pressure from Congress, in August the company announced a complex new system for identifying child-exploitative imagery on users’ iCloud backups. Apple insisted the new system poses no threat to private content, but privacy advocates accused the company of creating a backdoor that potentially allows authoritarian governments to demand broader content searches, which could result in the targeting of dissidents, journalists or other critics of the state. On Sept. 3, Apple announced it would delay implementation of the new system.
Still, it’s Facebook that seems to face the most constant skepticism among major tech platforms. It is using encryption to market itself as privacy-friendly, while saying little about the other ways it collects data, according to Lloyd Richardson, the director of IT at the Canadian Centre for Child Protection. “This whole idea that they’re doing it for personal protection of people is completely ludicrous,” Richardson said. “You’re trusting an app owned and written by Facebook to do exactly what they’re saying. Do you trust that entity to do that?” (On Sept. 2, Irish authorities announced that they are fining WhatsApp 225 million euros, about $267 million, for failing to properly disclose how the company shares user information with other Facebook platforms. WhatsApp is contesting the finding.)
Facebook’s emphasis on promoting WhatsApp as a paragon of privacy is evident in the December marketing document obtained by ProPublica. The “Brand Foundations” presentation says it was the product of a 21-member global team across all of Facebook, involving a half-dozen workshops, quantitative research, “stakeholder interviews” and “endless brainstorms.” Its aim: to offer “an emotional articulation” of WhatsApp’s benefits, “an inspirational toolkit that helps us tell our story,” and a “brand purpose to champion the deep human connection that leads to progress.” The marketing deck identifies a feeling of “closeness” as WhatsApp’s “ownable emotional territory,” saying the app delivers “the closest thing to an in-person conversation.”
WhatsApp should portray itself as “courageous,” according to another slide, because it’s “taking a strong, public stance that is not financially motivated on things we care about,” such as defending encryption and fighting misinformation. But the presentation also speaks of the need to “open the aperture of the brand to encompass our future business objectives. While privacy will remain important, we must accommodate for future innovations.”
WhatsApp is now in the midst of a major drive to make money. It has experienced a rocky start, in part because of broad suspicions of how WhatsApp will balance privacy and profits. An announced plan to begin running ads inside the app didn’t help; it was abandoned in late 2019, just days before it was set to launch. Early this January, WhatsApp unveiled a change in its privacy policy — accompanied by a one-month deadline to accept the policy or get cut off from the app. The move sparked a revolt, impelling tens of millions of users to flee to rivals such as Signal and Telegram.
The policy change focused on how messages and data would be handled when users communicate with a business in the ever-expanding array of WhatsApp Business offerings. Companies now could store their chats with users and use information about users for marketing purposes, including targeting them with ads on Facebook or Instagram.
Elon Musk tweeted “Use Signal,” and WhatsApp users rebelled. Facebook delayed for three months the requirement for users to approve the policy update. In the meantime, it struggled to convince users that the change would have no effect on the privacy protections for their personal communications, with a slightly modified version of its usual assurance: “WhatsApp cannot see your personal messages or hear your calls and neither can Facebook.” Just as when the company first bought WhatsApp years before, the message was the same: Trust us.
Gerade macht eine Nachricht aus den USA die Runde: Das Investigativmagazin ProPublica widmet dem Datenschutz bei WhatsApp einen ausführlichen Artikel und kommt zu dem Schluss, das Mutterunternehmen Facebook untergrabe die Privatsphäre der zwei Milliarden Nutzer:innen. So richtig diese Aussage ist, so problematisch ist das Framing der Autoren und vieler deutscher Medien, die die Meldung oberflächlich aufgreifen.
Im Hauptteil des Artikels geht es darum, dass Facebook ein Heer von Content-Moderator:innen beschäftigt, um gemeldete Inhalte in WhatsApp-Chats zu überprüfen. Das ist keine Neuigkeit, aber ProPublica kann erstmals ausführlicher darüber berichten, wie diese Arbeit abläuft. Dass potenziell jede WhatsApp-Nachricht von den Moderator:innen des Konzerns gelesen werden kann, stellen die Autoren dem Privacy-Versprechen des Messengers gegenüber: „No one outside of this chat, not even WhatsApp, can read or listen to them.”
Allerdings, und hier wird es problematisch, setzen die Autoren dann auf ein Framing, dass die Content-Moderation (die WhatsApp nicht so nennen will) als Schwächung der Ende-zu-Ende-Verschlüsselung darstellt. Ein ProPublica-Autor bezeichnete die Moderation sogar als „Backdoor“, was gemeinhin eine gezielt eingebaute Hintertür zum Umgehen von Verschlüsselung meint. Diverse Sicherheitsexpert:innen wie die Cybersecurity-Direktorin der Electronic Frontier Foundation, Eva Galperin, kritisieren deshalb die Berichterstattung.
Die Verschlüsselung tut, was sie soll
Wo also liegt das Problem? Klar ist: Mark Zuckerbergs 2018 gegebenes Versprechen, dass seine Firma keinerlei Kommunikationsinhalte aus WhatsApp-Chats lesen könne, ist irreführend. Jede Nachricht, jedes Bild und jedes Video, die von Chat-Teilnehmer:innen gemeldet werden, landen zur Überprüfung bei WhatsApp und deren Dienstleistern. Etwa 1000 Menschen seien in Austin, Dublin und Singapur rund um die Uhr im Einsatz, um die gemeldeten Inhalte zu sichten, berichtet ProPublica. Weil das Unternehmen das Privacy-Versprechen für sein Marketing benötigt, versteckt WhatsApp diese Info vor seinen Nutzer:innen.
Klar ist auch: Wie jede Form der Inhaltemoderation bringt dies erhebliche Probleme mit sich. So zeigen die Autoren nach Gesprächen mit diversen Quellen etwa, dass die Moderator:innen wenig Zeit für ihre schwerwiegenden Entscheidungen haben und mit teils missverständlichen Vorgaben arbeiten müssen. Wie bei der Moderation für Facebook und Instagram werden sie zudem von einem automatisierten System unterstützt, das mitunter fehlerhafte Vorschläge macht. Deshalb werden immer wieder Inhalte gesperrt, die eigentlich nicht gesperrt werden dürften, etwa harmlose Fotos oder Satire. Einen ordentlichen Widerspruchsmechanismus gibt es bei WhatsApp nicht und es ist ein Verdienst des Artikels, diese Schwierigkeiten ans Licht zu bringen.
Diese Probleme liegen jedoch nicht an einer mangelhaften Ende-zu-Ende-Verschlüsselung der WhatsApp-Nachrichten. Diese funktioniert technisch gesehen weiterhin gut. Die Nachrichten sind zunächst nur auf den Geräten der Kommunikationsteilnehmer:innen lesbar (sofern diese nicht durch kriminelle oder staatliche Hacker kompromittiert wurden). Die Nutzer:innen, die Inhalte aus Chats melden, leiten diese an WhatsApp weiter. Das kann jede:r tun und ist kein Verschlüsselungsproblem.
Die eigentliche Gefahr liegt woanders
Die Möglichkeit, missbräuchliche Inhalte zu melden, besteht bei WhatsApp schon seit Längerem. Das Meldesystem soll helfen, wenn etwa volksverhetztende Inhalte geteilt werden, Ex-Partner:innen bedroht oder in Gruppen zur Gewalt gegen Minderheiten aufgerufen wird. Es ist zwar ein Eingriff in private Kommunikation, aber man kann argumentieren, dass dieser in Abwägung mit den Gefahren gerechtfertigt ist. Selbstverständlich wäre WhatsApp in der Pflicht, seine Nutzer:innen besser darüber informieren, wie das Meldesystem funktioniert und dass ihre Nachrichten mit ein paar Klicks an Moderator:innen weitergeleitet werden können.
Die größere Gefahr für die Privatsphäre bei WhatsApp kommt jedoch von einer anderen Stelle: Es sind die Metadaten, die über Menschen ähnlich viel verraten wie die Inhalte ihrer Gespräche. Dazu gehört die Identität von Absender und Empfänger, ihre Telefonnummern und zugehörige Facebook-Konten, Profilfotos, Statusnachrichten sowie Akkustand des Telefons. Außerdem Informationen zum Kommunikationsverhalten: Wer kommuniziert mit wem? Wer nutzt die App wie häufig und wie lange?
Wie WhatsApp eine Whistleblowerin ans Messer lieferte
WhatsApp sammelt diese Daten im großen Stil, weil sie sich zu Geld machen lassen. Im Originalbericht von ProPublica kommt dieser Aspekt durchaus vor, in vielen deutschen Meldungen geht er leider unter. Tatsächlich berichtet das US-Medium sogar vom Fall einer Whistleblowerin, die ins Gefängnis musste, weil WhatsApp ihre Metadaten an das FBI weitergab. Natalie Edwards war im US-Finanzministerium angestellt und reichte Informationen über verdächtige Transaktionen an BuzzFeed News weiter. Entdeckt und verurteilt wurde sie unter anderem, weil die Strafverfolger nachweisen konnten, dass sie in regem WhatsApp-Kontakt mit dem BuzzFeed-Reporter stand.
Dem Bericht zufolge gibt WhatsApp in den USA derlei Metadaten regelmäßig an Ermittlungsbehörden weiter. Auch in Deutschland und Europa dürfte dies der Fall sein. Hinzukommt, dass nicht nur staatliche Stellen die verräterischen Informationen erhalten, sondern auch Facebook. Dort werden sie genutzt, um die Datenprofile der Nutzer:innen zu verfeinern und in weiten Teilen der Welt auch, um Werbeanzeigen besser zuschneiden zu können. Als der Datenkonzern den Messenger 2014 aufkaufte, versprach er der europäischen Wettbewerbsbehörde, dass dies technisch überhaupt nicht möglich sei. Eine dreiste Lüge, für die das Unternehmen mehr als 100 Millionen Euro Strafe zahlen musste.
On Oct. 28, hours after completing his $44 billion buyout of Twitter the night before, Mr. Musk gathered several human-resource executives in a “war room” in the company’s offices in San Francisco. Prepare for widespread layoffs, he told them, six people with knowledge of the discussion said. Twitter’s work force needed to be slashed immediately, he said, and those who were cut would not receive bonuses that were set to be paid on Nov. 1.
The executives warned their new boss that his plan could violate employment laws and breach contracts with workers, leading to employee lawsuits, the people said. But Mr. Musk’s team said he was used to going to court and paying penalties, and was not worried about the risks. So Twitter’s human-resource, accounting and legal departments scrambled to figure out how to comply with his command.
Two days later, Mr. Musk learned exactly how costly those potential fines and lawsuits could be, three people said. Delays were also piling up as managers haggled over which employees to let go. He decided to wait on cutting jobs until after Nov. 1.
The order for immediate layoffs, the ensuing panic and the about-face reflect the chaos that has engulfed Twitter since Mr. Musk took over the company two weeks ago. The 51-year-old barreled in with ideas about how the social media service should operate, but with no comprehensive plan to execute them. Then he quickly ran into the business, legal and financial complexities of running a platform that has been called a global town square.
The fallout has often been excruciating, according to 36 current and former Twitter employees and people close to the company, as well as internal documents and workplace chat logs. Some top executives were summarily fired by email. One engineering manager, upon being told to cut hundreds of workers, vomited into a trash can. Others slept in the office as they worked grueling schedules to meet Mr. Musk’s orders.
Twitter, which is under financial pressure from debt and a slumping economy, is now unrecognizable compared with what it was a month ago. Last week, Mr. Musk slashed 50 percent of the company’s 7,500 employees. Executive resignations have continued. Misinformation proliferated on the platform during Tuesday’s midterm elections. A key project to expand revenue from subscriptions hit snags. Some advertisers have been aghast.
Mr. Musk, who did not respond to a request for comment, told employees in a meeting on Thursday that Twitter’s situation was grim.
“There’s a massive negative cash flow, and bankruptcy is not out of the question,” he said, according to a recording heard by The New York Times.
Mr. Musk added that they would need to work strenuously to keep the company afloat. “Those who are able to go hard core and play to win, Twitter is a good place,” he said. “And those who are not, totally understand, but then Twitter is not for you.”
Image
Elon Musk posted a video of his entrance to Twitter headquarters on Oct. 26.Credit…Twitter, via Associated Press
‘Let that sink in!’
Mr. Musk arrived at Twitter’s San Francisco offices on Oct. 26, toting a white porcelain sink through the glass doors of the building. “Let that sink in!” he tweeted at the time, along with a video of his grand entrance.
Leslie Berland, Twitter’s chief marketing officer, encouraged employees to say hi to Mr. Musk and escorted him through the office. He was seen chatting with employees at the company coffee bar.
But the vibe quickly changed. The next day, Parag Agrawal, Twitter’s chief executive, and Ned Segal, the chief financial officer, were in the office, two people familiar with the situation said. Once they knew Mr. Musk’s acquisition of Twitter was closing that afternoon, they left the building, uncertain what the new owner would do.
Mr. Agrawal and Mr. Segal soon received emails saying they had been fired, two people familiar with the situation said. Vijaya Gadde, Twitter’s top legal and policy executive, and Sean Edgett, the general counsel, were also fired. Mr. Edgett, who was in Twitter’s offices at the time, was escorted out.
Image
Ned SegalCredit…Drew Angerer/Getty Images
Image
Parag AgrawalCredit…Kevin Dietsch/Getty Images
That evening, Twitter hosted a Halloween party called “Trick or Tweet” for employees and their families. Some workers dressed in costume and tried to keep the mood festive. Others cried and hugged one another.
Mr. Musk had brought his own advisers, many of whom had worked at his other businesses, such as the digital payments company PayPal and the electric carmaker Tesla. They parked themselves in the “war room,” on the second floor of a building attached to Twitter’s headquarters. The area, which Twitter used to fete big-spending advertisers and dignitaries, was stocked with company memorabilia.
The advisers included the venture capitalists David Sacks, Jason Calacanis and Sriram Krishnan; Mr. Musk’s personal lawyer Alex Spiro; his financial manager Jared Birchall; and Antonio Gracias, a former Tesla director. Joining in were engineers and others from Tesla; from Mr. Musk’s brain interface start-up, Neuralink; and from his tunneling company, the Boring Company.
At times, Mr. Musk was spotted with his 2-year-old son, X Æ A-12, at Twitter’s office as he greeted employees.
In meetings with Twitter executives, Mr. Musk was direct. At the Oct. 28 meeting with human-resource executives, he said he wanted to reduce the work force immediately, before a Nov. 1 date when employees would receive regularly scheduled retention bonuses in the form of vested stock. Tech companies often compensate employees with regular share grants, earned over time the longer they stay at the firm.
One Twitter team began creating a financial model to show the cost of the layoffs. Another built a model to demonstrate how much more Mr. Musk might pay in legal fees and fines if he proceeded with the rapid cuts, three people said.
On Oct. 30, Mr. Musk received word that the rapid approach could cost millions of dollars more than laying people off with their scheduled bonuses. He agreed to delay, four people said.
But he had a condition. Before paying the bonuses, Mr. Musk insisted on a payroll audit to confirm that Twitter’s employees were “real humans.” He voiced concerns that “ghost employees” who should not receive the money lingered in Twitter’s systems.
Mr. Musk tapped Robert Kaiden, Twitter’s chief accounting officer, to conduct the audit. Mr. Kaiden asked managers to verify that they knew certain employees and could confirm that they were human, according to three people and an internal document seen by The Times.
The Nov. 1 bonus date came and went with no mass layoffs. Mr. Kaiden was fired the next day and marched out of the building, five people with knowledge of the situation said.
A trip to New York
As Twitter managers compiled lists for layoffs, Mr. Musk flew to New York to meet with advertisers, who provide the bulk of Twitter’s revenue.
In some advertiser meetings, Mr. Musk proposed a system for Twitter users to choose the kind of content that the service exposed them to — akin to G to NC-17 movie ratings — implying that brands could then target their advertising on the platform better. He also committed to product improvements and more personalization for users and ads, two people with knowledge of the discussions said.
But his outreach was undercut by the departures of two New York-based Twitter executives — Ms. Berland and JP Maheu, a vice president in charge of advertising. They were well known in the advertising community.
Those Twitter executives “had great relationships with the senior-most people at the Fortune 500 — they were incredibly transparent and inclusive,” said Lou Paskalis, a longtime advertising executive. “Those things engender tremendous trust, and those things are now in question.”
Image
Leslie BerlandCredit…Xavi Torrent/Getty Images
Image
JP MaheuCredit…Astrid Stawiarz/Getty Images
Brands including Volkswagen Group, General Motors and United Airlines have said they will pause advertising on Twitter as they evaluate Mr. Musk’s ownership of the platform.
Mr. Musk elevated some managers at Twitter. He tapped Esther Crawford, a product manager, to revamp a subscription service called Twitter Blue. Mr. Musk wanted a new version of the service, which would cost $8 a month and include premium features and the verification check mark that was previously assigned for free to the accounts of celebrities, journalists and politicians to convey their authenticity.
He laid down a deadline: The team must finish Twitter Blue’s changes by Nov. 7 or its members would be fired.
Last week, Ms. Crawford shared a photo of herself sleeping at Twitter’s San Francisco offices in a sleeping bag and an eye mask, with the hashtag #SleepWhereYouWork.
Her message rubbed some colleagues the wrong way. They wondered in private chats why they should commit long working hours to a man who could fire them, according to five people and messages seen by The Times. On Twitter, Ms. Crawford responded to what she called “hecklers” by saying she had received supportive messages from other entrepreneurs and “builders of all types.”
The ax falls
The scope of layoffs was a moving target. Twitter managers were initially told to cut 25 percent of the work force, three people said. But Tesla engineers who reviewed Twitter’s code proposed deeper cuts to the engineering teams. Executives overseeing other parts of Twitter were told to expand their layoff lists.
Twitter executives also suggested assessing the lists for diversity and inclusion issues so the cuts would not hit people of color disproportionately and to avoid legal trouble. Mr. Musk’s team brushed aside the suggestion, two people said.
On Nov. 2, employees stumbled upon an open channel in the internal Slack messaging system where human resources and legal teams were discussing the layoffs. In a message seen by The Times, one employee said 3,738 workers could be laid off, or about half the work force. The message was widely shared internally.
That evening, Mr. Musk met with some advisers to settle on the reduction, according to a calendar invitation seen by The Times. They were joined by employees from Twitter’s human resources and staff from his other companies.
Anticipating the cuts, employees began bidding farewell to their colleagues, trading phone numbers and connecting on LinkedIn. They also pulled together documents and internal resources to help workers who survived the layoffs.
One engineering manager was approached by Mr. Musk’s advisers — or “goons,” as Twitter employees called them — with a list of hundreds of people he had to let go. He vomited into a trash can near his feet.
Late on Nov. 3, an email landed in employees’ inboxes. “In an effort to place Twitter on a healthy path, we will go through the difficult process of reducing our global work force,” the email, signed “Twitter,” said.
Pandemonium followed. While the note said employees would receive a follow-up email the next morning about whether they still had jobs, many found themselves locked out of email or Slack that night, an indication they had been laid off. Those who remained in Slack posted saluting emojis en masse as a send-off for co-workers.
The cuts were enormous. In Redbird, Twitter’s platform and infrastructure organization, Mr. Musk shed numerous managers. The unit also lost about 80 percent of its engineering staff, raising internal concerns about the company’s ability to keep its site up and running.
In Bluebird, Twitter’s consumer division, dozens of product managers were laid off, leaving just over a dozen of them. The new ratio of engineers to managers was 70 to 1, according to one estimate.
Image
Mr. Musk in New York last Friday, the day after Twitter employees received an email about mass layoffs.Credit…Andrew Kelly/Reuters
The aftermath
As layoffs unfolded, tech recruiters sensed opportunity. Top managers at rival companies such as Meta and Google sent messages to some of the employees being let go from Twitter, said two people who received the notes.
Most of Mr. Musk’s subordinates remained quiet throughout the process. But Mr. Calacanis, the venture capitalist, had been active on Twitter responding to product suggestions and concerns.
Last week, Mr. Musk dispatched a lieutenant to the “war room” to ask Mr. Calacanis, who was there, to cool it on Twitter and stop acting as if he were leading product development or policy, people familiar with the exchange said.
“To be clear, Elon is the product manager and CEO,” Mr. Calacanis later tweeted. “As a power user (and that’s all I am!) I’m really excited.”
By last Saturday, Mr. Musk’s advisers realized that the cuts may have been too deep, four people said. Some asked laid-off engineers, designers and product managers to return to their old jobs, three people familiar with the conversations said. The tech newsletter Platformer earlier reported the outreach.
At Goldbird, Twitter’s revenue division, the company had to bring back those who ran key money-generating products that “no one else knows how to operate,” people with knowledge of the business said. One manager agreed to try rehiring some laid-off workers, but expressed concerns that they were “weak, lazy, unmotivated and they may even be against an Elon Twitter,” two people familiar with the matter said.
On Monday, some Twitter employees arrived at work to find that certain systems they had relied on no longer worked. In San Francisco, an engineer discovered that some contracts with vendors that provide software for managing user data had been put on hold or had expired, and that the managers and executives who could fix the problem had been laid off or resigned.
On Wednesday, workers in Twitter’s New York office were unable to use the Wi-Fi after a server room overheated and knocked it offline, two people said.
Mr. Musk plans to begin making employees pay for lunch — which had been free — at the company cafeteria, two people said.
Image
Jason CalacanisCredit…Christie Hemm Klok for The New York Times
Image
Damien KieranCredit…Joshua Roberts/Reuters
An internal clash
Inside Twitter, some employees have clashed with Mr. Musk’s advisers.
This week, security executives disagreed with Mr. Musk’s team over how Twitter should meet its obligations to the Federal Trade Commission. Twitter had agreed to a settlement with the F.T.C. in 2011 over privacy violations, which requires the company to submit regular reports about its privacy practices and open its doors to audits.
On Wednesday, a day before a deadline for Twitter to submit a report to the F.T.C., Twitter’s chief information security officer, Lea Kissner; chief privacy officer, Damien Kieran; and chief compliance officer, Marianne Fogarty, resigned.
In internal messages later that day, an employee wrote about the resignations and suggested that internal privacy reviews of Twitter’s products were not proceeding as they should under the F.T.C. settlement.
Some engineers could be required to “self-certify” that their projects complied with the settlement, rather than relying on reviews from lawyers and executives, a shift that could lead to “major incidents,” the employee wrote.
“Elon has shown that his only priority with Twitter users is how to monetize them,” the person wrote in the message, which was viewed by The Times.
The employee added that Mr. Spiro, Mr. Musk’s lawyer, had said the billionaire was willing to take risks. Mr. Spiro, the employee said, told workers that “Elon puts rockets into space — he’s not afraid of the F.T.C.”
The F.T.C. said that it was tracking the developments at Twitter with “deep concern” and that “no C.E.O. or company is above the law.” Mr. Musk later sent employees an email saying Twitter will adhere to the F.T.C. settlement.
On Thursday, more Twitter executives resigned, including Kathleen Pacini, a human-resource leader, and Yoel Roth, the head of trust and safety.
At the meeting with employees that day, Mr. Musk tried to sound a note of optimism about Twitter’s future.
“Twitter can form an incredibly valuable service to the world and be the public town square,” he said, noting it should be a “battleground of ideas” where debate could “take the place of violence in a lot of cases.”
Constantly posting content on social media can erode your privacy—and sense of self.Photograph: Luka Milanovic/Getty Images
To be online is to be constantly exposed. While it may seem normal, it’s a level of exposure we’ve never dealt with before as human beings. We’re posting on Twitter, and people we’ve never met are responding with their thoughts and criticisms. People are looking at your latest Instagram selfie. They’re literally swiping on your face. Messages are piling up. It can sometimes feel like the whole world has its eyes on you.Being observed by so many people appears to have significant psychological effects. There are, of course, good things about this ability to connect with others. It was crucial during the height of the pandemic when we couldn’t be close to our loved ones, for example. However, experts say there are also numerous downsides, and these may be more complex and persistent than we realize.Studies have found that high levels of social media use are connected with an increased risk of symptoms of anxiety and depression. There appears to be substantial evidence connecting people’s mental health and their online habits. Furthermore, many psychologists believe people may be dealing with psychological effects that are pervasive but not always obvious.
“What we’re finding is people are spending way more time on screens than previously reported or than they believe they are,” says Larry Rosen, professor emeritus of psychology at California State University, Dominguez Hills. “It’s become somewhat of an epidemic.”Rosen has been studying the psychological effects of technology since 1984, and he says he’s watched things “spiral out of control.” He says people are receiving dozens of notifications every day and that they often feel they can’t escape their online lives.“Even when you’re not on the screens, the screens are in your head,” Rosen says.One value of privacy is that it gives us space to operate without judgment. When we’re using social media, there are often a lot of strangers viewing our content, liking it, commenting on it, and sharing it with their own communities. Any time we post something online, thus exposing a part of who we are, we don’t fully know how we’re being received in the virtual world. Fallon Goodman, an assistant professor of psychology at George Washington University, says not knowing what kind of impression you’re making online can cause stress and anxiety.
“When you post a picture, the only real data you get are people’s likes and comments. That’s not necessarily a true indication of what the world feels about your picture or your post,” Goodman says. “Now you’ve put yourself out there—in a semi-permanent way—and you have limited information about how that was received, so you have limited information about the evaluations people are making about you.”Anna Lembke, a professor of psychiatric and behavioral sciences at Stanford University, says we construct our identities through how we’re seen by others. Much of that identity is now formed on the internet, and that can be difficult to grapple with.“This virtual identity is a composition of all of these online interactions that we have. It is a very vulnerable identity because it exists in cyberspace. In a weird kind of way we don’t have control over it,” Lembke says. “We’re very exposed.”
Without the ability to find out how their identity is ricocheting around the virtual world, people often feel a fight-or-flight response when they’ve been online for many hours—and even after they’ve logged off.
“It’s kind of an adapted hyper-vigilance. As soon as you send something out into the virtual world, you’re sort of sitting on pins and needles waiting for a response,” Lembke says. “That alone—that kind of expectancy—is a state of hyperarousal. How will people respond to this? When will they respond? What will they say?”It would be one thing if only you saw any negative reactions, Lembke says, but they’re often available for everyone to see. She says this exacerbates feelings of shame and self-loathing that are already “endemic” in the modern world.We are social creatures, and our brains evolved to form communities, communicate with each other, and work together. We have not evolved to expose ourselves to the judgment of the whole world on a daily basis. These things affect everyone differently, but it’s clear many people regularly feel overwhelmed by this exposure level.
If we’re not careful, our online lives can become a source of chronic stress that subtly seeps into everything. Everyone needs some privacy, but we often don’t provide it for ourselves and end up feeling like we’re constantly battling invisible enemies.
There are things you can do for yourself, however. You can turn off your notifications for social media apps, reduce how much time you spend on them, limit when you allow yourself to use them, and more. Goodman says it sometimes helps to keep your phone in the other room so you’re not so easily tempted to pick it up.Lembke says we need to change how we think about social media and internet use as a society. She calls it a “collective” problem, not just an individual one.“We need to come up with a kind of cultural etiquette around what appropriate and healthy consumption is, just like we have for other consumptive problems,” Lembke says. “We have nonsmoking areas. We don’t eat ice cream for breakfast. We have all kinds of laws around who can buy and consume alcohol, who can go into a casino. We need guardrails for these digital products, especially for minors.”
By Andrew Hutchinson Content and Social Media Manager
Have you found yourself using Instagram way less of late? The once trendsetting social platform seems to have lost its luster, in large part due to Instagram’s insistence on pumping more content from accounts that you don’t follow into your main IG feed. The ‘inspiration’ for that approach is TikTok, which has seen great success by focusing on content, as opposed to creators, with the app opening to a ‘For You’ feed of algorithmically-selected clips, based on your viewing habits. Instagram, as usual, saw that as an opportunity, and it’s since been working to negate your direct input – i.e. the accounts that you’ve chosen to follow – by showing you more and more stuff that it thinks you’ll like. Which is annoying, and personally, I don’t find Instagram anywhere near as engaging as it once was.
And it seems many other users agree – according to a new report from The Wall Street Journal, Instagram engagement is declining, with Reels, in particular, seeing a significant drop-off in user engagement of late. As reported by WSJ, TikTok users are spending over 10x as many hours consuming content in that app as Instagram users currently spend viewing Reels. According to a leaked internal report, Reels engagement is also in decline, dropping 13.6% in recent months – while ‘most Reels users have no engagement whatsoever.’ Meta has lightly refuted the claims, by stating that the usage data doesn’t provide the full picture. Though it declined to add any more context – which is Meta’s usual process when it can’t dispel such with its own insight. Take, for example, total time spent in its apps. Back in 2016, as part of its regular performance reporting, Meta noted that people were spending more than 50 minutes per day, on average, using Facebook, Instagram and Messenger.
It hasn’t reported any official stats on this ever since, which many believe is because that number has been in steady decline, and Meta sees no value in reporting that it’s losing ground, and has been for years now. Meta, instead, is keen to talk about daily and monthly active users, where its figures are solid. But this almost feels like misdirection – Facebook and Instagram, in particular, have traditionally been based on building your social graph, and establishing a digital connection with the people that you know and want to stay connected with, and informed about.
As such, it makes sense that a lot of people log onto these apps each day just to see if their friends and family have shared anything new. That doesn’t, however, mean that they’re spending a lot of time in these apps. Which is another reason why Meta’s trying to push more interesting content into your main feed, and in between updates from your connections – because if it can hook those people that are just checking in, then logging straight back out, that could be a key way to get its engagement stats back on track. But it’s not working.
Again, Facebook and Instagram have spent years pushing you to establish connections with the people that you care about, even introducing an algorithm to ensure that you see the most important updates from these users and Pages every day. At one point, Facebook noted that an average user was eligible to see over 1,500 posts every day, based on the people and Pages they were connected to – which is way more than they could ever view in a single day. So it brought in the algorithm to help maximize engagement – which also had the added benefit of squeezing Page reach, and forcing more brands to pay up. But now, Facebook is actively working to add in even more content, cluttering your feed beyond the posts that you could already be shown, and making it harder than ever to see posts from the people you actually want to stay updated on. Hard to see how that serves the user interests.
And again, it seems that users are understandably frustrated by this, based on these latest engagement stats, and previously reported info from Facebook which showed that young users are spending less and less time in the app. Because it’s fundamentally going against its own ethos, purely for its own gain. Accept it or not, people go to different apps for different purpose, which is the whole point of differentiation and finding a niche in the industry. People go to TikTok for entertainment, not for connecting with friends (worth noting that TikTok has actually labeled itself an ‘entertainment app’, as opposed to a social network), while users go to Facebook and IG to see the latest updates from people they care about.
The focus is not the same, and in this new, more entertainment-aligned paradigm, Meta’s once all-powerful, unmatched social graph is no longer the market advantage that it once was. But Meta, desperately seeking to counter its engagement declines, keeps trying to get people to stick around, which is seemingly having the opposite effect. Of course, Meta needs to try, it needs to seek ways to negate user losses as best it can – it makes sense that it’s testing out these new approaches. But they’re not the solution. How, then, can Instagram and Facebook actually re-engage users and stem the tide of people drifting across to TikTok? There are no easy answers, but I’m tipping the next phase will involve exclusive contracts with popular creators, as they become the key pawns in the new platform wars. TikTok’s monetization systems are not as evolved, and YouTube and Meta could theoretically blow it out of the water if they could rope in the top stars from across the digital ecosphere. That could keep people coming to their apps instead, which could see TikTok engagement wither, like Vine before it.
But other than forcing people to spend more time on Facebook, by hijacking their favorite stars, there’s not a lot of compelling reasons for people to spend more time in Meta’s apps. At least, not right now, as they increasingly dilute any form of differentiation.
But essentially, it comes down to a major shift in user behaviors, away from following your friends, and seeing all the random stuff that they post, to following trends, and engaging with the most popular, most engaging content from across the platform, as opposed to walling off your own little space.
At one stage, the allure of social media was that it gave everyone their own soapbox, a means to share their voice, their opinion, to be their own celebrity in their own right, at least among their own networks. But over time, we’ve seen the negatives of that too. Over-sharing can lead to problems when it’s saved in the internet’s perfect memory for all time, while increasing division around political movements has also made people less inclined to share their own thoughts, for fear of unwanted criticism or misunderstanding. Which is why entertainment has now become the focus of the next generation – it’s less about personal insights and more about engaging in cultural trends. That’s why TikTok is winning, and why Facebook and Instagram are losing out, despite their frantic efforts.
Photograph: Luka Milanovic/Getty Images
Because it’s fundamentally going against its own ethos, purely for its own gain. Accept it or not, people go to different apps for different purpose, which is the whole point of differentiation and finding a niche in the industry. People go to TikTok for entertainment, not for connecting with friends (worth noting that TikTok has actually labeled itself an ‘entertainment app’, as opposed to a social network), while users go to Facebook and IG to see the latest updates from people they care about. 
TheIdea – The Innovation Agency 