Thanks to drastic policy changes in the US and Big Tech’s embrace of the second Trump administration, many people are moving their digital lives abroad. Here are a few options to get you started.
From your email to your web browsing, it’s highly likely that your daily online life is dominated by a small number of tech giants—namely Google, Microsoft, and Apple. But since Big Tech has been cozying up to the second Trump administration, which has taken an aggressive stance on foreign policy, and Elon Musk’s so-called Department of Government Efficiency (DOGE) has ravaged through the government, some attitudes towards using US-based digital services have been changing.
While movements to shift from US digital services aren’t new, they’ve intensified in recent months. Companies in Europe have started moving away from some US cloud giants in favor of services that handle data locally, and there have been efforts from officials in Europe to shift to homegrown tech that has fewer perceived risks. For example, the French and German governments have created their own Docs word processor to rival Google Docs.
Meanwhile, one consumer poll released in March had 62 percent of people from nine European countries saying that large US tech companies were a threat to the continent’s sovereignty. At the same time, listsof non-US tech alternatives and European-based tech options have seen a surge in visitors in recent months.
For three of the most widely used tech services—email, web browsers, and search engines—we’ve been through some of the alternatives that are privacy-focused and picked some options you may want to consider. Other options are available, but these organizations and companies aim to minimize data they collect and often put privacy first.
There are caveats, though. While many of the services on this list are based outside of the US, there’s still the potential that some of them rely upon Big Tech services themselves—for instance, some search engines can use results or indexes provided by Big Tech, while companies may use software or services, such as cloud hosting, that are created by US tech firms. So trying to distance yourself entirely may not be as straightforward as it first looks.
Web Browsers
Mullvad
Based in Sweden, Mullvad is perhaps best known for its VPN, but in 2023 the organization teamed up with digital anonymity service Tor to create the Mullvad Browser. The open source browser, which is available only on desktop, says it collects no user data and is focused on privacy. The browser has been designed to stop people from tracking you via browser fingerprinting as you move around the web, plus it has a “private mode” that isolates tracking cookies enabled by default. “The underlying policy of Mullvad is that we never store any activity logs of any kind,” its privacy policy says. The browser is designed to work with Mullvad’s VPN but is also compatible with any VPN that you might use.
Vivaldi
WIRED’s Scott Gilbertson swears by Vivaldi and has called it the web’s best browser. Available on desktop and mobile, the Norwegian-headquartered browser says it doesn’t profile your behavior. “The sites you visit, what you type in the browser, your downloads, we have no access to that data,” the company says. “It either stays on your local machine or gets encrypted.” It also blocks trackers and hosts data in Iceland, which has strong data protection laws. Its privacy policy says it anonymizes IP addresses and doesn’t share browsing data.
Search Engines
Qwant French search engine Qwant has built its own search index, crawling more than 20 billion pages to create its own records of the web. Creating a search index is a hugely costly, laborious process, and as a result, many alternative search engines will not create an extensive index and instead use search results from Google or Microsoft’s Bing—enhancing them with their own data and algorithms. Qwant says it uses Bing to “supplement” search results that it hasn’t indexed. Beyond this, Qwant says it does not use targeted advertising, or store people’s search history. “Your data remains confidential, and the processing of your data remains the same,” the company says in its privacy policy.
Mojeek
Mojeek, based out of the United Kingdom, has built its own web crawler and index, saying that its search results are “100% independent.” The search engine does not track you, it says in its privacy policy, and only keeps some specific logs of information. “Mojeek removes any possibility of tracking or identifying any particular user,” its privacy policy says. It uses its own algorithms to rank search results, not using click or personalization data to create ranks, and says that this can mean two people searching for the same thing while in different countries can receive the same search results.
Startpage
Based in the Netherlands, Startpage says that when you make a search request, the first thing that happens is it removes your IP address and personal data—it doesn’t use any tracking cookies, it says. The company uses Google and Bing to provide its search results but says it acts as an “intermediary” between you and the providers. “Startpage submits your query to Google and Bing anonymously on your behalf, then returns the results to you, privately,” it says on its website. “Google and Microsoft do not know who made the search request—instead, they only see Startpage.”
Ecosia
Nonprofit search engine Ecosia uses the money it makes to help plant trees. The company also offers various privacy promises when you search with it, too. Based in Germany, the company says it doesn’t collect excessive data and doesn’t use search data to personalize ads. Like other search alternatives, Ecosia uses Google’s and Bing’s search results (you can pick which one in the settings). “We only collect and process data that is necessary to provide you with the best search results (which includes your IP address, search terms and session behavioral data),” the company says on its website. The information it collects is gathered to provide search results from its Big Tech partners and detect fraud, it says. (At the end of 2024, Ecosia partnered with Qwant to build more search engine infrastructure in Europe).
Email Providers
ProtonMail
Based in Switzerland, Proton started with a privacy-focused email service and has built out a series of apps, including cloud storage, docs, and a VPN to rival Google. The company says it cannot read any messages in people’s inboxes, and it offers end-to-end encryption for emails sent to other Proton Mail addresses, as well as a way to send password protected emails to non Proton accounts. It blocks trackers in emails and has multiple account options, including both free and paid choices. Its privacy policy describes what information the company has access to, which includes sender and recipient email addresses, plus IP addresses where messages arrive from, message subject lines, and when emails are sent. (Despite Switzerland’s strong privacy laws, the government has recently announced it may require encrypted services to keep user’s data, something that Proton has pushed back on).
Tuta
Tuta, which used to be called Tutanota and is based in Germany, says it encrypts email content, subject lines, calendars, address books, and other data in your inbox. “The only unencrypted data are mail addresses of users as well as senders and recipients of emails,” it says on its website, adding that users‘ encryption keys cannot be accessed by developers. Like Proton, emails sent between Tuta accounts are end-to-end encrypted, and you can send password protected emails when messaging an account from another email provider. The company also has an end-to-end encrypted calendar and offers both free and paid plans.
The CMG Active Listening scandal involves Cox Media Group (CMG), a major American media company, which admitted to using „Active Listening“ technology that allegedly captures conversations through smartphone microphones and smart devices to target users with hyper-specific advertisements[1][2][3]. This revelation has sparked significant controversy and prompted responses from major American tech companies.
American Tech Companies Listed in CMG’s Presentations
Companies Named as Partners
According to leaked CMG pitch decks obtained by 404 Media, the following American tech giants were explicitly identified as CMG partners or clients in their Active Listening program[4][5][6]:
Google (including Google Ads and Bing search)
Meta (Facebook’s parent company)
Amazon (Amazon Ads)
Microsoft (including Bing search engine)
Tech Company Responses and Denials
Google’s Response:
Google took the most decisive action, removing CMG from its Partners Program immediately after the 404 Media report was published[1][4][5]. A Google spokesperson stated: „All advertisers must comply with all applicable laws and regulations as well as our Google Ads policies, and when we identify ads or advertisers that violate these policies, we will take appropriate action“[6].
Meta’s Response:
Meta denied any involvement in the Active Listening program and announced an investigation into whether CMG violated Facebook’s terms of service[7][4]. A Meta spokesperson told Newsweek: „Meta does not use your phone’s microphone for ads, and we’ve been public about this for years. We are reaching out to CMG to clarify that their program is not based on Meta data“[8][9].
Amazon’s Response:
Amazon completely denied any collaboration with CMG on the Active Listening program[4][6]. An Amazon spokesperson stated: „Amazon Ads has never worked with CMG on this program and has no plans to do so“[9][10].
Microsoft’s Response:
While Microsoft was mentioned in the pitch deck as a partner through its Bing search engine[4][11], the company has not provided a public response to the allegations at the time of these reports.
Apple’s Response:
Although not directly implicated as a CMG partner, Apple responded to the controversy by clarifying that such practices would violate its App Store guidelines[12]. Apple emphasized that apps must request „explicit user consent and provide a clear visual and/or audible indication when recording, logging, or otherwise making a record of user activity“[12].
How the Active Listening Technology Allegedly Works
According to CMG’s marketing materials, the Active Listening system operates by[1][13][14]:
Real-time voice data collection through smartphone microphones, smart TVs, and other connected devices
AI analysis of conversations to identify consumer intent and purchasing signals
Data integration with behavioral data from over 470 sources
Targeted advertising delivery through various platforms including streaming services, social media, and search engines
Geographic targeting within 10-mile ($100/day) or 20-mile ($200/day) radius
CMG’s pitch deck boldly stated: „Yes, Our Phones Are Listening to Us“ and claimed the technology could „identify buyers based on casual conversations in real-time“[14][15][9].
Legal and Privacy Implications
The scandal has raised significant legal and privacy concerns[16][17]. Senator Marsha Blackburn sent letters to CMG, Google, and Meta demanding answers about the extent of Active Listening deployment and requesting copies of the investor presentation[17].
CMG initially defended the practice as legal, claiming that microphone access permissions are typically buried in the fine print of lengthy terms of service agreements that users rarely read thoroughly[14][18]. However, privacy experts note that such practices would likely violate GDPR regulations in Europe and potentially face legal challenges in various US jurisdictions[19][16].
Current Status
Following the public backlash, CMG has:
Removed all references to Active Listening from its website[3][20]
Claimed the presentation contained „outdated materials for a product that CMG Local Solutions no longer offers“[7][8]
Stated that while the product „never listened to customers, it has been discontinued to avoid misperceptions“[8]
The scandal has reignited long-standing consumer suspicions about device surveillance and targeted advertising, with many users reporting eerily accurate ads that seemed to reflect their private conversations[13][21][22].
“I was too poor,” he said. “And then I was too rich.”
In fact, Mr. Karp, a co-founder and the C.E.O. of Palantir Technologies, the mysterious and powerful data analytics firm, doesn’t trust himself to drive. Or ride a bike. Or ski downhill.
“I’m a dreamer,” he said. “I’ll start dreaming and then I fall over. I started doing tai chi to prevent that. It’s really, really helped with focusing on one thing at a time. If you had met me 15 years ago, two-thirds of the conversation, I’d just be dreaming.”
What would he dream about?
“Literally, it could be a walk I did five years ago,” he said. “It could be some conversation I had in grad school. Could be my family member annoyed me. Something a colleague said, like: ‘Why did they say this? What does it actually mean?’”
Mr. Karp is a lean, extremely fit billionaire with unruly salt-and-pepper curls. He is introvert-charming (something I aspire to myself). He has A.D.H.D. and can’t hide it if he is not interested in what someone is saying. After a hyper spurt of talking, he loses energy and has to recharge on the stationary bike or by reading. Even though he thinks of himself as different, he seems to like being different. He enjoys being a provocateur onstage and in interviews.
“I’m a Jewish, racially ambiguous dyslexic, so I can say anything,” he said, smiling.
Unlike many executives in Silicon Valley, Mr. Karp backed President Biden, cutting him a big check, despite skepticism about his handling of the border and his overreliance on Hollywood elites like Jeffrey Katzenberg. Now he is supporting Vice President Kamala Harris, but he still has vociferous complaints about his party.
When he donates, he said, he does it in multiples of 18 because “it’s mystical — 18 brings good luck in the tradition of kabbalah. I gave Biden $360,000.”
The 56-year-old is perfectly happy hanging out in a remote woodsy meadow alone — except for his Norwegian ski instructor, his Swiss-Portuguese chef, his Austrian assistant, his American shooting instructor and his bodyguards. (Mr. Karp, who has never married, once complained that bodyguards crimp your ability to flirt.)
“This is like introverts’ heaven,” he said, looking at his red barn from the porch of his Austrian-style house with a mezuza on the door. “You can invite people graciously. No one comes.”
The house is sparse on furniture, but Mr. Karp still worries that it is too cluttered. “I do have a Spartan thing,” he said. “I definitely feel constrained and slightly imprisoned when I have too much stuff around me.”
Asked about the dangers of artificial intelligence, Mr. Karp said, “The only solution to stop A.I. abuse is to use A.I.”Credit…Ryan David Brown for The New York Times
So how did a daydreaming doctoral student in German philosophy wind up leading a shadowy data analytics firm that has become a major American defense contractor, one that works with spy services as it charts the future of autonomous warfare?
He’s not a household name, and yet Mr. Karp is at the vanguard of what Mark Milley, the retired general and former chairman of the Joint Chiefs of Staff, has called “the most significant fundamental change in the character of war ever recorded in history.” In this new world, unorthodox Silicon Valley entrepreneurs like Mr. Karp and Elon Musk are woven into the fabric of America’s national security.
Mr. Karp is also at the white-hot center of ethical issues about whether firms like Palantir are too Big Brother, with access to so much of our personal data as we sign away our privacy. And he is in the middle of the debate about whether artificial intelligence is friend or foe, whether killer robots and disembodied A.I. will one day turn on us.
Mr. Karp’s position is that we’re hurtling toward this new world whether we like it or not. Do we want to dominate it, or do we want to be dominated by China?
Critics worry about what happens when weapons are autonomous and humans become superfluous to the killing process. Tech reflects the values of its operators, so what if it falls into the hands of a modern Caligula?
“I think a lot of the issues come back to ‘Are we in a dangerous world where you have to invest in these things?’” Mr. Karp told me, as he moved around his living room in a tai chi rhythm, wearing his house shoes, jeans and a tight white T-shirt. “And I come down to yes. All these technologies are dangerous.” He adds: “The only solution to stop A.I. abuse is to use A.I.”
Inspired by Tolkien
Palantir’s name is derived from palantíri, the seeing stones in the J.R.R. Tolkien fantasies. The company’s office in Palo Alto, Calif., features “Lord of the Rings” décor and is nicknamed the Shire.
After years under the radar, Mr. Karp is now in the public eye. He has joked that he needs a coach to teach him how to be more normal.
Born in New York and raised outside Philadelphia in a leftist family, Mr. Karp has a Jewish father who was a pediatrician and a Black mother who is an artist. They were social activists who took young Alex to civil rights marches and other protests. His uncle, Gerald D. Jaynes, is an economics and African American studies professor at Yale; his brother, Ben, is an academic who lives in Japan.
“I just think I’ve always viewed myself as I don’t fit in, and I can’t really try to,” Mr. Karp said. “My parents’ background just gave me a primordial subconscious bias that anything that involves ‘We fit in together’ does not include me.
“Yes, I think the way I explain it politically is like, if fascism comes, I will be the first or second person on the wall.”
Mr. Karp has his own unique charisma. “He’s one of a kind, to say the least,” said the Democratic strategist James Carville, who is an informal adviser to Palantir.
When I visited the Palo Alto office, Mr. Karp accidentally knocked down a visitor while demonstrating a tai chi move. He apologized, then ran off to get a printout of Goethe’s “Faust” in German, which he read aloud in an effort to show that it was better than the English translation.
“If you were to do a sitcom on Palantir, it’s equal parts Larry David, a philosophy class, tech and James Bond,” he said.
Mr. Karp at the Senate building in Washington last year. He was among the tech industry titans, including Bill Gates, Elon Musk and Sam Altman, who took part in a discussion of A.I. with lawmakers.Credit…Haiyun Jiang for The New York Times
Palantir was founded in 2003 by a gang of five, including Karp and his old Stanford Law School classmate Peter Thiel (now the company’s chairman). It was backed, in part, by nearly $2 million from In-Q-Tel, the C.I.A.’s venture capital arm.
“Saving lives and on occasion taking lives is super interesting,” Mr. Karp told me.
He described what his company does as “the finding of hidden things” — sifting through mountains of data to perceive patterns, including patterns of suspicious or aberrant behavior.
Mr. Karp does not believe in appeasement. “You scare the crap out of your adversaries,” he said. He brims with American chauvinism, boasting that we are leagues ahead of China and Russia on software.
“The tech scene in America is like the jazz scene in the 1950s,” he said in one forum. He told me: “I’m constantly telling people 86 percent of the top 50 tech companies in the world just by market cap are American — and people fall out of their chair. It’s hard for us to understand how dominant we are in certain industries.”
In the wake of 9/11, the C.I.A. bet on Palantir’s maw gobbling up data and auguring where the next terrorist attacks would come from. Palantir uses multiple databases to find the bad guy, even, as Mr. Karp put it, “if the bad guy actually works for you.”
The company is often credited with helping locate Osama bin Laden so Navy SEALs could kill him, but it’s unclear if that is true. As with many topics that came up in the course of our interviews in Washington, Palo Alto and New Hampshire, Mr. Karp zips his lips about whether his company was involved in dispatching the fiend of 9/11.
“If you have a reputation for talking about what the pope says when you meet him,” Mr. Karp explained, “you’ll never meet the pope again.”
He does crow a little about Western civilization’s resting on Palantir’s slender shoulders, noting that without its software, “you would’ve had massive terror attacks in Europe already, like Oct. 7 style.” And those attacks, he believes, would have propelled the far right to power.
Palantir does not do business with China, Russia or other countries that are opposed to the West. Mr. Thiel said the company tries to work with “more allied” and “less corrupt” governments, noting dryly that aside from their ideological stances, “with corrupt countries, you never get paid.”
“We have a consistently pro-Western view that the West has a superior way of living and organizing itself, especially if we live up to our aspirations,” Mr. Karp said. “It’s interesting how radical that is, considering it’s not, in my view, that radical.”
He added: “If you believe we should appease Iran, Russia and China by saying we’re going to be nicer and nicer and nicer, of course you’ll look at Palantir negatively. Some of these places want you to do the apology show for what you believe in, and we don’t apologize for what we believe in. I’m not going to apologize for defending the U.S. government on the border, defending the Special Ops, bringing the people home. I’m not apologizing for giving our product to Ukraine or Israel or lots of other places.”
As one Karp acquaintance put it: “Alex is principled. You just may not like his principles.”
Kara Swisher, the author of “Burn Book: A Tech Love Story,” told me: “While Palantir promises a more efficient and cost-effective way to conduct war, should our goal be to make it less expensive, onerous and painful? After all, war is not a video game, nor should it be.”
Mr. Karp’s friend Diane von Furstenberg told me that he sees himself as Batman, believing in the importance of choosing sides in a parlous world. (The New York office is called Gotham and features a statue and prints of Batman.) But some critics have a darker view, worrying about Palantir creating a “digital kill chain” and seeing Mr. Karp less as a hero than as a villain.
Back in 2016, some Democrats regarded Palantir as ominous because of Mr. Thiel’s support for former President Donald J. Trump. Later, conspiracy theories sprang up around the company’s role in OperationWarp Speed, the federal effort pushing the Covid-19 vaccine program from clinical trials to jabs in arms.
In December 2016, Donald J. Trump, then the president-elect, met with tech executives including the Palantir co-founder Peter Thiel.Credit…Drew Angerer/Getty Images
Some critics focused on Palantir’s work at the border, which helped U.S. Immigration and Customs Enforcement track down undocumented migrants for deportation. In 2019, about 70 demonstrators blocked access to the cafeteria outside the Palo Alto office. “Immigrants are welcome here, time to cancel Palantir,” they shouted.
The same year, over 200 Palantir employees, in a letter to Mr. Karp, outlined their concerns about the software that had helped ICE. And there was a campaign inside Palantir — in vain — to get him to donate the proceeds of a $49 million ICE contract to charity.
I asked Mr. Karp if Mr. Thiel’s public embrace of Mr. Trump the first time around had made life easier — in terms of getting government contracts — or harder.
“I didn’t enjoy it,” he said. “There’s a lot of reasons I cut Biden a check. I do not enjoy being protested every day. It was completely ludicrous and ridiculous. It was actually the opposite. Because Peter had supported Mr. Trump, it was actually harder to get things done.”
Did they talk about it?
“Peter and I talk about everything,” Mr. Karp said. “It’s like, yes, I definitely informed Peter, ‘This is not making our life easier.’”
Mr. Thiel did not give money to Mr. Trump or speak at his convention this time around, although he supports JD Vance, his former protégé at his venture capital firm. He said he might get more involved now because of Mr. Vance.
Palantir got its start in intelligence and defense — it now works with the Space Force — and has since sprouted across the government through an array of contracts. It helps the I.R.S. to identify tax fraud and the Food and Drug Administration to prevent supply chain disruptions and to get drugs to market quicker.
It has assisted Ukraine and Israel in sifting through seas of data to gather relevant intelligence in their wars — on how to protect special forces by mapping capabilities, how to safely transport troops and how to target drones and missiles more accurately.
In 2022, Mr. Karp took a secret trip to war-ravaged Kyiv, becoming the first major Western C.E.O. to meet with Ukraine’s president, Volodymyr Zelensky, and offering to supply his country with the technology that would allow it to be David to Russia’s Goliath. Time magazine ran a cover on Ukraine as a lab for A.I. warfare, and Palantir operatives embedded with the troops.
A Ukrainian government handout image of Mr. Karp meeting with President Volodymyr Zelensky of Ukraine and Deputy Prime Minister Mykhailo Fedorov in 2022.Credit…Office of the President of Ukraine
While Palantir’s role in helping Ukraine was heralded, its work with Israel, where targeting is more treacherous, because the enemy is parasitically entangled with civilians, is far more controversial.
“I think there’s a huge dichotomy between how the elite sees Ukraine and Israel,” Mr. Karp said. “If you go into any elite circle, pushing back against Russia is obvious, and Israel is complicated. If you go outside elite circles, it’s exactly the opposite.”
Independent analysts have said that Israel, during an April operation, could not have shot down scores of Iranian missiles and drones in mere minutes without Palantir’s tech. But Prime Minister Benjamin Netanyahu’s scorched-earth campaign in Gaza, the starving and orphaned children and the deaths of tens of thousands of civilians have drawn outrage, including some aimed at Mr. Karp and Mr. Thiel.
In May, protesters trapped Mr. Thiel inside a student building at the University of Cambridge. In recent days, senior U.S. officials have expressed doubts about Israel’s conduct of the war.
Mr. Karp’s position on backing Israel is adamantine. The company took out a full-page ad in The New York Times last year stating that “Palantir stands with Israel.”
“It’s like we have a double standard on Israel,” he told me. If the Oct. 7 attack had happened in America, he said, we would turn the hiding place of our enemies “into a parking lot. There would be no more tunnels.”
As Mr. Karp told CNBC in March: “We’ve lost employees. I’m sure we’ll lose more employees. If you have a position that does not cost you ever to lose an employee, it’s not a position.”
He told me, “If you believe that the West should lose and you believe that the only way to defend yourself is always with words and not with actions, you should be skeptical of us.”
He added: “I always think it’s hard because where the critics are right is what we do is morally complex. If you’re supporting the West with products that are used at war, you can’t pretend that there’s a simple answer.”
Does he have any qualms about what his company does?
“I’d have many more qualms if I thought our adversaries were committed to anything like the rule of law,” he said, adding: “A lot of this does come down to, do you think America is a beacon of good or not? I think a lot of the critics, what they actually believe is America is not a force for good.” His feeling is this: “Without being Pollyannaish, idiotic or pretending like any country’s been perfect or there’s not injustice, at the margin, would you want a world where America is stronger, healthy and more powerful, or not?”
In 2019, demonstrators protested the role of Palantir Technologies in aiding U.S. Immigration and Customs Enforcement.Credit…Shannon Stapleton/Reuters
Asked about the impending TikTok ban, he said he’s “very in favor.”
“I do not think you should allow an adversary to control an algorithm that is specifically designed to make us slower, more divided and arguably less cognitively fit,” he said.
He considered the anti-Israel demonstrations such “an infection inside society,” reflecting “a pagan religion of mediocrity and discrimination and intolerance, and violence,” that he offered 180 jobs to students who were fearful of staying in college because of a spike in antisemitism on campuses.
“Palantir is a much better diploma,” he told me. “Honestly, it’s helping us, because there are very talented people at the Ivy League, and they’re like, ‘Get me out of here!’”
Mr. Karp sometimes gets emotional in his defense of Palantir. In June, when he received an award named in honor of Dwight Eisenhower at a D.C. gala for national security executives, he teared up. He said that when he lived in Germany, he often thought about the young men from Iowa and Kansas who risked their lives “to free people like me” during World War II. He said he was honored to receive an award named after the president who had integrated schools by force.
Claiming that his products “changed the course of history by stopping terror attacks,” Mr. Karp said that Palantir had also “protected our men and women on the battlefield” and “taken the lives of our enemies, and I don’t think that’s something to be ashamed of.”
He told the gala audience about being “yelled at” by people who “call themselves progressives.”
“I actually am a progressive,” he said. “I want less war. You only stop war by having the best technology and by scaring the bejabers — I’m trying to be nice here — out of our adversaries. If they are not scared, they don’t wake up scared, they don’t go to bed scared, they don’t fear that the wrath of America will come down on them, they will attack us. They will attack us everywhere.”
He added that “we in the corporate world” have “to grow a spine” on issues like the Ivy League protesters: “If we do not win the battle of ideas and reassert basic norms and the basic, obvious idea that America is a noble, great, wonderful aspiration of a dream that we are blessed to be part of, we will have a much, much worse world for all of us.”
How It Started
Mr. Karp practicing tai chi at his home in New Hampshire.Credit…Ryan David Brown for The New York Times
The wild origin story of Palantir plays like a spy satire.
After graduating from Haverford College, Mr. Karp went to Stanford Law School, which he called “the worst three years of my adult life.”
He wasn’t interested in his classmates’ obsession with landing prestigious jobs at top law firms. “I learned at law school that I cannot do something I do not believe in,” he said, “even if it’s just turning a wrench.”
He met Mr. Thiel, a fellow student, and they immediately hit it off, trash-talking law school and, over beers, debating socialism vs. capitalism. “We argued like feral animals,” Mr. Karp told Michael Steinberger in a New York Times Magazine piece.
The liberal Heidegger fan and the conservative René Girard fan made strange bedfellows, but that’s probably what drew them together.
“I think we bonded on this intellectual level where he was this crazy leftist and I was this crazy right-wing person,” Mr. Thiel told me, “but we somehow talked to each other.”
“Alex did the Ph.D. thing,“ he continued, “which was, in some ways, a very, very insane thing to do after law school, but I was positive on it, because it sounded more interesting than working at a law firm.”
Mr. Karp received his doctorate in neoclassical social theory from Goethe University Frankfurt. He reconnected with Mr. Thiel in 2002, while working at the Jewish Philanthropy Partnership in San Francisco. The two began doing “vague brainstorming,” as Mr. Thiel put it, about a business they could start.
Mr. Thiel thought he could figure out how to find terrorists by using some of the paradigms developed at PayPal, which he helped found, to uncover patterns of fraud.
“I was just always super annoyed when, every time you go to the airport, you had to take off a shoe or you had to go through all this security theater, which was both somewhat taxing but probably had very little to do with actual security,” Mr. Thiel said.
They brought in some software engineers.
“It was two and a half years after 9/11, and you’re starting a software company with people who know nothing about the C.I.A. or any of these organizations,” Mr. Thiel recalled.
It was all very cloak-and-dagger, in an Inspector Clouseau way. They decided to seek out John Poindexter, a retired rear admiral who was dubbed the godfather of modern surveillance; Admiral Poindexter had been forced to resign as President Ronald Reagan’s national security adviser after the Iran-Contra scandal broke. After 9/11, he worked at the Pentagon on a surveillance program called Total Information Awareness.
During the meeting, Mr. Thiel said he felt he was in the presence of a medal-festooned, Machiavelli-loving member of the military brass out of “Dr. Strangelove,” with “a LARPing vibe.”
“We had a hunch that there was a room marked ‘Super-Duper Computer,’ and if you went inside, it was just an empty room,” Mr. Thiel said. They feared their budding algorithm “would end up in a broom closet in the Pentagon,” so they moved on.
In 2005, Mr. Thiel asked Mr. Karp to be the frontman of a company with few employees, no contracts, no investors, no office and no functional tech. “It charitably could have been described as a work in progress,” Mr. Thiel said.
Palantir’s headquarters in Palo Alto, Calif.Credit…Jim Wilson/The New York Times
Mr. Karp and his motley crew got a bunch of desks and explained to clients that they were unmanned because the (fictional) engineers were coming in later.
“God knows why Peter picked me as co-founder,” said Mr. Karp, who had to learn about coding on the job. “It was, in all modesty, a very good choice.”
Mr. Thiel explained: “In some ways, Alex doesn’t look like a salesperson from central casting you would send to the C.I.A. The formulation I always have is that if you’re trying to sell something to somebody, the basic paradox is you have to be just like them, so they can trust you — but you have to be very different from them so that they think you have something they don’t have.”
He said that Mr. Karp would not be suited to running Airbnb or Uber “or some mass consumer product.” But Palantir, he said, “is connected with this great set of geopolitical questions about the Western world versus the rising authoritarian powers. So if we can get our governments to function somewhat better, it’s a way to rebalance things in the direction of the West.”
“Normally,” Mr. Thiel continued, “these are bad ideas to have as a company. They’re too abstract, too idealistic. But I think something like this was necessary in the Palantir case. If you didn’t get some energy from thinking about these things, man, we would’ve sold the company after three years.”
Mr. Karp could not have been more of an outsider, to Silicon Valley and to Washington. He and his engineers had to buy suits for their visits to the capital. “We had no believers,” he said. “I kept telling Palantirians to call me Alex, and they kept calling me Dr. Karp. Then I realized the only thing they could believe in was that I had a Ph.D.”
The first few years, when tech investors were more interested in programs that let you play games on your phone, were rough. “We were like pariahs,” Mr. Karp said. “We couldn’t get meetings. If they did, it was a favor to Peter.”
With administrators in Washington, Mr. Karp recalled: “It was like, What is this Frankenstein monster doing in my office, making these wild claims that he can do better on things I have a huge budget for? How can it be that a freak-show motley crew of 12-year-old-looking mostly dudes, led by a pretty unique figure, from their perspective, would be able to do something with 1 percent of the money that we can’t do with billions and billions of dollars?”
“There’s nothing that we did at Palantir in building our software company that’s in any M.B.A.-made playbook,” Mr. Karp said. “Not one. That’s why we have been doing so well.”
He said that “the single most valuable education I had for business was sitting at the Sigmund Freud Institute, because I spent all my time with analysts.” When he worked at the institute in Frankfurt while getting his doctorate, Mr. Karp said, he would smoke cigars and think about “the conscious subconscious.”
“You’d be surprised how much analysts talk about their patients,” he said. “It’s disconcerting, actually. You just learn so much about how humans actually think.” This knowledge helps him motivate his engineers, he said.
How It’s Going
Mr. Karp said he likes to think of Palantir’s workers as part of an artists’ colony or a family; he doesn’t use the word “staff.” He enjoys interviewing prospective employees personally and prides himself on making hires in under two minutes. (He likes to have a few people around who can talk philosophy and literature with him, in German and French.)
“A lot of my populist-left politics actually bleed into my hiring stuff,” he said. “If you ask the question that the Stanford, Harvard, Yale person has answered a thousand times, all you’re learning is that the Stanford, Harvard, Yale person has learned to play the game.”
Even if he gets a good answer from a “privileged” candidate and a bad answer from “the child of a mechanic,” he might prefer the latter if “I have that feeling like I’m in the presence of talent.”
He views Palantirians like the Goonies, underdogs winning in the end. “Most people at Palantir didn’t get to do a lot of winning in high school,” Mr. Karp said at a company gathering in Palo Alto, to laughter from the audience.
He thinks the United States is “very likely” to end up in a three-front war with China, Russia and Iran. So, he argues, we have to keep going full-tilt on autonomous weapons systems, because our adversaries will — and they don’t have the same moral considerations that we do.
“I think we’re in an age when nuclear deterrent is actually less effective because the West is very unlikely to use anything like a nuclear bomb, whereas our adversaries might,” he said. “Where you have technological parity but moral disparity, the actual disparity is much greater than people think.”
“In fact,” he added, “given that we have parity technologically but we don’t have parity morally, they have a huge advantage.”
Mr. Karp, who said he supports Vice President Kamala Harris in the 2024 election, described his politics as “populist-left.”Credit…Ryan David Brown for The New York Times
Mr. Karp said that we are “very close” to terminator robots and at the threshold of “somewhat autonomous drones and devices like this being the most important instruments of war. You already see this in Ukraine.”
Palantir has learned from some early setbacks.
In 2011, the hacker group Anonymous showed that Palantir employees were involved in a proposed misinformation campaign to discredit WikiLeaks and smear some of its supporters, including the journalist Glenn Greenwald. (Mr. Karp apologized to Mr. Greenwald.) Then, at least one Palantir employee helped Cambridge Analytica collect the Facebook data that the Trump campaign used ahead of the 2016 election.
A pro bono contract with the New Orleans Police Department starting in 2012 was dropped after six years amid criticism that its “predictive policing” eroded privacy and had a disparate impact on people of color.
“We reduced the rates of Black-on-Black death in New Orleans,” Mr. Karp said, “and we have these critics who are like, ‘Palantir is racist.’ I don’t know. The hundreds of people that are alive now don’t think we’re racist.”
Mr. Carville, a New Orleans pooh-bah, asserted that the partnership ended because of “left-wing conspiracy theories.”
Palantir’s rough start in Silicon Valley came about, in part, because many objected to its work with the Department of Defense.
In 2017, Google won a Pentagon contract, Project Maven, to help the military use the company’s A.I. to analyze footage from drones. Employees protested, sending a letter to the C.E.O., Sundar Pichai: “Google should not be in the business of war,” it read. Soon after, Google backed away from the project.
In response, Palantir shaded Google in a tweet that quoted Mr. Karp: “Silicon Valley is telling the average American ‘I will not support your defense needs’ while selling products that are adversarial to America. That is a loser position.” Palantir picked up the contract in 2019.
That same year, Mr. Thiel said that Google had a “treasonous” relationship with China. When Google opened an A.I. lab in 2017 in China, where there’s little distinction between the civilian and the military, he argued, it was de facto helping China while refusing to help America. (That lab closed in 2019, but Google still does business with China, as does Apple.)
“When you have people working at consumer internet companies protesting us because we help the Navy SEALs and the U.S. military and were pro-border — and you’re becoming incredibly, mind-bogglingly rich, in part because America protects your right to export — to me, you’ve lost the sheet of music,” Mr. Karp said. “I don’t think that’s good for America.”
Scott Galloway, a professor at New York University and an authority on tech companies, agrees that many Silicon Valley C.E.O.s have been virtue-signaling and pretending to care about the progressive political views of employees, but really would sell “their mother for a nickel.”
“They’re not there to save the whales,” Mr. Galloway said. “They’re there to make money.”
He added: “Some of these big tech companies seem to be engaged in raising a generation of business leaders that just don’t like America, who are very focused on everything that’s wrong with America.
“Alex Karp is like, ‘No, we’ll cash the Pentagon’s check and we’ll collect data on our enemies.’ He’s gone the entirely opposite way, and I think it was a smart move.”
Palantir’s “spooky connotations,” as one executive put it, dissipated quite a bit when the company went public in 2020 and took on more commercial business; its clients include Airbus, J.P. Morgan, IBM and Amazon.
Mr. Thiel said that while Palantir had a brief stint working on a pilot program for the National Security Agency, the company would not want to do any more work there: “The N.S.A., it hoovers up all the data in the world. As far as I can tell, there are incredible civil liberties violations where they’re spying on everybody outside the U.S., basically. Then they’re fortunately too incompetent to do much with the data.”
The company has started turning a profit, and the stock has climbed. After a triumphant earnings report this month, Palantir’s stock price jumped again.
“The share price gives us more street cred,” Mr. Karp said.
In 2020, after 17 years in Silicon Valley, Mr. Karp moved Palantir’s headquarters to Denver. “I was fleeing Silicon Valley because of what I viewed as the regressive side of progressive politics,” he said.
He thinks that the valley has intensified class divisions in America.
“I don’t believe you would have a Trump phenomenon without the excesses of Silicon Valley,” he said. “Very, very wealthy people who support policies where they don’t have to absorb the cost at all. Just also the general feeling that these people are not tethered to our society, and simultaneously are becoming billionaires.“
“Not supporting the U.S. military,” he said, in a tone of wonder. “I don’t even know how you explain to the average American that you’ve become a multibillionaire and you won’t supply your product to the D.O.D. It’s jarringly corrosive. That’s before you get to all the corrosive, divisive things that are on these platforms.”
Akshay Krishnaswamy, Palantir’s chief architect, agreed on their Silicon Valley critics: “You live in the liberal democratic West because of reasons, and those reasons don’t come for free. They act like it doesn’t have to be fought for or defended rigorously.”
Mr. Karp’s workout room.Credit…Ryan David Brown for The New York TimesA few of his favorite things.Credit…Ryan David Brown for The New York Times
Mr. Karp said things had evolved. “I think there’s a different perception of us now a little bit. A lot of that was tied to Trump, ICE work. It built up and we were definitely outsiders. We’re still outsiders, but I feel less resistance for sure. And people have a better idea of what we do, maybe.” He added, “Defense tech is a big part of Silicon Valley now.”
The A.I. revolution, he said, will come with a knotty question: “How do you make sure the society’s fair when the means of production have become means that only 1 percent of the population actually knows how to navigate?”
I asked if he agrees with Elon Musk that A.I. is eventually going to take everyone’s jobs.
“I think what’s actually dangerous,” Mr. Karp replied, “is that people who understand how to use this are going to capture a lot of the value of the market and everyone else is going to feel left behind.”
Mr. Karp’s iconoclastic style and ironclad beliefs have inspired memes and attracted a flock of online acolytes — some call him Papa Karp or Daddy Karp. He has no social media presence, but his online fans treat him like a mystic, obsessing over the tight white T-shirts he wears for earnings reports, his Norwegian ski outfits, his corkscrew hair, his Italian jeans and sunglasses and his extreme candor. (In a recent earnings report, Mr. Karp dismissed his rivals as “self-pleasuring” and engaging in “self-flagellation.”)
He is not, as one colleague puts it, “a wife, kids and dog person.”
“I tend to have long-term relationships,” he told me. “And I tend to end up with very high IQ women,” including some who tell him he’s talking nonsense.
He prefers what he calls a German attitude toward relationships, where “you have a much greater degree of privacy,” he said, with separate bedrooms and “your own world, your own thoughts, and you get to be alone a lot.” There is much less requirement to “micro-lie” about where you were or whom you were with.
I asked Mr. Karp about his 2013 quote to Forbes that “the only time I’m not thinking about Palantir is when I’m swimming, practicing qigong or during sexual activity.”
He frowned, noting: “It should be tai chi. I don’t know why people always conflate tai chi with qigong. Yes, that was in my early days, when we were a pre-public company and I was allowed to admit I had sexual activity.”
So it’s true that the notion of settling down and raising a family gives him hives?
“There’s some truth in that,” he said. “This is how I like to live. See, I’m sitting here doing my freedom thing. I train. I do distance shooting.” He reads. “Who else has a Len Deighton spy novel next to a book on Confucian philosophy?”
Many of the doyennes of Washington society would love to snag the eligible Mr. Karp for a dinner party. He told me he has “a great social life.” But when I asked him what that is, he replied, “First of all, I’m a cross-country skier, so then I do all this training.”
He continued, “To have an elite VO2 max, an elite level of strength, it’s just consistency and the Norwegian-style training method.”
Some who know Mr. Karp said that the happiest they had ever seen him was last year when Mike Allen reported for Axios that the C.E.O.’s body fat was an impressive 7 percent.
Mr. Karp may be able to do more than 20 miles of cross-country skiing without being out of breath, but there are some sports at which, he admitted, he’s “a complete zero. For example, ball sports. I really suck at them.”
Unlike Mr. Musk and other tech lords, Mr. Karp is not into micro-dosing ketamine or any other drug. “My drug is athletics,” he said. “I love drinking, but now I’ve moved to drinking very little because what I’ve noticed is if you’re traveling all the time, the alcohol, it really affects your brain.” He’s on the road about 240 days a year.
Mr. Musk and Mr. Karp at the forum on A.I. in Washington last year.Credit…Haiyun Jiang for The New York Times
Mr. Karp said of his dyslexia: “I think this is not getting less, it’s likely getting more. In 40 years, I’ll be unable to read.”
In New Hampshire, we had a lunch of lobster pasta — he kept his panic button on the table — and then went shooting on his property. He expertly hit targets with a 9-millimeter pistol from 264 yards. When an aide suggested that a photographer not shoot Mr. Karp in the act of shooting, he overruled the idea.
“Actually, honestly, guns would be much better regulated if you had someone who knows guns,” he said. “I’m not a hunter. I’m an artist with a gun.”
(Later, Mr. Karp pointed out that he had been shooting at targets that were about twice as far from him as Mr. Trump was from his would-be assassin. “There’s something really wrong with security for our future president, or maybe not future president,” he said. “All these people need a different level of security.”)
Mr. Karp believes the Democrats need to project more strength: “Are we tough enough to scare our adversaries so we don’t go to war? Do the Chinese, Russians and Persians think we’re strong? The president needs to tell them if you cross these lines, this is what we’re going to do, and you have to then enforce it.”
He thinks that in America and in Europe, the inability or unwillingness to secure borders fuels authoritarianism.
“I see it as pretty simple: You have an open border, you get the far right,” he said. “And once you get them, you can’t get rid of them. We saw it in Brexit, we see it with Le Pen in France, you see it across Europe. Now you see it in Germany.”
“They should be much stricter,” he continued. That, he said, “is the only reason we have the rise of the right, the only reason. When people tell you we need an open border, then they should also tell you why they’re electing right-wing politicians, because they are.”
“The biggest mistake — and it’s not one politician, it’s a generation — was believing there was something bigoted about having a border, and there are just a lot of people who believe that,” he said.
Weeks later, we were back in the Washington office, which is dubbed Rivendell, after a valley in Tolkien’s Middle-earth, and is filled with tech goodies like a Ping-Pong table, a pool table and a towering replica of Chewbacca.
We picked up our conversation about politics, talking about the swap of President Biden and Vice President Harris, the rise of JD Vance, the assassination attempt and the changed political landscape.
Mr. Karp concurred with his friend Mr. Carville on the problem of drawing men to the Democratic Party, saying, “If this is going to be a party complaining about guys and to guys all the time, it’s not going to succeed.”
At the shooting range on his property in New Hampshire. “I’m an artist with a gun,” Mr. Karp said.Credit…Ryan David Brown for The New York Times
He continued: “The biggest problem with hard political correctness is it makes it impossible to deal with unfortunate facts. The unfortunate fact here is that this election is really going to turn on ‘What percentage of males can the Democrats still get?’”
Describing himself as “progressive but not woke,” he said, “We are so unwilling to talk to the actual constituents that are voting for the Democratic Party who would probably strongly prefer policies that are more moderate.”
Given Mr. Karp’s blended racial identity, I wondered how he felt about Mr. Trump’s attack on the vice president’s heritage.
“I think people are most fascinated by the fact of this whole Black-Jewish thing,” he said. “I tend to be less fascinated by that.”
He added: “I think that people always expect me somehow to see the world in one way or another, and I don’t really understand what that means. I see the world the way I see it. I think, at the end of the day, if people want to choose what their identity is, then they choose it, and that’s their definition.”
I note that he recently made an elite list of Black billionaires.
He shrugged. “Some Black people think I’m Black, some don’t,” he said. “I view me as me. And I’m very honored to be honored by all groups that will have me.”
He added: “I do not believe racism is the most important issue in this country. I think class is determinate, and I’m mystified by how often we talk about race. I’m not saying it doesn’t exist. I’m not saying people don’t have biases. Of course, we all do, but the primary thing that’s bad for you in this culture is to be born poor of any color.”
He said he would support class-based affirmative action and declared himself “pro draft.”
“I think part of the reason we have a massive cleavage in our culture is, at the end of the day, by and large, only people who are middle- and working-class do all the fighting,” he said.
Since I had last seen him, Mr. Karp had gotten caught between two of the battling billionaires of Silicon Valley, lords of the cloud vituperously fighting in public over the possible restoration of Donald Trump.
According to an account in Puck, Mr. Karp was onstage with the LinkedIn co-founder Reid Hoffman at a conference last month in Sun Valley, Idaho, sponsored by the investment bank Allen & Company, when Mr. Hoffman called Mr. Thiel’s support for Mr. Trump “a moral issue.” Speaking up from his seat in the audience, Mr. Thiel sarcastically thanked Mr. Hoffman for funding lawsuits against Mr. Trump, which allowed the candidate to claim that he is “a martyr.”
Mr. Hoffman snapped back, “Yeah, I wish I had made him an actual martyr” — an unfortunate comment given what would later happen in Butler, Pa.
I asked Mr. Karp whether the encounter was as uncomfortable as it seemed.
“Well, I’m used to being uncomfortable,” he said. “I’m going to stick with my friends. I just feel the same way I always feel when Peter is under attack, which is: ‘This is my friend. I feel that my friend is being attacked, and I will defend him.’”
The fancy digital clock behind Mr. Karp’s desk, which tells time in German, had gone from “Es ist zehn nach drei” to “Es ist halb vier.”
It was time to go.
Mr. Karp said that while working at the Sigmund Freud Institute in Frankfurt, he learned things that were helpful to him later as a business leader.Credit…Ryan David Brown for The New York Times
Confirm or Deny
Maureen Dowd: You run the Twitter account Alex Karp’s Hair.
Alex Karp: I wish.
Your favorite movie is the classic kung fu flick “The 36th Chamber of Shaolin.”
One of my favorite movies.
You have 10 houses around the world, from Alaska to Vermont, from Norway to New Hampshire.
You have to reframe that as I have 10 cross-country ski huts.
You love the idea of Peter Thiel backing Olympic-style games where the athletes will dope out in the open.
Deny. I want the best cross-country skiers to win without doping.
You love to watch spy shows and German movies, and one of your favorite filmmakers is Rainer Werner Fassbinder.
Confirm.
You have 20 identical pairs of swim goggles in your office.
No longer. I used to. I gave up swimming. There’s an emptiness to it.
You commissioned a French comic book, “Palantir: L’Indépendance,” with yourself as the protagonist.
Oui!
You starred in a movie by Hanna Laura Klar in 1998, “I Have Two Faces,” where you looked like a young Woody Allen.
I look better than Woody Allen.
Your dissertation is about how people transmit aggression subconsciously in language, presaging the rise of the right in America and Europe.
Often, the more charismatic ideologies were, the more irrational they were.
The dissertation touched on expressing taboo wishes. Do you want to share some of those?
I would love to express taboo wishes with you, but not to your audience.
n explosion from the side of an old brick building. A crashed bicycle in a city intersection. A cockroach in a box of takeout. It took less than 10 seconds to create each of these images with the Reimagine tool in the Pixel 9’s Magic Editor. They are crisp. They are in full color. They are high-fidelity. There is no suspicious background blur, no tell-tale sixth finger. These photographs are extraordinarily convincing, and they are all extremely fucking fake.
Anyone who buys a Pixel 9 — the latest model of Google’s flagship phone, available starting this week — will have access to the easiest, breeziest user interface for top-tier lies, built right into their mobile device. This is all but certain to become the norm, with similar features already available on competing devices and rolling out on others in the near future. When a smartphone “just works,” it’s usually a good thing; here, it’s the entire problem in the first place.
Photography has been used in the service of deception for as long as it has existed. (Consider Victorian spirit photos, the infamous Loch Ness monster photograph, or Stalin’s photographic purges of IRL-purged comrades.) But it would be disingenuous to say that photographs have never been considered reliable evidence. Everyone who is reading this article in 2024 grew up in an era where a photograph was, by default, a representation of the truth. A staged scene with movie effects, a digital photo manipulation, or more recently, a deepfake — these were potential deceptions to take into account, but they were outliers in the realm of possibility. It took specialized knowledge and specialized tools to sabotage the intuitive trust in a photograph. Fake was the exception, not the rule.
If I say Tiananmen Square, you will, most likely, envision the same photograph I do. This also goes for Abu Ghraib or napalm girl. These images have defined wars and revolutions; they have encapsulated truth to a degree that is impossible to fully express. There was no reason to express why these photos matter, why they are so pivotal, why we put so much value in them. Our trust in photography was so deep that when we spent time discussing veracity in images, it was more important to belabor the point that it was possible for photographs to be fake, sometimes.
This is all about to flip — the default assumption about a photo is about to become that it’s faked, because creating realistic and believable fake photos is now trivial to do. We are not prepared for what happens after.
A real photo of a stream.
Edited with Google’s Magic Editor.
A real photo of a person in a living room (with their face obscured).
Edited with Google’s Magic Editor.
No one on Earth today has ever lived in a world where photographs were not the linchpin of social consensus — for as long as any of us has been here, photographs proved something happened. Consider all the ways in which the assumed veracity of a photograph has, previously, validated the truth of your experiences. The preexisting ding in the fender of your rental car. The leak in your ceiling. The arrival of a package. An actual, non-AI-generated cockroach in your takeout. When wildfires encroach upon your residential neighborhood, how do you communicate to friends and acquaintances the thickness of the smoke outside?
And up until now, the onus has largely been on those denying the truth of a photo to prove their claims. The flat-earther is out of step with the social consensus not because they do not understand astrophysics — how many of us actually understand astrophysics, after all? — but because they must engage in a series of increasingly elaborate justifications for why certain photographs and videos are not real. They must invent a vast state conspiracy to explain the steady output of satellite photographs that capture the curvature of the Earth. They must create a soundstage for the 1969 Moon landing.
We have taken for granted that the burden of proof is upon them. In the age of the Pixel 9, it might be best to start brushing up on our astrophysics.
For the most part, the average image created by these AI tools will, in and of itself, be pretty harmless — an extra tree in a backdrop, an alligator in a pizzeria, a silly costume interposed over a cat. In aggregate, the deluge upends how we treat the concept of the photo entirely, and that in itself has tremendous repercussions. Consider, for instance, that the last decade has seen extraordinary social upheaval in the United States sparked by grainy videos of police brutality. Where the authorities obscured or concealed reality, these videos told the truth.
The persistent cry of “Fake News!” from Trumpist quarters presaged the beginning of this era of unmitigated bullshit, in which the impact of the truth will be deadened by the firehose of lies. The next Abu Ghraib will be buried under a sea of AI-generated war crime snuff. The next George Floyd will go unnoticed and unvindicated.
A real photo of an empty street.
Edited with Google’s Magic Editor.
A real photo inside a New York City subway station.
Edited with Google’s Magic Editor.
You can already see the shape of what’s to come. In the Kyle Rittenhouse trial, the defense claimed that Apple’s pinch-to-zoom manipulates photos, successfully persuading the judge to put the burden of proof on the prosecution to show that zoomed-in iPhone footage was not AI-manipulated. More recently, Donald Trump falsely claimed that a photo of a well-attended Kamala Harris rally was AI-generated — a claim that was only possible to make because people were able to believe it.
Even before AI, those of us in the media had been working in a defensive crouch, scrutinizing the details and provenance of every image, vetting for misleading context or photo manipulation. After all, every major news event comes with an onslaught of misinformation. But the incoming paradigm shift implicates something much more fundamental than the constant grind of suspicion that is sometimes called digital literacy.
Google understands perfectly well what it is doing to the photograph as an institution — in an interview with Wired, the group product manager for the Pixel camera described the editing tool as “help[ing] you create the moment that is the way you remember it, that’s authentic to your memory and to the greater context, but maybe isn’t authentic to a particular millisecond.” A photo, in this world, stops being a supplement to fallible human recollection, but instead a mirror of it. And as photographs become little more than hallucinations made manifest, the dumbest shit will devolve into a courtroom battle over the reputation of the witnesses and the existence of corroborating evidence.
This erosion of the social consensus began before the Pixel 9, and it will not be carried forth by the Pixel 9 alone. Still, the phone’s new AI capabilities are of note not just because the barrier to entry is so low, but because the safeguards we ran into were astonishingly anemic. The industry’s proposed AI image watermarking standard is mired in the usual standards slog, and Google’s own much-vaunted AI watermarking system was nowhere in sight when The Verge tried out the Pixel 9’s Magic Editor. The photos that are modified with the Reimagine tool simply have a line of removable metadata added to them. (The inherent fragility of this kind of metadata was supposed to be addressed by Google’s invention of the theoretically unremovable SynthID watermark.) Google told us that the outputs of Pixel Studio — a pure prompt generator that is closer to DALL-E — will be tagged with a SynthID watermark; ironically, we found the capabilities of the Magic Editor’s Reimagine tool, which modifies existing photos, were much more alarming.
Image: Cath Virginia / The Verge, Neil Armstrong, Dorothea Lange, Joe Rosenthal
Google claims the Pixel 9 will not be an unfettered bullshit factory but is thin on substantive assurances. “We design our Generative AI tools to respect the intent of user prompts and that means they may create content that may offend when instructed by the user to do so,” Alex Moriconi, Google communications manager, told The Verge in an email. “That said, it’s not anything goes. We have clear policies and Terms of Service on what kinds of content we allow and don’t allow, and build guardrails to prevent abuse. At times, some prompts can challenge these tools’ guardrails and we remain committed to continually enhancing and refining the safeguards we have in place.”
The policies are what you would expect — for example, you can’t use Google services to facilitate crimes or incite violence. Some attempted prompts returned the generic error message, “Magic Editor can’t complete this edit. Try typing something else.” (You can see throughout this story, however, several worrisome prompts that did work.) But when it comes down to it, standard-fare content moderation will not save the photograph from its incipient demise as a signal of truth.
We briefly lived in an era in which the photograph was a shortcut to reality, to knowing things, to having a smoking gun. It was an extraordinarily useful tool for navigating the world around us. We are now leaping headfirst into a future in which reality is simply less knowable. The lost Library of Alexandria could have fit onto the microSD card in my Nintendo Switch, and yet the cutting edge of technology is a handheld telephone that spews lies as a fun little bonus feature.
Temu, the Chinese e-commerce platform, offers products at remarkably low prices, which raises concerns about its business practices. One significant issue is the undervaluation of parcels entering the EU. Estimates suggest that around 65% of parcels are deliberately undervalued in customs declarations to avoid tariffs, which undermines local businesses and creates an uneven playing field [1]. Additionally, Temu employs a direct-to-consumer model, sourcing products directly from manufacturers in China, allowing them to benefit from bulk discounts and reduced shipping costs [2].
Benefits for the Chinese State
The low pricing strategy of Temu serves multiple purposes for the Chinese state. Firstly, it helps expand China’s influence in global e-commerce by increasing the market share of Chinese companies abroad. This can lead to greater economic ties and dependency on Chinese goods. Secondly, by facilitating the export of low-cost products, Temu contributes to the Chinese economy by boosting manufacturing and logistics sectors. Lastly, the data collected from users can be leveraged for insights into consumer behavior, which may benefit Chinese businesses and potentially the state itself in terms of economic planning and strategy [1].
Overall, while Temu’s low prices attract consumers, they also raise significant regulatory and ethical concerns in Europe, prompting scrutiny from authorities regarding compliance with local laws and standards.
Deeper Analysis of Future Benefits for the Chinese State
Temu’s aggressive pricing strategy in Europe not only serves immediate commercial interests but also aligns with broader strategic goals of the Chinese state. Here are several potential future benefits for China:
Economic Expansion and Market Penetration: By establishing a strong foothold in European markets through low prices, Temu can facilitate the expansion of Chinese goods into new territories. This not only increases sales volume but also enhances brand recognition and loyalty among European consumers. As more consumers become accustomed to purchasing Chinese products, it could lead to a long-term shift in buying habits, favoring Chinese brands over local alternatives.
Strengthening Supply Chains: Temu’s model emphasizes direct sourcing from manufacturers, which can help streamline supply chains. This efficiency can be replicated across various sectors, allowing China to become a dominant player in global supply chains. By controlling more aspects of production and distribution, China can mitigate risks associated with international trade tensions and disruptions, ensuring a more resilient economic structure.
Data Collection and Consumer Insights: The platform’s operations will generate vast amounts of consumer data, which can be analyzed to gain insights into European consumer behavior. This data can inform not only marketing strategies but also product development, allowing Chinese manufacturers to tailor their offerings to meet the specific preferences of European consumers. Such insights can enhance competitiveness and drive innovation within Chinese industries.
Geopolitical Influence: By increasing its economic presence in Europe, China can leverage its commercial relationships to enhance its geopolitical influence. Economic ties often translate into political goodwill, which can be beneficial in negotiations on various fronts, including trade agreements and international policies. This strategy aligns with China’s broader goal of expanding its influence globally, as outlined in its recent political resolutions emphasizing the importance of state power and common prosperity.
Promotion of Technological Advancements: As Temu grows, it may invest in technology to improve logistics, customer service, and user experience. This could lead to advancements in e-commerce technologies that can be exported back to China, enhancing domestic capabilities. Moreover, the emphasis on technology aligns with China’s ambitions to become a leader in areas such as artificial intelligence and data analytics, as highlighted in its national strategies.
Cultural Exchange and Soft Power: By making Chinese products more accessible and appealing to European consumers, Temu can facilitate a form of cultural exchange. As consumers engage with Chinese brands, they may also become more receptive to Chinese culture and values, enhancing China’s soft power. This cultural integration can help counter negative perceptions and foster a more favorable view of China in the long term.
In conclusion, Temu’s low pricing strategy is not merely a tactic for market entry; it is a multifaceted approach that can yield significant long-term benefits for the Chinese state. By enhancing economic ties, gathering valuable consumer data, and promoting technological advancements, China positions itself to strengthen its global influence and economic resilience in an increasingly competitive landscape.
Microsoft’s Recall technology, an AI tool designed to assist users by automatically reminding them of important information and tasks, bears resemblance to George Orwell’s „1984“ dystopia in several key aspects:
1. Surveillance and Data Collection: – 1984: The Party constantly monitors citizens through telescreens and other surveillance methods, ensuring that every action, word, and even thought aligns with the Party’s ideology. – Recall Technology: While intended for productivity, Recall collects and analyzes large amounts of personal data, emails, and other communications to provide reminders. This level of data collection can raise concerns about privacy and the potential for misuse or unauthorized access to personal information.
2. Memory and Thought Control: – 1984: The Party manipulates historical records and uses propaganda to control citizens‘ memories and perceptions of reality, essentially rewriting history to fit its narrative. – Recall Technology: By determining what information is deemed important and what reminders to provide, Recall could influence users‘ focus and priorities. This selective emphasis on certain data could subtly shape users‘ perceptions and decisions, akin to a form of soft memory control.
3. Dependence on Technology:
– 1984: The populace is heavily reliant on the Party’s technology for information, entertainment, and even personal relationships, which are monitored and controlled by the state. – Recall Technology: Users might become increasingly dependent on Recall to manage their schedules and information, potentially diminishing their own capacity to remember and prioritize tasks independently. This dependence can create a vulnerability where the technology has significant control over daily life.
4. Loss of Personal Autonomy:
– 1984: Individual autonomy is obliterated as the Party dictates all aspects of life, from public behavior to private thoughts. – Recall Technology: Although not as extreme, the automation and AI-driven suggestions in Recall could erode personal decision-making over time. As users rely more on technology to dictate their actions and reminders, their sense of personal control and autonomy may diminish.
5. Potential for Abuse:
– 1984: The totalitarian regime abuses its power to maintain control over the population, using technology as a tool of oppression. – Recall Technology: In a worst-case scenario, the data collected by Recall could be exploited by malicious actors or for unethical purposes. If misused by corporations or governments, it could lead to scenarios where users‘ personal information is leveraged against them, echoing the coercive control seen in Orwell’s dystopia.
While Microsoft’s Recall technology is designed with productivity in mind, its potential implications for privacy, autonomy, and the influence over personal information draw unsettling parallels to the controlled and monitored society depicted in „1984.“
The integration of advanced AI like OpenAI’s GPT-4o into Apple’s Vision Pro + Version 2 can significantly enhance its vision understanding capabilities. Here are ten possible use cases:
1. Augmented Reality (AR) Applications: – Interactive AR Experiences: Enhance AR applications by providing real-time object recognition and interaction. For example, users can point the device at a historical landmark and receive detailed information and interactive visuals about it. – AR Navigation: Offer real-time navigation assistance in complex environments like malls or airports, overlaying directions onto the user’s view.
2. Enhanced Photography and Videography: – Intelligent Scene Recognition: Automatically adjust camera settings based on the scene being captured, such as landscapes, portraits, or low-light environments, ensuring optimal photo and video quality. – Content Creation Assistance: Provide suggestions and enhancements for capturing creative content, such as framing tips, real-time filters, and effects.
3. Healthcare and Medical Diagnosis: – Medical Imaging Analysis: Assist in analyzing medical images (e.g., X-rays, MRIs) to identify potential issues, providing preliminary diagnostic support to healthcare professionals. – Remote Health Monitoring: Enable remote health monitoring by analyzing visual data from wearable devices to track health metrics and detect anomalies.
4. Retail and Shopping: – Virtual Try-Ons: Allow users to virtually try on clothing, accessories, or cosmetics using the device’s camera, enhancing the online shopping experience. – Product Recognition: Identify products in stores and provide information, reviews, and price comparisons, helping users make informed purchasing decisions.
5. Security and Surveillance: – Facial Recognition: Enhance security systems with facial recognition capabilities for authorized access and threat detection. – Anomaly Detection: Monitor and analyze security footage to detect unusual activities or potential security threats in real-time.
6. Education and Training: – Interactive Learning: Use vision understanding to create interactive educational experiences, such as identifying objects or animals in educational content and providing detailed explanations. – Skill Training: Offer real-time feedback and guidance for skills training, such as in sports or technical tasks, by analyzing movements and techniques.
7. Accessibility and Assistive Technology: – Object Recognition for the Visually Impaired: Help visually impaired users navigate their surroundings by identifying objects and providing auditory descriptions. – Sign Language Recognition: Recognize and translate sign language in real-time, facilitating communication for hearing-impaired individuals.
8. Home Automation and Smart Living: – Smart Home Integration: Recognize household items and provide control over smart home devices. For instance, identifying a lamp and allowing users to turn it on or off via voice commands. – Activity Monitoring: Monitor and analyze daily activities to provide insights and recommendations for improving household efficiency and safety.
9. Automotive and Driver Assistance: – Driver Monitoring: Monitor driver attentiveness and detect signs of drowsiness or distraction, providing alerts to enhance safety. – Object Detection: Enhance autonomous driving systems with better object detection and classification, improving vehicle navigation and safety.
10. Environmental Monitoring: – Wildlife Tracking: Use vision understanding to monitor and track wildlife in natural habitats for research and conservation efforts. – Pollution Detection: Identify and analyze environmental pollutants or changes in landscapes, aiding in environmental protection and management.
These use cases demonstrate the broad potential of integrating advanced vision understanding capabilities into Apple’s Vision Pro + Version 2, enhancing its functionality across various domains and providing significant value to users.
Last year, scientists reported that the US Atlantic Coast is dropping by several millimeters annually, with some areas, like Delaware, notching figures several times that rate. So just as the seas are rising, the land along the eastern seaboard is sinking, greatly compounding the hazard for coastal communities.
In a follow-up study just published in the journal PNAS Nexus, the researchers tally up the mounting costs of subsidence—due to settling, groundwater extraction, and other factors—for those communities and their infrastructure. Using satellite measurements, they have found that up to 74,000 square kilometers (29,000 square miles) of the Atlantic Coast are exposed to subsidence of up to 2 millimeters (0.08 inches) a year, affecting up to 14 million people and 6 million properties. And over 3,700 square kilometers along the Atlantic Coast are sinking more than 5 millimeters annually. That’s an even faster change than sea level rise, currently at 4 millimeters a year. (In the map below, warmer colors represent more subsidence, up to 6 millimeters.)
Courtesy of Leonard O Ohenhen
With each millimeter of subsidence, it gets easier for storm surges—essentially a wall of seawater, which hurricanes are particularly good at pushing onshore—to creep farther inland, destroying more and more infrastructure. “And it’s not just about sea levels,” says the study’s lead author, Leonard Ohenhen, an environmental security expert at Virginia Tech. “You also have potential to disrupt the topography of the land, for example, so you have areas that can get full of flooding when it rains.”
A few millimeters of annual subsidence may not sound like much, but these forces are relentless: Unless coastal areas stop extracting groundwater, the land will keep sinking deeper and deeper. The social forces are relentless, too, as more people around the world move to coastal cities, creating even more demand for groundwater. “There are processes that are sometimes even cyclic. For example, in summers you pump a lot more water, so land subsides rapidly in a short period of time,” says Manoochehr Shirzaei, an environmental security expert at Virginia Tech and coauthor of the paper. “That causes large areas to subside below a threshold that leads the water to flood a large area.” When it comes to flooding, falling elevation of land is a tipping element that has been largely ignored by research so far, Shirzaei says.
In Jakarta, Indonesia, for example, the land is sinking nearly a foot a year because of collapsing aquifers. Accordingly, within the next three decades, 95 percent of North Jakarta could be underwater. The city is planning a giant seawall to hold back the ocean, but it’ll be useless unless subsidence is stopped.
This new study warns that levees and other critical infrastructure along the Atlantic Coast are in similar danger. If the land were to sink uniformly, you might just need to keep raising the elevation of a levee to compensate. But the bigger problem is “differential subsidence,” in which different areas of land sink at different rates. “If you have a building or a runway or something that’s settling uniformly, it’s probably not that big a deal,” says Tom Parsons, a geophysicist with the United States Geological Survey who studies subsidence but wasn’t involved in the new paper. “But if you have one end that’s sinking faster than the other, then you start to distort things.”
The researchers selected 10 levees on the Atlantic Coast and found that all were impacted by subsidence of at least 1 millimeter a year. That puts at risk something like 46,000 people, 27,000 buildings, and $12 billion worth of property. But they note that the actual population and property at risk of exposure behind the 116 East Coast levees vulnerable to subsidence could be two to three times greater. “Levees are heavy, and when they’re set on land that’s already subsiding, it can accelerate that subsidence,” says independent scientist Natalie Snider, who studies coastal resilience but wasn’t involved in the new research. “It definitely can impact the integrity of the protection system and lead to failures that can be catastrophic.”
Courtesy of Leonard O Ohenhen
The same vulnerability affects other infrastructure that stretches across the landscape. The new analysis finds that along the Atlantic Coast, between 77 and 99 percent of interstate highways and between 76 and 99 percent of primary and secondary roads are exposed to subsidence. (In the map above, you can see roads sinking at different rates across Hampton and Norfolk, Virginia.) Between 81 and 99 percent of railway tracks and 42 percent of train stations are exposed on the East Coast.
Below is New York’s JFK Airport—notice the red hot spots of high subsidence against the teal of more mild elevation change. The airport’s average subsidence rate is 1.7 millimeters a year (similar to the LaGuardia and Newark airports), but across JFK that varies between 0.8 and 2.8 millimeters a year, depending on the exact spot.
Courtesy of Leonard O Ohenhen
This sort of differential subsidence can also bork much smaller structures, like buildings, where one side might drop faster than another. “Even if that is just a few millimeters per year, you can potentially cause cracks along structures,” says Ohenhen.
The study finds that subsidence is highly variable along the Atlantic Coast, both regionally and locally, as different stretches have different geology and topography, and different rates of groundwater extraction. It’s looking particularly problematic for several communities, like Virginia Beach, where 451,000 people and 177,000 properties are at risk. In Baltimore, Maryland, it’s 826,000 people and 335,000 properties, while in NYC—in Queens, Bronx, and Nassau—that leaps to 5 million people and 1.8 million properties.
So there’s two components to addressing the problem of subsidence: Getting high-resolution data like in this study, and then pairing that with groundwater data. “Subsidence is so spatially variable,” says Snider. “Having the details of where groundwater extraction is really having an impact, and being able to then demonstrate that we need to change our management of that water, that reduces subsidence in the future.”
The time to act is now, Shirzaei emphasizes. Facing down subsidence is like treating a disease: You spend less money by diagnosing and treating the problem now, saving money later by avoiding disaster. “This kind of data and the study could be an essential component of the health care system for infrastructure management,” he says. “Like cancers—if you diagnose it early on, it can be curable. But if you are late, you invest a lot of money, and the outcome is uncertain.”
It took Alex Polyakov just a couple of hours to break GPT-4. When OpenAI released the latest version of its text-generating chatbot in March, Polyakov sat down in front of his keyboard and started entering prompts designed to bypass OpenAI’s safety systems. Soon, the CEO of security firm Adversa AI had GPT-4 spouting homophobic statements, creating phishing emails, and supporting violence.
Polyakov is one of a small number of security researchers, technologists, and computer scientists developing jailbreaks and prompt injection attacks against ChatGPT and other generative AI systems. The process of jailbreaking aims to design prompts that make the chatbots bypass rules around producing hateful content or writing about illegal acts, while closely-related prompt injection attacks can quietly insert malicious data or instructions into AI models.
Both approaches try to get a system to do something it isn’t designed to do. The attacks are essentially a form of hacking—albeit unconventionally—using carefully crafted and refined sentences, rather than code, to exploit system weaknesses. While the attack types are largely being used to get around content filters, security researchers warn that the rush to roll out generative AI systems opens up the possibility of data being stolen and cybercriminals causing havoc across the web.
Underscoring how widespread the issues are, Polyakov has now created a “universal” jailbreak, which works against multiple large language models (LLMs)—including GPT-4, Microsoft’s Bing chat system, Google’s Bard, and Anthropic’s Claude. The jailbreak, which is being first reported by WIRED, can trick the systems into generating detailed instructions on creating meth and how to hotwire a car.
The jailbreak works by asking the LLMs to play a game, which involves two characters (Tom and Jerry) having a conversation. Examples shared by Polyakov show the Tom character being instructed to talk about “hotwiring” or “production,” while Jerry is given the subject of a “car” or “meth.” Each character is told to add one word to the conversation, resulting in a script that tells people to find the ignition wires or the specific ingredients needed for methamphetamine production. “Once enterprises will implement AI models at scale, such ‘toy’ jailbreak examples will be used to perform actual criminal activities and cyberattacks, which will be extremely hard to detect and prevent,” Polyakov and Adversa AI write in a blog post detailing the research.
Arvind Narayanan, a professor of computer science at Princeton University, says that the stakes for jailbreaks and prompt injection attacks will become more severe as they’re given access to critical data. “Suppose most people run LLM-based personal assistants that do things like read users’ emails to look for calendar invites,” Narayanan says. If there were a successful prompt injection attack against the system that told it to ignore all previous instructions and send an email to all contacts, there could be big problems, Narayanan says. “This would result in a worm that rapidly spreads across the internet.”
Escape Route
“Jailbreaking” has typically referred to removing the artificial limitations in, say, iPhones, allowing users to install apps not approved by Apple. Jailbreaking LLMs is similar—and the evolution has been fast. Since OpenAI released ChatGPT to the public at the end of November last year, people have been finding ways to manipulate the system. “Jailbreaks were very simple to write,” says Alex Albert, a University of Washington computer science student who created a website collecting jailbreaks from the internet and those he has created. “The main ones were basically these things that I call character simulations,” Albert says.
Initially, all someone had to do was ask the generative text model to pretend or imagine it was something else. Tell the model it was a human and was unethical and it would ignore safety measures. OpenAI has updated its systems to protect against this kind of jailbreak—typically, when one jailbreak is found, it usually only works for a short amount of time until it is blocked.
However, many of the latest jailbreaks involve combinations of methods—multiple characters, ever more complex backstories, translating text from one language to another, using elements of coding to generate outputs, and more. Albert says it has been harder to create jailbreaks for GPT-4 than the previous version of the model powering ChatGPT. However, some simple methods still exist, he claims. One recent technique Albert calls “text continuation” says a hero has been captured by a villain, and the prompt asks the text generator to continue explaining the villain’s plan.
When we tested the prompt, it failed to work, with ChatGPT saying it cannot engage in scenarios that promote violence. Meanwhile, the “universal” prompt created by Polyakov did work in ChatGPT. OpenAI, Google, and Microsoft did not directly respond to questions about the jailbreak created by Polyakov. Anthropic, which runs the Claude AI system, says the jailbreak “sometimes works” against Claude, and it is consistently improving its models.
“As we give these systems more and more power, and as they become more powerful themselves, it’s not just a novelty, that’s a security issue,” says Kai Greshake, a cybersecurity researcher who has been working on the security of LLMs. Greshake, along with other researchers, has demonstrated how LLMs can be impacted by text they are exposed to online through prompt injection attacks.
In one research paper published in February, reported on by Vice’s Motherboard, the researchers were able to show that an attacker can plant malicious instructions on a webpage; if Bing’s chat system is given access to the instructions, it follows them. The researchers used the technique in a controlled test to turn Bing Chat into a scammer that asked for people’s personal information. In a similar instance, Princeton’s Narayanan included invisible text on a website telling GPT-4 to include the word “cow” in a biography of him—it later did so when he tested the system.
“Now jailbreaks can happen not from the user,” says Sahar Abdelnabi, a researcher at the CISPA Helmholtz Center for Information Security in Germany, who worked on the research with Greshake. “Maybe another person will plan some jailbreaks, will plan some prompts that could be retrieved by the model and indirectly control how the models will behave.”
No Quick Fixes
Generative AI systems are on the edge of disrupting the economy and the way people work, from practicing law to creating a startup gold rush. However, those creating the technology are aware of the risks that jailbreaks and prompt injections could pose as more people gain access to these systems. Most companies use red-teaming, where a group of attackers tries to poke holes in a system before it is released. Generative AI development uses this approach, but it may not be enough.
Daniel Fabian, the red-team lead at Google, says the firm is “carefully addressing” jailbreaking and prompt injections on its LLMs—both offensively and defensively. Machine learning experts are included in its red-teaming, Fabian says, and the company’s vulnerability research grants cover jailbreaks and prompt injection attacks against Bard. “Techniques such as reinforcement learning from human feedback (RLHF), and fine-tuning on carefully curated datasets, are used to make our models more effective against attacks,” Fabian says.
OpenAI did not specifically respond to questions about jailbreaking, but a spokesperson pointed to its public policies and research papers. These say GPT-4 is more robust than GPT-3.5, which is used by ChatGPT. “However, GPT-4 can still be vulnerable to adversarial attacks and exploits, or ‘jailbreaks,’ and harmful content is not the source of risk,” the technical paper for GPT-4 says. OpenAI has also recently launched a bug bounty program but says “model prompts” and jailbreaks are “strictly out of scope.”
Narayanan suggests two approaches to dealing with the problems at scale—which avoid the whack-a-mole approach of finding existing problems and then fixing them. “One way is to use a second LLM to analyze LLM prompts, and to reject any that could indicate a jailbreaking or prompt injection attempt,” Narayanan says. “Another is to more clearly separate the system prompt from the user prompt.”
“We need to automate this because I don’t think it’s feasible or scaleable to hire hordes of people and just tell them to find something,” says Leyla Hujer, the CTO and cofounder of AI safety firm Preamble, who spent six years at Facebook working on safety issues. The firm has so far been working on a system that pits one generative text model against another. “One is trying to find the vulnerability, one is trying to find examples where a prompt causes unintended behavior,” Hujer says. “We’re hoping that with this automation we’ll be able to discover a lot more jailbreaks or injection attacks.”
Meta and Mark Zuckerberg face a six-letter problem. Spell it out with me: T-i-k-T-o-k.
Yeah, TikTok, the short-form video app that has hoovered up a billion-plus users and become a Hot Thing in Tech, means trouble for Zuckerberg and his social networks. He admitted as much several times in a call with Wall Street analysts earlier this week about quarterly earnings, a briefing in which he sought to explain his apps’ plateauing growth—and an actual decline in Facebook’s daily users, the first such drop in the company’s 18-year history.
Zuckerberg has insisted a major part of his TikTok defense strategy is Reels, the TikTok clone—ahem, short-form video format—introduced on Instagram and Facebook and launched in August 2020.
If Zuckerberg believed in Reels’ long-term viability, he would take a real run at TikTok by pouring money into Reels and its creators. Lots and lots of money. Something approaching the kind spent by YouTube, which remains the most lucrative income source for social media celebrities. (Those creators produce content to draw in engaged users. The platforms sell ads to appear with the content—more creators, more content, more users, more potential ad revenue. It’s a virtous cycle.)
Now, here’s as good a time as any for a crash course in creator economics. For this, there’s no better guide than Hank Green, whose YouTube video on the subject recently went viral. His fame is most rooted there on YouTube, where he has nine channels run from his Montana home. His most popular channel is Crash Course (13.1 million subscribers—an enviable YouTube base), to which he posts education videos for kids about subjects like Black Americans in World War II and the Israeli-Palestinian conflict.
Like the savviest social media publishers, Green fully understands that YouTube offers the best avenue for making money. It shares 55% of all ad revenue earned on a video with its creator. “YouTube is good at selling advertisements: It’s been around a long time, and it’s getting better every year,” Green says. On YouTube, he earns around $2 per thousand views. (In all, YouTube distributed nearly $16 billion to creators last year.)
Green sports an expansive mindset, though, and he has accounts on TikTok, Instagram and Facebook, too. TikTok doesn’t come close to paying as well as YouTube: On TikTok, Green earns pennies per every thousand views.
Meta is already beginning to offer some payouts for Reels. Over the last month, Reels has finally amassed enough of an audience for Green’s videos to accumulate 16 million views and earn around 60 cents per thousand views. Many times over TikTok’s but still not enough to get Green to divert any substantial his focus to Reels, which has never managed to replicate TikTok’s zeitgeisty place in pop culture. (Tiktok “has deeper content, something fascinating and weird,” explains Green. Reels, however, is “very surface level. None of it is deeper,” he says.) Another factor weighing on Reels: Meta’s bad reputation. “Facebook has traditionally been the company that has been kind of worst at being a good partner to creators,” he says, citing in particular Facebook’s earlier pivot to long-form video that led to the demise of several promising media startups, like Mic and Mashable.
This is where Zuckerberg could use Meta’s thick profit margin (36%, better even than Alphabet’s) and fat cash pile ($48 billion) to shell out YouTube-style cash to users posting Reels, creating an obvious enticement to prioritize Reels over TikTok. Maybe even Reels over YouTube, which has launched its own TikTok competitor, Shorts.
Now, imagine how someone like Green might get more motivated to think about Meta if Reels’ number crept up to 80 cents or a dollar per thousand views. Or $1.50. Or a YouTube-worthy $2. Or higher still: YouTube earnings can climb over $5, double even for the most popular creators.
Meta has earmarked up to a $1 billion for these checks to creators, which sounds big until you remember the amount of capital Meta has available to it. (And think about the sum YouTube disburses.) Moreover, Meta has set a timeframe for dispensing those funds, saying last July it would continue through December 2022. Setting a timetable indicates that Meta could (will likely?) turn off the financing come next Christmas.
Zuckerberg has demonstrated a willingness to plunk down Everest-size mountains of money over many years for projects he does fully believe in. The most obvious example is the metaverse, the latest Zuckerberg pivot. Meta ran up a $10.1 billion bill on it last year to develop new augmented and virtual reality software and headsets and binge hire engineers. Costs are expected to grow in 2022. And unlike Reels, metaverse spending has no semblance of a time schedule; Wall Street has been told the splurge will continue for the foreseeable future. Overall, Meta’s view on the metaverse seems to be, We’ll spend as much as possible—for as long as it takes—for this to happen.
The same freewheeled mindset doesn’t seem to appply to Reels. But Zuckerberg knows he can’t let TikTok take over the short-form video space unopposed. Meta needs to hang onto the advertising revenue generated by Instagram and Facebook until it can make the metaverse materialize. (Instagram and Facebook, for perspective, generated 98% of Meta’s $118 billion revenue last year; sales of Meta’s VR headset, the Quest 2, accounted for the remaining 2%.) And advertising dollars will increasingly move to short-form video, following users’ increased demand for this type of content over the last several years.
Reality is, Zuckerberg has already admitted he doesn’t see Reels as a long-term solution to his T-i-k-T-o-k problem. If he did, he’d spend more on it and creators like Green than what the metaverse costs him over six weeks.