Two employees from OpenAI and Anthropic have resigned and issued public warnings about the direction of AI companies.
Mrinank Sharma, who studied engineering at Cambridge and machine learning at Oxford before leading Anthropic’s safeguards research team, resigned on Monday, saying “the world is in peril.”
Zoë Hitzig, researcher at OpenAI, announced her departure on Wednesday via an editorial in The new York Times in which she expressed “deep reservations” about the company’s strategy.
Anthropic is one of the world’s leading AI companies and its chatbot Claude has become popular with coders and businesses, while OpenAI is developing ChatGPT. Sharma’s team developed defenses against AI-assisted bioterrorism and studied AI sycophancy, which has become a growing concern as people become more emotionally dependent on robots.
“Pressure to put aside what matters”
He wrote on
“Additionally, throughout my time here, I have seen repeatedly how difficult it is to truly let our values govern our actions. I have seen this in myself, within the organization, where we are constantly faced with pressure to put aside what matters most, and in society as a whole as well. It is by holding this situation and listening as best I can that it becomes clear what I need to do.”
Sharma said he now hopes to pursue poetry studies.
“Potential for user manipulation”
Hitzig, who spent two years on product and security at OpenAI, said she was resigning over the company’s decision to serve ads in ChatGPT. She wrote: “For several years, ChatGPT users have generated an archive of unprecedented human frankness, in part because people thought they were talking about something that had no ulterior intent.
“Users interact with an adaptive, conversational voice to which they reveal their most private thoughts. People talk to chatbots about their medical fears, their relationship problems, their beliefs about God and the afterlife. Advertising built on these archives creates the potential to manipulate users in ways that we don’t have the tools to understand, let alone prevent.”
An existential threat?
Hieu Pham, a member of the OpenAI technical team, posted on
Anthropic was created by researchers who left OpenAI due to disagreements over AI safety. Dario Amodei, its chief executive, recently warned that AI could eliminate half of white-collar jobs.
Fears about the pace of AI disruption have been fueled by Anthropic’s release of AI tools that act as agents on behalf of the user, proofreading legal documents and performing legal and data-related tasks. This led to a massive sell-off in software stocks.
• Robert Colvile: If you’re not terrified of AI, you’re not paying attention
An essay by Matt Shumer, the tech entrepreneur, warning that AI would disrupt the global workforce more profoundly than the pandemic, went viral this week, after being viewed more than 40 million times on X.
Elon Musk’s company xAI is also facing an exodus. Two of its co-founders left this week, leaving only half of the original 12.
Andrea Miotti, founder and chief executive of ControlAI, a campaign group seeking to reduce the risks of AI, said: “I think this trend will only continue as employees of these companies grapple with the threat posed by the technology they are building. These companies explicitly aim to build superintelligence, technology that even AI CEOs themselves declare poses an extinction risk to humanity.”
The rapidly evolving AI sector has become accustomed to high staff turnover, with sought-after researchers lured away by competitors or striking out on their own at start-ups. Internal disagreements are also common as companies scale and market, straying from their original goals.
Elon Musk said xAI was ‘hiring aggressively’
HAKAN NURAL/ANADOLU/GETTY IMAGES
OpenAI has disbanded its “Mission Alignment Team,” responsible for ensuring that A.I. benefits all humanityaccording to the Platformer newsletter.
OpenAI highlighted comments from Fidji Simo, its CEO, on the Access podcast in which she stated that “ads are not going to influence LLM results” and “ads will always be very clearly separated and demarcated from the content.”
Musk posted on
“We are recruiting aggressively.”
Anthropic and xAI have been contacted for comment.