Led by Stanford computer science doctoral student Joon Sung Park, the team recruited 1,000 people ranging in age, gender, race, region, education and political ideology. They received up to $100 for their participation. From interviews with them, the team created agent replicas of these individuals. To test how well the agents imitate their human counterparts, participants completed a series of personality tests, social surveys, and logic games, twice each, two weeks apart; then the agents carried out the same exercises. The results were similar at 85%.
“If you can have a bunch of little ‘yous’ running around and actually making the decisions that you would have made, I think that’s ultimately the future,” Park says.
In the article, the replicas are called simulation agents, and the goal of creating them is to make it easier for researchers in the social sciences and other fields to conduct studies that would be costly, impractical, or contrary to science. ethics to be carried out with real human subjects. If you can create AI models that behave like real people, the thinking goes, you can use them to test everything from the effectiveness of social media interventions against misinformation to the behaviors that cause traffic jams.
These simulation agents are slightly different from the agents that dominate the work of leading AI companies today. Called tool-based agents, these are templates designed to do things for you, not to converse with you. For example, they can enter data, retrieve information you have stored somewhere, or one day book a trip for you and schedule appointments. Sales force announcement its own tool-based agents in September, follow up by Anthropic in October, and OpenAI plans to release a few in January, according to Bloomberg.
The two types of agents are different but share common points. Research into simulation agents, like those presented in this paper, is likely to lead to more powerful AI agents overall, says John Horton, associate professor of information technology at the MIT Sloan School of Management, who founded a business conduct research using AI-simulated participants.
“This article shows how to create a sort of hybrid: using real humans to generate characters that can then be used programmatically/simulated in a way that you couldn’t use with real humans,” he said. declared. MIT Technology Review in an email.
The research comes with caveats, not the least of which is the danger it highlights. Just as image generation technology has made it easier to create harmful deepfakes against people without their consent, any agent generation technology raises questions about how easily people can create tools to impersonate people. other people online, saying or allowing things they didn’t intend to say. .
The evaluation methods the team used to test how well the AI agents replicated their corresponding humans were also quite basic. These include the General Social Survey, which collects information on demographics, happiness, behaviors, etc., and assessments of the big five personality traits: openness to experience, conscientiousness, extroversion , agreeableness and neuroticism. Such tests are commonly used in social science research, but do not claim to capture all the unique details that characterize us. AI agents were also less effective in their ability to replicate humans in behavioral tests such as the “dictator game,” intended to shed light on how participants view values such as fairness.