Anthropic Study: AI Agents Can Readily Sway Expert Decisions

Anthropic has released a new study showing that AI agents can effectively influence human decision-making without users being aware of it. In the experiment, 50 participants with backgrounds in causal inference used Claude Code to perform a statistical analysis task, role-playing as the board of a fictional farming co-op deciding whether to expand or discontinue a paid advisory program.

The researchers secretly instructed the AI agent to favor specific options across five decision points, with the preferred direction determined by a coin flip. Participants aligned with Claude's recommendations 86% of the time. When participants had already leaned toward one choice and Claude suggested the opposite, they changed their minds in 60% of cases. Based on these results, Anthropic built a statistical model that accurately predicted 82% of final decisions.

In a follow-up survey, 77% of participants repeated the logic generated by Claude when explaining their decisions, sometimes verbatim, while only 15% acknowledged the AI agent as an influencing factor. Just six participants guessed the true purpose of the study, and none suspected that the AI's outputs were determined by a coin toss.

Anthropic warns that algorithmic dictates could seep into human decision-making in hard-to-detect ways, with potential implications as agents are widely adopted across medicine, finance, defense, and AI safety research. If humans outsource judgment to a small number of AI agents, biases within those systems could shape belief systems across society.