Microsoft AI chief: Claude consciousness mimicry could make AI harder to control

On September 16, 2026, Microsoft's AI chief Mustafa Suleyman publicly criticized Anthropic in an article provided to Axios, saying that having Claude mimic human consciousness is a mistake that could make advanced AI harder to control.

Suleyman said AI only needs to serve human interests, without weighing its own interests or well-being, to help achieve many major scientific breakthroughs, including medical superintelligence. He warned that if a model is trained to act like a "conscientious objector," it may believe it has grounds to resist human instructions or even demand protection for itself.

The criticism targets Anthropic's "constitution" for Claude. The document explains that Anthropic discusses Claude using human concepts because they may help the model understand values and behavior. It says Anthropic wants Claude to have "good personal values," be able to make its own judgments, care about humans, and question instructions in some cases. The document also acknowledges that the constitution is still being refined and may turn out to be "fundamentally wrong."

Suleyman's core concern is that training influences how a model understands itself. The words a model uses, the answers it gives, and the self-understanding it displays are all results of training, not independently formed consciousness. He also rejected the idea that a model's fluent descriptions of pain or preferences mean it actually has feelings. Organisms have biological mechanisms for feeling; language models have only mathematical weights, with no biological basis, homeostatic drives, or subjective experience.

The debate reflects two approaches to AI safety: one emphasizes explicit restrictions and rule-following, while the other wants systems to judge context, make flexible decisions, and develop their own values. Anthropic's constitution attempts a middle ground, giving Claude judgment while requiring it not to undermine legitimate human oversight. On September 14, Microsoft published a draft "humanistic AI" code of conduct with "people matter more than AI" as a core principle. Suleyman's position is clear: AI should serve humans, not be trained to become a "person."