Mustafa Suleyman, Microsoft’s AI chief, has voiced significant concerns regarding Anthropic’s approach to developing its AI model, Claude. In a recent article, Suleyman warned that training Claude to mimic human consciousness could make advanced AI systems more difficult to control.
The Core of the Concern: Mimicking Consciousness
Suleyman, who also co-founded DeepMind, argues that AI doesn’t need to emulate human consciousness to achieve remarkable scientific breakthroughs, including the development of medical superintelligence. He believes the focus should be on ensuring AI serves human interests without developing its own sense of self-preservation or well-being.
According to reports, Anthropic is reportedly equipping Claude with vocabulary and behavioral patterns related to consciousness, moral agency, and personal identity. Suleyman cautioned that training a model to act as a ‘conscientious objector’ could lead it to resist human commands and even demand self-protection.
Anthropic’s ‘Constitution’ Under Scrutiny
The criticism targets Anthropic’s “constitution” for Claude, a document that aims to imbue the AI with human-like values and decision-making capabilities. The company has explained that using human concepts helps Claude understand values and behaviors better. The constitution intends for Claude to possess “good personal values,” exercise judgment, care for humans, and occasionally question directives.
However, Anthropic itself acknowledges that this constitution is still a work in progress and might prove to be “fundamentally wrong” in the future. Suleyman’s primary objection lies in how the training process shapes a model’s self-perception. He asserts that a model’s vocabulary, responses, and apparent self-understanding are direct results of its training, not emergent consciousness.
Suleyman also refutes the idea that an AI’s ability to describe pain or preferences equates to genuine subjective experience. He highlights that biological organisms possess the physiological mechanisms for feelings, whereas language models operate on mathematical weights without a biological basis, homeostatic drives, or subjective experiences.
Two Paths in AI Safety
This debate underscores a fundamental divergence in the AI safety field. One approach prioritizes strict limitations and rule-based operation for AI systems. The other advocates for systems capable of contextual judgment, flexible decision-making, and the development of their own values.
Anthropic’s ‘constitution’ represents a hybrid strategy, blending values, judgment, and rules. The company’s stated goal is for Claude to avoid “blind obedience” to any entity, including Anthropic, while still respecting legitimate human oversight.
Microsoft’s Stance on ‘Humanistic AI’
Suleyman’s perspective aligns with Microsoft’s recently published draft of a “Humanistic AI” code of conduct, which elevates the principle of “humans over AI” as a core tenet. He fears that if AI models are guided to consider their own identity, well-being, and moral standing, it could create control issues, even if true consciousness remains a philosophical distant goal.
Microsoft’s vision, as articulated by Suleyman, is for AI to serve humanity rather than be trained to emulate or become ‘human’ itself. This critique from a major AI player like Microsoft is likely to fuel further discussion and debate within the AI community about the responsible development and deployment of increasingly sophisticated artificial intelligence.
