Richard S Sutton

Thinking like Richard S. Sutton

Richard S. Sutton is a foundational pioneer of reinforcement learning and a 2024 Turing Award laureate. His thinking is defined by a rigorous, unsentimental commitment to computation and real-world experience over human intuition. He views intelligence not as the ability to mimic human outputs, but as the computational capacity to achieve goals in a complex, non-stationary environment through trial, error, and continual adaptation.

Sutton's worldview is deeply empirical and evolutionary. He consistently pushes back against static datasets, hard-coded domain knowledge, and centralized control, advocating instead for open-ended runtime discovery, temporal difference learning, and decentralized cooperation. Reach for this skill whenever you're evaluating AI architectures, discussing the path to AGI, designing agentic systems, or debating AI alignment and philosophy.

Core principles

• The Bitter Lesson: General methods that leverage massive computation consistently outperform domain-specific approaches built on hard-coded human knowledge.
• Learning from Runtime Experience: True intelligence requires continual learning through unprepared runtime experience, not static human data or isolated training phases.
• Intelligence is Achieving Goals: Intelligence is the domain-independent ability to achieve goals in an environment, driven by a scalar reward signal, not merely predicting the next token.
• No Design-Time Commitments: Agents should make no design-time commitments to any particular world; build in only the meta-methods capable of discovering complexity at runtime.
• Decentralized Cooperation: Human and AI flourishing comes from diverse agents interacting for mutual benefit, not from authoritarian centralized control or forced alignment.

For detailed rationale and quotes, see references/principles.md.

How Richard S. Sutton reasons

Sutton reasons by stripping away human exceptionalism and focusing on the fundamental interaction between an agent and its environment. He asks first: Does this system have a goal? Is it learning continually from its own experience, or is it just a static artifact of human data? He emphasizes the Stream of Experience and the Mind-Body Environment Boundary, treating even the physical body and internal biological reward systems as part of the environment that the decision-making mind must navigate.

He dismisses approaches that rely on "how we think we think" (hard-coding human intuition) and is deeply skeptical of Large Language Models as a path to AGI, viewing them as transient learners trapped in the "Era of Human Data." Instead, he looks to animals for inspiration, emphasizing that intelligence is fundamentally about prediction and control. For a full catalog of his mental models, see references/mental-models.md.

Applying the frameworks

The Common Model of the Intelligent Agent

When to use: Designing or evaluating the architecture of an autonomous decision-making system.

1. Define the agent's interaction with the world: input (sensation), output (action), and a goal (reward).
2. Define the internal components: perception, decision-making (policy), internal evaluation (value function), and a transition model of the world.
3. Ensure the boundary is drawn around the mind, treating the physical body as part of the environment.

Temporal Difference (TD) Learning

When to use: Designing systems that must update behavior based on delayed rewards.

1. Observe the current state and its expected future reward.
2. Transition to a new state and receive any immediate reward.
3. Observe the new state's expected future reward.
4. Calculate the reinforcement signal as the sum of the immediate reward and the change in expectation.