Preview
Open Original
Published on November 6, 2025 8:36 AM GMT
Inspired by Daniel and Romeo https://x.com/DKokotajlo/status/1986231876852527270, I’m posting a memo on Takeoff, which I plan to expand into a proper article.
This memo comes from notes I took from a talk from Daniel, which have been synthesized by Claude for breviy:
Key Predictions
- Transformative AI timeline: Late 2028
- Key prediction: All existing benchmarks solved by 2030
- p(doom): 10%
Current Gaps
- Reasoning capabilities (mostly solved)
- Agency and long-horizon skills:
- Error correction
- Goal pursuit</l…
Published on November 6, 2025 8:36 AM GMT
Inspired by Daniel and Romeo https://x.com/DKokotajlo/status/1986231876852527270, I’m posting a memo on Takeoff, which I plan to expand into a proper article.
This memo comes from notes I took from a talk from Daniel, which have been synthesized by Claude for breviy:
Key Predictions
- Transformative AI timeline: Late 2028
- Key prediction: All existing benchmarks solved by 2030
- p(doom): 10%
Current Gaps
- Reasoning capabilities (mostly solved)
- Agency and long-horizon skills:
- Error correction
- Goal pursuit
- Hierarchical planning
- Data-efficient learning
Proposed Solutions
- New models and architectures (e.g., recurrence)
- Language Model Programs (LMP)
- Scaffolding systems
- AI bureaucracies
- Increased inference compute
- “Just train it to have those skills” approach
Stages of AI Development
The RE→RS→G→ASI Framework
- Research Engineer for AI Research (we are entering here with Kosmos)
- “Devin actually works”
- Significant speedup in AI research
- Research Scientist
- “CEO talks to the cluster”
- Autonomous white paper production
- Genius
- Qualitatively better than human researchers
- Artificial Superintelligence
- Capable of anything
- Equivalent to 100k human teams
Technical Specifications:
Compute Requirements
- 10^14 text predictions/rollouts on agentic paths
- 10^23 floating point operations for transformative capabilities
- GPT-4 pre-training: ~2.15 × 10^25 FLOPS for comparison
Time Horizons
- t-AGI (10 second AGI to 10 day AGI)
- Year-AGI scaling
- Annual 1 OOM (order of magnitude) improvements expected
Critical Questions and Concerns
On Full Automation
- Robotics gap: “If TAI is Industrial Revolution level, what about physical automation?”
- Job stickiness: Institutional and social lags in job displacement
- “Wet robots”: Humans as interim solution
The Capitalism-AI Nexus
- “Paperclips are a metaphor for money”
- “Capitalism is the ultimate Turing test”
- AI systems are “incentivized to be constrained by human values”
Societal Predictions
Early prediction market odds:
- 1/3 Futurama scenario
- 1/4 Fizzle out
- 1/5 Dystopia
Risk Scenarios
- Cyberpunk 2077-style hacking risks
- 43% of Americans believe civil war is somewhat likely
- Democracy at its peak by voter count (2024)
- Unprecedented political deepfakes
- Cultural homogenization
- Human disempowerment (sci-fi scenarios)
- Potential for singleton control
- “OpenAI coming for everybody’s jobs is a solace” -
Basic Critiques
- Logarithmic growth in atoms, exponential growth in bits
- “Folk theorem - bigger (model) is better”
- “We’re very different from current machines”
- “Brains are way more complex than even the most sophisticated NNs”
- Adaptation and time remain key differentiators
Alternative Approaches
- Virtual societies and digital twins
- AI safety as cancer immune response
- Automated social science research
- a deceptively simple solution to alignment: “Train them to be honest”
Discuss