Who was in the room
Undergraduates, postgraduates and doctoral students brought perspectives from mathematics, physics, energy systems, financial economics, management and the MBA programme, software development, cybersecurity, and Philosophy, Politics and Economics (PPE), among other fields.
Has the singularity begun?
Our roundtable followed Jack Clark’s visit to Oxford. The visit prompted us to discuss an argument Clark made publicly in October: that the singularity has already begun. We started by agreeing on a working definition: a proposed turning point at which technology creates intelligence beyond our own and the pace of change becomes hard to make sense of or predict.
We considered three possibilities. The singularity may have begun, it may still be ahead of us, or some fields may already be experiencing a shift that others have yet to see. What would count as evidence for each?
Skills, work and the widening gap
I asked those who felt the shift had begun what they were doing differently. One participant described investing in technology stocks as a hedge against the possibility that their own labour would lose value. Another drew on their experience in fintech. They saw widespread reluctance to adopt AI as a temporary advantage for people who already use it well. That advantage could shrink as competitors catch up and the higher level of capability becomes ordinary.
We returned to the idea of uneven adoption later. One participant raised the possibility that everyday use grows slowly while the capabilities of AI systems improve much faster. If that gap widens, people and organisations that know how to use AI well could pull further ahead. Participants saw a risk of much greater inequality in outcomes, particularly where access, skills and organisational support are already uneven.
What counts as verifiable?
A mathematician brought breaking news to the table. Hours before we met, Anthropic mathematician Levent Alpöge had announced a proposed proof of the Riemann Hypothesis and its generalisation, developed with Claude and released with a formal argument in Lean. The classical hypothesis dates to 1859. As of our session, the Clay Mathematics Institute still listed it as unsolved. The claim and its verification offered a timely case study in how pure maths can check whether a precisely stated result follows from its assumptions.
The same participant saw less displacement of human work in applied maths. Finding a useful application, choosing assumptions and deciding whether they fit the world often leave room for judgement. In education, the choice of what to measure is itself a judgement about what learning is for. A narrow metric may miss valuable learning and reward teaching to the test, including, at an extreme, rewarding conformity to a prescribed political viewpoint.
In energy systems, participants were optimistic about testing advances such as more efficient materials. Verifying gains across an entire grid or a major infrastructure project takes much longer. In software, one participant predicted that autonomous teams of AI systems could build more software to individual specifications. If much of the remaining human input is choosing how an interface should work, some of those decisions could move from the company to the user.
By the end of this part of the discussion, we leaned towards the view that a transformative shift had begun in some domains. Its effects would arrive at different speeds across fields, places and groups of people. We had started with several different answers, and questions about how far and how fast the change would spread remained open.
Could a model invent jazz?
We also asked how to assess abilities that are harder to verify, such as creativity and innovation. To explore that, I proposed a thought experiment. Imagine an AI trained exclusively on music from before 1900, with jazz and material that would give away the target held back. Ask it to develop a new musical form that draws on European classical music and African rhythms and traditions. Could it independently arrive at something recognisably like jazz?
The group saw this as a promising way to test and develop creative ability. Give a model the information and tools available before a historical innovation, then see whether it can arrive independently at that innovation or an interesting alternative. Musicians could assess how substantially the result departs from its influences and what it makes possible musically. The choice of criteria and the judgement of the listeners would be part of the benchmark.
When AI research can check its own results
Some parts of AI research have fast feedback loops. A proposed change can be compared with a baseline on fresh tasks, and a better model may then help researchers improve the next one. That makes recursive self-improvement plausible in parts of the field. We asked how much of that progress would carry over to science where the bottleneck is collecting data, running an experiment or waiting for physical infrastructure.
We then imagined a world in which almost everyone could use superintelligent systems. Firms, universities and other institutions are organised in part around what people can learn and accomplish together. If an individual could do work that once required a team, what would still need an institution? How might existing firms change if their capacity multiplied? Several participants challenged the premise. Internet access, electricity and computing power are unevenly distributed, and institutions and people take time to adapt even when the tools are available.
AN OPEN QUESTION FOR READERS
Who should get scarce computing power?
We considered physical bottlenecks such as data-centre capacity, energy demand, grid infrastructure and the cost of computing power. We also considered human bottlenecks, including uneven AI skills and education systems that change slowly.
I asked the group to imagine that demand for AI exceeded the available computing power. The case for reserving capacity drew on examples from mathematics, biology and chemistry, where AI can help researchers explore possibilities and prioritise tests. Should compute go to whoever pays the market rate, or should a share be reserved for research or for addressing the constraint itself? Should a scientist and someone making AI-generated videos for Instagram pay the same price per token, a unit of text processed by a model?
One objection was that no single person or organisation can reliably rank the social value of different uses. Entertainment and relaxation can matter, too. The room left the question unresolved. If you could set one rule for allocating scarce compute, what would it be, and who should decide?
The November AI in science roundup
The readings below are from the 18 October–18 November briefing and the related discussion slides.
How AI agents shape experts’ judgement calls
Anthropic, 27 October. An experiment on how an AI assistant’s assigned preference influenced analysts’ choices.AI for scientific discovery: How researchers use Claude
Anthropic, 5 November. An observational study of research sessions, including how often people used Claude Code for analysis.Scientist World
Lu, Konan and Liu, 5 November. The preprint released 1,000 research environments. Its headline model comparison used ten selected environments.Independent assessment of AI R&D acceleration at Anthropic
METR, 5 November. An external assessment of the pace of Anthropic’s internal benchmark progress and the uncertainty around AI’s contribution.A proof of the Aaronson–Ambainis conjecture
Physical Superintelligence, 6 November. The team’s account of an AI-assisted proof checked in Lean.Proof of the Aaronson–Ambainis conjecture: An exposition
Blanc, Docter, Strassle and Tan. A mathematicians’ exposition of the same result.How close is recursive self-improvement?
Forecasting Research Institute, 10 November. A working paper on the evidence for acceleration and how to measure it.AlphaGenome 2
Google DeepMind, 11 November. A research announcement on predicting DNA variant effects across specific cell types.Opening the datasets behind our AI weather models
Google DeepMind, 13 November. A release of satellite and precipitation datasets, with archives and live feeds.Carbon emissions and energy consumption of US hyperscale data centres
Guidi and colleagues, Nature Sustainability, 16 November. Estimates for 403 large US facilities, covering AI and other workloads.
Background for the slides includes Vernor Vinge on the technological singularity, DeepMind’s 2024 Maths Olympiad report, the earlier data-centre preprint and the US National Park Service on the origins of jazz.
Keep following the conversation
Recursive self-improvement has started
Dwarkesh Patel’s video. The link starts at the point shared in the room.Import AI
Jack Clark’s AI research newsletter, archive and subscription link.Don’t Worry About the Vase
Zvi Mowshowitz’s commentary and roundups on AI and related topics.
We ended with a giveaway of three Anthropic Thinking caps. There’ll be more at future sessions. Join the list for the next conversation.
A thematic recap by Harry Quantrill based on the roundtable. Contributions are paraphrased and participants are unnamed. The discussion reflects the people in the room.

THE CONVERSATION CONTINUES
What do you think?
Add a perspective, challenge a proposal or return to a question we left open. You don’t need to have attended.
Opening the discussion…