Abstracts

Sunsets, safety and singularities

Things are changing in mathematics, and in the world, due to AI. Mathematicians, physicists and academics of all stripes can help our socities understand and navigate these changes. I’ll speak a bit about my own journey into the field of AI alignment from pure math, what I think is deep and interesting about the problem, and I’ll present some of the work of my collaborators at Timaeus and elsewhere. I hope to convince you that it is possible to work hard, think deep and make a real difference in this space, even on short timelines.

Geometry and learning dynamics of Deep Linear Networks

Deep Linear Networks (DLNs) provide a mathematically tractable setting in which to study the geometry and learning dynamics of deep neural networks. Although their input-output map is linear, their parameterization exhibits a rich algebraic structure. In this talk, I will review important results on the loss landscape of DLNs: every critical point is either a saddle point or a global minimum. I will then explain how tools from quiver representation theory refine the variety of global minima into a stratification indexed by geometric invariants, such as rank patterns. Finally, I will discuss how depth, width, and initialization organize the learning dynamics into distinct regimes, including a saddle-to-saddle regime associated with better generalization properties and a linear regime associated with memorization.

AI Safety: interests and responsibility

In this participatory and discussion-based session, we will reflect on themes from the workshop and the wider field of AI Alignment through the lens of interests and responsibility. Recognising that human societies accommodate a diversity of values and priorities, sometimes conflicting and often not mathematically solvable, we will ask how research goals are established, by whom, and whom these affect and how. We will pair these with questions about the interests served by AI Alignment and how research in this field affects questions of trust, risk, consent, harm and benefit, responsibility, and accountability.

The path from QFT to AI alignment

In this informal talk, I will give an account on how — starting from ongoing discussions within the broad QFT community — I got interested in and ended up taking a full-time job to work on alignment research. To make the discussion concrete, I will briefly discuss a few strange (and to date unexplained) behaviours observed in LLMs, and how I think mathematics and physics can help making sense of those. I will end by outlining various ways to get involved in the research effort to make AI safe.