On Thursday October 15, 2026, from 12:00–13:30 PM (UTC-6), or 18:00–19:30 PM (UTC), our Discord community will hold a study group meeting!
This is a study group organised by members of our community. The aim of this study group is to share questions, solve exercises, and discuss thoughts on AGI. The focus of this next meeting is described below.
Meeting link: https://meet.google.com/qir-iujx-fpi
Study group Google doc: https://docs.google.com/document/d/1vYVT7Z2ZoHmIqDGyY6y9n6m2pYvKvrPEgXX_09TTmAI/edit?tab=t.0
Core book
Hutter, Quarel & Catt (2024). Open the book PDF | Author’s resource page | Book DOI
Preparation for the first meeting
A flexible reading pace: Chapter 2 is broad and technical. It is absolutely fine if you are not able to get through every section before Thursday. The additional topics and readings are listed later in this document so that, when time permits, you can return to unfinished material before our next meeting, alongside any further readings shared for that session.
Note: Page references are to printed book pages (PDF page = printed page + 15).
Core book readings
(Try completing at least these if you aren’t able to complete everything in chapter 2)
| READ IN THE BOOK | FOCUS |
|---|---|
| Chapter 1, pp. 3–11 | Motivation for UAI; prediction versus action; Occam’s razor; Solomonoff induction; the AIXI agent. Skim pp. 12–15 for the roadmap. |
| Chapter 2 intro, pp. 16–18; §2.1.2, pp. 20–22 | Why bits and prefix-free encodings matter. |
| §2.2.5, pp. 37–40; §2.4, pp. 67–74 | Conditional probability, priors, likelihoods, posterior beliefs and predictive distributions. |
| §2.5.1, pp. 76–78 | Shannon entropy and uncertainty in a source. |
| §2.7.1–2.7.2, pp. 102–106 | Describing simplicity mathematically through Kolmogorov complexity. |
Videos and supporting resources
Bayes’ Theorem, Clearly Explained!!!! — StatQuest Required · ~12 minutes
Relates to §2.4. Follow how evidence updates the probability of a hypothesis.
Reinventing Entropy | Compression is Intelligence Part 1 — 3Blue1Brown Required · watch 14:47–31:14
Relates to §2.5. Why information and code length are linked to probability.
Optional videos for deeper intuition
Marcus Hutter: Universal Artificial Intelligence, AIXI, and AGI — Lex Fridman #75 Optional
Suggested excerpts: 05:48–20:06 (Occam, Solomonoff, Kolmogorov); 26:03–40:00 (intelligence, AIXI).
Halting Problem in Python — Computerphile Optional
Relates to §2.6; an accessible demonstration of a key limit of computation.
Read and interact
Seeing Theory: Bayesian Inference — Brown University
Use the “Prior to Posterior” coin-flip experiment to see how beliefs change after new observations.
Official UAI book resource page — Marcus Hutter
Book PDF, shorter introductory materials, slides and related resources.
Information — Stanford Encyclopedia of Philosophy
Optional conceptual background on information, algorithmic complexity and Solomonoff induction; see §6.5.
Left-over Chapter 2 supplementary readings
Binary-string foundations — §2.1.1, §2.1.3–2.1.4 (pp. 18–23)
Finite/infinite strings, encodings, cylinder sets and exercises. Use the book as the principal reference; compare the role of code lengths with MacKay’s chapter on integer codes.
Supplementary reading: MacKay, ch. 7 (Codes for Integers)
Probability, expectation and convergence — §2.2.1–2.2.4, §2.2.6–2.2.9 (pp. 23–59)
Probability axioms and measures, random variables, expectation and variance, probability inequalities, laws of large numbers, and different types of convergence.
Supplementary reading: MIT OCW 6.041 lecture notes (L1, L5–6, L15, L23) • Seeing Theory: Probability Distributions
Statistical inference and likelihood — §2.3 (pp. 60–66)
How sample observations lead to parameter estimates; maximum likelihood and consistency.
Supplementary reading: Seeing Theory: Frequentist Inference • Seeing Theory textbook draft, ch. 4
Codes, divergence and compression — §2.5.2–2.5.7 (pp. 79–88)
Shannon–Fano coding, KL divergence, the Kraft inequality, source coding and arithmetic coding.
Supplementary reading: MacKay, chs. 4–6 • Goodfellow et al., ch. 3, §3.13
Computability and the halting problem — §2.6 (pp. 89–101)
Turing machines, undecidability, semicomputable functions and the arithmetic hierarchy.
Supplementary reading: MIT OCW 18.404J notes (lectures 5, 6, 8) • SEP: Computability and Complexity
Kolmogorov complexity, MDL and limits — §2.7.3–2.7.7 (pp. 107–118)
Complexity bounds, the minimum description length principle, approximation, and its relation to Shannon entropy.
Supplementary reading: SEP: Information, §6.5 • Grünwald: MDL tutorial, ch. 1
Distances, semimeasures and probability-zero events — §§2.8–2.9 (pp. 118–135)
Probability-distribution distances (including KL and Hellinger), subprobability/semimeasure issues, conditioning on zero-probability events, plus the chapter’s history and references. The book itself is the primary source for the specialized semimeasure discussion.
Supplementary reading: Goodfellow et al., §3.13 (KL divergence) • UAI book PDF, §§2.8–2.9
Leave a Reply