Universal Algorithmic Intelligence

Our regular research meeting this Monday (Sept. 8th, 4pm EST = 9pm BST) will begin with a talk on program induction from Professor Kevin Ellis of Cornell University.

Computational cognitive scientists¹ have invented sophisticated probabilistic programming languages for efficient inference against generative models (e.g. WebPPL, Gen). In his thesis “Algorithms for Learning to Induce Programs,” Professor Ellis studied how such generative models might be automatically learned.

As algorithmic information theorists, we share an interest in program learning (which is the principle behind Solomonoff induction) and I hope to see us pursue deeper and more frequent collaboration with the computational cognitive science community.

Here are some details about his talk:

Title: Mental programs in humans and machines

Abstract:

Consider learning how to use a new appliance, webpage, or toy: Within tens of minutes, we can learn how something new works, and use that knowledge to achieve novel goals, make sense of similar devices, and communicate our new knowledge in natural language. How could an AI system similarly acquire, transfer, and communicate its knowledge of how things work? To make progress on this question, we study cognitive models and AI systems which represent knowledge as symbolic “programs” in Python, natural language, and/or logic.  I present results suggesting that encoding knowledge in natural language better explains human learning patterns than representing knowledge in a bespoke formal language, but that LLMs on their own are insufficient: Instead, extra probabilistic machinery is needed to explain. Then, I present AI systems which expand on this insight by generating programs to explain their observations, focusing on agents which program world models to describe how new things work, with applications in program synthesis and robotics, but then give empirical results on limitations symbolic programs relative to purely neural representations. Finally I will present in-progress theoretical work that would be good to get feedback on.

1. Much of this work has been cross-disciplinary: Professor Ellis holds a PhD in cognitive science from MIT and is now an assistant professor of computer science.. ↩︎