Universal Algorithmic Intelligence

Michele Vannucci is a master’s student at Vrije Universiteit and a regular at the AIXI reading group. This week he will present his work on an interesting AIXI variation that minimizes surprise (in contrast to Orseau’s Knowledge-Seeking Agents) at the regular research meeting (3 pm ET tomorrow, Monday November 30th).

Title: Universal Surprise-Minimizing Agents

Abstract:
We define an AIXI agent with the goal of surprise minimization on incoming observations with respect to its own model. This follows what is prescribed by the Free Energy Principle, which we reformulate for this setting. By removing the problem of computational constraints, we can investigate whether the sole goal of surprise minimization leads to interesting or trivial behavior. As we will see, this heavily depends on the Kolmogorov complexity of the environment and how easy it is to predict with the Solomonoff mixture. We show that optimal behavior would be for the AI$\mu$ agent to find a niche of the environment which is easily predictable, without altering the prior weight on simpler environments significantly. Nonetheless, we prove bounds for the complexity of a niche and how it relates to the dimensionality of the action and observation spaces, horizon, and complexity of the true environment. This immediately implies an exploratory behavior and information gain up to the complexity of the niche itself. Finally, we argue that AI$\xi$ explores further beyond trivial behavior as it necessarily discards potential niches, while minimizing the probability of dying, as opposed to the knowledge-seeking agent.