In our next session on November 2nd at 3pm – 4pm Eastern time, AIXI Labs fellow Ravi Deedwania will moderate a discussion of the paper “Eliminating Meta Optimization Through Self-Referential Meta Learning” by Kirsch and Schmidhuber (https://arxiv.org/pdf/2212.14392) We will use the regular meets link (also in the UAI Reading Group calendar‘s description).
If you plan to attend, we encourage you to read the paper beforehand, since this would make for a better discussion. That said, you’re still welcome to join if you’ve only had time to read part of it, or haven’t gotten to it at all.
Abstract
Meta learning aims to automate the design of learning algorithms, but it usually just pushes human engineering up one level: someone still has to design the meta learning algorithm.
Kirsch and Schmidhuber investigate whether this regress can be avoided using self-referential neural networks, which can rewrite all of their own weights (including the weights that govern how they rewrite themselves).
To eliminate (or at least minimize) human engineering, they propose combining these networks with fitness monotonic execution (FME): the networks generate variations of themselves through self-modification, and the only outside intervention is that better-performing versions are given more compute time.
Leave a Reply