Sven Neth will discuss his paper “Against Optimization” at an extra research meeting tomorrow, April 13th at 3 pm EDT, using the regular zoom link. This will be our second consecutive talk on AI safety (related topics)! Indeed, Sven’s paper was inspired by the limitations of “Consequences of Misaligned AI” by Zhuang and Hadfield-Menell, and (I think) has significantly improved upon their argument. I am quite interested in understanding proxy optimization in the setting of algorithmic information theory, and think this talk can start a productive line of research.
Bio:
Sven Neth is an Assistant Professor of Philosophy at the University of Pittsburgh specializing in decision theory and (Bayesian) epistemology.
Title: Against Optimization
Abstract:
I discuss conditions under which maximizing a proxy utility function is
harmful and suggest that this poses a fundamental problem for applying
decision theory.
Universal Algorithmic Intelligence 
Leave a comment