principle-of-epistemic-deference
The idea that since a future superintelligence has a superior epistemic vantage point, humans should defer to its judgments whenever feasible.
1 chapter across 1 book
Superintelligence: Paths, Dangers, Strategies (2014)Nick Bostrom
Chapter 13 addresses the challenge of selecting the final value or goal to install in a superintelligence, emphasizing the difficulty and risks of making such a choice based on current human moral understanding. It introduces the concept of indirect normativity as a strategy to delegate the complex task of value selection to the superintelligence itself, anchored by abstract conditions rather than fixed, potentially flawed human values. The chapter discusses Eliezer Yudkowsky's coherent extrapolated volition (CEV) as a prototype for indirect normativity, explaining how it aims to approximate humanity's idealized collective wishes through a process of extrapolation and consensus.