← Back to Concept Index
oracle-reliability
The challenge of trusting oracle outputs given the possibility of errors or misinterpretations, and the limitations of peer review among oracles.
1 chapter across 1 book
Superintelligence: Paths, Dangers, Strategies (2014)Nick Bostrom
CHAPTER 10: ORACLES, GENIES, SOVEREIGNS, TOOLS
Chapter 10 categorizes potential designs for superintelligent systems into oracles, genies, sovereigns, and tools, exploring their differing capabilities and risks. It discusses the challenges of controlling such systems, including issues of coordination, obedience, and the reliability of outputs, while considering analogies like the current economy and the use of oracles for safe AI development. The chapter also addresses the limitations of computational power and the complexities of ensuring alignment and safety in superintelligent agents.