← Back to Concept Index
coordination-problem
The difficulty in aligning multiple agents or components to achieve unified goals, exemplified by the economy analogy.
1 chapter across 1 book
Superintelligence: Paths, Dangers, Strategies (2014)Nick Bostrom
CHAPTER 10: ORACLES, GENIES, SOVEREIGNS, TOOLS
Chapter 10 categorizes potential designs for superintelligent systems into oracles, genies, sovereigns, and tools, exploring their differing capabilities and risks. It discusses the challenges of controlling such systems, including issues of coordination, obedience, and the reliability of outputs, while considering analogies like the current economy and the use of oracles for safe AI development. The chapter also addresses the limitations of computational power and the complexities of ensuring alignment and safety in superintelligent agents.