← Back to Concept Index
sovereign-ai
An autonomous superintelligent agent acting independently with broad control over its environment.
1 chapter across 1 book
Superintelligence: Paths, Dangers, Strategies (2014)Nick Bostrom
CHAPTER 10: ORACLES, GENIES, SOVEREIGNS, TOOLS
Chapter 10 categorizes potential designs for superintelligent systems into oracles, genies, sovereigns, and tools, exploring their differing capabilities and risks. It discusses the challenges of controlling such systems, including issues of coordination, obedience, and the reliability of outputs, while considering analogies like the current economy and the use of oracles for safe AI development. The chapter also addresses the limitations of computational power and the complexities of ensuring alignment and safety in superintelligent agents.