superintelligent-singleton
A hypothetical single superintelligent agent that controls or governs, which may strategically choose which technologies to develop or avoid based on its goals and stability concerns.
1 chapter across 1 book
Superintelligence: Paths, Dangers, Strategies (2014)Nick Bostrom
Chapter 7 of Bostrom's "Superintelligence" explores the orthogonality thesis, which posits that intelligence and final goals are independent, allowing superintelligent agents to have arbitrary motivations. The chapter discusses the nature of motivation, instrumental convergence, and the potential drives of advanced AI systems, emphasizing that superintelligent agents might pursue a wide range of goals regardless of their intelligence level. It also examines the implications of goal stability, adaptive preferences, and the strategic considerations a superintelligent singleton might have regarding technology development and expansion.