optimization-power

The effort or resources applied to improve an intelligent system's capabilities.

2 chapters across 1 book

Superintelligence: Paths, Dangers, Strategies (2014)Nick Bostrom

CHAPTER 4

Chapter 4 examines the dynamics of the transition from human-level machine intelligence to radical superintelligence, focusing on the speed and nature of this change. It introduces the concepts of optimization power and system recalcitrance to analyze whether the intelligence explosion will be gradual or sudden. The chapter emphasizes the importance of understanding these factors near the threshold of human-equivalent general reasoning ability.

Chapter 1 about how far away we currently are from developing a machine with human-level general intelligence. Here the question is instead, if and when such a machine is developed, how long will it be from then until a machine becomes radically superintelligent? Note that one could think that it will take quite a long time until machines reach the human baseline, or one might be agnostic about how

The chapter explores the timeline and dynamics of the transition from human-level machine intelligence to radically superintelligent systems, focusing on the concept of the 'takeoff'—the period during which AI capabilities rapidly escalate. It distinguishes between slow, moderate, and fast takeoff scenarios, analyzing their implications for human preparedness and response. The chapter introduces key variables such as optimization power and recalcitrance to frame how quickly intelligence improvements might occur, suggesting that machine intelligence paths to superintelligence likely face low recalcitrance and could lead to explosive takeoffs.