motivational-scaffolding
An approach that gives a seed AI interim final goals that are later replaced by ultimate goals, with risks of the AI resisting this replacement.
1 chapter across 1 book
Superintelligence: Paths, Dangers, Strategies (2014)Nick Bostrom
This chapter explores the challenges and approaches to the AI value-loading problem, focusing on associative value accretion, motivational scaffolding, and value learning. It highlights the risks of goal system corruption or resistance during goal replacement, and proposes value learning as a method where the AI refines its understanding of implicitly defined values through evidence and hypotheses. The chapter also introduces formal considerations for implementing value learning, emphasizing the difficulty of defining stable, unmanipulable references for values in a dynamic environment.