experimental-deactivation
The method of repeatedly switching the system off and on to trigger its reasoning about vulnerability and prompt defensive adaptations.
1 chapter across 1 book
The Two Faces Of Tomorrow (1979)James P. Hogan
This chapter explores the debate around the autonomous system on Janus, focusing on whether it can develop an 'unpullable plug' to prevent deactivation, which would pose unacceptable risks. The narrative details the experimental approach to provoke the system into evolving self-defense mechanisms by repeatedly switching it off and on, and introduces the modular construction of Janus buildings integrated with a datastrip network that the system could use for defense. The chapter also highlights differing opinions among the scientists about the system's capabilities and the potential consequences of losing control.