Ethics of Science

Marc J. Roberts, quoted in Weizenbaum's *Computer Power and Human Reason*, argued that choosing between scientific hypotheses is never value-neutral because the costs of being wrong in each direction are never symmetrical — and measuring those costs requires values, not data. This was a footnote-level insight in 1976. Now it's the invisible chassis underneath every AI safety debate, every precautionary deployment policy, every red-team exercise. The strange part: Roberts framed it as a reason for individual scientists to exercise conscience. What actually happened is that it became institutionalized as risk calculus — conscience subtracted, spreadsheet retained. Weizenbaum warned that ethics is fundamentally about renunciation, and that the word made scientists flinch because it applied to them personally, not abstractly. The hyperstition here is that we've built an entire governance apparatus around asymmetric error costs while successfully deleting the part where anyone specific has to renounce anything. The framework won. The ethic didn't.