algorithmic-opacity

Chapter 9 of Weapons of Math Destruction examines the use of mathematical models in evaluating teacher performance, specifically focusing on the Washington, D.C. school district's value-added model (VAM) implemented through the IMPACT evaluation system. The chapter highlights the complexity and opacity of these models, their reliance on limited data, and their failure to incorporate meaningful feedback, resulting in unfair and destructive outcomes such as the dismissal of effective teachers based on flawed algorithmic scores. This example illustrates the broader dangers of WMDs, which perpetuate bias and punish individuals without accountability or transparency.

This chapter explores the widespread use of automated personality tests by companies during the hiring process, focusing on the case of Kyle Behm, who was repeatedly rejected for minimum-wage jobs due to such tests despite his qualifications. It critiques these tests as opaque, legally questionable, and poor predictors of job performance, highlighting how they serve primarily to exclude applicants cheaply rather than identify the best employees. The chapter also contrasts the feedback-driven analytics in professional sports with the stagnant and unaccountable use of hiring algorithms in low-wage labor markets.

Chapter 10 explores how powerful tech companies like Facebook and Google use opaque algorithms to influence civic life, including voting behavior and political opinions. It highlights Facebook's experiments on voter turnout and emotional contagion, illustrating the vast but hidden power these platforms wield over public discourse and democracy. The chapter also raises concerns about the potential for these algorithms to become political weapons of math destruction, despite the lack of current evidence of malicious intent.