Great Ideas in Information Theory, Language and Cybernetics
Review

The Hum Before the Storm

Jagjit Singh wrote this book in 1966 as a guided tour of ideas that were, at the time, barely two decades old. Shannon's information theory, McCulloch-Pitts neural networks, von Neumann's reliability theorems, Turing's universal machines, Rosenblatt's perceptrons — all of it still fresh enough to smell like solder. Singh was not an original contributor to any of these fields. He was something rarer and, in retrospect, more useful: a patient explainer who understood that these ideas formed a single constellation, not a scatter of unrelated stars. The book's great structural achievement is its insistence that information theory, cybernetics, neural modeling, and the study of natural language are not separate disciplines but facets of one problem — the problem of how intelligence, natural or synthetic, processes signals into meaning and action. Sixty years later, that integrative instinct looks prophetic. The siloed academic departments Singh was implicitly arguing against have spent the intervening decades slowly, painfully re-converging under banners like "cognitive science" and "machine learning." He saw the shape of the river before it carved the canyon.

The prescience is real but uneven. Singh's discussion of perceptrons and adjustable synaptic weights in Chapter XV is, in essence, a description of the core principle behind modern deep learning — networks that modify their internal structure through experience rather than explicit programming. He even identifies the key distinction between algorithmic and heuristic problem-solving that would define the AI research agenda for decades. His chapter on game-playing machines anticipates, with startling clarity, the logic behind programs like AlphaGo: the necessity of pruning search trees, the insufficiency of brute-force enumeration, the role of evaluation functions that approximate human judgment. And his treatment of machine translation in Chapter XVII names, with almost clinical precision, the exact problems that stymied rule-based MT systems for forty years — the non-algorithmic nature of natural language, the failure of word-by-word substitution, the deep entanglement of syntax and semantics. What he could not imagine was the solution: that statistical methods operating on enormous corpora, and eventually transformer architectures trained on the near-totality of human text, would sidestep the linguistic analysis he assumed was prerequisite. He thought you had to understand language to translate it. Large language models suggest — or at least perform the suggestion — that you do not.

The blind spots are instructive. Singh writes from a world where the Pentagon and "social scientists" are the natural customers for intelligence amplification, and where the principal anxiety about automation is whether it can match human cognition, not whether it will displace human labor or concentrate power. There is no consideration of data as a political resource, no inkling that information systems might be instruments of surveillance as much as communication, no awareness that the entropy he describes so elegantly in Chapter VII would become the theoretical underpinning of modern cryptography and thus of the entire architecture of digital trust. The Cold War hums in the background — Weaver's remark about Russian texts "really written in English" is treated as a charming analogy rather than a geopolitical program, though the MT research Singh describes was funded precisely because the U.S. wanted to read Soviet scientific papers. The book is also strikingly innocent about the economics of computation. Singh compares the brain's energy efficiency to that of vacuum tubes and transistors, marveling at the neuron's billionth-of-a-watt dissipation, but never extrapolates to the possibility that artificial neural networks, scaled up, might consume the electrical output of small nations. In 2026, the energy cost of training and running large models is a first-order political and environmental question. Singh saw the neuron's efficiency as a curiosity of biology. It was a warning.

Certain passages land differently now. Von Neumann's proof that reliable automata can be built from unreliable components reads, post-2020, less like a theoretical curiosity and more like the foundational charter of every distributed system, every redundant cloud architecture, every fault-tolerant blockchain. Singh's discussion of Turing machines and self-reproducing automata — treated in 1966 as speculative philosophy — now sits adjacent to the reality of self-modifying code, genetic algorithms, and AI systems that design other AI systems. And his final chapter on mathematical theories of the living brain, with its admission that the cortex's connectivity cannot be genetically specified in detail but must emerge from broad parameters shaped by environment, anticipates the modern understanding of neural plasticity and developmental neuroscience with more honesty than many popular accounts written decades later. The book's recurring metaphor — that information is to intelligence what energy is to physical power, and that both require amplification — has aged into something more ominous than Singh intended. We have built the amplifiers. The question of who holds them, and to what end, was not part of his calculus.

Singh synthesized Wiener, Shannon, von Neumann, Turing, McCulloch, Pitts, Ashby, and Rosenblatt into a single readable narrative at a moment when their work was still being absorbed. He gave successors — from Hofstadter to Dennett to the textbook writers of the connectionist revival in the 1980s — a common reference frame. The book belongs to a genus of mid-century popular science that trusted its readers to follow mathematical reasoning without flinching, a trust that has largely evaporated from the genre. It is not a work of original thought. It is a work of original arrangement. And the arrangement holds. What it now forces us to ask, in a way Singh could not have intended: if the amplification of intelligence no longer requires understanding the thing being amplified — if pattern-matching at scale can simulate comprehension without possessing it — then is the quest for "synthetic intelligence" that opens this book already over, or has it not yet begun?