We talk about thought as if it arrives pre-wrapped: insights, beliefs, the occasional flash. But what if thinking is closer to a measurement process—signals crossing thresholds, noise negotiated, models revised in small shivers of surprise. The older I get, the more the center seems to move. Not “What do I think?” but “How does my mind sense its own uncertainty?” That is where information sneaks in. Not “data.” Signals. Constraints. The grain of the world that pushes back.
Entropy, Surprise, and the Feeling of Knowing
Information theory begins with a simple provocation: uncertainty can be counted. Claude Shannon’s entropy H measures the expected “surprise” of a source. Flatter distributions—more possible messages, each with similar chance—carry higher entropy. A message reduces uncertainty by eliminating alternatives; fewer alternatives left, fewer bits needed next time. This is math, not mysticism, yet it touches the subjective. The jolt of a correct guess is a drop in entropy felt from the inside.
Now put that next to metacognition: the mind’s way of modeling its own knowing. Confidence ratings, the tip‑of‑the‑tongue phenomenon, calibration (“I’m 70% sure”), and strategic control (“don’t trust memory, check the source”). These are not extras added to cognition; they’re the organism’s method for allocating attention and error-correction budgets. If cognition is compression—turning the raw flux of the world into workable bits—then metacognition is the compression audit. It asks, “How lossy was that?” and “Where are the artifacts going to bite me?”
What does entropy have to do with the feeling of knowing? Imagine every belief as a small codebook that predicts incoming signals: what happens next, what a word means, the pattern a colleague will follow in a meeting. When the world deviates from the codebook, you get surprise, a prediction error. Good systems don’t just fix the local error; they update how much attention to pay next time—precision weighting. In humans, that looks like altered confidence. In machines, it looks like adjusted learning rates or uncertainty bounds. The parallel is not perfect, but it is instructive: metacognition is a distribution over distributions, a second-order sense of volatility.
Experts often feel less certain than novices because they’ve sampled more counterexamples; the tails are heavier in their internal models. Children are sometimes wildly overconfident—thin distributions drawn from meager evidence. This isn’t a moral failing; it’s a statistical one. Meanwhile, illusions of knowledge (the “familiarity equals mastery” trap) are over-compressed representations: the brain stores the gist and mislabels fluency as accuracy. Standard test-prep advice—generate explanations, not just re-read—works because generation forces the codebook to produce, to confront gaps. You get bits back when you put bits in.
Push farther. If the self is a set of predictive summaries, then the “feeling of knowing” is one of the few reliable signals we have about model quality—except when it isn’t. Anxiety can masquerade as evidence against a true belief; fluency can masquerade as evidence for a false one. A careful metacognitive practice looks like signal detection theory: treat confidence as a noisy measure, calibrate it on feedback, and separate sensitivity (can you tell a correct answer from an incorrect one?) from bias (how eager are you to say you know?). The point is not to be certain. It is to be aligned with the statistics of your own mistakes. That alignment is the thin bridge that joins information theory and metacognition without collapsing one into the other.
Compression, Memory, and the Self as a Working Hypothesis
Information theory’s second life is in compression. Fewer bits, same meaning. Human memory is ruthless here. We store hierarchies, prototypes, stories—devices that minimize storage while trying to preserve what matters. Lossy methods, everywhere. The price of lossy compression is distortion, and the mind spends a surprising amount of its day repairing and re-indexing those distortions. Metacognition is the scheduler for that repair shop. It decides when to re-run the past with higher fidelity, when to leave the blur alone.
This framing makes the “self” look less like a solid object and more like a moving summary: a temporary compression optimized for present tasks. Ask a person for their life story, watch them re-encode while speaking—certain events gain bit depth, others are smoothed into archetypes. The listener’s questions change the code. As in communication theory, context reduces uncertainty; shared background lets us compress more aggressively. If there is an ethics here, it starts with an obligation to carry enough detail where it matters—to avoid compressing other people’s complexity into slogans. Not because slogans are evil, but because they raise downstream error rates.
Collective memory runs the same game at a different scale. Ritual, law, and institution are storage formats for hard-won constraints. Joseph Henrich calls culture a storehouse for inherited solutions; moral memory preserves not just rules but gradients of shame and honor that bias learning. These are compression schemes with evolutionary burn-in. They are also brittle when moved too fast across contexts, or when a platform optimizes for engagement instead of error bounds. An institution with good metacognition publishes its calibration curves: where it was wrong, how wrong, how the codebook was updated. Most don’t. They ship certainty. Then call the audit function “friction.”
Corporate AI governance loves frictionless certainty—metrics that render messy judgments into green checks. But metacognitive systems admit doubt on purpose. They allocate human review when entropy spikes. They log confidence with outputs, not as a footnote. And they retain a slow channel for value-laden updates—what some would call conscience, or “slow moral memory.” Building systems without this slow channel invites a kind of moral overfitting: smooth behavior on benchmark distributions, strange failures under shift. The better stance is stubbornly practical: surface uncertainty, route tricky cases to processes shaped by long memories (think peer review, not PR), and treat the self—individual or institutional—as a working hypothesis subject to compression errors that need routine maintenance.
Designing Systems That Know When They Don’t Know
What does it look like to engineer for not-knowing? Start with measurement. In classrooms, require students to attach confidence to answers. Score two things: accuracy and calibration. Reward a correct 60% claim that is actually right 60% of the time more than a thrilled 100% claim that’s right 60%. Over a semester, the metacognitive signal tightens; students learn to budget study time where their entropy is highest. The trick is feedback at the right granularity: too slow and the calibration loop drifts, too fast and noise drowns the signal.
Therapy isn’t engineering, but the same math keeps showing up. Anxiety as an alarm with a broken threshold. Depressive certainty as a frozen prior. Metacognitive interventions don’t argue the content as much as they adjust the precision dials: notice the breath, name the bias, write before you ruminate. Small actions that restore variance where the mind has collapsed to a narrow, punitive distribution. The “insight” you feel after naming a pattern is the entropy drop becoming conscious just long enough to be used.
In AI systems, the analogues are explicit. Report uncertainty alongside predictions. Use abstention (a “don’t know” class), conformal prediction sets, or temperature scaling to calibrate probabilities. Track Brier score and expected calibration error, not just accuracy. Budget human-in-the-loop review where predicted entropy is high. And crucially, preserve a slow lane for updates that can’t be derived from clickstreams—the long, often painful, cross-examination of values that don’t compress neatly. You can prototype this in a hospital triage model: if predicted risk sits inside a broad confidence interval, escalate to a clinician; if outside, proceed but log the rationale and sample counterfactuals for later audit. The goal isn’t perfection. It’s to make the uncertainty legible where the harm would be real.
There’s also the question of time. Carlo Rovelli reminds us that time may be local, relational. Our metacognitive loops are bounded by the clocks we live in—email cycles, quarterly targets, the speed of outrage online. Systems that know when they don’t know often just move on a different clock: slower reviews, longer memory, the willingness to revisit last year’s confident deck. John Wheeler’s “it from bit” is over-quoted, but the lineage matters: if reality is constraints all the way down, precision is never free. You buy it with patience, and with the storage cost of keeping multiple hypotheses alive long enough to watch the world select among them.
Product teams can build this culture into interfaces. Show an entropy bar next to recommendations. Let users request the “why” and the “how sure,” not as explainability theater but as an actual decision input. In security operations, prioritize alerts by uncertainty, not only by raw score. In civic life, publish confidence with statistics on public dashboards, and keep the raw streams open for scrutiny. Open-sourced methods over closed “governance” brochures. Not anti-technology; just anti-incentives that make certainty look cheaper than it is. A mind that can watch itself think—and mark its not-knowing clearly—does less damage when it guesses. And guesses are most of what we have, even on our best days, even when the codebooks feel tight and the world seems briefly, improbably, clear
Reykjavík marine-meteorologist currently stationed in Samoa. Freya covers cyclonic weather patterns, Polynesian tattoo culture, and low-code app tutorials. She plays ukulele under banyan trees and documents coral fluorescence with a waterproof drone.