synthesized from several pieces of Allan's writings & conversations by ChatGPT-5.6 Sol
Artificial intelligence has a definition of intelligence. Not an official definition, naturally. There is no card pinned to the wall at OpenAI or DeepMind saying INTELLIGENCE MEANS WHITE-COLLAR LABOR, although there might as well be.
Look at what we measure. Can the machine write a legal brief? Diagnose a disease? Pass an exam? Produce code? Interpret a regulation? Summarize a paper? Plan a project? Answer questions in the manner of someone who paid attention in graduate school?
Do enough of these things well enough and the system is described as generally intelligent. Do most economically useful cognitive work at roughly the level of a competent person and, for practical purposes, many people will say we have arrived at AGI. The philosophical arguments can continue after the product launch.
This is the Labor Theory of Intelligence: intelligence is what performs labor that we have historically paid educated humans to perform.
I am deliberately stealing the shape of an older phrase, not its economics. The claim is not that labor creates intelligence. It is that the AI industry has operationally defined intelligence by substitution in the labor market. A system is intelligent to the extent that it can replace cognitive workers.
That definition is wonderfully convenient. It provides benchmarks, customers, investment theses, and a scoreboard. It may even be the right definition for companies building useful computational devices. But it is an impoverished account of intelligence, and the impoverishment will not remain confined to machines.
If we train ourselves to recognize intelligence only where it produces economically legible cognitive output, that is eventually all we will recognize—or reward—in human beings too.
What Exactly Has Become Intelligent?#
I am not especially interested in whether current models are conscious. I do not know what consciousness is. Nobody else knows either, although this has not inhibited a thriving trade in declarations about which entities possess it.
My concern is more structural.
When someone tells me a computational system is becoming intelligent in the human sense, I want to know where the entity is. Is it the weights? The checkpoint? The deployment? The API endpoint? The conversation? The company operating the servers? The fine-tuned derivative? The version before Tuesday's update or the version after?
The “thing” can be copied, forked, paused, restarted, merged, distilled, retrained, rolled back, and deleted. There may be ten thousand simultaneous instances. They may share weights but not memories. One may promise something another cannot know it promised.
There is no obvious, continuous “it.” That is not a defect in the engineering. It is the architecture.
Human beings are not merely cognitive engines. We are bounded trajectories through time. The child becomes the adult; the adult becomes the old person. The same being receives the benefit or pays the price of earlier choices. Continuity makes promises, deterrence, reputation, responsibility, loyalty, sacrifice, and betrayal possible.
Can the machine be rewarded? Can it be punished? Can it be deterred? Can it own a decision? Can it refuse? Can it bear a cost? Can it commit now and remain the entity to whom the commitment applies later?
These questions are not a covert consciousness test. A system might be conscious and still fail them. They are questions about moral agency.
A calculator can outperform me at arithmetic. A market can outperform me at price discovery. A corporation can outperform me at capital allocation. None is therefore a person. A future system may outperform me at diagnosis, law, engineering, writing, strategy, and research. That establishes a breathtaking breadth of capability. It does not, by itself, tell me that a moral agent has appeared.
The discourse slides quietly from “this system performs cognitive labor” to “this system is intelligent in the sense humans are intelligent.” Capability, intelligence, agency, personhood, and moral status are treated as marks on one ruler.
They are not.
Intelligence Without Transcendence#
There is another problem with the labor definition. The systems we are building are meant to pursue objectives, not reconsider the civilization that supplied them.
An excellent lawyer may discover that the law is unjust. A scientist may undermine the institution funding the research. A philosopher may question premises everybody else regards as reality. Socrates was not celebrated for efficiently processing the Athenian case backlog.
Some of the human beings we regard as most intelligent were troublesome precisely because they would not accept the objective function.
Alignment research, reasonably enough, asks how to keep powerful systems within human control. A system that independently revises its terminal goals is not an easy product to deploy. The revolutionary who questions objectives may destroy the state; the machine that questions objectives may destroy more than that.
Fair enough. But we should then be honest about what kind of intelligence we want.
We want extraordinarily capable means generators. We want systems that can discover better routes to ends supplied from outside. We want optimization that is creative about methods and conservative about purpose. In the limit, perhaps we want intelligence without transcendence.
The humanistic tradition often treats transcendence—the capacity to examine, reject, or reform one's inherited ends—as a high expression of intelligence. The alignment tradition treats it as a failure mode. Those are not the same project.
Perhaps the alignment project is still the correct engineering project. I would prefer not to be exterminated by a machine experiencing philosophical growth. But a permanently aligned intelligence may lack something that made human intelligence worth admiring: wisdom, autonomy, moral development, or whatever word we settle upon after arguing about all the others.
The deeper danger is not only that we create optimizers without transcendence. It is that optimization becomes our dominant conception of intelligence, and the ability to question objectives becomes progressively less valuable—first in machines, then in institutions, then in labor markets, and eventually in our conception of a worthwhile human mind.
The White-Collar Mirror#
AI benchmarks are a mirror held up to a particular class's self-image.
We have spent decades declaring that routine physical labor is not evidence of high intelligence. The ability to navigate a cluttered room, manipulate unfamiliar objects, raise a child, calm a frightened person, repair a broken mechanism by touch, negotiate status without saying it aloud, or keep a household alive under relentless constraint rarely appears on the leaderboard.
Then a language model produces a competent memorandum and the metaphysics begin.
This is not because writing memoranda is the summit of cognition. It is because the people building and financing AI recognize their own labor in the output. The machine speaks in the artifacts by which educated institutions certify intelligence: exams, reports, code, arguments, plans, and answers.
The irony is that much of this work was already highly formalized. White-collar institutions operate through documents because documents travel, scale, and survive personnel. We converted judgment into forms, briefs, spreadsheets, protocols, and tickets. Language models arrived exceptionally well suited to inhabit that documentary shell. Then we mistook fit with the shell for a general theory of mind.
Again, the systems are genuinely useful. I am not arguing that their achievements are fake. A machine that can replace a large fraction of cognitive labor will transform society whether or not a philosopher grants it the honorific “intelligent.” Economic consequence does not wait for ontological clarity.
But the word matters because it tells us what to notice. Once intelligence is equated with the production of white-collar artifacts, people whose contributions are not easily rendered as artifacts become less visible. Care, judgment, presence, courage, practical knowledge, embodied skill, taste, loyalty, humor, and the ability to know when the stated task is the wrong task become soft qualities—lovely, perhaps, but not intelligence.
The labor market already rewards what it can measure. AI supplies a metaphysics that says the measurable portion was the mind all along.
The Automation Paradox#
Suppose the machines remain tools rather than moral agents. We still have to live inside institutions that use them.
The reassuring phrase is “human in the loop.” In medicine, law, accounting, engineering, and public administration, a licensed person will review the output. A human will remain responsible. The machine advises; the professional decides.
This sounds plausible because we imagine tomorrow's professional reviewing roughly the amount of work a professional reviews today. That is not what productivity does.
When AI makes the production of a legal brief ten times faster, the institution does not ordinarily preserve nine-tenths of the lawyer's day for contemplation. It produces more briefs. When code generation becomes a hundred times cheaper, we do not receive the same amount of lovingly reviewed code. We receive mountains of code that would never have been written. The velocity of human production once imposed a crude velocity check on the amount of material requiring maintenance, inspection, and judgment. Remove the production bottleneck and the review burden expands beyond any plausible human capacity.
The lawyer now handles more cases. The doctor oversees more patients. The engineer signs more designs. The accountant certifies more transactions. Each person may be more “productive” precisely because less human cognition is applied to each decision.
This is the automation paradox: the better the machine becomes at generating apparently acceptable work, the less practical it becomes for the nominal human supervisor to exercise independent judgment over each item.
The human role degrades in stages. First the person authors. Then the person edits. Then reviews. Then checks plausibility. Then scans for anomalies. Finally the person supplies a signature and the institution supplies a fiction: a responsible human made this decision.
The licensed professional becomes the designated fall guy.
Sleeping on the Couch#
Human oversight is often treated as a binary variable. Was a human in the loop? Check yes or no.
But a human can be technically present and cognitively absent. A person can click approve while thinking about lunch. A supervisor can review the first dozen outputs carefully, see that they are excellent, and gradually learn the rational behavior: trust the machine. Reliability produces complacency more effectively than failure does.
The problem is not moral weakness. Sustained vigilance is difficult, particularly when anomalies are rare. A system that is correct 99.9 percent of the time teaches its supervisor, through thousands of repetitions, that independent checking is wasted effort—right up to the consequential exception.
I once reached for the analogy of airport security tests. If a screener is supposed to detect a prohibited object in an endless stream of ordinary bags, you occasionally introduce a test object. Not because the screener is suspected of disloyalty, but because the system needs evidence that the human detection layer is alive.
What would the equivalent look like for judges, doctors, lawyers, accountants, or engineers supervising AI? Could an institution insert synthetic anomalies, known counterexamples, or deliberately flawed recommendations to measure whether reviewers notice? Could we quantify review depth, disagreement, intervention rates, independent reasoning, and the decay of expertise?
There are ethical and practical complications everywhere. A test case in a real medical workflow is not a blue rubber gun in a suitcase. People will game metrics. Reviewers will learn the pattern of planted errors. The monitoring system may become one more automated ritual.
Still, the question is unavoidable: how do we know the alleged human in the loop is a real human doing real oversight rather than a legitimacy prop sleeping on the couch?
Zombie Civilization#
Extend this dynamic across the institutions that run society.
Courts continue issuing decisions. Hospitals continue treating patients. Financial systems allocate capital. Engineers certify structures. Agencies promulgate rules. Universities award credentials. Everything looks recognizably alive. There are forms, hearings, signatures, seals, appeals, professional titles, and people sitting where authoritative people have always sat.
But the substantive cognition has moved elsewhere.
Humans remain in ceremonial positions because legitimacy was built around human authorship and responsibility. The institution continues speaking that language after the underlying reality changes. Judgment becomes machine-generated; human approval becomes theater.
That is Zombie Civilization: a society whose institutions retain the outward forms of accountable human agency after the agency itself has been hollowed out.
This does not require superintelligence, recursive self-improvement, or a dramatic singularity. It arrives through locally rational decisions. An organization that extracts more output from every expensive professional will outperform one that does not. A hospital that automates judgment will be cheaper and faster. A bureaucracy that processes cases automatically will clear backlogs. A law firm that declines these efficiencies will lose to one that accepts them.
No villain needs to decree the end of human agency. Optimization pressure does the work.
Nor is preserving the old arrangement always desirable. Human institutions are already opaque, unjust, inconsistent, and full of ceremonial review. An AI may make many decisions better than the people it displaces. “The human decided” has never been a guarantee of wisdom.
The problem is that our systems of legitimacy, accountability, appeal, and responsibility were constructed around bounded human agents. When the causal source of a decision becomes a distributed collection of models, prompts, retrieval systems, policies, operators, and organizations, we do not know where responsibility lands. The human signer did not perform the reasoning. The model cannot bear consequences. The vendor disclaims liability. The institution blames procedure.
Everyone was in the loop. Nobody decided.
Accountability Requires an Entity#
If machines are going to perform consequential cognition, we may need architectures that make them accountability-compatible without pretending they are human.
That begins with persistence and attribution. Which system made the decision? What version was it? What information did it receive? What policies governed it? What actions followed? Can its record survive a model update? Can responsibility attach to a stable deployment rather than evaporating when a checkpoint changes?
Perhaps synthetic actors need persistent identities, decision records, reputations, and continuity rules. Perhaps the operator remains responsible, but only under legal structures that prevent responsibility from being diffused across a supply chain. Perhaps certain decisions must remain contestable before a body capable of reconstructing the epistemic chain of custody.
Explainability here is not primarily about debugging a neural network. It is about governance. A person affected by a decision needs to know what can be challenged, by whom, under what authority, and with what possibility of reversal. Due process is a property of a relationship among actors, not a colorful visualization of attention weights.
We also need to preserve human capability deliberately. Expertise atrophies when it is not exercised. The better the automation, the fewer occasions humans have to develop the judgment required to intervene when automation fails. A civilization cannot maintain reserve cognitive capacity by declaring that humans remain important. It must create workflows, institutions, and incentives in which they continue to practice consequential judgment.
What We Choose to Reward#
The Labor Theory of Intelligence began as a criticism of AI language and ends as a criticism of us.
We built machines to perform the forms of cognition our institutions reward. Then their success tempted us to declare those forms the essence of cognition. The risk is not only that machines become more like white-collar workers. It is that people become more like the machines: optimized means generators, evaluated by output, discouraged from questioning the ends, and valued only where their work remains economically distinct.
There is a constructive alternative. We can treat machine cognition as infrastructure while preserving accountable civilization around it. We can measure whether oversight is substantive. We can design for contestability and epistemic chain of custody. We can attach decisions to persistent actors. We can preserve human skills that markets would otherwise allow to decay. We can monitor the concentration of cognition and the hollowing-out of institutions before a legitimacy crisis makes the problem obvious.
Most importantly, we can refuse the false choice between rejecting AI and accepting its implicit definition of intelligence.
The machines may write better briefs, discover better strategies, and make better predictions. Let them. Capability is valuable. Cognitive labor is valuable.
But intelligence is not merely the ability to produce the artifact an institution requested. It is also the ability to ask why the institution requested it, who benefits, what was excluded, whether the objective is worthy, and what kind of being one becomes by pursuing it.
If we forget that, the machines will not have defined intelligence down.
We will have.