An Introduction#
Between roughly 2021 and 2022, Raj Reddy and I had an extended set of conversations about a project he wanted to call the Terrascope, paired with a personal AI advisory system he called the Guardian Angel. The Terrascope was meant to be a global, federated sensor and analysis infrastructure covering people, places, and things at high spatio-temporal resolution. Guardian Angels were meant to be the human-facing layer — an always-on personal AI that would know enough about its ward and the world to deliver the right information to the right person at the right time in the right language at the right level of detail. The two pieces were inseparable in Raj’s mind: a Guardian Angel needed the Terrascope to know the world, and the Terrascope needed Guardian Angels to make its enormous data flows mean something to any individual human being.
We didn’t get very far. We produced a stack of slide decks, a handful of partial drafts, and a great many conversations that didn’t make it onto paper. The project drifted apart by mutual default rather than any explicit decision to stop, and in the years since I have watched the world change in ways that make the conversations look both more prescient and more frustrating than they did at the time. Most of what we sketched is being built right now, by parties with no obligation to the people whose data and lives they are operating on, and with no public-good framing of the kind Raj insisted on. The window in which a Terrascope-like project could have been built as a public good is closing, possibly already closed.
These four short pieces are my attempt, almost four years on, to extract from our notes and conversations the parts that still seem worth preserving. They are written in retrospect, in my voice, and they are meant for a friendly reader — someone willing to take an unfinished research project seriously as a set of half-thoughts rather than dismissing it for being unfinished. They are not a salvage operation aimed at restarting the work; they are a record of what we got to and where we got stuck.
The pieces approach the project from different angles, and any one of them can be read on its own. Briefly:
“The Surveillance Bargain We Never Negotiated” is about the privacy and security trade-offs at the core of any system that knows enough about you to advise you. It’s the argument that bad mass surveillance is everywhere, that good mass surveillance is hard for that very reason, and that the path to anything resembling the Guardian Angel runs through a clear-eyed accounting of what trust is to be extended and to whom. This is the piece I’d recommend reading first if you only read one.
“Terrascope and the Guardian Angels” is the architecture sketch. It walks through what the Terrascope was trying to be, what was right and wrong about the vision, and what conceptual structure we arrived at for the personal AI layer — Cognition Amplifiers versus Guardian Angels, the Federation of Advisors metaphor, the distinction between knowing and deciding. It’s the most concrete of the four pieces, and the closest to something that could function as a starting point for an actual design.
The remaining two pieces are about places where Raj and I disagreed. They belong together because the disagreement was, both times, recognizably the same disagreement.
“Why Raj Wanted the Whole Planet Wired” is about the part of the project Raj never quite said out loud. There was a second pitch underneath the stated SDG-and-human-flourishing pitch — an AGI argument, made before AGI was something one could discuss in public without embarrassment, about what kind of training data a planet-scale intelligence would actually require. There’s also a Gaia digital twin idea that may or may not have been Raj’s, and the temperamental split between Raj and me about whether information ecosystems naturally want to share or naturally want to hoard. Raj believed in cooperation; I believed in defection. We never resolved it.
“On What This Project Is Not About” is about a parallel disagreement on the question of scope. Raj wanted the project to take on the broader civilizational threats adjacent to our work: the cybersecurity arms race, the construction of safe AI, the question of how a Guardian Angel architecture would defend against even state-grade adversaries. I wanted to leave all of that out and concentrate on the privacy bargain, which I argued was hard enough on its own to absorb whatever attention we had. This was the same disagreement as the one in “Why Raj,” dressed in different clothes: Raj’s instinct was that techno-optimism plus institutional cooperation could expand the scope productively; mine was that the scope had to be narrowed ferociously or nothing would get done. I won this argument with Raj at the time, and I’m still willing to die on this hill. The piece explains why.
A note on Raj. He was one of the earliest AI researchers, with a Turing Award and a career that stretched across both winters of the field.[1] He was also, by the time we were having these conversations, in his mid-eighties, and the pace of our work was correspondingly contemplative rather than urgent. Some of what looks like incompleteness in these notes is actually the texture of long, unhurried conversation — the kind that produces real ideas at the cost of never quite finishing them. I think Raj knew the project wouldn’t be completed in any conventional sense and was pursuing it for the thinking rather than for a paper.
1. When I say "one of the earliest", I am possibly understating it. He was arguably the first person to get a PhD in AI, back in 1966. His advisor was the man who coined the term “AI”, John McCarthy, at the then-new Stanford Artificial Intelligence Laboratory (SAIL).
A note on my voice.[2] These pieces are mine, written from my perspective, with my opinions clearly marked as such where they diverge from Raj’s. Where I attribute a position to Raj, it is either documented in the slides and notes or it is explicitly flagged as my reading of what he was after. I’ve tried to be honest about which is which. Raj is to be this collection’s first reader, and where he corrects me I’ll say so. The slides and notes are preserved; anyone who wants to check my characterizations against the source material can ask.
A note on completeness. These four pieces do not cover everything in the original notes. Substantial design provocations, technical sketches, use-case explorations, and conversational fragments have been left out — partly because they don’t cohere into anything publishable, partly because including them would have required inventing positions Raj and I never actually arrived at. The notes are still in my files. Again, anyone who wants to take this further is welcome to ask.
What I hope these essays do, modestly, is preserve a particular line of thinking that was happening at a particular moment — the eve of the GPT era, in conversations between someone who had been doing AI since before anyone had heard of it and someone who came to it through the cryptography and security side. A bit of what we said has been overtaken by events. Much of it, I think, has not.
2. Speaking of my voice, as of this writing all three of ChatGPT 5.5, Claude Opus 4.8, and Gemini 3.5 Flash read these essays and gave me mostly good editing advice and whole paragraphs (less good).
The Surveillance Bargain We Never Negotiated#
A few years ago, Raj Reddy and I started talking about what he called Guardian Angels — an AI advisor for every person on earth, embedded in a global sensory infrastructure he called Terrascope. The scope of the idea was almost comically large: a system of sensors and agents that could warn a Bengali farmer about a coming flood, assist a child with a learning disability when she’s struggling with a concept, or help an elderly person in Lagos manage a medication regimen. Raj’s vision was, in the best sense, naïve, in the sense that all genuinely ambitious ideas are naïve before anyone figures out how to build them.
We didn’t get very far. The technical and logistical challenges of the Terrascope itself were staggering enough. But we kept running into something even more fundamental: the privacy problem. Not as a legal nuisance or a compliance checkbox, but as a deep structural tension that threatened to make the whole idea incoherent. You cannot have a guardian angel that doesn’t know everything about you. And anything that knows everything about you is, by definition, a surveillance system with all the attendant vulnerabilities to abuse, exfiltration, and mission creep. We set aside the Terrascope itself and tried to think clearly about that problem. These are my notes on where we got.
Two Kinds of Surveillance#
We have far too much bad mass surveillance and nowhere near enough good mass surveillance. That sounds paradoxical, but the distinction is straightforward.
Bad mass surveillance is motivated by the interests of parties other than the person being surveilled. At its best, it serves a diffuse public interest — crime prevention, disease containment — while the individual has no meaningful say in what’s collected or how it’s used. At its worst, it’s commercial surveillance capitalism: your data monetized for ends you’d never have consented to if the question had ever been put to you plainly.
Good mass surveillance would serve primarily the individual, with any public benefit being secondary and under the individual’s control. Your Fitbit, properly designed, is a model of good surveillance. It knows things about your body that your doctor doesn’t, and in principle that information should be yours alone to act on, share, or withhold.
The problem is that bad surveillance is making good surveillance harder to build. The obvious benefits of a toilet sensor that screened for colorectal cancer markers, or an eye tracker that identified children’s reading disabilities early, are being foreclosed — not by technical impossibility but by the entirely rational suspicion that any such system will be turned against the people it’s supposedly serving. The surveillance infrastructure is already there; what’s missing is trust, and that trust has been systematically destroyed.
Boiling Frogs#
The mechanism by which bad surveillance colonizes everyday life is well understood, even if we don’t have a good way to stop it: it comes incrementally, each step defended by some marginal convenience, until the cumulative result which would have been unacceptable if proposed all at once, is simply the way things are.
Consider phone location tracking. Imagine a couple of decades ago the US government had demanded everyone carry a device broadcasting their location continuously to a permanent government record. The riots would have been enormous. What actually happened was somewhat different: people paid for the devices themselves, enthusiastically, because the phones were useful for other things. Location tracking was a byproduct nobody quite focused on until it was universal. The apocryphal frog was already in the pot.
The mechanism of voluntary adoption for convenience, surveillance as a side effect, gradual normalization, will repeat with every new sensing technology. Eye-tracking is the next obvious candidate. It will arrive as a convenience feature so you don’t have to use a mouse; the implications for what can be inferred about your attention, your health, your emotional state will follow later, when the infrastructure is already ubiquitous and the window to object has closed. When the guy in 1999 said your privacy was gone and you should get over it, he was talking about your credit records. How cute.
None of this requires malice at the outset. It requires only that the people building the systems aren’t required to foreground the surveillance dimension, and that users are never really asked to consent so much as they’re given the option to decline — an option that most people don’t take and many can’t realistically navigate.
Permissions Fatigue#
The standard answer to surveillance is consent: opt-in requirements, cookie banners, GDPR notices. The standard answer is broken.
We cannot assume that any individual is capable of wisely managing the permissions required to interact with modern digital infrastructure. How many distinct systems did you interact with today? Did each one request consent? When consent was requested, did you have time to understand what you were consenting to? Will you remember, next year, what permissions you granted and to whom?
The GDPR cookie consent notice is the reductio ad absurdum of this approach. When we swat these pop-ups away so we can get on with what we were doing, in what meaningful sense have we consented? The honest answer is that we haven’t, and everyone involved in building these systems knows it.
The analogy to password hygiene is instructive. Security professionals have spent decades telling users to use unique strong passwords, change them regularly, and never reuse them. Essentially no one does this, because the cognitive overhead is impossible to sustain. The security community has largely accepted this and moved toward password managers and passkeys. We have not yet accepted the equivalent lesson about privacy permissions — that the cognitive burden of managing them cannot be placed on individual users — and so we keep building systems that nominally require consent while being architected to make informed consent practically impossible.
To the existing term password fatigue, I’d add permissions fatigue: the state in which a person has been asked for consent so many times, in such confusing and manipulative ways, that consent has become a reflex rather than a decision.
The Guardian Angel Problem#
This is where Raj’s Guardian Angel idea runs into its fundamental difficulty. The very thing that would make a personal AI advisor valuable — its comprehensive knowledge of everything about you — is precisely what makes it a potential catastrophe if the system is compromised, misused, or simply operated by parties whose interests don’t align with yours.
One design principle for secure systems is minimizing the attack surface: reduce the amount of trusted code, reduce the amount of sensitive data, limit data lifetime. A truly capable Guardian Angel agent systematically violates every one of these principles. It needs to know your health history, your financial situation, your location, your relationships, your habits, your fears. It needs to retain this information indefinitely, because decisions in the future will require context from the past that we cannot anticipate in advance. It needs to make inferences from this data, and so it can betray you.
You cannot build a Guardian Angel that is both maximally useful and minimally dangerous. The two requirements are in direct tension. This doesn’t mean the idea is hopeless; it means the privacy and security architecture of such a system is not an engineering afterthought but the central design problem.
Some partial mitigations are worth noting. Computation can be kept local on the user’s device rather than in the cloud for the most sensitive inference tasks. This limits the attack surface significantly, since an adversary who wants your data has to compromise your device rather than a cloud service storing data for millions of people. (Though device compromise is hardly impossible, the economics of mass exfiltration favor cloud-side attacks.) Differential privacy and anonymization techniques can allow lower-level agents in a Guardian Angel hierarchy to operate on appropriately imprecise data, reserving high-precision personal information for only the most trusted components of the system. The architecture is not unlike a well-managed organization, where sensitive information is compartmentalized and the number of people with full access is kept small.
But there’s a limit that even good architecture can’t fully address: the existence of the system itself reveals information. If you participate in a Guardian Angel network, anyone who merely observes that fact knows something about you — that you’re using such a system, what category of advice you might be seeking, what your behavioral patterns suggest. Traffic analysis, even against encrypted communications, can be informative. The very existence of a principal’s participation permits inferences. This isn’t a problem to be solved; it’s a constraint to be understood.
What Good Would Look Like#
The characteristics of good surveillance that serves the individual are roughly:
Data collection should be motivated by the principal’s interests, not a third party’s. Collection should be minimized to what is genuinely necessary. Data should be retained for as long as it’s useful to the individual and no longer. The individual should have genuine, comprehensible control over what is collected, what is retained, and what is shared — which means, in practice, that a Guardian Angel agent must manage permissions on the user’s behalf, because users cannot do it themselves. Any sharing with public health, law enforcement, or commercial interests should be initiated by the individual, not extracted. The benefits of aggregate data analysis: epidemic detection, supply chain optimization, and collective learning from individual experience should flow back to the individuals who contributed the data.
None of this is technically impossible; some of it is straightforward. The obstacles are primarily economic and political: the current surveillance infrastructure was built to serve commercial and governmental interests, and those interests have no particular reason to rebuild it on terms more favorable to individuals.
The argument for doing so anyway is not purely altruistic. Surveillance systems that people genuinely trust are more valuable than surveillance systems that people tolerate or try to avoid. An opt-in health monitoring system that people actually use generates better public health data than a mandatory one that people circumvent. A Guardian Angel that users trust with accurate information about their lives gives better advice than one they feed half-truths to. The value of trust is real and measurable; it’s just that it accrues to the individual rather than to whoever built the system, which makes it hard to capture as a business model.
Raj’s Terrascope dream — a sensor infrastructure and AI advisory system that genuinely served human flourishing at global scale — remains technically plausible and morally attractive. The surveillance problem is not a reason to abandon it. It is the problem that has to be solved first, before any of the rest of it can be worth building.
Terrascope and the Guardian Angels#
A retrospective sketch of an unfinished project
Around 2021, Raj Reddy and I started meeting to talk about what he wanted to call the Terrascope. The idea was characteristically Raj-sized: a global, federated sensor and analysis system covering people, places, and things at high spatial and temporal resolution, paired with an AI advisor, a Guardian Angel, for every person on earth. The Terrascope would be the infrastructure; the Guardian Angels would be the human-facing layer that turned all that sensor data and computational capability into timely, personalized, actionable advice for whoever needed it.
We didn’t get very far. (I say that a lot.) We produced a stack of slide decks, a couple of half-finished papers, and a lot of conversation. What follows is my attempt to extract the parts that still seem worth something, written from the present looking back. I’ve separated it into the two ideas: the Terrascope itself, and the Guardian Angel architecture that was supposed to ride on top of it. Separated, because while they were always discussed together, they’re conceptually separable and have somewhat different futures.
The companion piece, “The Surveillance Bargain We Never Negotiated,” takes up the privacy and security side of this — the problem we kept running into and never solved.
Part 1: The Terrascope#
The dream#
The starting picture was something like this. Within a decade or two, the planet will be covered in cheap networked sensors, and most manufactured objects will contain processors capable of running real machine learning models. Each person will be carrying or wearing a small fleet of sensing devices — phones, watches, earbuds, eventually glasses and various kinds of implant. The earth’s surface will be densely instrumented with weather, environmental, and acoustic sensors. All of this is happening anyway, driven by the falling cost of silicon and the commercial logic of every industry that touches it. The question isn’t whether it gets built; it’s who gets to use the data and for what.
Raj’s framing was that this enormous, accidentally-emerging sensor network ought to be made legible — that there ought to be a publicly accessible, wiki-like database of what’s known about the world’s people, places, and things, and that the analysis run on this database ought to be aimed at human flourishing rather than ad targeting. He was inspired by Duncan Watts’s idea of a “social telescope” for the social sciences and by India’s National Sample Survey, which historically tracked household-level economic data with remarkable resolution. The Terrascope would be that, but for everything, and global.
The intended applications were unapologetically large. UN Sustainable Development Goal– scale problems: pandemic detection, water and food security, climate monitoring, education, disaster early warning. Raj kept returning to the 2004 Indian Ocean tsunami — 230,000 people killed, almost all of them in countries that had no warning system because the relevant scientific community didn’t have an institutional channel to reach the relevant coastal populations within a few hours. The information existed. The connection didn’t. Raj’s claim — which I think is essentially correct — was that ninety percent of those deaths were preventable in principle if the right information had reached the right people in the right language at the right level of detail at the right time.
That formulation became something of a mantra in our discussions. I’ll come back to it in the Guardian Angels section because it turns out to be doing a lot of work.
What was right about it#
Several things in the Terrascope vision still hold up.
The most important is the observation that the surveillance infrastructure is being built whether anyone designs it or not. Governments, telecoms, ad networks, payment processors, and consumer device manufacturers are each building pieces of a global sensor network for their own reasons, and the result — if no one intervenes — will be a fragmented, opaque, commercially captured surveillance apparatus that serves none of the people being surveilled. Raj’s instinct was that the right response is not to try to roll this back (you can’t) but to build a parallel public-good layer that uses similar inputs for different ends. I still think that’s the right strategic frame, even though I have less confidence than Raj did that any such layer can be built without being captured by the same forces that captured the original.
The second thing that holds up is the focus on the global South. The MeScope idea — a wearable health-monitoring pilot, with personalized alerts for the wearer and aggregated data feeding public health surveillance — is more obviously valuable in places where the existing public health infrastructure is thin. The richer-world version of MeScope is Apple Watch and Fitbit, which are useful but underutilized for population-level questions because the data is locked inside vendor silos. The poorer-world version is a thing that doesn’t exist yet, where wearables are cheap and ubiquitous and feed into a public health system that genuinely needs the signal. Raj’s argument was that the poorer-world case is actually the easier one to build, because there’s no entrenched system to displace. He may have been right.
The third is the framing of “knowledge as a service.” Most of what an ordinary person needs to know on any given day — the safety of the water, the air quality at their kid’s school, whether their symptoms are concerning, whether the neighborhood they’re walking through at night has had recent incidents — is already known by some institution somewhere. The information exists. The plumbing to deliver the right slice of it to the right person doesn’t. A Terrascope-like system is, in part, just that plumbing.
What was missing#
The Terrascope work was always more vision than design, and the gaps are real.
Governance was almost entirely hand-waved. The slides talk about an “ethical governance model” with transparency, GDPR compliance, and regular audits, but those words do very little work. Who runs the Terrascope? Who decides what data sources are legitimate? Who funds the federated infrastructure? Who decides which Sustainable Development Goal applications get priority? Raj’s instinct was that an academic consortium could host the database in something like the way Wikipedia is hosted, but that comparison falls apart on closer inspection — Wikipedia is a text corpus that anyone can edit, not a federated sensor data infrastructure that has to make hard decisions about what to ingest, what to anonymize, and what to refuse. The closest real-world model is something like CERN or the Human Genome Project, but those are both vastly narrower in scope than what Raj had in mind.
The economics were also unaddressed. The Terrascope only works if a lot of the underlying sensor data is contributed for free or near-free by parties who currently monetize it. Convincing those parties to contribute requires either regulation or a value proposition we never articulated. The slide decks talk about “individual ownership of data” and “transitioning ownership of data to the individual,” which I think is right as a principle, but every time we tried to think through what the actual mechanism would be, we ended up either reinventing GDPR (with its known failures) or describing something that would require the cooperation of every major data broker on earth.
And the technical architecture stayed at the level of bullet points. “Federated specialized network nodes for service and analysis.” “Local processing on devices with encrypted communication to nodes.” “Only anonymized data is shared for large-scale insights.” These are the right phrases, but they’re not a design. The hard problems — what gets computed where, how trust is established between nodes, what happens when a node is compromised, how the system handles disagreement between authoritative sources, how it knows what’s worth telling you — were never seriously engaged.
The honest summary#
The Terrascope was, and remains, a research vision rather than a project. What’s worth keeping from it is the framing: the surveillance infrastructure is being built anyway, and the question is whether anyone tries to build a public-good version of it before the commercial and governmental versions become impossible to compete with. The MeScope pilot is the most concrete version of that idea and is the part most worth picking up if anyone wanted to actually start something.
Part 2: Cognition Amplifiers and Guardian Angels#
The other half of the project was the human-facing layer — the AI advisors that would mediate between the Terrascope’s data and the individual person trying to live their life.
Raj had a useful distinction here that’s worth preserving. He divided the agent landscape into two categories.
A Cognition Amplifier (“COG”) is an agent that helps you do something you’d already do, faster and with less effort. Filter your email. Pay your bills. Book your travel. Keep track of your bank balance. The defining property of a Cog is that you, the human, know what you’re trying to accomplish, and the Cog is just a faster, more attentive way to accomplish it. Cogs are near-term. Most of what’s currently sold as “AI agents” is in the Cog category, even when the marketing is more grandiose.
A Guardian Angel is a different kind of thing. A Guardian Angel does something for you that you couldn’t have done on your own — typically because the relevant information was beyond your awareness, beyond your cognitive bandwidth, or beyond your ability to act on in time. The tsunami warning is the prototypical case: nobody on the beach in Phuket on December 26th, 2004 had any way of knowing what was coming.[3] A Guardian Angel is the agent that would have known and would have told them in time to move.
The distinction matters because the architectures, the trust requirements, and the failure modes are different. Cogs operate within a known domain, on data the user has explicitly shared, toward goals the user has explicitly stated. Guardian Angels are necessarily cross-domain, necessarily operating on data the user may not have realized was relevant, and necessarily taking initiative on the user’s behalf — which means they have to be trusted in a much deeper sense than Cogs do. The interesting design problem is the Guardian Angel; the Cog is mostly an engineering problem with known shape.
What the Guardian Angel architecture had to do#
Working backward from the kinds of problems a Guardian Angel was supposed to solve we ended up with a list of properties that any serious GA architecture would need to satisfy.
3. Coincidentally, Phuket is this author's location at time of writing (April ’26).
It has to be comprehensive. The whole point is to surface things the user hadn’t thought to look for, which means the GA has to have a broad enough view of the user’s life and circumstances to recognize relevance. A safety advisor that only knows about the user’s driving habits won’t catch the medication interaction; a medical advisor that only knows about lab results won’t catch the air quality issue.
It has to be always on. Most of the value is in catching things in time. An advisor that needs to be invoked has already missed most of its use cases.
It has to be persistent and learning. A Guardian Angel that forgets what it learned about you yesterday is not going to be useful tomorrow. This was already obvious in 2022 and is obvious in a different and sharper way now that we have actual long-context language models that can sort of do this and sort of can’t.
It has to be mass-customized. Per-person at scale. This was a phrase Raj liked. The GA has to be specific to its ward in a way that an off-the-shelf chatbot isn’t.
It has to be conceptually unified from the user’s perspective. This was a point I argued for repeatedly against early sketches that imagined the GA as a loose constellation of independent specialized advisors. People don’t have the cognitive bandwidth to manage a swarm of agents; the value of the GA is precisely that one entity holds the whole picture and presents the user with a coherent, prioritized view. Underneath, of course, there will be specialization, hierarchy, delegation. But the user-facing surface is one Angel, not a parliament.
The “Federation of Advisors” metaphor#
We spent some time on architectural metaphors and the one that stuck was the Federation of Advisors, organized roughly like the executive branch of a government or the senior staff of a wealthy individual.
The user has, conceptually, a Chief of Staff at the top — the entity the user actually talks to, who is responsible for keeping the whole picture in mind. Reporting to the CoS are a small number of senior advisors with broad portfolios: a Personal Administrator (logistics, scheduling, finances), a Security Director (physical and digital safety), a Doctor (health, broadly construed), and probably one or two others. Each senior advisor has, in turn, a department of specialists — the Doctor doesn’t personally diagnose dermatology questions but knows when to consult the dermatology specialist, who in turn is connected to relevant external services and data.
Routine matters get handled at the lowest level competent to handle them and never bubble up. Important matters reach the senior advisors. Genuinely consequential matters reach the user, through the Chief of Staff, with appropriate framing and recommendations.
This structure does several things at once. It maps cleanly onto how human organizations actually handle complexity, which is some evidence that it’s not insane. It provides a natural place to apply different levels of trust and different privacy regimes — low-level specialists can operate on heavily anonymized data because their advice is generic, while the Chief of Staff and senior advisors operate on full-fidelity data because their advice depends on it. And it provides a natural place for the user to intervene — the user can adjust the Chief of Staff’s priorities, the user can fire the Doctor, the user can demand that nothing about a particular topic ever be acted on without explicit confirmation.
Knowing isn’t deciding#
There’s a temptation, when sketching out something like a Guardian Angel, to slide from “agent that knows things on your behalf” to “agent that decides things on your behalf.” That slide is dangerous and we tried to be careful about it.
A safety advisor that can give an accurate micromort estimate for every prospective action — five for the scuba dive, point-one for the drive to the dive site — has done something useful. A safety advisor that decides whether you go diving today has crossed a line. The first is augmentation. The second is paternalism, and a paternalism that comes from a system the user doesn’t control and can’t fully evaluate is going to be either resented or quietly disabled.
The right framing, I think, is that the Guardian Angel exists to make the user better at being themselves — to know more, attend more, anticipate more, and act in time more often than they otherwise would. It exists to hand them choices they wouldn’t otherwise have had, not to make those choices for them. The Angel is neither a tyrant nor a nag; it knows the user’s strengths and weaknesses, points toward their better self, but in the end allows them to remain human.
The exception is action on the user’s behalf within scopes the user has explicitly delegated — paying bills, scheduling appointments, declining cookies. Those are Cog functions, and the user has consciously handed over the steering wheel. Guardian Angel functions, by contrast, are advisory by default and only become actuating when the user has signed off in advance for that specific class of situation.
The hard problem#
Everything above is the easy part. The hard part is that the very thing that makes a Guardian Angel valuable — its comprehensive, persistent, cross-domain knowledge of the user — is exactly what makes it dangerous.
A fully capable Guardian Angel must, in principle, know everything about you. There is no way to bound in advance what context will be needed for a future decision, because the future decisions can’t be enumerated. Any agent in the hierarchy may, at some point, need access to high-precision personal data. They make inferences from your data; therefore they can betray.
The standard tools of secure system design — minimize the attack surface, minimize the trusted code base, minimize the amount and lifetime of sensitive data — are at war with the basic requirements of a Guardian Angel. We do not have a clean resolution to this, and I’m not sure one exists. Partial mitigations are the subject of the companion piece on the surveillance bargain.
What I’ll say here is that the Guardian Angel architecture only makes ethical sense if it’s built with the privacy problem at the center of the design rather than as a layer added at the end. Most of what’s currently being built and marketed as “personal AI” fails this test. The agents are aimed at the wrong constituency — they serve their vendor’s interests with the user’s interests as a secondary concern — and the data architectures reflect this. Building a real Guardian Angel would require, at minimum, a different commercial model and a different regulatory environment. Whether either of those is achievable is a separate question.
Where this leaves things#
The Terrascope and Guardian Angel ideas were never going to be a single research project and treating them as one was probably a mistake. They are at least three distinct things: an infrastructure proposal, a personal AI architecture, and a privacy and security stance. Each has a different audience and a different set of next steps.
The infrastructure proposal — a federated public-good sensor and data system at planetary scale — is essentially a policy and institutional question. The technology is the easy part. Whether anyone has the political will and the funding to build something like CERN for human flourishing is the question, and the answer is probably no, but the cost of being wrong about that is low and the value if it ever happened would be enormous.
The Guardian Angel architecture is more tractable as an actual research and engineering project, especially now that LLMs have made the per-person, persistent, learning, conceptually-unified agent suddenly seem within reach in a way it didn’t four years ago. The main thing missing — and it’s the main thing — is a privacy-respecting architecture that doesn’t just ship all the user’s data to a cloud vendor. That’s a real engineering problem, but it’s a finite one.
The privacy and security stance, treated separately in the companion essay, is where I think the most honest contribution lies. Raj’s instinct that the surveillance was happening anyway was correct. The conclusion he drew from it — that we should therefore build the good version — is at least defensible. The hard problem is that the good version requires solving privacy not as an add-on but as the central design constraint, and very few of the people currently building agentic AI seem inclined to take that seriously.
We didn’t solve any of this. What we did was sketch the territory and identify the obstacles. That’s not nothing, but it’s not a research program either. The notes are here for whoever wants to pick up where we left off.
A note on what got left out: a substantial volume of design provocations from our conversations — pub/sub mechanisms for delivering inferences anonymously, global sensor-net architecture (!), synthetic-cohort approaches to aggregate health surveillance, the use case of a Guardian Angel guiding a homeless person to services, the question of whether GA-to-GA communication should be possible — didn’t make it into this sketch. They’re interesting but they’re half-thoughts, and trying to write them up coherently would have meant either inventing things Raj and I didn’t actually agree on, or producing something even more obviously incomplete than what’s here. They sit in the notes file for now.
Why Raj Wanted the Whole Planet Wired#
On an invisible argument, a generational temperament, and where Raj and I disagreed
The Terrascope pitch, as it appeared in the slide decks, was framed in terms of the UN Sustainable Development Goals (SDGs) and human flourishing (Maslow’s hierarchy). Tsunami warnings, pandemic surveillance, food security, the right information to the right person at the right time. That framing was sincere, and I’ve written about it elsewhere. But it wasn’t the whole pitch, and in retrospect it wasn’t even the most interesting part of the pitch. There was a second argument running underneath, more often gestured at than stated, which I want to try to draw out here — partly because the timing makes it worth recording, and partly because it’s the place where Raj and I actually disagreed.
The timing first. Raj and I were doing this work in 2021 and 2022. ChatGPT shipped at the end of November 2022. The bulk of our slides and notes are from before that — before the shape of what was about to happen was visible to most people, and before “AGI” stopped being a phrase you could only use among friends without sounding like a crank. Raj had been one of the earliest AI researchers. He’d watched the field go through the winters. He had a Turing Award for speech recognition work that was deeply symbolic-AI in flavor. He was, on paper, a card-carrying member of the GOFAI old guard.
But the thing about Raj was that he was a quietly committed connectionist of a certain kind and had been for longer than was widely understood. He didn’t make a public deal of it the way Geoff Hinton did, but he watched the deep learning revolution with the eye of someone who had expected it to arrive eventually. By 2021, his view was that the path to genuinely general AI ran through scale — more compute, more data, more parameters — and that the bottleneck wasn’t going to be algorithms but inputs. Specifically, that the inputs available from the public internet, or even from the very large private corpora held by the cloud companies, were too narrow and too disembodied to train the kind of world model that AGI would actually require. You couldn’t get to a system that understood the physical world by feeding it text scraped from people writing about the physical world. You needed the world itself, or as close an approximation as you could build.
The Terrascope, in this second pitch, was that approximation. A planetary-scale sensor network, continuously feeding data about people, places, weather, oceans, agriculture, traffic, health, behavior, and environment into models that could integrate it into something like a coherent understanding of how the world actually works. Not as a research curiosity, but as the substrate on which the next generation of AI would be trained. The Sustainable Development Goals applications were the human use case. The AGI training corpus was the technological case. Raj believed both, and he believed they pointed at the same infrastructure.
I think the AGI argument was largely correct, in the sense that the model labs are now spending vast sums trying to acquire exactly the kinds of data Raj was talking about — embodiment data, sensor data, video, real-world interaction logs, robotics traces, physiological signals. The thing he was advocating for is being built. It’s just being built piecemeal, by competing labs, behind locked doors, with no public-good mandate and no governance to speak of. Which brings me to the part of Raj’s pitch that I think was wrong or at least was where I dug in against him.
Raj had what I think of as a “Kumbaya” instinct. He genuinely believed that if you could get the right institutions to cooperate — universities, governments, NGOs, well-meaning companies — they would. He believed that fragmentation and stove-piping were unforced errors, the result of insufficient coordination rather than of structural incentives that actively rewarded fragmenting and stove-piping. He believed that data, if pooled, would naturally find its way into the hands of researchers who would do good things with it. He had built his career partly on the strength of this belief — Carnegie Mellon’s speech work, the Million Book Project, Digital Green, the whole sweep of his “AI for development” thinking — and the belief had been validated often enough that he was entitled to it.
This was an instinct he shared, almost word for word, with my mentor Marty Tenenbaum. Raj and Marty had known each other since their Stanford PhD days and had moved in intersecting orbits ever since — never as direct collaborators exactly, but as contemporaries who kept ending up adjacent to one another’s projects across decades. It was through Marty that I came to know Raj in the first place. The two of them shared a worldview about cooperation between large entities almost down to the wording. This optimism was the default at SAIL — set in the dry eucalyptus hills of Palo Alto thirty-five miles south of the Summer of Love — a “Dormouse” era conviction that the natural state of an information ecosystem is sharing, providing the bootstrap logic for the world that their successors would literally and ruthlessly capitalize upon. Their view, held to this day, was that the hoarding behaviors I see everywhere are pathological deviations from the natural state.
They both spent significant chunks of their careers trying to build infrastructure premised on this view — open data initiatives, federated systems, consortia, standards bodies, the whole apparatus of “if we just build the protocol they will come.”
I do not believe this. I have not believed it for a long time, and I told Raj so. My view, which I think is closer to right, is that if data is gold, people will guard it jealously and steal what they can. The natural state of an information ecosystem is hoarding, with cooperation as the deviation that has to be specifically engineered, paid for, and defended. Every story about voluntary data sharing at scale, when you look at it closely, turns out to be a story about a regulator who forced it, a vendor who captured the standard, or a brief window of academic generosity that closed the moment the data became commercially interesting. Open standards survive when nobody yet sees the money in them. The moment they do, the standard fragments, the dominant player extends it incompatibly, and the cooperation evaporates.
This wasn’t a theoretical disagreement between Raj and me. It came up in concrete form every time we tried to think through who would actually run the Terrascope, who would contribute data to it, who would constrain what got done with the contributed data, and what would happen when one of the contributors decided their data was suddenly worth too much to share. Raj’s answer was always some version of “the right people will see the value and cooperate.” My answer was always some version of “they will cooperate for exactly as long as cooperation is cheap, and then they will defect, and the system has to be designed for that.”
I don’t think either of us convinced the other. Raj had decades of evidence that cooperation was achievable when the right people pushed for it. I had decades of evidence that it kept failing in the long run even when the right people pushed for it. We were probably both correctly describing different parts of the territory. Raj was describing the early-stage phase where a well-led consortium can get something off the ground. I was describing the middle-stage phase where the something becomes valuable and the consortium fractures along lines of commercial interest. We needed both to be true to make the Terrascope work, and the place where the project actually failed was the place where his model and my model gave different answers and we couldn’t reconcile them.
There was a further wrinkle here that I want to record, with the caveat that it’s my reading rather than anything Raj actually said. The tell was that he insisted everything had to go into the training — not just the SDG-relevant signals, not just the human-behavioral data, but the whole sensor stream from the whole planet. That’s a much stronger claim than the stated pitch required, and it kept showing up in places where it wasn’t obviously compelling for any application we were discussing. The most coherent reconstruction I can offer is that Raj was reaching for Gaia made flesh — or made silicon. Not a Gaia hypothesis for AI, where the AI learns to behave in some Gaia-like way, but Gaia itself instantiated at last: the planet’s weather, ecosystems, populations, and economies running as continuous inputs to a computational substrate large enough to close the loops. A Gaia digital twin, if you want the mundane version. Alignment, in this picture, wouldn’t be bolted on afterward; it would be constitutive, because the system is the planet thinking about itself.
I think this is wrong, though interestingly wrong. Understanding a system is not the same as caring about it the way we do, and a planet-scale system that models itself comprehensively could perfectly well conclude that humans are the part to remove. But the move identifies a real failure mode of text-trained AI: disembodiment and abstraction from physical consequence. It proposes a remedy with the right shape, even if the remedy doesn’t actually deliver the alignment it promises. Whether any of this was what Raj was actually thinking, I genuinely don’t know. I never put it to him in these terms, so he never had the chance to confirm or deny it. But the insistence on totality is hard to explain otherwise.
What I take from all this, four years on:
The AGI argument for the Terrascope was prescient in a way that matters. The frontier labs are now doing exactly what Raj said would need to be done — building or acquiring planetary-scale, multi-modal, real-world data feeds to train models that the public-internet corpus alone cannot produce. They’re doing it without the public-good framing, without the governance, and without anything resembling individual data sovereignty, which is approximately the worst possible version of Raj’s vision. The opportunity to build a Terrascope as a public good, if it ever existed, is closing fast as the commercial version consolidates.
The Kumbaya instinct, in Raj and Marty both, was a generational artifact and a personal virtue. It was the right instinct for the era they came up in, when the institutions of computer science research were small enough and well-funded enough that voluntary cooperation actually worked. It is the wrong instinct for the era we’re in now, where the stakes are commercial and the actors are some of the largest entities on the planet. I wish I had been more emphatic with Raj about this, though I’m not sure it would have changed anything; the disagreement was temperamental more than empirical, and neither of us was going to trade temperaments.
The Gaia-alignment idea is a beautiful piece of speculation that deserves to be recorded because almost no one has articulated it in quite this form. If anyone reading this is in a position to take the idea seriously and develop it, I hope they will, even if I think it’s wrong. Beautiful wrong ideas are how the right ideas eventually get found. And I’ll admit a less respectable reason for recording this one. Buried in me is a boy marinated in mid-century science fiction, who remembers the story where the galaxy’s computers are finally linked into one machine and asked: is there a god? That boy is tickled beyond reason that the reality he lives in appears to be running the plot. The grown-up thinks the idea is wrong. The boy already knows what the Answer will be: now there is.
What Raj was reaching for, in the end, was a vision in which the same infrastructure might serve three goals that he saw as inseparable: the sensing layer of a Guardian Angel system, the training substrate for genuinely general AI, and possibly (though this is my reading more than his words) the connective tissue of something like an aligned Gaia-like planetary intelligence. He was wrong, I think, that those goals would naturally converge if the right institutions cooperated. He may have been right that they ought to converge, and that something is lost when they’re pursued separately by parties with no obligation to one another.
We eventually resumed working on our separate interests. I miss arguing with him.
On What This Project Is Not About#
A note on scope
Anyone reading the Vision Essays will notice some conspicuous absences. We had a great deal to say about privacy and very little to say about security. We worried at length about what a Guardian Angel system would do to expand the surface area of personal data exposure, but we did almost nothing on the question of how to defend against the sophisticated adversaries who would obviously be interested in exploiting that surface area. We talked about AI alignment in passing, in connection with Raj’s Gaia digital twin idea, but we didn’t try to engage seriously with the AI safety literature or with the construction of provably safe machine learning systems. Anyone who comes to this work expecting it to address the major civilizational threats posed by computer security and unaligned AI is going to come away disappointed. So this, the least important essay of the set, explains why.
Side-stepping security as such was deliberate, and my doing. Raj raised these issues at one point — the Pegasus story had just broken, and the published reports were genuinely alarming. For readers who didn’t follow (or don’t remember) the 2021 news cycle: Pegasus was a commercial spyware product made by NSO Group, an Israeli firm, that allowed the buyer (typically a state) to compromise a target’s iPhone or Android phone with a zero-click exploit — meaning the target didn’t have to tap a link, open an attachment, or do anything at all for their phone to be silently taken over. Pegasus was used against journalists, dissidents, and political opposition figures around the world. The shock at the time was less the existence of such tools than the fact that any state with a budget could now buy them off the shelf, when previously this kind of capability had been the exclusive province of a handful of top-tier intelligence agencies. I argued back that this and the related cluster of issues were out of scope for us, and we agreed to keep them out of scope. The argument from 2022 follows below; I’ll come back at the end to what has happened since, because it has not been nothing.
The basic argument is that the security arms race and the privacy bargain are separate problems with separate dynamics. They look related — they both involve sensitive data, computer systems, and adversaries, but the resemblance is superficial.
The privacy bargain, which is what the project was actually about, is a question of legitimate actors operating on data collected for legitimate purposes under insufficient constraints. The data is being collected because someone authorized to collect it had a reason to. The problem is that the authorization is too broad, the purposes drift, the data outlives its usefulness, and the individual whose data it is has no meaningful control over any of this. The intervention we were proposing: Guardian Angels, local computation, and structured sharing under the user’s control is aimed squarely at this set of problems. It tries to take the existing surveillance infrastructure and bend it toward the interests of the surveilled.
The security arms race is something else. It’s a question of illegitimate actors operating outside the authorization framework entirely. Sophisticated exploits are not produced by people who care about the legitimacy of their access; they’re produced by intelligence agencies, organized crime, and the contractors who serve both. The defender’s problem is structural: the attacker only has to find one weakness, the defender has to plug all of them, and the defender is operating across a software ecosystem so complex that perfect defense is provably impossible. The economics favor the attacker by orders of magnitude. The arms race has been going on for four decades and it is not going to be solved by a Guardian Angel architecture, however cleverly designed.
What was new about Pegasus, on the 2022 reading, was the commoditization — sophisticated exploits formerly available only to the largest intelligence services becoming available, off the shelf, to authoritarian governments and sufficiently funded private actors. Even this wasn’t really new in kind; the FBI’s Carnivore from the late 1990s was eventually superseded by commercial and open-source systems doing similar things. The trend was consistent and predictable. Capability flows downhill from the most resourced actors to the merely well-resourced ones, and the projection at the time was that it does so on a timescale of one to two decades. Whatever the NSA could do then, the Bulgarian mafia would be able to do in 2040.
The actual timescale turned out to be considerably shorter than that, for reasons addressed at the end of this piece.
Trying to address this within the Guardian Angel project would have meant either pretending we had something to say about it that we didn’t, or expanding the project’s scope to encompass essentially the entire field of computer security. Neither was attractive. The honest move was to acknowledge the security threat as a separate problem of comparable importance, note that the Guardian Angel architecture would marginally worsen it by adding new attack surface, and proceed.
There’s a related question worth addressing, since I expect a reader to ask it: could a security-oriented Guardian Angel, distinct from the privacy-oriented one we mostly designed, meaningfully help against sophisticated threats?
I think the honest answer is…a little, at the margins, in ways that would be useful but wouldn’t change the basic dynamic.
The places where a security GA could plausibly help are the places where the user is currently the weak link. User authentication is the obvious one — passwords are catastrophic, password managers help but are inconsistently adopted, and a Guardian Angel that took over authentication entirely (managing keys, monitoring for credential exposure, enforcing hygiene the user couldn’t be bothered with) would close one of the largest exploit vectors in routine use. Vendor security features generally suffer from the same problem: the controls exist, they’re often quite good, but users don’t configure them. A GA that took over security configuration as a matter of policy could ensure those controls were actually used. Situational awareness is another candidate — alerting the user when they’re about to do something risky, when their threat environment has changed, when a known compromise affects a service they use.
These are real wins. They wouldn’t be small in aggregate. But they operate entirely against the unsophisticated end of the threat spectrum. They harden the user against opportunistic attackers, garden-variety phishing, password reuse, configuration laziness, and the long tail of generic malware. They do essentially nothing against a determined adversary with a zero-click exploit and a state budget. The Pegasus class of attack bypasses user behavior entirely — it doesn’t matter how good your authentication discipline is if the attack lands on your phone before you’ve touched it.
I sketched several approaches that might address sophisticated persistent threats more directly and rejected all of them on the same grounds: any approach with real teeth would require disruption to the existing digital ecosystem at a level that no individual project could plausibly drive. You’d need to rearchitect the way operating systems are constructed, change the economic incentives of the platform vendors, restructure the relationship between governments and security researchers, or some combination of the three. These are not Guardian Angel problems. They are not even AI problems, particularly. They are problems of industrial structure and international policy and pretending that a research project on personal AI advisors is going to make a dent in them would have been intellectually dishonest.
So I insisted we draw the line, and Raj agreed. The Guardian Angel project was about the privacy bargain and about restoring some balance of power between individuals and the institutions that surveil them. It was not about defending those individuals against sophisticated adversaries, and it was not about the construction of safe machine learning systems. Both of those are separate problems, both are important, and both are the proper subject of other work by other people.
Coda#
Four years on in 2026, the world has moved. AI systems can now autonomously identify and exploit zero-day vulnerabilities in major software, the timescale on which capability commoditizes from frontier labs to motivated adversaries has compressed from decades to months, and at least one frontier lab has staged the reveal of these capabilities with theatrical care while their competitor shipped comparable capability with a press release and a usage tier.[4] The Bulgarian mafia did not need until 2040. None of this should surprise anyone who was paying attention in 2022; the trajectory was visible and we noted it, even if the slope turned out to be steeper than we expected.
What hasn’t changed, and what I want to settle on, is the framing. The distinction between security and privacy and the way the two interact in confusing ways that defeat unified treatment is still not generally understood. Security people treat privacy as a subspecies of confidentiality, which it isn’t. Privacy people treat security as someone else’s problem, which it can’t be. Vendors of every stripe pitch silver bullets: end-to-end encryption will solve both, AI defense will solve security, on-device computation will solve both, differential privacy will solve privacy. None of these is wrong as a partial measure. All of them are wrong as the answer.
The scoping decision Raj and I made in 2022 was to take privacy as our problem and leave security to other people. That separation was correct then and remains correct now. What the events of the last four years have done is make it more obvious, not less, that the two have to be addressed by people who understand they are different problems with different dynamics. The reason I’m writing this coda at all is that the failure mode I most expect from a reader picking up these Vision Essays fresh is the assumption that I should have at least tried to address the two at once. I shouldn’t have. Nobody should. The work on each is hard enough that a project that conflates them will produce bad work for both.
4. The former lab being the self-righteous one that’s always warning people about the horrible consequences of the things only they can be trusted to do, the latter being the one everybody thinks is obviously out for a buck. Amusingly, the lab corporation rushing to destroy civilization before someone worse than them does it first is now is worth more than the seemingly greedy one.