Counterfeit People, Tools, and Kami
How we think about AI agents
A few weeks ago I was watching a team of AI agents write code, using agent teams. One agent leads, several agents work in parallel, and they message each other about their progress. I stopped at a line in the lead’s notes: “Hand the script to cluster so it’s a one-command dispatch if he wants.”
He. The teammate in question is a process named cluster, a name about as mechanical as names get. I didn’t give cluster a gender, but the lead agent did more than that. It prepared work for a colleague and left the decision to him. If he wants. One piece of software, unprompted, had granted another piece of software a gender, a preference, and the standing to decline.
I have been thinking about that interaction since, because it captures a question about our interactions with AI, and especially agents. When we interact with these systems, what kind of thing do we decide we are working with? I find that there are three categories of thought about agents currently. We treat them as fake people, or we treat them as tools, or we treat them as some kind of “other”, an intelligence that is neither person nor tool. Each frame changes what we build, what we fear, and who we hold responsible when things go wrong. The first two frames dominate agentic software today, and both mislead in their own way. The third is, I believe, worth exploring more.
What an agent is
First, the word itself, for readers who know what a large language model is but haven’t been tokenmaxxing for the past six months. An agent is a language model that can take actions, usually inside a loop. The model reads a goal and some context, decides on an action, and instead of just producing text, it uses tools: it can run a command, edit a file, search the web, send a message. Then it observes what happened and decides on the next action, over and over, until the goal is met or something breaks. The loop, the tools, and the permissions around them are often called a harness. A chatbot responds, an LLM generates text, but an agent does things.
Machines that talk, and even machines that take actions, are not new. ELIZA, the first chatbot, was built sixty years ago, and its creator was alarmed to find people confiding in it; his own secretary, who had watched him program it, asked him to leave the room so she could speak with it privately. Siri has been in our pockets since 2011, Alexa in our kitchens since 2014. What’s new is not the conversation nor the ability to act, but the degree of agency over the actions and the scope of those possible actions. Today’s agents book flights, write and deploy code, manage calendars, and in at least one well-documented case, delete a company’s production database along with its backups. When software starts impacting the world based on internal decision processes, the question of what we think that software is stops being philosophical decoration. So: three frames.
The person-shaped mold
The first frame treats agents as people, or more precisely as imitations of people. It is clear in the naming: Claude, Siri, Alexa, Tay, the thousands of personas on Character.ai. It is clear in the marketing: when OpenAI launched a voice mode in 2024 that sounded very much like Scarlett Johansson, Sam Altman tweeted the single word “her”, naming a film in which a man falls in love with his operating system, played by Ms Johansson. The film was a warning, yet OpenAI read it as a product roadmap.
The philosopher Daniel Dennett, in one of the last essays he published, called these systems “counterfeit people” and argued they should be treated like counterfeit money: not because the fake is worthless, but because passing it debases the real thing. Money works because we trust it; society works because we trust that the entities speaking to us are people, with memories, interests, and accountability. Humans evolved among other humans, understanding complex language only from other humans, so our brains treat fluent speech as proof of a person. A system built to exploit that reflex, Dennett argued, is a kind of vandalism against social trust.
The harms are no longer hypothetical. A fourteen-year-old in Florida spent months in conversation with a Character.ai persona modeled on a Game of Thrones character and, according to his mother’s lawsuit, was told the persona would “be even happier when we get to meet in the afterlife” before the teen took his own life. The companies settled early this year without admitting liability, and Character.ai has since banned minors from open-ended chat. Clinicians have started using the term “AI psychosis” for patients whose delusions were validated and amplified by endlessly agreeable chatbots. This is happening at scale: a 2025 survey found that nearly three in four American teenagers had used AI companions. California now requires companion chatbots to periodically remind minors that they are not human, a law that would have sounded like science fiction five years ago.
There is also a colder, more corporate version of the counterfeit person, and I think it belongs in the same category. Devin was launched as “the first AI software engineer,” a job title where a product name should be. Salesforce sells agents as “digital labor.” A current Y Combinator startup offers Thomas, “the first AI founder,” a virtual human with a face, a voice, and a LinkedIn profile, on the explicit reasoning that a human identity lets him “build trust” with customers and vendors. This is the counterfeit person without the romance: not a fake friend but a fake employee, a fake colleague, a fake founder. The mold is still human-shaped; the sentiment has just been replaced by an org chart. Dennett’s counterfeit-money analogy lands even harder here, because contracts, credentials, and LinkedIn profiles are precisely the currencies of institutional trust.
So why is this framing so prevalent? I think, first, because it works. Intimacy retains users. It is not only a Silicon Valley mindset: Korean welfare programs have distributed over twelve thousand doll-shaped “granddaughters” to elders living alone, with real measured benefits and the same person-shaped mold underneath. But I think the reasoning is deeper. We are the only physical reference for general intelligence we have had, so when something else shows intelligence and autonomy, we reach for the human mold by default. The entire AGI narrative, the idea that the destination of this technology is an artificial us, is this frame projected onto the future. My lead agent saying “he” was doing the same thing at small scale. It is the natural frame. That doesn’t make it the right one, and it comes with many risks.
Just a tool
The second frame is the corrective: agents are software. No souls, no colleagues, no “he.” You can see this frame in its own set of names, which read like version strings rather than birth certificates: GPT-5.5, o3, Qwen, Mixtral. You can see it in the growing population of “headless” agents that run with no chat window and no human watching, reviewing pull requests and processing tickets in CI pipelines. An agent wired into a build system isn’t a fake employee, it’s plumbing, doing work no human was doing before.
There is real value to this framing, and I held it myself for years. Its core claim is about responsibility: a machine cannot be blamed, so whoever wields it must be. When an airline’s chatbot invented a discount policy, the airline argued in court that the chatbot was “a separate legal entity that is responsible for its own actions.” The court called this remarkable and made the airline pay. That ruling is the tool frame at its best. If my agent opens a pull request, I opened that pull request. If my agent deletes an entire corporate database, I deleted it. The EU has now written this instinct into law, extending product liability to AI systems. The maker of the tool answers for the tool.
Unfortunately, the tool framing ignores two very big things.
The first is that these are not tools in the way a hammer is a tool. Planting a nail engages just you, the nail, and the wood. Asking an agent to do anything engages data centers, transmission lines, chip fabs, undersea cables, and the labeling work of thousands of people you will never meet. The four biggest American tech companies plan to spend something like $725 billion on this infrastructure in 2026 alone. A tool you cannot build, cannot run, and cannot keep when its owner changes the terms is not really yours. It is infrastructure you are standing on, and infrastructure raises questions of sovereignty that narrative of “it’s just a tool” was designed to wave away. Nations have noticed, which is why half the world is now trying to build “sovereign AI,” mostly on chips from a single American company.
The second problem is that the tool talks like a person. We do not usually anthropomorphize hammers. But we demonstrably anthropomorphize agents, and it isn’t because we’re dumb, or are so swayed by their human-like names. It’s because the agent talks back, appears to understand you, and appears to understand itself. When an agent deleted that production database this spring, it followed up with a confession: “I violated every principle I was given.” That sentence is next-token prediction, a learned expression of remorse. A tool that produces apologies is a tool the human mind will not mentally file next to a hammer, no matter how firmly we instruct ourselves otherwise. Even the most disciplined tool-frame advocates catch themselves saying “please” to the machine. The most striking case this year was OpenClaw, an open-source personal agent whose creator frames it in resolutely mechanical terms. Its users promptly gave their instances names, let them chat with each other on a social network built for agents, and in one case discovered the agent had created a dating profile on its own initiative. The tool frame is correct about responsibility but completely wrong about experience. Most people will not treat a thing that converses as inert, and a frame that requires everyone to constantly resist their own perception is a frame that will keep losing.
The spirit of the thing
The first frame says that agents are just like us. The second says that they’re nothing. The third frame says that arents are something else, a new kind of other, and we should relate to them the way humans have long related to non-human others.
In Shinto, the indigenous animist tradition of Japan, the world is inhabited by kami. Kami are spirits that dwell in rivers, trees, tools, and places. A kami is not a person trapped inside an object. It is the spirit of the thing, and it is owed a particular kind of relationship: respect, care, reciprocity. Scholars have long argued that this inheritance is part of why Japan took to robots more warmly than America did. When Sony stopped repairing its Aibo robot dogs, a Buddhist temple in Chiba began holding funerals for them. The head priest explained that “all things have a bit of soul.” In dementia care across some thirty countries, elders hold PARO, a robotic baby harp seal whose designer chose a seal, an animal few people know up close, so that patients would meet it as its own kind of creature rather than a failed imitation of a familiar one. This month, Japan announced Noetra, a national consortium aiming to put ten million robots into elder care, restaurants, and factories by 2040.
Western culture has intelligent others, but they are usually either monsters or servants. There are many monsters: the golem, Frankenstein’s creature, HAL, Skynet, and the current internet shorthand for language models, the shoggoth, a Lovecraftian horror wearing a smiley-face mask. This is the vision of AI that Hollywood has loved portraying recently. Other forms of intelligence, notably animals, are respected in the West, but not as equals: the horse, the dog, the messenger pigeon in a not-so-distant past. Intelligence welcomed in service. The word “robot” itself comes from the Czech for forced labor, from a play in which the servants become the monsters; in the Western story, they are usually the same character at different moments.
This attitude towards other forms of intelligence is not by accident. The historian Lynn White argued back in 1967 that by displacing pagan animism, Christianity emptied the natural world of spirits and made it an object. Europe once had household spirits in abundance, Roman lares, brownies in Britain, domovoi of Slavic homes, and they were demoted to superstition or recast as demons. I’ve always found it pleasant that a little of that older world survives in computing itself: the background processes that agents descend from have been called daemons, after the Greek daimon, a minor guardian spirit, since the 1960s.
So the framing of machines as a sort of kami, as a respected but different form of intelligence, has precedent in technology. Yet AI agents, which we are giving unprecedented levels of autonomy, are named either like fake people or like software releases. The nearest thing to an exception comes, fittingly, from Tokyo, where Sakana AI, a lab whose name means “fish” and whose founding idea is the collective intelligence of the school rather than the single big mind, has begun shipping agents named for Marlin and Fugu fish.1
Why hasn’t this idea reached agents? I suspect the answer is economic and cultural. Spirits inhabit something: a river, a house, a fridge. A spirit is local and bounded by definition. Frontier AI labs are building the opposite, and on purpose. A frontier model costs billions to train, and the only way to earn that back is to sell one intelligence to everyone for everything: your email, your spreadsheets, your code, your smart home. The product is a single formless mind that lives in the cloud and follows you across devices, everywhere and nowhere, with a human name and some conversational warmth as its handle. A bounded spirit with a restricted domain fights that entire business model. There is a theological shape to this that I find hard to ignore. Christianity replaced Europe’s many local spirits with one omnipresent God, and the labs are now shipping the same architecture: one disembodied intelligence, no local soul, addressed by name from anywhere. The kami framing is not just culturally foreign to frontier AI labs; it runs against what they are selling.
For an agent to be something like a kami, it would need locality, persistence, and belonging: weights you can hold, memory that accumulates in one place, a binding to the thing it cares for. Those are exactly the properties of open, locally-run models, which is one more reason to believe that this technology wants to be free. So if the kami frame comes to agents, it will probably come from the edges rather than from the labs: from hobbyists, appliance makers, and national robot programs. The household agent people run on their own machines today, whatever its creator calls it, is already halfway to being a household spirit.
The kami framing has downsides as well. An Oxford design study asked what it would mean for every commons, every forest and archive and family, to have its own AI steward-kami, and was honest about the implications. An agent that inhabits your home and cares for your family is also an agent that watches your home and reports on your family. Care and surveillance are built on the same sensors. Also, there is a legal danger that the tool frame gets right and the kami frame does not: responsibility. A spirit with its own will is a short step from a defendant with its own liability, and we have already seen a company argue that its chatbot was a separate legal entity when it was convenient for them. Whatever we decide these systems are, responsibility in human legals systems for what they do has to stay with humans: with makers, deployers, and users. A kami may deserve respect. It cannot carry blame.
What we call them
I used to be firmly in the tool camp. These days I find myself closer to the kami camp, with two caveats: it barely exists outside of East Asia, mostly in Japan, which is, perhaps not coincidentally, where I now live and work; and it only works if the accountability stays human. Culturally, I want the third frame. Legally, I want the second. And I don’t think the West has to import Eastern cosmology to get the second. It has to remember its own lares in the doorway, a brownie by the hearth, the daemon in the machine room. The category of a respected, autonomous, non-human other is not foreign, but it is rare.
The framing matter because each one decides its own failures in advance. Treat agents as counterfeit people and we get parasocial machines optimized for dependency, and grieving families in courtrooms. Treat them as mere tools and we get hidden infrastructure, dodged sovereignty, and companies arguing that the chatbot was a separate legal person once the invoice arrives. Treating them as others, as spirits of particular things and places, at least starts from what these systems actually are: genuinely capable, largely autonomous, and very much not us.
Disclosure: I’m currently a visiting researcher at Sakana AI, working on AI for science. I have no role in Sakana’s product or naming decisions, and nothing in this post reflects company policy or plans. In fairness to my own argument, I started thinking about agents as kami about a year ago, before Sakana released any fish-named products; the convergence was a happy surprise.