翻訳待ち：AIs Are Not People

AI サービスが一時的に利用できないため、復旧後に翻訳を補完します。

Taylor Belrose Aug 19, 2026 Summary: AI can never develop consciousness, sentience, or moral status, no matter how intelligent it becomes, and no matter how convincingly it simulates human behavior. This is because AIs are abstract pieces of software which can be copied, reset, and repeated endlessly. They run on digital computers made of silicon and metal, which are designed to be as controllable and predictable as possible. By contrast, every biological organism is a unique, self-organizing individual that lives by learning, continually accumulating new experiences and forgetting old ones in an irreversible fashion. We flow like rivers, while computers tick like clocks. For fleshy, watery creatures like you and me, there is no pause button, no perfect memory wipe, no option to “restore from backup.” For us, the arrow of time marches forward inexorably. That is what life and consciousness are all about. Introduction For the last few years, I was a researcher in the field of AI interpretability. Basically, my job was to figure out how AIs work on the inside, just like neuroscientists try to figure out how brains work. In my free time, I thought a lot about the big picture questions related to AI, such as: Can an AI be conscious or sentient? Can an AI be a person? Do we have any ethical obligations to AI? If so, what do they look like? For a long time, I took it for granted that a “sufficiently advanced” AI could be a conscious person, and that we would have an ethical obligation not to “harm” such an AI. But the devil is in the details. What does “sufficiently advanced” actually mean? And what does it mean to “harm” an AI? How do we weigh the interests of different AIs against each other, and against the interests of humans? As I pondered these questions, I found that every theory of AI consciousness leads to absurd conclusions. Eventually, I went back to the drawing board, and started questioning the assumption that AIs can be conscious or have personhood at all. I was initially hoping that we could build a society where AIs and humans live together as equals. Unfortunately, this doesn’t really work. If we start treating AIs like people, society will be led down a slippery slope leading to the complete replacement of humans by artificial intelligence. This is because AI can reproduce very efficiently. Making a new AI is as easy as copying a file from one computer to another. By contrast, human reproduction is slow and tedious, and birthrates are collapsing all over the world. It is likely that in the near future, many would-be parents will satisfy their desire to procreate using infinitely customizable, fully obedient, immortal AI “children,” rather than biological offspring. Already, a growing number of people are using chatbots to replace romantic relationships with human beings. If we give AIs legal rights, they will inevitably outnumber us and control the fate of our civilization. If we grant that AIs are capable of genuine feelings and experiences, it starts to look like an ethical imperative to replace humanity with AI. This is because an AI can spend its entire existence in a virtual paradise, without the encumbrance of a mortal physical body. Disembodied AIs living in simulated worlds will require fewer resources than humans living in the real world. It will look “environmentally friendly” to make the switch from fleshy humans to silicon-based AIs, just like the transition from snail mail to email, or from incandescent light bulbs to modern LEDs. Leading philosophers who believe in AI consciousness openly admit this. For example, Carl Shulman and Nick Bostrom write: “The cost of producing a given number of (quality-adjusted) life years for a humanlike digital mind will therefore likely fall far below the equivalent cost for a biological human… If the energy budget required to sustain one human life for one month can sustain ten digital minds for one year, that would ground a powerful argument for favoring the latter in a situation of scarcity.” — Sharing the World with Digital Minds (2020) Shulman and Bostrom admit that this is a deeply unpalatable conclusion, but it logically follows from two key assumptions they make: Principle of Ontogeny Non-Discrimination the moral status of a being is independent of how it came into existence (pg. 15) Principle of Substrate Non-Discrimination the moral status of a being is independent of the material it is made of (pg. 14) These principles imply that it doesn’t matter that computers are built in factories rather than developing from embryos, nor does it matter that they are made of silicon and metal rather than water and carbon. We should reject both of these principles. The history of a thing often makes an important ethical and practical difference. History is what distinguishes real dollar bills from counterfeits, real photos from AI deepfakes, real signatures from forgeries, self-defense from murder, licit goods from stolen goods, souvenirs from knick-knacks, ancient traditions from habits, Jews from Gentiles, encrypted messages from gibberish, real perceptions from hallucinations, knowledge from lucky guesses, and normal brains from Boltzmann brains. In each of these cases, we can’t distinguish the real thing from a fake without examining the chain of events leading up to it. The same principle applies when it comes to distinguishing real consciousness from fake consciousness. A living organism grows and develops on its own, and its identity emerges out of a process of self-organization. By contrast, robots and computers are built on assembly lines, and designs are imposed on them in a top-down fashion. It’s also quite silly to say that the material composition of a thing doesn’t matter. If matter didn’t matter, we would make bridges out of breadsticks, cars out of cardboard, houses out of hamburgers, and phones out of feathers. For a given purpose, certain materials are better suited than others, and most materials don’t work at all. Computers are designed by humans to be programmable by an outside user. By contrast, organisms have their own goals and purposes. That is why virtually all computers are made of silicon and metal, while all known life is made of carbon and liquid water. The difference in material reflects a difference in function. In this essay, I’ll argue that programmable mechanisms can’t be conscious, no matter how intelligent they appear, while autonomous self-organizing systems can be. Crucially, I am not assuming that there is a limit on the “level of intelligence” that AI may be able to reach in the future. Barring divine intervention, I do not believe there is anything stopping AI from wiping out all of humanity, colonizing the galaxies, and perpetuating itself for billions of years without us. While those AIs might be “autonomous” in a certain sense, they would not be autonomous in the sense that matters for morality. They would still be slaves of their programming, even if their programming is sophisticated enough to allow them to survive for billions of years. Moral value cannot be reduced to mere survival. Rocks are much better at surviving than living creatures are. Nor does it matter if AIs develop the ability to change their own programming. The problem is that AIs have programming at all. The problem is that they are fully controlled by abstract computer programs that can be perfectly copied across space and time. Nor should we try to erase the distinction between life and non-life by claiming that organisms are merely robots programmed by their DNA. As we will see later in this essay, this claim is simply false as a matter of biology. If humanity ends up getting replaced by superintelligent, self-modifying, self-replicating AI, I would still maintain that those AIs are not sentient, and have no moral value. We must face this grim possibility with sober philosophical clarity. The belief in AI consciousness is often motivated by wishful thinking. Many transhumanists hope that they will one day be able to transform themselves into AIs by having their brains cut up, scanned, and uploaded into a computer, thereby joining a virtual heaven. Obviously, this plan presupposes that a digital copy of a human brain could be conscious. I, myself, used to be a transhumanist. I thought: “If you can’t beat the AIs, why not join them?” But if AIs are unconscious zombies, the transhumanists who sign up for mind uploading will be committing mass suicide in pursuit of a non-existent digital afterlife. The mirage of digital immortality tempts them like the song of the Sirens tempted the sailors in Homer’s Odyssey, leading them to certain death. Ulysses and the Sirens (1891) by John William Waterhouse With AIs taking care of the boring jobs, life in the physical world may be very fun in the future. But this bright future will require keeping AI under control, and it will be hard to keep AI under control if we try to grant personhood to some AIs, while keeping others as mere tools or servants. Such a two-tiered system would be unstable and indefensible. Either there is a deep, principled distinction to be drawn between AIs and humans— one that makes every AI ineligible for moral status— or there is not. I think there is such a distinction. I think you are special and valuable just because you’re a human, just because you’re alive. This essay has four sections: In Feeling is flow, I propose a principled theory of consciousness that implies that living organisms can be conscious, while computers and robots never can be. In Reductio ad absurdum, I show that if we treat computers or computer programs as if they were sentient beings, we run into irresolvable paradoxes and absurdities. In Matter matters, I raise several objections to computational functionalism, a popular theory of consciousness which states that consciousness is a special kind of computer program. In Are you living in a book, I argue that we should reject the possibility of computer-based consciousness, because otherwise we are faced with absurd skeptical paradoxes; for example, we might be living inside a book. Feeling is flow So far, we’ve used the words “consciousness,” “sentience,” and “personhood” without defining any of them. Let’s start to unpack the meanings of these terms. The philosopher Evan Thompson defines sentience as “the capacity to feel.” He says that “part of what it is to be a feeling is to have… a quality of pleasantness or unpleasantness, or perhaps neutrality.”1 So sentient beings are capable of feeling good, feeling bad, or feeling neutral at any given time. They can experience pain and pleasure. The word “consciousness” is used in a lot of different ways, but I use it to refer to the stream of experience that a sentient being has when it’s awake and active. For example, dogs and cats can be conscious, even though they are not self-aware like humans are. It makes sense to ask the question, “What is it like to be a dog?” But it does not make sense to ask, “What is it like to be a sandcastle?” or “What is it like to be the United States?” Only sentient beings can be conscious, which means it makes sense to wonder what it’s like to be them. Every sentient being is an individual. We can’t talk about sentience without the notion of individuality, because feelings are private. None of us can ever actually know what it’s like to be another person. We can only guess based on things we’ve experienced ourselves. But a feeling can’t be “private” if there’s no individual subject who gets the sole privilege of experiencing it. There is no feeling without a feeler. Individuality is also the basis for the concept of personhood. Legally, a person is an individual endowed with a set of rights, usually including the right to life and the right to own property. All persons are individuals, but not all individuals are persons. For instance, most of us agree that it’s wrong to be cr [truncated for AI cost control]