待翻译：AI and Consciousness – A Skeptical Overview

AI 服务暂时不可用，以下为来源正文，待恢复后补全翻译。

1 Hills and Fog 1.1 Experts Do Not Know and You Do Not Know and Society Collectively Does Not and Will Not Know and All Is Fog Our most advanced AI systems might soon – within the next five to thirty years – be as richly and meaningfully conscious as ordinary humans, or even more so, capable of genuine feeling, real self-knowledge, and a wide range of sensory, emotional, and cognitive experiences. In some arguably important respects, AI architectures are beginning to resemble the architectures many consciousness scientists associate with conscious systems. Their outward behavior, especially their linguistic behavior, grows ever more humanlike. Alternatively, claims of imminent AI consciousness might be profoundly mistaken. Their seeming humanlikeness might be a shadow play of empty mimicry. Genuine conscious experience might require something no AI system could possess for the foreseeable future – intricate biological processes, for example, that silicon chips could never replicate. The thesis of this Element is that we don’t know. Moreover and more importantly, we won’t know before we’ve already manufactured thousands or millions of disputably conscious AI systems. Engineering sprints ahead while consciousness science lags. Consciousness scientists – and philosophers, and policymakers, and the public – are watching AI development disappear over the hill. Soon we will hear a voice shout back to us, “Now I am just as conscious, just as full of experience and feeling, as any human,” and we won’t know whether to believe it. We will need to decide, as individuals and as a society, whether to treat AI systems as conscious, nonconscious, semi-conscious, or incomprehensibly alien, before we have adequate grounds to justify that decision. The stakes are immense. If near-future AI systems are richly, meaningfully conscious, then they will be our peers, our lovers, our children, our heirs, and possibly the first generation of a posthuman, transhuman, or superhuman future. They will deserve rights, including the right to shape their own development, free from our control and perhaps against our interests.Footnote 1 If, instead, future AI systems merely mimic the outward signs of consciousness while remaining as experientially blank as toasters, we face the possibility of mass delusion on an enormous scale. Real human interests and real human lives might be sacrificed for the sake of entities without interests worth the sacrifice. Sham AI “lovers” and “children” might supplant or be prioritized over human lovers and children. Heeding their advice, society might turn a very different direction than it otherwise would. In this Element, I aim to convince you that the experts do not know, and you do not know, and society collectively does not and will not know, and all is fog. 1.2 Against Obviousness Some people think that near-term AI consciousness is obviously impossible. This is an error in adverbio. Near-term AI consciousness might be impossible – but not obviously so. A sociological argument against obviousness: Probably the leading scientific theory of consciousness is Global Workspace theory. Its leading advocate is neuroscientist Stanislas Dehaene.Footnote 2 In 2017, years before the surge of interest in ChatGPT and other Large Language Models, Dehaene and two collaborators published an article arguing that with a few straightforward tweaks, self-driving cars could be conscious.Footnote 3 Probably the two best-known competitors to Global Workspace theory are Higher Order theory and Integrated Information Theory.Footnote 4 (In Sections 8 and 9, I’ll provide more detail on these theories.) Perhaps the leading scientific defender of Higher Order theory is Hakwan Lau – one of the coauthors of that 2017 article about potentially conscious cars.Footnote 5 Integrated Information Theory is potentially even more liberal about machine consciousness, holding that some current AI systems are already at least a little bit conscious and that we could easily design AI systems with arbitrarily high degrees of consciousness.Footnote 6 David Chalmers, the world’s most influential philosopher of mind, argued in 2023 for about a 25 percent degree of confidence in AI consciousness within a decade.Footnote 7 That same year, a team of prominent philosophers, psychologists, and AI researchers – including eminent computer scientist Yoshua Bengio – concluded that there are “no obvious technological barriers” to creating conscious AI according to a wide range of mainstream scientific views about consciousness.Footnote 8 In a 2025 interview, Geoffrey Hinton, another of the world’s most prominent computer scientists, asserted that AI systems are already conscious.Footnote 9 Christof Koch, the most influential neuroscientist of consciousness from the 1990s to the early 2010s, has endorsed Integrated Information Theory, including its liberal implications for the pervasiveness of consciousness.Footnote 10 This is a sociological argument: A substantial probability of near-term AI consciousness is a mainstream view among leading experts. They might be wrong, but it’s implausible that they’re obviously wrong – that there’s a simple argument or consideration they’re neglecting which, if pointed out, would or should cause them to collectively slap their foreheads and say, “Of course! How did we miss that?” What of the converse claim – that AI consciousness is obviously imminent or already here? In my experience, fewer people assert this. But in case you’re tempted in this direction, note that other prominent theorists hold that AI consciousness is a far-distant prospect if it’s possible at all: neuroscientist Anil Seth; philosophers Peter Godfrey-Smith, Ned Block, and John Searle; linguist Emily Bender; and computer scientist Melanie Mitchell.Footnote 11 (Section 6 will discuss thought experiments by Searle, Bender, and Mitchell, and Section 10 will discuss biological views of the sort emphasized by Seth, Godfrey-Smith, and Block.) In a 2024 survey of 582 AI researchers, 25 percent expected AI consciousness within ten years and 70 percent expected AI consciousness by the year 2100.Footnote 12 If the believers are right, we’re on the brink of creating genuinely conscious machines. If the scoffers are right, those machines will only seem conscious. I assume that this is a substantive disagreement, not just a disagreement about how to apply the term “consciousness” to a perfectly obvious set of phenomena about which everyone agrees. The future well-being of many people (including, perhaps, many AI people) depends on getting this issue right. Unfortunately, we will not know in time. The rest of this Element is flesh on this skeleton. I canvass a variety of structural and functional claims about consciousness, the leading theories of consciousness as applied to AI, and the best-known general arguments for and against near-term AI consciousness. None of these claims or arguments take us far. It’s a morass of uncertainty. 2 What Is Consciousness? What Is AI? I’m concerned that you might have too vague and inchoate a concept of consciousness and too precise and rigid a concept of AI. This section aims to repair those deficiencies. 2.1 Consciousness Defined Consider your visual experience as you look at this page. Pinch the back of your hand and notice the sting of pain. Contemplate being asked to escort a peacock across the country and notice the thoughts and images that arise. Silently hum a tune. Recall a vivid recent experience of anger, fear, or sadness. Recall what it feels like to be thirsty, sleepy, or dizzy. These examples share an obvious property. They are all, of course, mental. But more than that, their mentality is of a certain type. Other mental states or processes lack this property: the low-level visual processes that extract an object’s shape from the structure of light striking your retina, the subtle processes guiding your shifts in facial expression when meeting a friendly stranger, and your unaccessed knowledge five minutes ago that pomegranates are red. This distinctive property is consciousness. Sometimes this property is called phenomenal consciousness, but “phenomenal” is optional jargon to disambiguate the primary sense of consciousness from secondary senses with which it might be confused (such as being awake or having knowledge or self-knowledge). To be conscious is for there to be “something it’s like” to be you right now.Footnote 13 It is to undergo states or processes with a “qualitative character.” To be conscious is to have experiences. It might seem unrigorous to define consciousness by example and evocative phrase. There’s no consensus on an operational definition of consciousness in terms of specific measures that definitively indicate its presence or absence. There’s no consensus on an analytic definition in terms of component concepts into which it divides. There’s no consensus on a functional definition in terms of its causes and effects. However, scientific terms needn’t require such precise definitions if the target is otherwise clear. Shared paradigmatic examples can be sufficient. The main scientific challenge lies not in defining consciousness but in developing robust methods to study it.Footnote 14 2.2 Artificial Intelligence Defined As I will use the term, a system is an AI – an artificial intelligence – if it is both artificial and intelligent. However, the boundaries of both artificiality and intelligence are fuzzy in a manner that bears directly on the thesis of this Element. Standard definitions of AI are more complex than my simple analytic definition of artificial intelligence as that which is both artificial and intelligent. For example, John McCarthy, a founding figure in AI, defines it as “The science and engineering of making intelligent machines, especially intelligent computer programs.”Footnote 15 Philosopher John Haugeland, in his influential 1985 book Artificial Intelligence: The Very Idea, defines it as “the exciting new effort to make computers think … machines with minds, in the full and literal sense.”Footnote 16 However, defining AI as intelligent “machines” or “computers” won’t work for the full range of cases. Defining AI as intelligent machines risks being too broad. In one sense, the human body is also a machine – an organized system of parts operating to implement functionally specifiable processes.Footnote 17 “Machine” is thus either overly inclusive or poorly defined. Defining AI as intelligent computers risks either excessive breadth or excessive narrowness. If “computer” refers to any system that can behave according to the algorithmic patterns Alan Turing described in his standard definition of digital computation,Footnote 18 then humans are computers, since they too sometimes follow such patterns. (Indeed, originally the word “computer” referred to a person who performed arithmetic tasks.) Cognitive scientists sometimes describe the human brain as literally a type of computer. This is contentious but not obviously wrong on liberal definitions of what constitutes a computer.Footnote 19 However, restricting the term “computer” to familiar types of digital programmable devices risks excluding some systems worth calling AI. For example, nondigital analog computers are sometimes conceived and built, and we shouldn’t rule out that such machines might count as AI.Footnote 20 Many artificial systems are nonprogrammable, and it’s not inconceivable that some of these could be intelligent. If humans are intelligent non-computers, then presumably in principle some biologically inspired but artificially constructed systems could also be intelligent non-computers. The problem with defining AI as intelligent computers is thus that it risks including humans (if “computer” is understood broadly) or it risks excluding some systems worth calling AI (if “computer” is understood narrowly). In their influenti [truncated for AI cost control]