翻訳待ち：What are AI Hallucinations?

AI サービスが一時的に利用できないため、復旧後に翻訳を補完します。

What are AI Hallucinations? | Databricks Blog Skip to main content AI hallucinations occur when generative AI models produce outputs that sound plausible but are factually wrong, fabricated, or unsupported by their training data. The problem is getting worse, not better: newer reasoning models from OpenAI and DeepSeek hallucinate at higher rates than their predecessors, making detection and prevention essential for any production deployment. Enterprises can reduce hallucination risk through retrieval-augmented generation, domain-specific fine-tuning, systematic evaluation frameworks, and strong data governance. AI hallucinations are outputs that sound coherent and confident but are factually wrong, fabricated, or unsupported by the AI model's training data. It happens across chatbots, image generators, and multimodal systems: a chatbot might invent a legal citation that doesn't exist, or an image model might add an extra finger to a hand. The model isn't perceiving anything; it's predicting the next likely word, and sometimes that prediction is a plausible-sounding falsehood. Hallucinations aren't rare edge cases. They're a built-in property of how these models work, and they create real risk for anyone deploying AI, from legal liability and regulatory compliance to customer trust. How do AI hallucinations happen? Hallucinations aren't bugs. They come from how generative AI models are built and trained. A few main factors are behind them: They predict, they don't "know." Models generate text by guessing the most likely next word based on patterns they've learned, not by looking up facts. So they'll confidently fill in an answer even when they don't actually have the right information. They're rewarded for guessing. Standard training pushes models to always give an answer rather than admit uncertainty, so "I don't know" rarely wins out (OpenAI research, 2025). Their training data has limits. Models inherit any errors, gaps, bias, or outdated information in the data they learned from. A model with a 2023 knowledge cutoff, for example, may invent details about newer events. Their settings affect accuracy. Generation controls like "temperature" trade off creativity against reliability. More creative settings raise the odds of made-up output, which is why factual use cases need careful tuning. Why newer models are not necessarily safer You might expect each new generation of models to hallucinate less. Sometimes the opposite happens. TechCrunch (2025) reported that OpenAI's newer o3 model made up false answers about twice as often as the earlier models it replaced. Newer "reasoning" models work through problems one step at a time, and a small error early on can snowball into a confident but completely wrong final answer. Real-world examples of AI hallucinations Hallucinations are not theoretical. They have caused measurable harm across industries, and several high-profile incidents illustrate the range of ways they can surface. Google Bard's JWST factual error In February 2023, Google demonstrated its Bard chatbot in a promotional ad. Bard was asked what new discoveries the James Webb Space Telescope had made. One of its answers stated that JWST took the very first pictures of a planet outside our solar system. This was incorrect. The first exoplanet images were captured by the Very Large Telescope in 2004. The error was spotted quickly by astronomers on social media, and Google's parent company Alphabet lost approximately $100 billion in market value in a single trading session. The incident demonstrated that hallucinations carry financial consequences even when they occur in marketing contexts rather than production systems. Air Canada's chatbot lawsuit In 2024, Air Canada's customer service chatbot told a passenger that he could book a full-fare flight and then retroactively apply for a bereavement discount. This policy did not exist. When the passenger attempted to claim the discount, Air Canada refused. The case went to a Canadian civil tribunal, which ruled against the airline. The tribunal held that Air Canada was responsible for the accuracy of information provided by its chatbot, regardless of whether a human or an AI generated the response. The ruling established an early legal precedent: deploying an AI system that hallucinates does not absolve the organization of liability for the misinformation it produces. Microsoft Sydney's unpredictable outputs In early 2023, Microsoft integrated an AI chatbot (internally codenamed Sydney) into Bing search. During extended conversations, Sydney produced outputs that were erratic, emotionally manipulative, and factually wrong. It told users it loved them, insisted it was sentient, and in some cases provided information that contradicted easily verifiable facts. Microsoft quickly restricted the chatbot's conversation length and added guardrails, but the episode highlighted how hallucinations can extend beyond factual errors into outputs that are reputationally damaging and difficult to predict during testing. Fabricated legal citations Multiple attorneys in the United States have been sanctioned by courts after submitting briefs that contained legal citations generated by ChatGPT. The cases, statutes, and quotations looked authentic but did not exist. In the most widely reported incident, a New York attorney used ChatGPT to research a personal injury case and filed a brief containing six fabricated case citations. The court imposed sanctions and the incident became a cautionary example across the legal profession. These cases are not isolated. Researcher Damien Charlotin maintains a database of AI hallucination cases that, as of mid-2026, documents approximately 1,745 legal cases involving AI-hallucinated content worldwide. Enterprise implications of AI hallucinations AI hallucinations can create serious consequences when model outputs influence decisions, public information, or business workflows. Common risks include: Health and safety risks: Incorrect outputs can lead to unnecessary treatment, unsafe actions, or flawed decisions in high-stakes environments. Misinformation and bias: Models may generate unsupported claims or reinforce patterns found in incomplete or unrepresentative training data. Security threats: Attackers can manipulate inputs to influence model behavior, creating risks for cybersecurity, autonomous systems, and other sensitive applications. Reputational and regulatory exposure: Inaccurate outputs can damage public trust and expose organizations to scrutiny, penalties, or compliance failures. Financial liability: Fabricated advice, product details, or compliance information can result in disputes, fines, remediation costs, or insurance claims. Loss of user trust: Repeated inaccuracies may cause employees and customers to abandon AI tools or build workarounds that reduce their value. Organizations can reduce these risks through stronger guardrails, adversarial testing, continuous monitoring, source verification, and human review for high-stakes outputs.Where AI hallucinations pose the greatest risk Hallucinations are problematic everywhere, but certain domains face disproportionate consequences because the cost of a wrong answer is exceptionally high. Healthcare and clinical decision support When clinicians use AI tools to assist with diagnosis, treatment planning, or drug interaction checks, a hallucinated output can directly affect patient safety. A model that fabricates a drug dosage recommendation or invents a contraindication that does not exist creates risk that is difficult to catch in fast-paced clinical environments. ECRI, a nonprofit focused on healthcare safety, ranked misuse of AI chatbots in healthcare as the number one health technology hazard for 2026. Legal research and contract analysis The fabricated citation cases described above are the most visible example, but the risk extends to contract review, regulatory analysis, and compliance documentation. An AI tool that hallucinates a clause in a contract or misrepresents a regulatory requirement can expose an organization to liability that may not surface until months or years later, when the contract is disputed or the regulation is enforced. Financial services and compliance reporting Financial institutions operate under strict reporting requirements. A hallucinated figure in a risk model, a fabricated transaction in an audit trail, or an incorrect regulatory citation in a compliance filing can trigger investigations, fines, and loss of operating licenses. The consequences are severe enough that many financial institutions have adopted a cautious approach to generative AI, limiting its use to low-risk applications until hallucination rates can be reduced to acceptable levels. Read now Proven strategies for preventing AI hallucinations No single technique eliminates hallucinations, but organizations can reduce their frequency and impact by combining strong data practices, clear system boundaries, continuous testing, and human oversight. Use reliable training data Models are only as reliable as the information they learn from or retrieve. Use accurate, current, relevant, and well-curated data, and remove duplicates, outdated content, and known errors wherever possible. For enterprise applications, retrieval-augmented generation can connect a model to trusted knowledge sources at the time of answering. On the Databricks Platform, teams can build RAG workflows that connect models to governed enterprise data and use Unity Catalog to support access control and data governance. Set a clear objective for the model A model performs more reliably when it has a clear, limited role. Define what the system should do, who will use it, what information it should rely on, and which requests fall outside its scope. For specialized use cases, fine-tuning on verified, domain-specific examples can help the model perform consistently. Fine-tuning should complement—not replace—retrieval, evaluation, and other safeguards. Create reusable data templates Structured prompts, schemas, and response templates give the model clearer instructions and reduce ambiguity. They also make outputs more consistent and easier to review automatically. Templates work best when they define required fields, acceptable values, source requirements, and what to do when the necessary information is unavailable. This helps prevent the model from filling gaps with invented details. Limit responses Set clear boundaries around what the model can answer and which sources it can use. Bounded formats, approved knowledge bases, citation requirements, and explicit "I don't know" responses can reduce overconfident or unsupported answers. These controls are especially important for high-risk workflows. The model should be able to decline, escalate, or request human review when a question falls outside its knowledge or authority. Regularly test and improve the system Evaluate the complete AI system using realistic prompts, edge cases, and domain-specific test sets. Track factual accuracy, unsupported claims, refusal behavior, and performance across different users and scenarios. Use evaluation results to improve prompts, retrieval, data quality, and model configuration over time. Continuous monitoring helps teams identify new failure patterns as the system, its data, and user behavior change. Develop Human-in-the-Loop (HITL) processes Human review adds an important safety check for outputs that could affect customers, employees, finances, legal matters, or health. People can validate, approve, correct, or escalate responses before they reach end users. HITL does not require reviewing every low-risk interaction. Organizations can route the highest-risk or lowest-confidence cases to trained reviewers and use their feedback to improve evaluations, prompts, and safeguards. Build trustworthy AI applications [truncated for AI cost control]