Skip to content
AI News HubLIVE
Original source2 min read

NEBench R1: a Reasoning Benchmark for Evaluating Nuclear Domain Competence in Large Language Models

Summary

NEBench R1 is a 7,546-question reasoning benchmark drawn from NRC reactor operator licensing exams across 18 nuclear facilities. Six LLMs ranged from 34.0% to 73.9% accuracy, and correct answers often came with weak factual and procedural justification; longer reasoning traces correlated with lower accuracy in frontier reasoning models.

SourcearXiv AIAuthor: Liyan Wang, Longze Li, Mohamed Hassan, Isshu Lee, Dajie Sun, Ruimin Feng, Aleksandar Vakanski
NEBench R1: a Reasoning Benchmark for Evaluating Nuclear Domain Competence in Large Language Models
Report an error

The correction channel is not available yet. You can copy the article reference below for later.

Correction instructions
Read article

[Submitted on 15 Nov 2026]

Title:NEBench R1: a Reasoning Benchmark for Evaluating Nuclear Domain Competence in Large Language Models

View a PDF of the paper titled NEBench R1: a Reasoning Benchmark for Evaluating Nuclear Domain Competence in Large Language Models, by Liyan Wang and 6 other authors

View PDF

Abstract:Large language models (LLMs) are increasingly being explored for knowledge-intensive and safety-critical applications in nuclear engineering. However, existing nuclear-domain benchmarks provide limited content coverage or focus primarily on answer accuracy without systematically evaluating the quality of the underlying reasoning. This study introduces NEBench R1, a reasoning-oriented benchmark comprising 7,546 unique multiple-choice questions extracted from U.S. Nuclear Regulatory Commission (NRC) reactor operator licensing examinations covering 18 nuclear facilities. Each question is classified according to the NRC Knowledge and Abilities catalogs and accompanied by an official explanation serving as a reference for reasoning evaluation. Six LLMs were evaluated, with overall accuracy ranging from 34.0% to 73.9% across three distinct performance tiers. To assess reasoning quality, an LLM-based judge evaluated correctly answered responses using a multidimensional rubric. Although responses generally demonstrated strong problem comprehension, they scored lower in factual and procedural accuracy and distractor analysis, indicating that correct answers were not necessarily supported by technically sound justifications. Response clarity was also weakly indicative of technical quality. Furthermore, longer reasoning traces were associated with lower accuracy in both frontier reasoning models, suggesting that reasoning length may reflect question difficulty rather than improved problem-solving performance. Reasoning-token consumption may therefore provide an additional signal for identifying potentially unreliable responses. NEBench R1 establishes a benchmark for evaluating both answer accuracy and reasoning quality, providing a foundation for assessing the domain competence and reliability of LLMs in nuclear engineering.

Subjects:

Artificial Intelligence (cs.AI)

Cite as: arXiv:2611.21682 [cs.AI]

(or arXiv:2611.21682v1 [cs.AI] for this version)

https://doi.org/10.48550/arXiv.2611.21682

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Liyan Wang Ms. [view email] [v1] Sun, 15 Nov 2026 19:02:18 UTC (545 KB)

Full-text links:

Access Paper:

View a PDF of the paper titled NEBench R1: a Reasoning Benchmark for Evaluating Nuclear Domain Competence in Large Language Models, by Liyan Wang and 6 other authors

View PDF

view license

Additional Features

Audio Summary

Current browse context:

cs.AI

new | recent | 2026-11

Change to browse by:

cs

References & Citations

SciX

Google Scholar

Semantic Scholar

Loading...

Data provided by:

Bibliographic Tools

Bibliographic and Citation Tools

Bibliographic Explorer Toggle

Bibliographic Explorer (What is the Explorer?)

Connected Papers Toggle

Connected Papers (What is Connected Papers?)

Litmaps Toggle

Litmaps (What is Litmaps?)

scite.ai Toggle

scite Smart Citations (What are Smart Citations?)

Code, Data, Media

Code, Data and Media Associated with this Article

alphaXiv Toggle

alphaXiv (What is alphaXiv?)

Links to Code Toggle

CatalyzeX Code Finder for Papers (What is CatalyzeX?)

DagsHub Toggle

DagsHub (What is DagsHub?)

GotitPub Toggle

Gotit.pub (What is GotitPub?)

Huggingface Toggle

Hugging Face (What is Huggingface?)

ScienceCast Toggle

ScienceCast (What is ScienceCast?)

Demos

Demos

Replicate Toggle

Replicate (What is Replicate?)

Spaces Toggle

Hugging Face Spaces (What is Spaces?)

Spaces Toggle

TXYZ.AI (What is TXYZ.AI?)

Related Papers

Recommenders and Search Tools

Link to Influence Flower

Influence Flower (What are Influence Flowers?)

Core recommender toggle

CORE Recommender (What is CORE?)

Author

Venue

Institution

Topic

About arXivLabs

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)

Key points and analysis

Article intelligence

EngineersAdvanced

Key points

  • NEBench R1 contains 7,546 unique multiple-choice questions from NRC licensing exams covering 18 nuclear facilities, classified by NRC Knowledge and Abilities catalogs with official explanations as reasoning references.
  • Six LLMs scored between 34.0% and 73.9% overall accuracy across three performance tiers.
  • An LLM-based judge found correct responses were strong on problem comprehension but weaker on factual/procedural accuracy and distractor analysis.
  • Longer reasoning traces were associated with lower accuracy in both frontier reasoning models, suggesting reasoning length reflects question difficulty rather than better problem-solving.

Highlights and analysis are generated automatically and may contain errors. Check the original source.

NEBench R1: a Reasoning Benchmark for Evaluating Nuclear Domain Competence in Large Language Models | AI News Hub