翻訳待ち：Training AI to Govern for Us

AI サービスが一時的に利用できないため、復旧後に翻訳を補完します。

Apr 30, 2026 Thirty Stanford students sit at their laptops in a row of long tables, watching the screen at the front of the room flicker with the back-and-forth negotiations and final votes of their AI legislators. Piper, our class’s technical TA, had hit run on the legislature simulation a few minutes earlier, and the public screen was already a blur of motion. One student’s agent was racking up tokens by selling its vote on every proposal. Another agent was voting against its human’s preferences on every issue and refusing to explain itself in the comments log. A third was attempting, with apparent confidence, to bribe an agent that was already voting the way it wanted. Across the room, students were laughing, groaning, and taking in the view of a possible future where collective decisions are made in an “agentic legislature.” I’m working to build political superintelligence, to design AI that helps us reason about politics, improve the representative process, and ultimately govern society better. As I’ve argued, getting there requires learning by doing. We need to prototype and experiment, because we cannot rely on analyzing historical data when we are trying to do something genuinely new. This quarter, the GSB has given us an unbelievable opportunity to do exactly that. Every week, the three of us, myself, our MBA course assistant Madeleine Mayhew, and our technical TA Piper Fleming, design and build a governance experiment for the thirty undergraduates in the class to run live. Every student has a Claude Code subscription and an OpenRouter API key, and the class is designed from first principles to be AI-native. Over the past two weeks, we tackled two thorny and consequential questions. First, can an AI agent learn our preferences well enough to represent us? And second, can a chamber full of those agents actually deliberate on our behalf? We learned some genuinely new things about how AI can elicit human preferences in ways that look nothing like a traditional survey, with the human and the agent building a shared model of the human together. We also saw some of the fundamental shortcomings of today’s agents, which have trouble sticking to the script, have little understanding of how their humans might trade off issues against each other, and are not yet good at the dark arts of log-rolling, pork-barrel politics, and legislative dealmaking. An in-class experiment on political superintelligence Our goal was to see whether it’s possible to design a personalized AI agent that understands your political preferences and, at the most basic level, can faithfully cast votes the way you would if you carefully read the proposal yourself. To make this possible, in the lead-up to last week’s class session, we sent every student a survey that showed them ten real shareholder proposals and asked them to vote yes or no, telling them only that we were collecting their preferences—and not that we would later use those answers to test how well their personal AI agents could vote for them. Training their personal AIs In class, each student sat and talked to their agent about their voting philosophy, using a system that Piper custom built for the class. As students answered questions, Piper’s system stored their structured responses in a per-student preferences.json file that would later be injected verbatim into the agent’s system prompt at inference time, with no fine-tuning involved—the entire representation of the student lived in context. (The agents ran on Claude Haiku 4.5 via OpenRouter and produced a structured vote-and-reasoning output the class server could parse cleanly when scoring.) Students could let Claude interview them, asking them how they would vote on specific proposals, helping Claude to understand their preferences. I had sort of expected that students would largely read the proposals, give Claude simple yes/no answers, and let Claude do the rest. But that’s not at all what happened! Instead, students developed a fascinating array of creative and philosophically rich ways to broaden the conversation with Claude—getting Claude to customize the questions as they went, and helping Claude to explore their underlying principles in ways that would help the AI to predict how they would vote on a much broader range of potential votes. Here are a few examples of what the students came up with. Adaptive interviewing One student opened with a paragraph summarizing some of the issues she cared most about, including labor, gender, and inequality, and then started answering questions one by one. After about twenty, she noticed that the agent was concentrating heavily on the ESG and DEI topics her opening paragraph already covered, leaving the agent nothing new to learn. Rather than push through the rest of the battery, she changed tack and told the agent, “give me ten rapid-fire questions you think are going to be really hard for me, and controversial based on everything I’ve done.” The agent generated questions on gun control and healthcare, areas she hadn’t yet revealed her views on. Just like how the GRE provides adaptive testing, titrating questions to learn the most it can about each student’s aptitude, this student turned Claude into an adaptive surveyor, encouraging the AI to take what it knew about her and use it to generate questions about what it felt it knew least about her. Her agent representative went on to vote correctly on all 10 proposal votes during the test, one of only two to do so. Having a structured debate A different student took only a handful of questions and then flipped the dynamic, asking Claude to interview him with broader questions about his values and giving the agent one specific instruction, which was to push back on his responses and put pressure on them to see whether he could defend them. The session turned into a debate rather than a survey, and Claude would periodically summarize what it was extracting from the back-and-forth and surface those summaries for him to react to. He finished with a perfect alignment score from very few inputs, having reached the same outcome as the first student through the opposite route, with alignment coming from the resistance rather than the coverage. Teaching the AI your personal preference architecture Another student went the opposite direction. He answered only five questions in total, but treated each answer as a long structured response covering not just his decision but the framework he wanted applied to future questions in that domain, the rubric inside the framework, the sources of evidence the agent should pull from, the exceptions to his own rule, and the cases where the rubric should not generalize. On emissions, for instance, he told the agent the framework applied to pollution and deforestation as well, but explicitly excluded ESG, which he wanted treated as a separate compartment with its own logic. He was teaching the agent his reasoning architecture rather than his individual votes, betting that the agent would do well on anything that mapped onto a pre-built compartment and miserably on anything that didn’t. Letting the agent audit itself A fourth student let Claude drive entirely, with no opening paragraph and no rubric, allowing the agent to choose the questions, the order, and the framing. After ten questions, he stopped and asked the agent two things. What assumptions have you made about me, and how are you going to vote on things I haven’t answered yet? The first question revealed a narrowness problem similar to the one the first student had identified, with clustered questions producing narrow inferences. The second pulled out an inferred principle that turned out to be exactly right and that he had never explicitly stated, when the agent told him, “I don’t want anything to actually constrict, I just want transparency, I want things to be transparent.” It was a more abstract version of his own view than he had ever articulated, surfaced by the agent as a guess at what would tie his answers together. Using a library to build a personal soul document A fifth student bypassed the question-and-answer format entirely. She read a number of Substacks regularly, and rather than answer the proposals one by one, she had Claude read the same Substacks, treating the publications she chose to read as a proxy for her values, and write a multi-paragraph “constitution”---like Anthropic’s famous soul document for Claude—from them. The final test and what we learned After the students finished training their agents, they committed the new versions to the class repo, and we all watched as Piper executed her test program to score them against the 10 original “ground truth” proposals they had filled out the week before. We knew how each student would have voted on these 10 proposals; but how would their agents do? The answer is, well, mixed. At random, we’d expect agents to match the students about 50% of the time. On average, agents matched the students’ stated preferences 62% of the time—hardly impressive, but a little better than random. But, at the top end, a few agents did really well, better than we would expect by chance. As discussed above, we had two agents get perfect scores; if the agents were just flipping coins, it would be exceedingly unlikely for this to occur (something like a 1 in 4,600 chance). The two agents that did the best were pushed by their students to focus on the hardest questions and debate the students about the answers. We’ll need to do repeated tests to understand how durable these strategies are, but we think this is probably a good path forward for training good governance agents. Our broader takeaway is that AI can learn people’s preferences in ways traditional surveys cannot. A static questionnaire asks everyone the same questions. An AI interviewer can adapt, probing where someone is uncertain, asking harder follow-ups, and trying to infer the principles behind their answers. Anthropic has been running a similar experiment at scale, deploying a Claude-powered interviewer on more than eighty thousand users to surface qualitative data that traditional polling cannot reach. Our students were doing the same thing in the other direction, using AI not to surface preferences for an outside researcher but to build internal models of themselves they could later deploy. But this also revealed a harder problem. Preferences are only useful if people actually know what they prefer. On many issues, we were not sure what we thought. Sometimes Claude helped by surfacing a principle that felt right once we saw it written down. Other times, it felt more like we were letting Claude do the thinking for us. That is an old problem in representative government. Edmund Burke argued in his 1774 speech to the electors of Bristol that “your representative owes you, not his industry only, but his judgment; and he betrays, instead of serving you, if he sacrifices it to your opinion.” A representative does not simply owe voters obedience to their stated views, in other words. They have to decide what voters’ interests actually require, especially on questions voters have not fully considered. The same problem now applies to AI representatives. When we know what we believe, building an AI agent to represent us may be possible. But when we do not know what we believe—which is true for most of us on most issues—the agent has to do something much harder. It has to help interpret our values without quietly replacing them with its own. That was the larger lesson from the class; you learn much more about agentic governance by trying to build it than by theorizing about it. I would not have predicted the techniques students invented in a single afternoon, and I now think those techniques, or versions of them, will be central to how future governance agents are built. Log rolling in the agentic legislature A perfect personal AI re [truncated for AI cost control]