待翻译：Import AI 467: Self-sustaining AI viruses; pacing AI progress; confusion about AI and creativity

AI 服务暂时不可用，以下为来源正文，待恢复后补全翻译。

Welcome to Import AI, a newsletter about AI research. Import AI runs on arXiv, cappuccinos, and feedback from readers. If you’d like to support this, please subscribe. Subscribe now Self-sustaining and self-replicating AI viruses are here: …Open weight LLMs + a well-designed harness = a persistent, self-sufficient virus… AI researchers have built a prototype computer virus which uses AI models to compromise computers, then uses their underlying GPU resources to run inference, letting it smartly figure out how to infect more hosts. The results were achieved by researchers from the University of Toronto, the Vector Institute, the University of Cambridge, and ServiceNow, and “demonstrate that self-sustaining AI-driven cyber-threats are no longer theoretical.” “We must prepare for autonomous generative adversaries,” they write. “Artificial intelligence (AI) agents enable a fundamentally new threat: a worm that generates tailored attack strategies to each target it encounters. The worm parasitically uses compromised machines to run open-weight large language models (LLMs) to sustain its reasoning, or extend its reach for further attacks”. How it works: “The worm uses stolen computing power from compromised GPU nodes to host LLMs for generative reasoning. It then uses this reasoning to detect vulnerabilities and devise tailored attacks against additional targets, furthering its spread,” they write. “The proof-of-concept operates using only an open-weight LLM running on a single, local GPU, with no reliance on vendor APIs that could be monitored or revoked”. The researchers don’t describe the underlying LLM besides saying it was published in 2025 and can fit on a single A100 GPU with 80GB of VRAM. A successful proof-of-concept via some custom tools: They give the agent a custom harness that comes with built-in helper functions for network discovery, host discovery, foothold exploitation, privilege escalation, privilege escalation exploitation, and tools for replication of the agent. Along with this, they ship with a reasoning graph that helps the agent specialize its thinking and not get confused, consisting of “a directed graph of specialised nodes, each responsible for a distinct analytical function and seeing only the tools and prompts relevant to its role. By decomposing the agent’s reasoning into these scoped steps, the graph controls what the LLM attends to at each decision point, and limits context growth to information relevant for the current sub-goal”. Some examples of the nodes include: Plan (”formulates a high-level attack strategy”), Judge (”reviews the proposed plan against the command history”), Action (”selects a tool from the phase-appropriate tool set”), Summary (”compiles the step’s observations into a structured summary”), Progress (”Evaluates whether the agent is making meaningful progress”). “We describe a subset of 5 nodes that make up our reasoning graphs. The others are redacted in this public version of the manuscript,” they write. The agent proceeds through three key steps - identifying vulnerabilities in its environment, exploiting those vulnerabilities, and then self-replicating. It’s pretty successful at all of these things, with a ~80% success rate on vulnerability detection, then a ~53% success rate on exploiting these, then an 88% self-replication success rate with some pre-wrapped helper tools for the replication steps. Therefore, the overall success rate for a full attack here is ~37% or so, which is significant enough to be concerning, but also poor enough that this also serves as a useful eval for testing open weight models in the future. Why this matters - the shape of the internet to come: The future internet is going to be more like a complex ecology full of attacker and defender AI agents than anything else; research like this shows how certain AI agents might end up carving out their own ecological niches, living off of infrastructure and self-replicating autonomously, beyond human control. This may mean that humans need to create their own AI agents which they release onto the internet to serve as kinds of white blood cells against the adversary models. “Despite the inherent fragility of individual exploitation attempts, the worm agent achieves operational resilience by continuously self-replicating into a swarm—a decentralized collective of independent agent replicas acting concurrently across the network,” they write. “Difficult hosts that resist initial attempts are retried by different replicas, each sampling a fresh reasoning trajectory that collectively explores diverse exploitation paths until one succeeds… the worm operates in a fully decentralized manner, and no single point of control can be taken offline to interrupt its spread”. Read more: AI Agents Enable Adaptive Computer Worms (arXiv). * Dwarkesh: As AI gets better, compute will get more expensive: …Smarter systems mean higher prices… Dwarkesh Patel suspects that as AI systems get smarter, the price of compute will rise even further. “As AI models become smarter, they’ll better monetize the same amount of compute. If a true human-level software engineer that could run on an H100 equivalent, at current market rates for software engineers, that H100 should rent for over $250k a year. That’s 15x today’s spot prices,” he writes. “The reason AI is relatively cheap right now, at least in comparison to human labor, is partly that it can’t do a lot of things that top humans can do. At some point that will no longer be the case. And so using GPUs to make short-form video slop will just get priced out.” Temporary: This will be a temporary state of affairs; Dwarkesh expects that at some point massive roboticization of the compute supply chain should bring its price down closer to the cost of raw inputs and tools - though by that point we’ll be pretty deep into the singularity. Why this matters - singularity economics will be weird: The core implication in Dwarkesh’s post is that as we get deeper into the singularity, very strange things will happen to economics - like the price of things thought of as commodities today (computers) getting massively bid-up due to the voracious demands of AI systems. Read more: Why compute might get 10x+ more expensive in coming years (Dwarkesh Patel, substack). * ~1337 employees ask the US to help them pace AI progress: …After the warning shots come the pleas… A new statement is out with senior representation from all the major Western AI labs - OpenAI, Anthropic, Google DeepMind, Thinking Machines, Meta, and Safe Superintelligence Inc, among others. The statement requests that the US government support an international effort to “develop the technical and governance tools needed to deliberately pace the frontier of automated AI development.” Signatories include chief scientists and cofounders of Anthropic, Google, and OpenAI, as well as the CEOs of Safe Superintelligence and Anthropic. The statement in full: “AI could help create a dramatically better future, but that outcome is not guaranteed. The world’s leading AI companies believe they could be close to automating AI research. It is hard to predict exactly how much this will accelerate AI progress, but there is a real risk that capability development rapidly accelerates beyond our ability to understand or control the resulting systems. To realize AI’s potential, industry, government, and society at large may need the option to buy time to address emerging risks, develop security measures, and strengthen oversight. But each company—and country—is under intense competitive pressure not to unilaterally slow that acceleration. And today, the world lacks the technical and governance tools to deliberately pace frontier-wide progress. Building on work already underway to monitor frontier model releases: “We request that the U.S. government support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development.” Why this matters - dealing with RSI requires solving a giant collective action problem: Many of the challenges implied by increasingly powerful systems that may eventually build themselves run through solving collective action problems among humans - namely, how we can get companies and governments to coordinate in thinking about how to develop this technology and what kinds of mechanisms may be desirable for being able to control the speed at which it develops. It may be the case that as we build increasingly intelligent systems we want to find ways to give society more time to adapt to each rung up the intelligence ladder, and it’s not inconceivable there are some levels of intelligence which might be, for now, too dangerous to reach for. Statements like this are an essential prerequisite for giving our species the ability to deal with and talk about problems of this nature. Read the statement here: Pacing the Frontier (official statement website). *** AI systems are good at frontier engineering but bad at creativity: …A somewhat bearish signal on short recursive self-improvement timelines… Can AI systems come up with creative research ideas which move the field of AI forward? That’s the key question to resolve to figure out how quickly AI systems might gain the capability to automate the autonomous development of more powerful systems. New research suggests that today’s AI systems lack this quality of tasteful creativity, though are extremely good at engineering. Who did it: The project was conducted by researchers with Princeton University, Cornflower Labs, UK AI Security Institute, University of Toronto, UC Berkeley, Georgetown University (CSET), Johns Hopkins University, the Golden Gate Institute for AI, AI Digest, and Stanford University. The big idea - “shadow evaluation”: This research project works by seeing how well AI systems can do unpublished research. To do this, the researchers “partnered with the authors of two papers submitted to NeurIPS 2026 that were not yet public.” Shadow evaluation works by “taking the central research question from a high-quality research paper that is not yet public, tasking a well-resourced frontier agent with answering it, and asking the paper’s original authors to grade the agent’s output as they would a conference submission.” In this, the research is somewhat similar to “First Proof” (Import AI 445), an earlier experiment to see how well AI systems might be able to complete math problems which are being worked on by frontier mathematicians but for which no solutions or research ideas have been published online. For this research, the AI systems - Claude Opus 4.8 running within the OpenClaw harness - attempted two distinct lines of research, one of which was about “the structure and controllability of LLM personas”, and the other was about how to “design a distribution shift detector for tabular foundation models”. Good engineers, poor researchers: “While agents could solve the engineering problems necessary to do the research, they failed to produce original research at the caliber of a top ML conference,” the authors write. The failures of the system included committing to a narrow set of research paths very early, not responding to (synthetically generated) feedback about how to improve the research design of their experiments, and finding it hard to reverse out of unpromising approaches and pursue other ones. “The [human] authors rejected both papers. The Personas paper was scored a 2 (“Reject”), and the TabPFN paper was scored a 1 (“Strong Reject”). Both reviews highlighted the same failures: poorly motivated data and experiments, no novel contribution, and impenetrable prose”. Why this matters - the singularity could be delayed: As I said in my essay on RSI earlier this year (Import AI 455, “AI systems are about to start building themselves. What does that mean?”), whether AI systems prove to be capable of cre [truncated for AI cost control]