Stop Waiting for a Bigger Context Window

Engineering July 7, 2026

At INT21, we are all-in on self-improving multi-agent systems. We have built SwarmOS, our cloud-native platform for running specialized agents, and our first product, PTX Kernel Factory. The biggest change is not simply a larger context window. It is what frontier models make possible when they are orchestrated: turning one enormous context problem into a coordinated team of smaller, evidence-seeking tasks.

About a year ago, I was exploring how AI agents could generate CUTLASS C++ kernels, NVIDIA’s building blocks for high-performance GPU computation. By my count, the entire CUTLASS codebase represented roughly five million tokens. At the time, the best production model available to us offered a one-million-token context window.

The central blocker was never code generation. It was finding and preserving the right evidence across the repository.

Rather than wait for a magical five- or ten-million-token model, I ran the one-million-token model several times in parallel. Each agent studied a different portion of the codebase, and combined their findings as the final step.

It was a simple architecture, but it established the principle behind our work today:

When context stops fitting vertically, scale it horizontally.

A Bigger Window Is Not Better Context

Even when millions of tokens technically fit inside a model, the model must still separate signal from noise. A bigger window introduces more irrelevant information, more intermediate output, and more competition for attention.

Multi-agent systems address this structurally. Specialized agents explore different parts of a codebase, investigate the same question from independent angles, and return distilled findings to a coordinating agent. When a subproblem is still too large, it gets divided again.

The goal is not infinite context. It is effective context.

A General Solution for Complex Problems

At INT21, we use SwarmOS not only for hard engineering problems, such as expert-level PTX generation, but also to understand complex business landscapes.

To test this outside a codebase, we pointed SwarmOS at a different kind of long-context problem:

The research ran autonomously using public information. The system involved 27 agents, performed 166 web searches, visited more than 200 web pages across 73 unique domains, and ran for about two hours.

In total, it consumed 119 million tokens.

We are sharing the report in this article because we believe this is a topic many people will want to understand more deeply. But the report is also a demonstration of the broader point: multi-agent orchestration is the real long-context breakthrough.

Long Context Is Becoming a Systems Problem

So, are the latest AI generations solving long contexts?

Not by making context infinite, but in an agentic way.

It is helping solve long context by making it divisible, searchable, and composable.

That is why INT21 is all-in on multi-agent systems. At INT21, we are building Self-Improving Compute Infrastructure, and SwarmOS is the operating system behind a massive number of agents.

PTX Kernel Factory is now in beta for teams working on GPU kernel generation and AI compute infrastructure. Accepted participants receive limited-time free access and $100 in credits. Join the beta ->.

Huawei, CXMT and the China AI-stack pressure path

Evidence snapshot: 2026-07-06 UTC Research support only. Not investment, legal, procurement, operational, or tax advice.

Download full report + evidence packet (3.0MB, 103 files)

The research supports a plausible regional pressure scenario, not the sensational claim that China has already broken NVIDIA, HBM and neocloud scarcity economics worldwide.

Swarm stats

27 agents, 119M tokens

166 searches, 200+ public pages, 73 unique domains, and about two hours of elapsed runtime.

27agents in the system

166web searches conducted

200+web pages visited

73unique domains

~2happroximate runtime

119Mtokens consumed

Mechanism map

A pressure chain, not a completed displacement

The strongest mechanism is not a sudden global cancellation of Western GPU-cloud contracts. It is a chain that starts with China-local substitution and low token prices, then travels into renewal terms, new-build economics, financing spreads and private marks.

Huawei/Ascend surfaces appear real enough for selected China-local or Huawei-controlled workloads. CXMT is a real DRAM-scale actor, but public evidence does not prove scaled CXMT/mainland HBM inside shipped Ascend systems. China power helps domestic scale but is too small a cost line to be the standalone shock. Low Chinese model/API prices are the cleanest demand-side pressure vector.

Huawei systems

Ascend, Atlas, CANN, ModelArts and selected workload support reject a vaporware reading.

Memory bottleneck

HBM provenance and package throughput remain the decisive physical supply question.

China scale

Electricity and policy incentives enable domestic scale but do not alone reset global prices.

Token price anchor

Chinese and open-weight prices pressure buyer expectations and inference margins.

Financial transmission

New builds, renewals, private marks and project finance are more exposed than protected backlog.

What the report rejects: "Nothing to see here" and "China has already broken NVIDIA/HBM/neocloud economics."

What it supports: a bounded dislocation path that can matter before global physical substitution is proven.

Evidence quality

Large evidence base, with three decisive gaps

The conclusion is strongest where public facts come from official documents, filings, public pricing, regulation and observable software support. It is weakest where the decisive proof would require private contracts, physical teardown evidence, account-gated cloud quotes or live benchmark access.

Official docs

Strong for stated specs, regions, prices and service existence.

high

Filings

Strong for RPO, debt, capex, concentration and official risk disclosures.

high

Regulators

Strong for control regimes and official statistics, not transaction-specific advice.

high

Code/docs

Good for software existence and support matrix, weak for production TCO.

medium

Secondary reports

Useful where primary evidence is absent, downgraded for decisive HBM claims.

mixed

Threshold math

Useful for pressure bands and break-even logic, not a valuation or contract model.

bounded

HBMmemory provenance and stack throughput remain the most important uncertainty

TCObillable Ascend cloud economics require matched price, quota and benchmark proof

Fungibilitypolicy, data and procurement limits make the thesis regional-first

Contractsrenewals, new builds, private marks and financing carry the clearest financial channel

Calibrated findings

Where the thesis is strongest and weakest

Finding Supported Not supported Confidence

Regional-first pressure China domestic substitution, selected Ascend workloads, token price anchoring and valuation sensitivity. Immediate global physical AI-infrastructure black swan. Medium-high

Huawei/Ascend stack Commercially and technically real for selected workloads. Broad CUDA-equivalent global fungibility or public-proof lower TCO. Medium

CXMT/HBM bottleneck No public proof of scaled CXMT/mainland HBM in shipped Ascend systems. Confirmed domestic HBM bottleneck break. Medium-high

China electricity Domestic scale enabler and western-hub advantage. Cheap electricity alone breaking neocloud economics. High against power-only shock

Model/token prices Low official/public prices create a real price-anchor vector. Durable enterprise-grade global substitution. Medium-high for list prices

Neocloud channel Contract-cushioned near term, but sensitive in renewals, new builds, private marks and financing. Immediate broad reported-revenue collapse. Medium-high mechanism

Financial thresholds

Valuation pressure can arrive before revenue collapse

The model shows why the shock is about marginal economics. Protected backlog can cushion near-term revenue. New or exposed capacity can turn unattractive quickly when rental prices, utilization and financing move together.

New-capacity economic margin by scenario

Base renewal proxy

26%

Mild pressure

2%

Hard renewal pressure

-53%

Black-swan unprotected

-297%

Capex/down-cost offset

8%

Break-even rental price grid

Capex/debt

45% util

55% util

65% util

75% util

$42k, 10%

$4.48/hr

$3.66/hr

$3.10/hr

$2.69/hr

$35k, 7%

$3.66/hr

$2.99/hr

$2.53/hr

$2.20/hr

$55k, 12%

$5.85/hr

$4.78/hr

$4.05/hr

$3.51/hr

Ascend public TCO

Visible prices improve the case, but do not settle it

The L3 refresh found public ModelArts Snt9b1/2/3 list prices. That narrows one blocker. But Snt9b23, CloudMatrix and AICS price, quota, availability and matched independent benchmark evidence remain gated or absent.

Scenario Implied cost per 1M output tokens Interpretation

Snt9b3 at 4.578 per ModelArts PU-hour plus vLLM Llama8B offline~$1.26Competitive versus some Western public list prices if throughput is comparable.

Snt9b3 at 4.832 per ModelArts PU-hour plus vLLM Llama8B offline~$1.33Region price and currency-label caveats remain.

Same-throughput CoreWeave H100/H200~$1.69 / ~$1.73Public list-price baseline, not committed enterprise contract pricing.

Same-throughput Nebius H100/H200~$1.06 / ~$1.24Snt9b3 is not clearly cheaper than the lowest public Western baselines.

TCO conclusion: Ascend is a credible selected-workload China/APAC regional substitution and price anchor.

Still missing: invoice-grade Snt9b23/CloudMatrix/AICS prices, actual quota, real availability, SLA attachments and matched benchmarks.

Monitoring map

What would upgrade or weaken the thesis

Upgrade indicators

Huawei/CXMT disclosure of HBM output, yield, qualification or shipments.

Open or licensed teardowns showing CXMT/mainland HBM in Ascend packages.

Invoice-grade Snt9b23/CloudMatrix/AICS price, quota and SLA evidence.

Matched independent Ascend versus H100/H200/B200 benchmarks.

Filings or contract evidence showing repricing, de-booking, financing stress or private mark-downs tied to China-stack pressure.

Weakening indicators

More teardowns point to foreign or stockpiled HBM dependence.

Snt9b23/CloudMatrix prices stay gated, expensive, unavailable or weak on matched benchmarks.

Chinese model/API prices rise or fail cost-per-success tests.

Policy constraints tighten enough to block adoption outside China.

Neoclouds renew strongly and show no RPO, utilization, credit or financing deterioration.