Skip to content
AI News HubLIVE
Original source2 min read

Pixels to Tokens: Token Space Efficient Active Learning for Low-Budget Semantic Segmentation

Summary

TEAL is the first pixel-level active learning framework built on ViT representations, performing diversity-based selection directly on the native token lattice and refining with margin-based uncertainty on a finer decoder grid. It improves up to +24.36 mIoU over the strongest baseline with one labeled pixel per image and reaches 97% of fully supervised performance with just 10 pixels.

Pixels to Tokens: Token Space Efficient Active Learning for Low-Budget Semantic Segmentation
Report an error

The correction channel is not available yet. You can copy the article reference below for later.

Correction instructions
Read article

content type paperpublished October 2026

Pixels to Tokens: Token Space Efficient Active Learning for Low-Budget Semantic Segmentation

AuthorsUtku Cicek†, Mahip Singh†, Ziyao Shang†, Kimathi Kaai, Pedro Pablo Guerrero Vela, C Thomas, Alexander Wong, Sirisha Rambhatla†

View publication

While foundation models and vision transformers (ViTs) have redefined visual representations, current active learning (AL) strategies for these backbones largely operate at the patch or region level, where annotation requirements remain high due to extensive pixel labeling. Extending ViT-based AL to the pixel-level, low-budget regime is uniquely challenging: with as little as one pixel query per image, a method must simultaneously identify the most informative locations and train a reliable decoder head from an extremely sparse signal. We introduce TEAL (Token-space Efficient Active Learning), the first pixel-level AL framework built around ViT representations. TEAL uses a frozen ViT backbone and performs diversity-based selection directly on its native token lattice, avoiding the artifacts introduced by interpolating token embeddings to dense pixel resolutions. Our framework applies a two-level MaxHerding strategy over multi-layer token descriptors to select representative candidates, followed by margin-based uncertainty refinement on a finer decoder grid. Across CamVid, Cityscapes, ADE20K, and Pascal Context, TEAL consistently outperforms previous baselines under extreme label scarcity, improving over the strongest baseline by up to +24.36mIoU with a single labeled pixel (px) per image, and reaching up to 97% of fully supervised performance with just 10 px. To rigorously establish the source of the gains, we further show that the improvements persist when all baselines are equipped with the same DINOv3 backbone, demonstrating that coupling token-space selection with uncertainty refinement on the decoder grid makes ViT token geometry an effective foundation for extremely low-budget active segmentation.

† University of Waterloo

VideoFlexTok: Flexible-Length Coarse-to-Fine Video Tokenization

July 2, 2026research area Computer Visionconference ICML

Visual tokenizers map high-dimensional raw pixels into a compressed representation for downstream modeling. Beyond compression, tokenizers dictate what information is preserved and how it is organized. A de facto standard approach to video tokenization is to represent a video as a spatiotemporal 3D grid of tokens, each capturing the corresponding local information in the original signal. This requires the downstream model that consumes the…

Read more

Training a Tokenizer for Free with Private Federated Learning

May 23, 2022research area Privacy, research area Speech and Natural Language Processingconference ACL

Federated learning with differential privacy, i.e. private federated learning (PFL), makes it possible to train models on private data distributed across users’ devices without harming privacy. PFL is efficient for models, such as neural networks, that have a fixed number of parameters, and thus a fixed-dimensional gradient vector. Such models include neural-net language models, but not tokenizers, the topic of this work. Training a tokenizer…

Read more

Key points and analysis

Article intelligence

EngineersAdvanced

Key points

  • TEAL operates directly on a frozen ViT's native token lattice, avoiding artifacts from interpolating token embeddings to dense pixel resolutions.
  • A two-level MaxHerding strategy selects representative candidates over multi-layer token descriptors, followed by margin-based uncertainty refinement on a finer decoder grid.
  • On CamVid, Cityscapes, ADE20K, and Pascal Context, TEAL improves up to +24.36 mIoU with one labeled pixel per image and reaches 97% of fully supervised performance with 10 pixels.
  • Gains persist when all baselines use the same DINOv3 backbone, demonstrating that coupling token-space selection with decoder-grid uncertainty refinement is key.

Highlights and analysis are generated automatically and may contain errors. Check the original source.