Skip to content
AI News HubLIVE
Original source2 min read

FAVE: Foveated Adaptive Visual Encoding for Efficient Fine-Grained Visual Understanding

Summary

FAVE is a lightweight variable-resolution ViT inspired by human active vision that separates where to look from what to encode, encoding externally selected regions at high acuity while preserving native geometry and improving fine-grained understanding of small objects, text, and attributes at lower compute.

SourcearXiv Computer VisionAuthor: Amitangshu Mukherjee, Kaushik Roy
FAVE: Foveated Adaptive Visual Encoding for Efficient Fine-Grained Visual Understanding
Report an error

The correction channel is not available yet. You can copy the article reference below for later.

Correction instructions
Read article

[Submitted on 3 Sep 2026]

Title:FAVE: Foveated Adaptive Visual Encoding for Efficient Fine-Grained Visual Understanding

View a PDF of the paper titled FAVE: Foveated Adaptive Visual Encoding for Efficient Fine-Grained Visual Understanding, by Amitangshu Mukherjee and 1 other authors

View PDF HTML (experimental)

Abstract:Fine-grained visual understanding depends on local detail, yet visual encoders face a trade-off between costly full-image high-resolution processing and compact global encoding that can weaken such evidence. Inspired by human active vision, we separate where to look from what to encode. We focus on the latter and introduce FAVE (Foveated Adaptive Visual Encoding), a lightweight variable-resolution ViT that encodes externally selected regions at high acuity while preserving native geometry. We first isolate this encoding problem using oracle ground-truth crops in a controlled small-object regime. On ImageNet objects with a native maximum side of 96 pixels, FAVE improves Top-1 by 9.4 points over a fixed-resolution ViT on the same crop window with 12.7 times lower FLOPs. Increasing global resolution or backbone capacity does not recover the same operating point. We then integrate FAVE as a complementary local branch in FastVLM. Its local tokens are combined with FastVLM's global visual tokens, while the original global pathway and language model remain frozen. With at most 16 additional local tokens, FAVE improves TextVQA by 1.60 points and achieves a 3.3 times controlled TTFT speedup over SmolVLM2-2.2B. On GQA attribute questions, it improves FastVLM-1.5B by 1.31 points, extending the benefit beyond text while narrowing the gap to FastVLM-7B. Together, these results show that selectively allocating high-acuity local capacity provides an efficient complement to broader global representations and model scaling for fine-grained understanding of small objects, text, and attributes.

Comments: 21 pages, 6 figures, 7 tables; includes supplementary material

Subjects:

Computer Vision and Pattern Recognition (cs.CV)

Cite as: arXiv:2609.04392 [cs.CV]

(or arXiv:2609.04392v1 [cs.CV] for this version)

https://doi.org/10.48550/arXiv.2609.04392

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Amitangshu Mukherjee [view email] [v1] Thu, 3 Sep 2026 18:58:41 UTC (2,769 KB)

Full-text links:

Access Paper:

View a PDF of the paper titled FAVE: Foveated Adaptive Visual Encoding for Efficient Fine-Grained Visual Understanding, by Amitangshu Mukherjee and 1 other authors

View PDF

HTML (experimental)

TeX Source

view license

Current browse context:

cs.CV

new | recent | 2026-09

Change to browse by:

cs

References & Citations

NASA ADS

Google Scholar

Semantic Scholar

Loading...

Data provided by:

Bibliographic Tools

Bibliographic and Citation Tools

Bibliographic Explorer Toggle

Bibliographic Explorer (What is the Explorer?)

Connected Papers Toggle

Connected Papers (What is Connected Papers?)

Litmaps Toggle

Litmaps (What is Litmaps?)

scite.ai Toggle

scite Smart Citations (What are Smart Citations?)

Code, Data, Media

Code, Data and Media Associated with this Article

alphaXiv Toggle

alphaXiv (What is alphaXiv?)

Links to Code Toggle

CatalyzeX Code Finder for Papers (What is CatalyzeX?)

DagsHub Toggle

DagsHub (What is DagsHub?)

GotitPub Toggle

Gotit.pub (What is GotitPub?)

Huggingface Toggle

Hugging Face (What is Huggingface?)

ScienceCast Toggle

ScienceCast (What is ScienceCast?)

Demos

Demos

Replicate Toggle

Replicate (What is Replicate?)

Spaces Toggle

Hugging Face Spaces (What is Spaces?)

Spaces Toggle

TXYZ.AI (What is TXYZ.AI?)

Related Papers

Recommenders and Search Tools

Link to Influence Flower

Influence Flower (What are Influence Flowers?)

Core recommender toggle

CORE Recommender (What is CORE?)

Author

Venue

Institution

Topic

About arXivLabs

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)

Key points and analysis

Article intelligence

EngineersAdvanced

Key points

  • On ImageNet objects with a native maximum side of 96 pixels, FAVE improves Top-1 by 9.4 points over a fixed-resolution ViT on the same crop window with 12.7x lower FLOPs.
  • As a complementary local branch in FastVLM, at most 16 additional local tokens improve TextVQA by 1.60 points and enable a 3.3x controlled TTFT speedup over SmolVLM2-2.2B.
  • On GQA attribute questions, FAVE improves FastVLM-1.5B by 1.31 points, narrowing the gap to FastVLM-7B.
  • The results suggest selectively allocating high-acuity local capacity is an efficient complement to broader global representations and model scaling.

Highlights and analysis are generated automatically and may contain errors. Check the original source.