2026-05-26原文

Brain-to-Image Retrieval and Reconstruction via Multimodal EEG Alignment

Researchers propose a brain-to-image system that decodes visual stimuli from EEG signals recorded during natural image viewing. It handles EEG-to-image retrieval (86.30% Top-1 accuracy among 200 candidates) and EEG-to-image reconstruction (CLIP score 0.903). The method uses multi-level blurring, EVNet features, InfoNCE loss, and multi-modal CLIP alignment with SDXL-Turbo generation, demonstrating feasibility of decoding rich visual representations from EEG.

Article intelligence

EngineersAdvanced

Key points

EEG-to-image retrieval achieves 86.30% Top-1 and 98.55% Top-5 accuracy over 200 candidate images.
EEG-to-image reconstruction uses CognitionCapturerPro with multi-modal CLIP embeddings and SDXL-Turbo, achieving CLIP score 0.903.
Study demonstrates that modern multi-modal alignment and generative modeling can decode visual information from EEG signals.

Why it matters

This matters because EEG-to-image retrieval achieves 86.30% Top-1 and 98.55% Top-5 accuracy over 200 candidate images.

Technical impact

May affect model selection, inference cost, product capability, and evaluation benchmarks.

[2605.23996] Brain-to-Image Retrieval and Reconstruction via Multimodal EEG Alignment

[Submitted on 18 May 2026]

Title:Brain-to-Image Retrieval and Reconstruction via Multimodal EEG Alignment

View a PDF of the paper titled Brain-to-Image Retrieval and Reconstruction via Multimodal EEG Alignment, by Chi Kit Wong and 2 other authors

View PDF HTML (experimental)

Abstract:We present a brain-to-image system that decodes visual stimuli from EEG signals recorded during natural image viewing. Our system addresses two tasks: (1) EEG-to-image retrieval, which ranks the correct stimulus image among 200 candidates given an EEG segment, and (2) EEG-to-image reconstruction, which generates an image consistent with the perceived stimulus. For retrieval, we implement a multi-level blurring approach improved with biologically inspired EVNet features and trained with the InfoNCE loss. Evaluated over 10 random seeds for a single subject, the retrieval model achieves a mean final-epoch Top-1 accuracy of 86.30% and Top-5 accuracy of 98.55%. For reconstruction, we implement CognitionCapturerPro, which aligns EEG representations to multi-modal CLIP embeddings, including image, text, depth, and edge embeddings, and synthesizes images with SDXL-Turbo conditioned via IP-Adapter. Averaged over 10 seeds, the reconstruction model achieves a CLIP score of 0.903 using ViT-H-14, a CLIP score of 0.870 using ViT-L/14, and an SSIM of 0.409. These results demonstrate the feasibility of decoding rich visual representations from EEG signals using modern multi-modal alignment and generative modeling techniques.

Comments: 16 pages, 5 figures. Code available at: this https URL

Subjects:

Computer Vision and Pattern Recognition (cs.CV); Image and Video Processing (eess.IV)

Cite as: arXiv:2605.23996 [cs.CV]

(or arXiv:2605.23996v1 [cs.CV] for this version)

https://doi.org/10.48550/arXiv.2605.23996

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Chi Kit Wong [view email] [v1] Mon, 18 May 2026 05:33:44 UTC (4,650 KB)

Full-text links:

Access Paper:

View a PDF of the paper titled Brain-to-Image Retrieval and Reconstruction via Multimodal EEG Alignment, by Chi Kit Wong and 2 other authors

View PDF

HTML (experimental)

TeX Source

view license

Current browse context:

cs.CV

new | recent | 2026-05

Change to browse by:

cs eess eess.IV

References & Citations

NASA ADS

Google Scholar

Semantic Scholar

Data provided by:

Bibliographic Tools

Bibliographic and Citation Tools

Bibliographic Explorer Toggle

Bibliographic Explorer (What is the Explorer?)

Connected Papers Toggle

Connected Papers (What is Connected Papers?)

Litmaps Toggle

Litmaps (What is Litmaps?)

scite.ai Toggle

scite Smart Citations (What are Smart Citations?)

Code, Data, Media

Code, Data and Media Associated with this Article

alphaXiv Toggle

alphaXiv (What is alphaXiv?)

Links to Code Toggle

CatalyzeX Code Finder for Papers (What is CatalyzeX?)

DagsHub Toggle

DagsHub (What is DagsHub?)

GotitPub Toggle

Gotit.pub (What is GotitPub?)

Huggingface Toggle

Hugging Face (What is Huggingface?)

ScienceCast Toggle

ScienceCast (What is ScienceCast?)

Demos

Replicate Toggle

Replicate (What is Replicate?)

Spaces Toggle

Hugging Face Spaces (What is Spaces?)

Spaces Toggle

TXYZ.AI (What is TXYZ.AI?)