2026-04-30 00:00 UTCOriginal source2 min readUpdated: 2026-06-27 00:25 UTC

Bootstrapping Sign Language Annotations with Sign Language Models

Researchers at Apple and Gallaudet University developed a pseudo-annotation pipeline to address the scarcity of high-quality annotated sign language data. Their approach uses a fingerspelling recognizer, isolated sign recognizer, and a K-Shot LLM to generate likely annotations from signed video and English input. They achieved state-of-the-art results on FSBoard (6.7% CER) and ASL Citizen (74% top-1 accuracy) and are releasing nearly 500 human-annotated videos and over 300 hours of pseudo-annotations.

SourceApple Machine Learning Research

Article intelligence

EngineersAdvanced

Key points

Lack of annotated data limits AI sign language interpretation; new datasets like ASL STEM Wiki and FLEURS-ASL have hundreds of hours but are underutilized due to annotation costs.
The pipeline uses a fingerspelling recognizer, isolated sign recognizer (ISR), and K-Shot LLM to produce ranked annotations with time intervals.
State-of-the-art performance: 6.7% CER on FSBoard and 74% top-1 accuracy on ASL Citizen.
Professional interpreter annotated nearly 500 videos; over 300 hours of pseudo-annotations and human labels are being released.

Why it matters

This matters because lack of annotated data limits AI sign language interpretation; new datasets like ASL STEM Wiki and FLEURS-ASL have hundreds of hours but are underutilized due to annotation costs.

Technical impact

May affect model selection, inference cost, product capability, and evaluation benchmarks.

This panel is AI-generated and reviewed for accuracy.

Bootstrapping Sign Language Annotations with Sign Language Models - Apple Machine Learning Research

Machine Learning Research

Open MenuClose Menu

Overview

Research Highlights

Publications

Events

Work with us

research area Accessibility, research area Computer Visionconference CVPR

content type paperpublished April 2026

Bootstrapping Sign Language Annotations with Sign Language Models

AuthorsColin Lea, Vasileios Baltatzis, Connor Gillis, Raja Kushalnagar†, Lorna Quandt†, Leah Findlater

View publication

Copy Bibtex

AI-driven sign language interpretation is limited by a lack of high-quality annotated data. New datasets including ASL STEM Wiki and FLEURS-ASL contain professional interpreters and 100s of hours of data but remain only partially annotated and thus underutilized, in part due to the prohibitive costs of annotating at this scale. In this work, we develop a pseudo-annotation pipeline that takes signed video and English as input and outputs a ranked set of likely annotations, including time intervals, for glosses, fingerspelled words, and sign classifiers. Our pipeline uses sparse predictions from our fingerspelling recognizer and isolated sign recognizer (ISR), along with a K-Shot LLM approach, to estimate these annotations. In service of this pipeline, we establish simple yet effective baseline fingerspelling and ISR models, achieving state-of-the-art on FSBoard (6.7% CER) and on ASL Citizen datasets (74% top-1 accuracy). To validate and provide a gold-standard benchmark, a professional interpreter annotated nearly 500 videos from ASL STEM Wiki with sequence-level gloss labels containing glosses, classifiers, and fingerspelling signs. These human annotations and over 300 hours of pseudo-annotations are being released in supplemental material.

† Gallaudet University

** Work done while at Apple