Skip to content
AI News HubLIVE
Original source2 min read

VITA-QinYu: Expressive Spoken Language Model for Role-Playing and Singing

Summary

VITA-QinYu is the first end-to-end expressive spoken language model that supports natural conversation, role-playing, and singing. It uses a hybrid speech-text paradigm with multi-codebook audio tokens to enhance paralinguistic representation while maintaining modality separation. Trained on 15.8K hours of synthetic data, it outperforms peer models on role-playing and singing benchmarks and achieves state-of-the-art conversational accuracy and fluency. Code and models are open-sourced.

SourcearXiv Computational LinguisticsAuthor: Jiacheng Xu, Heting Gao, Liufei Xie, Zhenchuan Yang, Lijiang Li, Yiting Chen, Bin Zhang, Meng Chen, Chaoyu Fu, Weifeng Zhao, Wenjiang Zhou
VITA-QinYu: Expressive Spoken Language Model for Role-Playing and Singing
Report an error

The correction channel is not available yet. You can copy the article reference below for later.

Correction instructions
Read article

[2605.06765] VITA-QinYu: Expressive Spoken Language Model for Role-Playing and Singing

[Submitted on 7 May 2026]

Title:VITA-QinYu: Expressive Spoken Language Model for Role-Playing and Singing

View a PDF of the paper titled VITA-QinYu: Expressive Spoken Language Model for Role-Playing and Singing, by Jiacheng Xu and 10 other authors

View PDF HTML (experimental)

Abstract:Human speech conveys expressiveness beyond linguistic content, including personality, mood, or performance elements, such as a comforting tone or humming a song, which we formalize as role-playing and singing. We present VITA-QinYu, the first expressive end-to-end (E2E) spoken language model (SLM) that goes beyond natural conversation to support both role-playing and singing generation. VITA-QinYu adopts a hybrid speech-text paradigm that extends interleaved text-audio modeling with multi-codebook audio tokens, a design enabling richer paralinguistic representation while preserving a clear separation between modalities to avoid interference. We further develop a comprehensive data generation pipeline to synthesize a total of 15.8K hours of natural conversation, role-playing, and singing data for training. VITA-QinYu demonstrates superior expressiveness, outperforming peer SLMs by 7 percentage points on objective role-playing benchmarks, and surpassing peer models by 0.13 points on a 5-point MOS scale for singing. Simultaneously, it achieves state-of-the-art conversational accuracy and fluency, exceeding prior SLMs by 1.38 and 4.98 percentage points on the C3 and URO benchmarks, respectively. We open-source our code and models and provide an easy-to-use demo with full-stack support for streaming and full-duplex interaction.

Comments: this https URL

Subjects:

Computation and Language (cs.CL); Artificial Intelligence (cs.AI)

Cite as: arXiv:2605.06765 [cs.CL]

(or arXiv:2605.06765v1 [cs.CL] for this version)

https://doi.org/10.48550/arXiv.2605.06765

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Chaoyou Fu [view email] [v1] Thu, 7 May 2026 17:59:56 UTC (2,203 KB)

Full-text links:

Access Paper:

View a PDF of the paper titled VITA-QinYu: Expressive Spoken Language Model for Role-Playing and Singing, by Jiacheng Xu and 10 other authors

View PDF

HTML (experimental)

TeX Source

view license

Current browse context:

cs.CL

new | recent | 2026-05

Change to browse by:

cs cs.AI

References & Citations

NASA ADS

Google Scholar

Semantic Scholar

Loading...

Data provided by:

Bibliographic Tools

Bibliographic and Citation Tools

Bibliographic Explorer Toggle

Bibliographic Explorer (What is the Explorer?)

Connected Papers Toggle

Connected Papers (What is Connected Papers?)

Litmaps Toggle

Litmaps (What is Litmaps?)

scite.ai Toggle

scite Smart Citations (What are Smart Citations?)

Code, Data, Media

Code, Data and Media Associated with this Article

alphaXiv Toggle

alphaXiv (What is alphaXiv?)

Links to Code Toggle

CatalyzeX Code Finder for Papers (What is CatalyzeX?)

DagsHub Toggle

DagsHub (What is DagsHub?)

GotitPub Toggle

Gotit.pub (What is GotitPub?)

Huggingface Toggle

Hugging Face (What is Huggingface?)

ScienceCast Toggle

ScienceCast (What is ScienceCast?)

Demos

Demos

Replicate Toggle

Replicate (What is Replicate?)

Spaces Toggle

Hugging Face Spaces (What is Spaces?)

Spaces Toggle

TXYZ.AI (What is TXYZ.AI?)

Related Papers

Recommenders and Search Tools

Link to Influence Flower

Influence Flower (What are Influence Flowers?)

Core recommender toggle

CORE Recommender (What is CORE?)

Author

Venue

Institution

Topic

About arXivLabs

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)

Key points and analysis

Article intelligence

EngineersAdvanced

Key points

  • VITA-QinYu is the first end-to-end expressive SLM supporting role-playing and singing.
  • It employs a hybrid speech-text architecture with multi-codebook audio tokens for rich paralinguistics.
  • Trained on 15.8K hours of synthetic data, it surpasses existing models on role-playing and singing metrics.
  • It achieves state-of-the-art on conversational benchmarks and open-sources code and models.

Highlights and analysis are generated automatically and may contain errors. Check the original source.

More recent stories on this topic · Models