Skip to content
AI News HubLIVE
Original source2 min read

CamSpokenNLI: A Pilot Resource for Cross-Variety Transfer Evaluation in Cameroon Pidgin English and Camfranglais

Summary

The paper introduces CamSpokenNLI, a pilot spoken NLI dataset of 114 scenario-matched items in each of Cameroon Pidgin English and Camfranglais, recorded by one native speaker from Buea. XLM-RoBERTa baselines reach 44.7% and 46.2% on reference text but are statistically indistinguishable from hypothesis-only input; disjoint-item cross-variety transfer scores 40.1% and 45.6%, while a counterpart-leak audit inflates them to 60-64%. Zero-shot Whisper transcription is poor, with best word error rates of 66.6% and 45.5%, and Whisper Large-v3 mislabels 93 of 114 Pidgin recordings as Yoruba or Swahili.

SourcearXiv Computational LinguisticsAuthor: Tiani Pekins Ebika
CamSpokenNLI: A Pilot Resource for Cross-Variety Transfer Evaluation in Cameroon Pidgin English and Camfranglais
Report an error

The correction channel is not available yet. You can copy the article reference below for later.

Correction instructions
Read article

[Submitted on 12 Nov 2026]

Title:CamSpokenNLI: A Pilot Resource for Cross-Variety Transfer Evaluation in Cameroon Pidgin English and Camfranglais

View a PDF of the paper titled CamSpokenNLI: A Pilot Resource for Cross-Variety Transfer Evaluation in Cameroon Pidgin English and Camfranglais, by Tiani Pekins Ebika

View PDF

Abstract:Cameroon Pidgin English (a West African Pidgin English variety) and Camfranglais are widely used in Cameroon, yet to our knowledge, neither has a dedicated spoken Natural Language Inference (NLI) resource. We introduce CamSpokenNLI, a pilot dataset of 114 scenario-matched spoken NLI items in each variety (228 in total), recorded by one native speaker from Buea. Each item pairs premise audio and text with a hypothesis, an NLI label, and English glosses. The author assigned labels, so the dataset is an exploratory resource rather than a mature benchmark. The NLI experiments use the reference text; the audio is evaluated separately. XLM-RoBERTa baselines with five-fold item-level cross-validation reach 44.7% on Pidgin and 46.2% on Camfranglais (chance 33.3%). Still, neither is statistically distinguishable from hypothesis-only input, so hypothesis wording may account for much of the accuracy. Under disjoint-item cross-variety transfer, where a test item's counterpart is never seen in training, accuracy is 40.1% (Pidgin to Camfranglais) and 45.6% (Camfranglais to Pidgin). A counterpart-leak audit raises this to 60-64%, illustrating how counterpart leakage can overstate transfer. Zero-shot Whisper transcription of the recordings performs poorly: the best word error rates are 66.6% for Pidgin and 45.5% for Camfranglais, and with automatic language identification, Whisper Large-v3 labels 93 of 114 Pidgin recordings as Yoruba or Swahili. Together, the results suggest that such resources need disjoint-item evaluation and hypothesis-only controls, and that speech recognition must be adapted to these varieties before spoken NLI can be tested end to end.

Comments: 26 pages, 3 figures, 3 numbered tables

Subjects:

Computation and Language (cs.CL); Sound (cs.SD)

ACM classes: I.2.7

Cite as: arXiv:2611.17573 [cs.CL]

(or arXiv:2611.17573v1 [cs.CL] for this version)

https://doi.org/10.48550/arXiv.2611.17573

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Tiani Pekins Ebika [view email] [v1] Thu, 12 Nov 2026 22:51:31 UTC (95 KB)

Full-text links:

Access Paper:

View a PDF of the paper titled CamSpokenNLI: A Pilot Resource for Cross-Variety Transfer Evaluation in Cameroon Pidgin English and Camfranglais, by Tiani Pekins Ebika

View PDF

TeX Source

view license

Additional Features

Audio Summary

Current browse context:

cs.CL

new | recent | 2026-11

Change to browse by:

cs cs.SD

References & Citations

SciX

Google Scholar

Semantic Scholar

Loading...

Data provided by:

Bibliographic Tools

Bibliographic and Citation Tools

Bibliographic Explorer Toggle

Bibliographic Explorer (What is the Explorer?)

Connected Papers Toggle

Connected Papers (What is Connected Papers?)

Litmaps Toggle

Litmaps (What is Litmaps?)

scite.ai Toggle

scite Smart Citations (What are Smart Citations?)

Code, Data, Media

Code, Data and Media Associated with this Article

alphaXiv Toggle

alphaXiv (What is alphaXiv?)

Links to Code Toggle

CatalyzeX Code Finder for Papers (What is CatalyzeX?)

DagsHub Toggle

DagsHub (What is DagsHub?)

GotitPub Toggle

Gotit.pub (What is GotitPub?)

Huggingface Toggle

Hugging Face (What is Huggingface?)

ScienceCast Toggle

ScienceCast (What is ScienceCast?)

Demos

Demos

Replicate Toggle

Replicate (What is Replicate?)

Spaces Toggle

Hugging Face Spaces (What is Spaces?)

Spaces Toggle

TXYZ.AI (What is TXYZ.AI?)

Related Papers

Recommenders and Search Tools

Link to Influence Flower

Influence Flower (What are Influence Flowers?)

Core recommender toggle

CORE Recommender (What is CORE?)

Author

Venue

Institution

Topic

About arXivLabs

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)

Key points and analysis

Article intelligence

InvestorsAdvanced

Key points

  • Releases 228 scenario-matched spoken NLI items (114 per variety) with premise audio, text, hypothesis, labels, and English glosses.
  • XLM-RoBERTa five-fold text baselines hit 44.7% on Pidgin and 46.2% on Camfranglais but are not distinguishable from hypothesis-only models.
  • Disjoint-item transfer yields 40.1% and 45.6%, and a leak audit shows counterpart overlap can raise scores to 60-64%.
  • Zero-shot Whisper performs poorly: best WER is 66.6% and 45.5%, and language ID misclassifies most Pidgin recordings.

Highlights and analysis are generated automatically and may contain errors. Check the original source.