Skip to content
AI News HubLIVE
Original source2 min read

Small Open-Weight Language Models for Crowd Decision-Making: Capabilities and Limits of Output-Level Fusion and Linguistic Aggregation

Summary

A new arXiv paper evaluates small open-weight language models (8-36B parameters) for crowd decision-making on restaurant review classification. Repeating queries and output-level fusion stabilize results but do not eliminate sensitivity to participant sampling; systems output candidate sets rather than single winners. Three larger models match a hosted reference, suggesting local CDM is feasible.

SourcearXiv Computational LinguisticsAuthor: David Herrera-Poyatos, Cristina Zuheros, Carlos Pel\'aez-Gonz\'alez, Andr\'es Herrera-Poyatos, Rosana Montes, Francisco Herrera
Small Open-Weight Language Models for Crowd Decision-Making: Capabilities and Limits of Output-Level Fusion and Linguistic Aggregation
Report an error

The correction channel is not available yet. You can copy the article reference below for later.

Correction instructions
Read article

[Submitted on 12 Nov 2026]

Title:Small Open-Weight Language Models for Crowd Decision-Making: Capabilities and Limits of Output-Level Fusion and Linguistic Aggregation

View a PDF of the paper titled Small Open-Weight Language Models for Crowd Decision-Making: Capabilities and Limits of Output-Level Fusion and Linguistic Aggregation, by David Herrera-Poyatos and 5 other authors

View PDF HTML (experimental)

Abstract:Small open-weight language models can run crowd decision-making (CDM) on an institution's own infrastructure, which keeps the checkpoint, the inference settings and the opinions under its control. Their judgements vary across repeated queries; whether repeating and aggregating them, often through the 2-tuple linguistic representation, settles the decision rather than the labels is untested. Five open-weight models of 8-36B total parameters on one consumer GPU and a hosted reference classify 474 TripAdvisor reviews of four London restaurants ten times each; three open-weight models repeat the protocol on 600 reviews of twelve Madrid restaurants. Continuous evaluations preserve resolution: snapping each review leaves 5 distinct values against 21-65, and snapping at every level at most two collective values. Sensitivity to the participants persists once run-to-run variability is reduced: in London, redrawing the reviewers keeps the winner on 47-81% of replicas against 86-100% for redrawing the runs. With simultaneous intervals over every pair of alternatives under crowd resampling, no system on either corpus names a single winner: in London every system keeps all four restaurants as candidates, and in Madrid the candidate sets hold five to eight of the twelve. Extra runs and output-level fusion stabilize the winner but do not remove that sensitivity, and the fusion does not beat its best member. The three larger open-weight models name the hosted reference's London winner and at least match its per-review correlation with the reviewers' stars, so small open-weight models can support local CDM, reported as candidate sets with margins rather than forced rankings.

Subjects:

Computation and Language (cs.CL)

Cite as: arXiv:2611.17589 [cs.CL]

(or arXiv:2611.17589v1 [cs.CL] for this version)

https://doi.org/10.48550/arXiv.2611.17589

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: David Herrera-Poyatos [view email] [v1] Thu, 12 Nov 2026 23:15:00 UTC (363 KB)

Full-text links:

Access Paper:

View a PDF of the paper titled Small Open-Weight Language Models for Crowd Decision-Making: Capabilities and Limits of Output-Level Fusion and Linguistic Aggregation, by David Herrera-Poyatos and 5 other authors

View PDF

HTML (experimental)

TeX Source

view license

Additional Features

Audio Summary

Current browse context:

cs.CL

new | recent | 2026-11

Change to browse by:

cs

References & Citations

SciX

Google Scholar

Semantic Scholar

Loading...

Data provided by:

Bibliographic Tools

Bibliographic and Citation Tools

Bibliographic Explorer Toggle

Bibliographic Explorer (What is the Explorer?)

Connected Papers Toggle

Connected Papers (What is Connected Papers?)

Litmaps Toggle

Litmaps (What is Litmaps?)

scite.ai Toggle

scite Smart Citations (What are Smart Citations?)

Code, Data, Media

Code, Data and Media Associated with this Article

alphaXiv Toggle

alphaXiv (What is alphaXiv?)

Links to Code Toggle

CatalyzeX Code Finder for Papers (What is CatalyzeX?)

DagsHub Toggle

DagsHub (What is DagsHub?)

GotitPub Toggle

Gotit.pub (What is GotitPub?)

Huggingface Toggle

Hugging Face (What is Huggingface?)

ScienceCast Toggle

ScienceCast (What is ScienceCast?)

Demos

Demos

Replicate Toggle

Replicate (What is Replicate?)

Spaces Toggle

Hugging Face Spaces (What is Spaces?)

Spaces Toggle

TXYZ.AI (What is TXYZ.AI?)

Related Papers

Recommenders and Search Tools

Link to Influence Flower

Influence Flower (What are Influence Flowers?)

Core recommender toggle

CORE Recommender (What is CORE?)

Author

Venue

Institution

Topic

About arXivLabs

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)

Key points and analysis

Article intelligence

EngineersAdvanced

Key points

  • Five open-weight models (8-36B) on one consumer GPU classified 474 London reviews ten times each; three models repeated on 600 Madrid reviews.
  • Continuous evaluations preserve resolution: snapping leaves 5 distinct values versus 21-65, and snapping at every level yields at most two collective values.
  • Sensitivity to participants persists: redrawing reviewers keeps the winner in 47-81% of replicas versus 86-100% for redrawing runs.
  • No system names a single winner under crowd resampling; extra runs and fusion stabilize the winner but do not remove sensitivity, and fusion does not beat its best member.

Highlights and analysis are generated automatically and may contain errors. Check the original source.