Skip to content
AI News HubLIVE
Original source2 min read

ICARE: cross-domain text classification with incremental class-aware representation and distillation learning

Summary

ICARE is a novel framework that combines a representation-learning-based instance selection module with feature- and logit-level knowledge distillation to mitigate catastrophic forgetting in continual multi-task natural language inference, achieving significant improvements in target accuracy and reductions in forgetting on benchmark text classification datasets.

SourcearXiv Computational LinguisticsAuthor: Son Duong, Truong X. Tran, Hai-Anh Tran
ICARE: cross-domain text classification with incremental class-aware representation and distillation learning
Report an error

The correction channel is not available yet. You can copy the article reference below for later.

Correction instructions
Read article

[Submitted on 11 Nov 2026]

Title:ICARE: cross-domain text classification with incremental class-aware representation and distillation learning

View a PDF of the paper titled ICARE: cross-domain text classification with incremental class-aware representation and distillation learning, by Son Duong and 2 other authors

View PDF HTML (experimental)

Abstract:The rise of powerful, fine-tuned large language models (LLMs) necessitates robust natural language inference (NLI) systems. These NLI systems ideally train across various tasks, encompassing diverse concepts like sentiment analysis, news classification, and coreference resolution. However, this continual, multi-tasking approach can lead to catastrophic forgetting, where previously learned tasks are lost, especially in domains lacking specific labels. Rehearsing past examples with labels can further exacerbate this issue by causing overfitting, hindering the system's ability to classify new categories. To tackle these challenges, this research proposes ICARE, a novel framework with two key components: an instance selection module and a knowledge distillation method. The former is based on representation learning, which meticulously selects training examples to present to the NLI system. The latter, knowledge distillation with feature and logit-level transfer, facilitates knowledge transfer between tasks. This leverages pruning techniques and optimized objective functions to transfer both essential feature-level and decision-making (logit-level) knowledge, mitigating catastrophic forgetting while adapting to new patterns. Extensive evaluations on benchmark text classification datasets have demonstrated ICARE's effectiveness. The framework achieves a significant improvement in target accuracy and reduction in the ``forgetting metric''. These results solidify ICARE's ability to preserve and leverage learned knowledge across diverse tasks, ultimately enhancing the adaptability and performance of NLI systems. The source code is available at this https URL

Subjects:

Computation and Language (cs.CL); Machine Learning (cs.LG)

Cite as: arXiv:2611.24959 [cs.CL]

(or arXiv:2611.24959v1 [cs.CL] for this version)

https://doi.org/10.48550/arXiv.2611.24959

arXiv-issued DOI via DataCite (pending registration)

Journal reference: International Journal of Machine Learning and Cybernetics, vol. 16, pp. 5277-5294, 2025

Related DOI:

https://doi.org/10.1007/s13042-025-02572-6

DOI(s) linking to related resources

Submission history

From: Truong Tran [view email] [v1] Wed, 11 Nov 2026 13:42:02 UTC (488 KB)

Full-text links:

Access Paper:

View a PDF of the paper titled ICARE: cross-domain text classification with incremental class-aware representation and distillation learning, by Son Duong and 2 other authors

View PDF

HTML (experimental)

TeX Source

view license

Additional Features

Audio Summary

Current browse context:

cs.CL

new | recent | 2026-11

Change to browse by:

cs cs.LG

References & Citations

SciX

Google Scholar

Semantic Scholar

Loading...

Data provided by:

Bibliographic Tools

Bibliographic and Citation Tools

Bibliographic Explorer Toggle

Bibliographic Explorer (What is the Explorer?)

Connected Papers Toggle

Connected Papers (What is Connected Papers?)

Litmaps Toggle

Litmaps (What is Litmaps?)

scite.ai Toggle

scite Smart Citations (What are Smart Citations?)

Code, Data, Media

Code, Data and Media Associated with this Article

alphaXiv Toggle

alphaXiv (What is alphaXiv?)

Links to Code Toggle

CatalyzeX Code Finder for Papers (What is CatalyzeX?)

DagsHub Toggle

DagsHub (What is DagsHub?)

GotitPub Toggle

Gotit.pub (What is GotitPub?)

Huggingface Toggle

Hugging Face (What is Huggingface?)

ScienceCast Toggle

ScienceCast (What is ScienceCast?)

Demos

Demos

Replicate Toggle

Replicate (What is Replicate?)

Spaces Toggle

Hugging Face Spaces (What is Spaces?)

Spaces Toggle

TXYZ.AI (What is TXYZ.AI?)

Related Papers

Recommenders and Search Tools

Link to Influence Flower

Influence Flower (What are Influence Flowers?)

Core recommender toggle

CORE Recommender (What is CORE?)

Author

Venue

Institution

Topic

About arXivLabs

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)

Key points and analysis

Article intelligence

InvestorsAdvanced

Key points

  • ICARE consists of two core components: an instance selection module and a knowledge distillation method.
  • Knowledge distillation transfers both feature-level and logit-level knowledge using pruning techniques and optimized objective functions.
  • Evaluations on benchmark text classification datasets show significant improvement in target accuracy and reduction in the forgetting metric.
  • The source code is available on GitHub, and the paper is published in the International Journal of Machine Learning and Cybernetics.

Highlights and analysis are generated automatically and may contain errors. Check the original source.