Evaluating multilingual encoder models for few-shot named entity recognition tasks

Abstract

This work provides a thorough analysis of few-shot learning approaches in the realm of multilingual named entity recognition (NER). Our research is driven by the need to enhance linguistic inclusivity and performance efficiency across diverse languages. We focus on benchmarking a selection of prominent encoder models including XLM-RoBERTa (XLM-R), multilingual BERT (mBERT), DistilBERT, character architecture for eNcoders IN embeddings (CANINE), and multilingual text-to-text transfer transformer (mT5), to illuminate their capabilities and limitations within few-shot learning paradigms, particularly for underrepresented languages. Results indicate that models like XLM-R and mT5 demonstrate superior adaptability and accuracy, outperforming others in complex linguistic settings, which suggests their potential in supporting more inclusive artificial intelligence (AI) technologies. The impact of this study extends beyond academic interest, offering pivotal insights for the development of more inclusive, adaptable and efficient NER systems. By advancing our understanding of few-shot learning in multilingual contexts, this work contributes to the broader goal of creating AI applications that are linguistically diverse and more reflective of global communication patterns. These results provide crucial insights for advancing entity recognition capabilities across diverse artificial intelligence systems, facilitating development of more precise, equitable, and sophisticated linguistic processing frameworks.