dialect-identification

A suite of Arabic natural language processing tools developed by the CAMeL Lab at New York University Abu Dhabi.

TunBERT is the first release of a pre-trained BERT model for the Tunisian dialect using a Tunisian Common-Crawl-based dataset. TunBERT was applied to three NLP downstream tasks: Sentiment Analysis (SA), Tunisian Dialect Identification (TDI) and Reading Comprehension Question-Answering (RCQA)

Language and Speech Technology for Central Kurdish Varieties (LREC-COLING 2024)

VarDial19 shared task: Discriminating between Mainland and Taiwan Variation of Mandarin Chinese (DMT)

A tool that predicts the dialect of English of an SMS message using recurrent neural networks supplemented with data from Google Trends.

Dialect-aware grapheme-to-phoneme conversion for German using Transformer + XLM-R. Context-aware, multi-dialect support with CTC+CE training. Built with PyTorch Lightning & Hydra.

using AraBert to classify different Arabic dialects. ranked fourth in WANLP2020 workshop.

Chinese dialect identification using audio embeddings from LLMs.

Fine-tuning SDAIA's ALLaM LLM to understand Arabic dialects — identify, translate to MSA, reply in-dialect. ALLaM Challenge finalist; led a 4-person team.

Dialect Identification in Indic Languages

Arabic Dialect Identification Dataset: 5,000-sentence evaluation benchmark across Egyptian, Gulf, Levantine, Maghrebi, and MSA

5-Class Arabic dialect identification system distinguishing Egyptian, Gulf, Levantine, Maghrebi, and MSA across 5,000 sentences.

Write Arabic in a genuine Palestinian voice, not Gulf, Egyptian or stiff formal Arabic.

Sub-regional Libyan Arabic speech corpus + DID benchmark: pipeline, manifests, cached-embedding release (ArabicNLP 2026)

A modernized, lightweight port of CAMeL Tools designed to minimize heavy dependencies.

[Interspeech19] Computational Paralinguistics ChallengE (ComParE)

Binary dialect classification: Standard vs Kathiawadi Gujarati.

log MFSC based classification of British English dialects from the IViE(Intonational Variation in English) corpus dataset

Integrated Arabic NLP system for sentiment analysis, topic categorization, and dialect identification using TF-IDF and Scikit-learn pipelines.