We use a simplified character level sequence-to-sequence deep learning model that we apply to MSA, CA, and two varieties of DA, namely Moroccan and Tunisian. Though the model does not require any feature engineering, it beats all previous state-of-the-art results for all the Arabic varieties that we test on. Our system achieves word error rates of 4.5% for MSA, 3.7% for CA, 1.4% for Moroccan dialect, and 2.5% for Tunisian.
DEMOS
QCRI's combined Arabic speech recognition, natural language processing (NLP), and dialect identification pipeline. It features modern web technologies to capture live audio, transcribes Arabic audio, NLP processes the transcripts, and identifies the dialect of the speaker. For transcription, we use QATS, which is a Kaldi-based ASR system that uses Time Delay Neural Networks (TDNN). For NLP, we use a SOTA Arabic NLP toolkit that employs various deep neural network and SVM based models. Finally, our dialect identification system uses multi-modality from both acoustic and linguistic input.
SHOW METhe demo describes the best QCRI submission to the shared task of automatic Arabic dialect classification into 5 Arabic variants, namely Egyptian, Gulf, Levantine, North-African (Maghrebi),and Modern Standard Arabic (MSA). The relatively small training set is automatically generated from an ASR system. To avoid over-fitting on such small data, we selected and designed features that capture the morphological essence of the different dialects. We submitted four runs to theArabic sub-task. We used a combined feature vector of character bigrams, trigrams,4-grams, and 5-grams. The submitted run used SVM with a linear kernel. We got the best accuracy of 0.5136 and the third best weighted F1 score, with a difference of less than 0.002.