Sila Consumer Sentiment Index - UAE, May 2021

Subscribe to Sila CSI reports to your inbox

---

About the D/A Sila Consumer Sentiment Index: The Sila Consumer Sentiment Index (Sila CSI) is an index of over 45 million data points on social media that measures the public sentiment about the economy, business and employment in Saudi and the UAE. It excludes news, and only focuses on conversations about those particular topics. The language used is then analysed using natural language processing and AI to determine sentiment in Arabic dialects. Index scores are out of 100 (a score of 100 means 100% positive, a score of 59.5 would mean 59.5% positive, 40.5% negative). About D/A and Sila: D/A has built out the region’s only sentiment platform that natively works in Arabic dialect (different Khaleej dialects in addition to broader region), Sila, and within it has a sentiment index that pools together the positive and negative discussion on social media about key items of concern to consumers. We exclude news sharing from this analysis and instead look to opinion. Put simply, we use a proprietary Natural Language Processing (NLP) model to understand what consumers are feeling towards a topic, at scale. The basis of the data is a continuous analysis of around 45mn tweets over the last 17 months (excluding news articles) from the UAE and KSA that allows us to better understand consumers’ feelings, in real-time, in their language. Technical Methodology: The sentiment model is based on the Natural Language Process - NLP techniques (MLM) and BERT-Base Multilingual Cased model for the Arabic language and is trained using a custom implementation of TensorFlow. The model involves a 3-way classification (positive, neutral, and negative). For the training and testing phases, we used ArapTweet, (a dataset of tweets from 11 Arab regions from 16 different countries, for a total of 45,000 tweets and news). and the data were divided into 3 phases: train (70%), Dev (15%), and Test (15%). The input parameters are the tweet (text), the number of words, and the weight of the business lexicon corpus. The model was trained with +31,000 tweets with 4 layers and 56 hidden units and 32 adjustment parameters. On Train set, the fine-tuned model obtains 86.09% on accuracy and 87.46% on F1 score. On the Test set, we acquired 88.19% acc and 86.51% F1 score. The testing and tuning process of the model is carried out every 2 months in order to improve the corpus and adjust the precision of the model. For explanation: 86% of the F1 score means that there is no evidence of assigning a sentiment, (this can happen when the text is very short or there is no congruence in the text or a mixture of languages), for our model, after excluding these cases, the precision ranges between 92.1% and 94.67% of successful cases, that is, the error ranges between 5.33% and 7.90% Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then runs the entire masked sentence through the model and has to predict the masked words. This is different from traditional recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like GPT which internally mask the future tokens. It allows the model to learn a bidirectional representation of the sentence.