Comparative Analysis of Naive Bayes and Support Vector Machine for Sentiment Classification of Indonesian-Language Mobile Application Reviews on Google Play Store

Abstract

This study conducted a comparative performance evaluation of Multinomial Naive Bayes and Support Vector Machine (SVM) with a linear kernel in classifying the sentiment of Indonesian-language mobile application reviews collected from the Google Play Store. A total of 2,847 reviews targeting the GoPay digital wallet application were gathered via web scraping using the google-play-scraper library. After preprocessing, including case folding, cleansing, tokenization, stopword removal, and stemming using the Sastrawi library, the final dataset comprised 2,634 usable reviews. Sentiment labeling was conducted automatically based on star ratings: ratings of 4 and 5 were assigned as positive (1,841 reviews, 69.9%), while ratings of 1 and 2 were assigned as negative (793 reviews, 30.1%). Feature extraction used TF-IDF with a vocabulary size of 8,432 unique terms. Model training used an 80:20 train-test split with stratified sampling. SVM parameters were set to kernel=linear and C=1.0; Naive Bayes used alpha=1.0 (Laplace smoothing). Experimental results show that SVM achieved an accuracy of 88.3%, precision of 0.89, recall of 0.88, and F1-score of 0.88, while Naive Bayes obtained an accuracy of 82.1%, precision of 0.84, recall of 0.82, and F1-score of 0.83. SVM demonstrated superior performance across all four evaluation metrics, with the largest gap observed in the F1-score for the negative class (SVM: 0.71 vs. Naive Bayes: 0.56). These findings confirm that SVM is more robust against class imbalance in informal Indonesian-language review data.