Design and Evaluation of a Sentiment and Topic Modeling Dashboard for Mining Google Play Reviews of a Public Service Mobile Application

Abstract

App-store reviews are a rich source of user feedback for improving mobile service quality, yet they are large in volume, unstructured, and can change rapidly after version updates, making manual triage difficult to perform consistently. This condition requires an approach that can transform review text into prioritized, decision-ready information through a monitoring-oriented information system. Objective: This study aims to design and evaluate an app-review analytics information system that integrates sentiment classification and topic modeling to capture user perception, identify dominant issues, and support faster and more measurable prioritization of app improvements. Methodology: The study employs a quantitative design using an applied case study and a design-and-evaluate approach. Secondary data were collected from 5,000 Google Play reviews over 12 months (text, rating, timestamp), and primary data were obtained through dashboard usability testing with 20 participants using the System Usability Scale (SUS). The analysis includes text preprocessing, TF–IDF-based sentiment classification with multiple model comparisons, LDA topic modeling to extract recurring issues, and integration of outputs into a dashboard for trend monitoring and reporting. Findings: The best-performing model (SVM) achieved Accuracy = 0.86 and Macro-F1 = 0.84 for three-class sentiment classification (positive–neutral–negative). Topic modeling produced 10 dominant topics, with negative sentiment most strongly concentrated in core service issues such as login/OTP failures (72% negative), post-update crashes (69%), payment/transaction errors (65%), and server downtime (63%). The dashboard achieved a usability score of SUS = 82.3, indicating strong operational readiness for monitoring and triage. Implications: The results provide a practical basis for app managers to conduct periodic monitoring, convert dominant complaints into prioritized maintenance backlogs, and assess the impact of updates through sentiment and topic trends. The approach can be adopted as a routine decision-support mechanism for product owners, helpdesk teams, and developers. Originality/Value: This study’s novelty lies in its end-to-end integration of NLP analytics (sentiment and topics) with an information system artifact (dashboard), complemented by standardized usability validation. Unlike studies that stop at model outputs, this work demonstrates how analytics can be operationalized into a continuously usable monitoring system for app-maintenance decision-making.