User Feedback to Actionable Insights: A Fine-grained Multilabel Approach for Classifying App Reviews

Chirag Jain, Preethu Rose Anish


Abstract
The rapid proliferation of mobile applications has resulted in a vast volume of user-generated reviews, which often contain critical insights into bugs, feature demands, and overall user satisfaction. Efficiently organizing and interpreting this feedback is essential for developers seeking to enhance app quality and user experience. In this study, we propose a fine-grained, multilabel classification schema for app reviews, structured as a two-level taxonomy. The first level categorizes reviews into three primary classes: Bug Report, Feature Request, and App Satisfaction Feedback. The second level further refines these categories into subtypes addressing specific concerns such as performance, usability and privacy. To automate this classification, we fine-tune several state-of-the-art large language models (LLMs), including LLaMA, Gemma, Mistral, Qwen, on a manually annotated dataset spanning 10 diverse application domains. Among these, LLaMA 3.1 achieved the highest performance, with weighted F1 scores of 0.90 and 0.79 at the first and second taxonomy levels, respectively. To assess scalability, we applied the fine-tuned LLaMA 3.1 model to label an additional 10,000 unlabeled reviews, creating a silver-standard corpus. This taxonomy-driven approach enables more nuanced analysis of user feedback, supporting targeted app improvements and fostering enhanced user satisfaction.
Anthology ID:
2026.aacl-long.230
Volume:
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
November
Year:
2026
Address:
Hengqin, China
Editors:
Yue Zhang, Ruifeng Xu, Ayu Purwarianti, Tong Rong, Jun Suzuki
Venues:
AACL | IJCNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
5164–5180
Language:
URL:
https://aclanthology.org/2026.aacl-long.230/
DOI:
Bibkey:
Cite (ACL):
Chirag Jain and Preethu Rose Anish. 2026. User Feedback to Actionable Insights: A Fine-grained Multilabel Approach for Classifying App Reviews. In Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5164–5180, Hengqin, China. Association for Computational Linguistics.
Cite (Informal):
User Feedback to Actionable Insights: A Fine-grained Multilabel Approach for Classifying App Reviews (Jain & Anish, AACL-IJCNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.aacl-long.230.pdf