AVA-TMP: A Human-in-the-Loop Multi-layer Dynamic Topic Modeling Pipeline

Viseth Sean, Padideh Danaee, Yang Yang, Hakan Kardes


Abstract
A phone call is still one of the primary preferred channels for seniors to express their needs, ask questions, and inform potential problems to their health insurance plans. Alignment Healthis a next-generation, consumer-centric organization that is providing a variety of Medicare Advantage Products for seniors. We combine our proprietary technology platform, AVA, and our high-touch clinical model to provide seniors with care as it should be: high quality, low cost, and accompanied by a vastly improved consumer experience. Our members have the ability to connect with our member services and concierge teams 24/7 for a wide variety of ever-changing reasons through different channels, such as phone, email, and messages. We strive to provide an excellent member experience and ensure our members are getting the help and information they need at every touch —ideally, even before they reach us. This requires ongoing monitoring of reasons for contacting us, ensuring agents are equipped with the right tools and information to serve members, and coming up with proactive strategies to eliminate the need for the call when possible. We developed an NLP-based dynamic call reason tagging and reporting pipeline with an optimized human-in-the-loop approach to enable accurate call reason reporting and monitoring with the ability to see high-level trends as well as drill down into more granular sub-reasons. Our system produces 96.4% precision and 30%-50% better recall in tagging calls with proper reasons. We have also consistently achieved a 60+ Net Promoter Score (NPS) score, which illustrates high consumer satisfaction.
Anthology ID:
2022.dash-1.11
Volume:
Proceedings of the Fourth Workshop on Data Science with Human-in-the-Loop (Language Advances)
Month:
December
Year:
2022
Address:
Abu Dhabi, United Arab Emirates (Hybrid)
Editors:
Eduard Dragut, Yunyao Li, Lucian Popa, Slobodan Vucetic, Shashank Srivastava
Venue:
DaSH
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
81–87
Language:
URL:
https://aclanthology.org/2022.dash-1.11
DOI:
Bibkey:
Cite (ACL):
Viseth Sean, Padideh Danaee, Yang Yang, and Hakan Kardes. 2022. AVA-TMP: A Human-in-the-Loop Multi-layer Dynamic Topic Modeling Pipeline. In Proceedings of the Fourth Workshop on Data Science with Human-in-the-Loop (Language Advances), pages 81–87, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
Cite (Informal):
AVA-TMP: A Human-in-the-Loop Multi-layer Dynamic Topic Modeling Pipeline (Sean et al., DaSH 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.dash-1.11.pdf
Video:
 https://aclanthology.org/2022.dash-1.11.mp4