Tiantian Feng
Author directory2026
ChildVox: A Speech, Audio, and Large Audio-Language Model Benchmark in Understanding and Characterizing Sound across Childhood
Tiantian Feng | Anfeng Xu | Xuan Shi | Aditya Kommineni | Shakhrul Iman Siam | Megan Micheletti | Zhonghao Shi | Helen Tager-Flusberg | Mi Zhang | Lynn K Perry | Catherine Lord | Daniel Messinger | Shrikanth Narayanan
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Tiantian Feng | Anfeng Xu | Xuan Shi | Aditya Kommineni | Shakhrul Iman Siam | Megan Micheletti | Zhonghao Shi | Helen Tager-Flusberg | Mi Zhang | Lynn K Perry | Catherine Lord | Daniel Messinger | Shrikanth Narayanan
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
We present ChildVox, a novel benchmark for characterizing the diverse acoustic signals through which children communicate. Specifically, ChildVox follows the full developmental trajectory from birth through school age, covering physiological sounds, non-linguistic vocalizations, canonical syllables, and spoken language. ChildVox integrates more than 20 sub-tasks across 17 child-centered audio and speech datasets, enabling systematic cross-corpus and cross-domain comparison. We evaluate a representative range of audio and speech foundation models, including self-supervised, ASR-oriented, and large audio-language models, on tasks including physiological sound classification, vocalization and canonical syllables modeling, and speech quality assessment and recognition. Benchmark results show that ChildVox provides a suite of high-performance models in recognizing a wide range of acoustic signals from children, supporting downstream applications such as characterizing children’s language levels and tracking speech production with age. Our benchmark models on public datasets are released at: https://github.com/tiantiaf0627/childvox-release.
RespiraMFM: A Multimodal Foundation Model with Contrastive Audio-Language Alignment for Respiratory Disease Identification
Shakhrul Iman Siam | Tiantian Feng | Jiankun Zhang | Shrikanth Narayanan | Mi Zhang
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Shakhrul Iman Siam | Tiantian Feng | Jiankun Zhang | Shrikanth Narayanan | Mi Zhang
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Respiratory diseases remain a leading cause of global mortality, where timely and accurate diagnosis is critical to improving patient outcomes and reducing healthcare burdens. While prior work has explored audio-based models for respiratory disease detection, such unimodal approaches often suffer from limited generalizability and diagnostic precision. In this paper, we propose RespiraMFM, a Multimodal Foundation Model that integrates respiratory sounds with patient medical history and symptoms to enhance diagnostic accuracy and disease detection capabilities. We introduce an effective contrastive alignment strategy for audio-text multimodal integration, allowing the model to learn better cross-modal representations between respiratory sounds and corresponding textual clinical information. We evaluate RespiraMFM across five major respiratory diseases using seven real-world datasets in both supervised fine-tuning and zero-shot settings, achieving a 9.15% improvement in AUROC on supervised tasks and a 20.98% gain on zero-shot tasks over existing baselines. These findings underscore the potential of our framework to advance early diagnosis and improve clinical decision-making in respiratory disease management.
2025
Creating a Lens of Chinese Culture: A Multimodal Dataset for Chinese Pun Rebus Art Understanding
Tuo Zhang | Tiantian Feng | Yibin Ni | Mengqin Cao | Ruying Liu | Kiana Avestimehr | Katharine Butler | Yanjun Weng | Mi Zhang | Shrikanth Narayanan | Salman Avestimehr
Findings of the Association for Computational Linguistics: ACL 2025
Tuo Zhang | Tiantian Feng | Yibin Ni | Mengqin Cao | Ruying Liu | Kiana Avestimehr | Katharine Butler | Yanjun Weng | Mi Zhang | Shrikanth Narayanan | Salman Avestimehr
Findings of the Association for Computational Linguistics: ACL 2025
Large vision-language models (VLMs) have demonstrated remarkable abilities in understanding everyday content. However, their performance in the domain of art, particularly culturally rich art forms, remains less explored. As a pearl of human wisdom and creativity, art encapsulates complex cultural narratives and symbolism. In this paper, we offer the Pun Rebus Art Dataset, a multimodal dataset for art understanding deeply rooted in traditional Chinese culture. We focus on three primary tasks: identifying salient visual elements, matching elements with their symbolic meanings, and explanations for the conveyed messages. Our evaluation reveals that state-of-the-art VLMs struggle with these tasks, often providing biased and hallucinated explanations and showing limited improvement through in-context learning. By releasing the Pun Rebus Art Dataset, we aim to facilitate the development of VLMs that can better understand and interpret culturally specific content, promoting greater inclusiveness beyond English-based corpora. The dataset and evaluation code are available at this link.
2020
Screenplay Quality Assessment: Can We Predict Who Gets Nominated?
Ming-Chang Chiu | Tiantian Feng | Xiang Ren | Shrikanth Narayanan
Proceedings of the First Joint Workshop on Narrative Understanding, Storylines, and Events
Ming-Chang Chiu | Tiantian Feng | Xiang Ren | Shrikanth Narayanan
Proceedings of the First Joint Workshop on Narrative Understanding, Storylines, and Events
Deciding which scripts to turn into movies is a costly and time-consuming process for filmmakers. Thus, building a tool to aid script selection, an initial phase in movie production, can be very beneficial. Toward that goal, in this work, we present a method to evaluate the quality of a screenplay based on linguistic cues. We address this in a two-fold approach: (1) we define the task as predicting nominations of scripts at major film awards with the hypothesis that the peer-recognized scripts should have a greater chance to succeed. (2) based on industry opinions and narratology, we extract and integrate domain-specific features into common classification techniques. We face two challenges (1) scripts are much longer than other document datasets (2) nominated scripts are limited and thus difficult to collect. However, with narratology-inspired modeling and domain features, our approach offers clear improvements over strong baselines. Our work provides a new approach for future work in screenplay analysis.