Jianming Wu


pdf bib
Contrastive Response Pairs for Automatic Evaluation of Non-task-oriented Neural Conversational Models
Koshiro Okano | Yu Suzuki | Masaya Kawamura | Tsuneo Kato | Akihiro Tamura | Jianming Wu
Proceedings of the 22nd Annual Meeting of the Special Interest Group on Discourse and Dialogue

Responses generated by neural conversational models (NCMs) for non-task-oriented systems are difficult to evaluate. We propose contrastive response pairs (CRPs) for automatically evaluating responses from non-task-oriented NCMs. We conducted an error analysis on responses generated by an encoder-decoder recurrent neural network (RNN) type NCM and created three types of CRPs corresponding to the three most frequent errors found in the analysis. Three NCMs of different response quality were objectively evaluated with the CRPs and compared to a subjective assessment. The correctness obtained by the three types of CRPs were consistent with the results of the subjective assessment.


pdf bib
Utterance Intent Classification of a Spoken Dialogue System with Efficiently Untied Recursive Autoencoders
Tsuneo Kato | Atsushi Nagai | Naoki Noda | Ryosuke Sumitomo | Jianming Wu | Seiichi Yamamoto
Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue

Recursive autoencoders (RAEs) for compositionality of a vector space model were applied to utterance intent classification of a smartphone-based Japanese-language spoken dialogue system. Though the RAEs express a nonlinear operation on the vectors of child nodes, the operation is considered to be different intrinsically depending on types of child nodes. To relax the difference, a data-driven untying of autoencoders (AEs) is proposed. The experimental result of the utterance intent classification showed an improved accuracy with the proposed method compared with the basic tied RAE and untied RAE based on a manual rule.