A Rationale-Centric Framework for Human-in-the-loop Machine Learning

Jinghui Lu, Linyi Yang, Brian Namee, Yue Zhang


Abstract
We present a novel rational-centric framework with human-in-the-loop – Rationales-centric Double-robustness Learning (RDL) – to boost model out-of-distribution performance in few-shot learning scenarios. By using static semi-factual generation and dynamic human-intervened correction, RDL, acting like a sensible “inductive bias”, exploits rationales (i.e. phrases that cause the prediction), human interventions and semi-factual augmentations to decouple spurious associations and bias models towards generally applicable underlying distributions, which enables fast and accurate generalisation. Experimental results show that RDL leads to significant prediction benefits on both in-distribution and out-of-distribution tests, especially for few-shot learning scenarios, compared to many state-of-the-art benchmarks. We also perform extensive ablation studies to support in-depth analyses of each component in our framework.
Anthology ID:
2022.acl-long.481
Volume:
Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
May
Year:
2022
Address:
Dublin, Ireland
Editors:
Smaranda Muresan, Preslav Nakov, Aline Villavicencio
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
6986–6996
Language:
URL:
https://aclanthology.org/2022.acl-long.481
DOI:
10.18653/v1/2022.acl-long.481
Bibkey:
Cite (ACL):
Jinghui Lu, Linyi Yang, Brian Namee, and Yue Zhang. 2022. A Rationale-Centric Framework for Human-in-the-loop Machine Learning. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6986–6996, Dublin, Ireland. Association for Computational Linguistics.
Cite (Informal):
A Rationale-Centric Framework for Human-in-the-loop Machine Learning (Lu et al., ACL 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.acl-long.481.pdf
Video:
 https://aclanthology.org/2022.acl-long.481.mp4
Code
 GeorgeLuImmortal/RDL-Rationales-centric-Double-robustness-Learning
Data
IMDb Movie ReviewsSSTSST-2