GreaseVision: Rewriting the Rules of the Interface

Siddhartha Datta, Konrad Kollnig, Nigel Shadbolt


Abstract
Digital harms can manifest across any interface. Key problems in addressing these harms include the high individuality of harms and the fast-changing nature of digital systems. We put forth GreaseVision, a collaborative human-in-the-loop learning framework that enables end-users to analyze their screenomes to annotate harms as well as render overlay interventions. We evaluate HITL intervention development with a set of completed tasks in a cognitive walkthrough, and test scalability with one-shot element removal and fine-tuning hate speech classification models. The contribution of the framework and tool allow individual end-users to study their usage history and create personalized interventions. Our contribution also enables researchers to study the distribution of multi-modal harms and interventions at scale.
Anthology ID:
2022.woah-1.3
Volume:
Proceedings of the Sixth Workshop on Online Abuse and Harms (WOAH)
Month:
July
Year:
2022
Address:
Seattle, Washington (Hybrid)
Editors:
Kanika Narang, Aida Mostafazadeh Davani, Lambert Mathias, Bertie Vidgen, Zeerak Talat
Venue:
WOAH
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
24–28
Language:
URL:
https://aclanthology.org/2022.woah-1.3
DOI:
10.18653/v1/2022.woah-1.3
Bibkey:
Cite (ACL):
Siddhartha Datta, Konrad Kollnig, and Nigel Shadbolt. 2022. GreaseVision: Rewriting the Rules of the Interface. In Proceedings of the Sixth Workshop on Online Abuse and Harms (WOAH), pages 24–28, Seattle, Washington (Hybrid). Association for Computational Linguistics.
Cite (Informal):
GreaseVision: Rewriting the Rules of the Interface (Datta et al., WOAH 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.woah-1.3.pdf