CoVA: Context-aware Visual Attention for Webpage Information Extraction

Anurendra Kumar, Keval Morabia, William Wang, Kevin Chang, Alex Schwing


Abstract
Webpage information extraction (WIE) is an important step to create knowledge bases. For this, classical WIE methods leverage the Document Object Model (DOM) tree of a website. However, use of the DOM tree poses significant challenges as context and appearance are encoded in an abstract manner. To address this challenge we propose to reformulate WIE as a context-aware Webpage Object Detection task. Specifically, we develop a Context-aware Visual Attention-based (CoVA) detection pipeline which combines appearance features with syntactical structure from the DOM tree. To study the approach we collect a new large-scale datase of e-commerce websites for which we manually annotate every web element with four labels: product price, product title, product image and others. On this dataset we show that the proposed CoVA approach is a new challenging baseline which improves upon prior state-of-the-art methods.
Anthology ID:
2022.ecnlp-1.11
Volume:
Proceedings of the Fifth Workshop on e-Commerce and NLP (ECNLP 5)
Month:
May
Year:
2022
Address:
Dublin, Ireland
Editors:
Shervin Malmasi, Oleg Rokhlenko, Nicola Ueffing, Ido Guy, Eugene Agichtein, Surya Kallumadi
Venue:
ECNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
80–90
Language:
URL:
https://aclanthology.org/2022.ecnlp-1.11
DOI:
10.18653/v1/2022.ecnlp-1.11
Bibkey:
Cite (ACL):
Anurendra Kumar, Keval Morabia, William Wang, Kevin Chang, and Alex Schwing. 2022. CoVA: Context-aware Visual Attention for Webpage Information Extraction. In Proceedings of the Fifth Workshop on e-Commerce and NLP (ECNLP 5), pages 80–90, Dublin, Ireland. Association for Computational Linguistics.
Cite (Informal):
CoVA: Context-aware Visual Attention for Webpage Information Extraction (Kumar et al., ECNLP 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.ecnlp-1.11.pdf
Video:
 https://aclanthology.org/2022.ecnlp-1.11.mp4
Code
 kevalmorabia97/cova-web-object-detection
Data
CoVA