DoSA : A System to Accelerate Annotations on Business Documents with Human-in-the-Loop

Neelesh Shukla, Msp Raja, Raghu Katikeri, Amit Vaid


Abstract
Business documents come in a variety of structures, formats and information needs which makes information extraction a challenging task. Due to these variations, having a document generic model which can work well across all types of documents for all the use cases seems far-fetched. For document-specific models, we would need customized document-specific labels. We introduce DoSA (Document Specific Automated Annotations), which helps annotators in generating initial annotations automatically using our novel bootstrap approach by leveraging document generic datasets and models. These initial annotations can further be reviewed by a human for correctness. An initial document-specific model can be trained and its inference can be used as feedback for generating more automated annotations. These automated annotations can be reviewed by humanin-the-loop for the correctness and a new improved model can be trained using the current model as pre-trained model before going for the next iteration. In this paper, our scope is limited to Form like documents due to limited availability of generic annotated datasets, but this idea can be extended to a variety of other documents as more datasets are built. An opensource ready-to-use implementation is made available on GitHub.
Anthology ID:
2022.dash-1.4
Volume:
Proceedings of the Fourth Workshop on Data Science with Human-in-the-Loop (Language Advances)
Month:
December
Year:
2022
Address:
Abu Dhabi, United Arab Emirates (Hybrid)
Editors:
Eduard Dragut, Yunyao Li, Lucian Popa, Slobodan Vucetic, Shashank Srivastava
Venue:
DaSH
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
23–27
Language:
URL:
https://aclanthology.org/2022.dash-1.4
DOI:
Bibkey:
Cite (ACL):
Neelesh Shukla, Msp Raja, Raghu Katikeri, and Amit Vaid. 2022. DoSA : A System to Accelerate Annotations on Business Documents with Human-in-the-Loop. In Proceedings of the Fourth Workshop on Data Science with Human-in-the-Loop (Language Advances), pages 23–27, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
Cite (Informal):
DoSA : A System to Accelerate Annotations on Business Documents with Human-in-the-Loop (Shukla et al., DaSH 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.dash-1.4.pdf
Video:
 https://aclanthology.org/2022.dash-1.4.mp4