Sentence Boundary Detection on Line Breaks in Japanese

Yuta Hayashibe, Kensuke Mitsuzawa


Abstract
For NLP, sentence boundary detection (SBD) is an essential task to decompose a text into sentences. Most of the previous studies have used a simple rule that uses only typical characters as sentence boundaries. However, some characters may or may not be sentence boundaries depending on the context. We focused on line breaks in them. We newly constructed annotated corpora, implemented sentence boundary detectors, and analyzed performance of SBD in several settings.
Anthology ID:
2020.wnut-1.10
Original:
2020.wnut-1.10v1
Version 2:
2020.wnut-1.10v2
Volume:
Proceedings of the Sixth Workshop on Noisy User-generated Text (W-NUT 2020)
Month:
November
Year:
2020
Address:
Online
Editors:
Wei Xu, Alan Ritter, Tim Baldwin, Afshin Rahimi
Venue:
WNUT
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
71–75
Language:
URL:
https://aclanthology.org/2020.wnut-1.10
DOI:
10.18653/v1/2020.wnut-1.10
Bibkey:
Cite (ACL):
Yuta Hayashibe and Kensuke Mitsuzawa. 2020. Sentence Boundary Detection on Line Breaks in Japanese. In Proceedings of the Sixth Workshop on Noisy User-generated Text (W-NUT 2020), pages 71–75, Online. Association for Computational Linguistics.
Cite (Informal):
Sentence Boundary Detection on Line Breaks in Japanese (Hayashibe & Mitsuzawa, WNUT 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.wnut-1.10.pdf