Tools and methods for objective or contextual evaluation of topic segmentation

Laurianne Sitbon, Patrice Bellot


Abstract
In this paper we discuss the way of evaluating topic segmentation, from mathematical measures on variously constructed reference corpus to contextual evaluation depending on different topic segmentation usages. We present an overview of the different ways of building reference corpora and of mathematically evaluating segmentation methods, and then we focus on three tasks which may involve a topic segmentation: text extraction, information retrieval and document presentation. We have developed two graphical interfaces, one for an intrinsic comparison, and the other one dedicated to an evaluation in an information retrieval context. These tools will be very soon distributed under GPL licences on the Technolangue project web page.
Anthology ID:
L06-1241
Volume:
Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC’06)
Month:
May
Year:
2006
Address:
Genoa, Italy
Editors:
Nicoletta Calzolari, Khalid Choukri, Aldo Gangemi, Bente Maegaard, Joseph Mariani, Jan Odijk, Daniel Tapias
Venue:
LREC
SIG:
Publisher:
European Language Resources Association (ELRA)
Note:
Pages:
Language:
URL:
http://www.lrec-conf.org/proceedings/lrec2006/pdf/410_pdf.pdf
DOI:
Bibkey:
Cite (ACL):
Laurianne Sitbon and Patrice Bellot. 2006. Tools and methods for objective or contextual evaluation of topic segmentation. In Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC’06), Genoa, Italy. European Language Resources Association (ELRA).
Cite (Informal):
Tools and methods for objective or contextual evaluation of topic segmentation (Sitbon & Bellot, LREC 2006)
Copy Citation:
PDF:
http://www.lrec-conf.org/proceedings/lrec2006/pdf/410_pdf.pdf