Simone Teufel


2021

pdf bib
End-to-End Argument Mining as Biaffine Dependency Parsing
Yuxiao Ye | Simone Teufel
Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume

Non-neural approaches to argument mining (AM) are often pipelined and require heavy feature-engineering. In this paper, we propose a neural end-to-end approach to AM which is based on dependency parsing, in contrast to the current state-of-the-art which relies on relation extraction. Our biaffine AM dependency parser significantly outperforms the state-of-the-art, performing at F1 = 73.5% for component identification and F1 = 46.4% for relation identification. One of the advantages of treating AM as biaffine dependency parsing is the simple neural architecture that results. The idea of treating AM as dependency parsing is not new, but has previously been abandoned as it was lagging far behind the state-of-the-art. In a thorough analysis, we investigate the factors that contribute to the success of our model: the biaffine model itself, our representation for the dependency structure of arguments, different encoders in the biaffine model, and syntactic information additionally fed to the model. Our work demonstrates that dependency parsing for AM, an overlooked idea from the past, deserves more attention in the future.

pdf bib
Parsing Argumentative Structure in English-as-Foreign-Language Essays
Jan Wira Gotama Putra | Simone Teufel | Takenobu Tokunaga
Proceedings of the 16th Workshop on Innovative Use of NLP for Building Educational Applications

This paper presents a study on parsing the argumentative structure in English-as-foreign-language (EFL) essays, which are inherently noisy. The parsing process consists of two steps, linking related sentences and then labelling their relations. We experiment with several deep learning architectures to address each task independently. In the sentence linking task, a biaffine model performed the best. In the relation labelling task, a fine-tuned BERT model performed the best. Two sentence encoders are employed, and we observed that non-fine-tuning models generally performed better when using Sentence-BERT as opposed to BERT encoder. We trained our models using two types of parallel texts: original noisy EFL essays and those improved by annotators, then evaluate them on the original essays. The experiment shows that an end-to-end in-domain system achieved an accuracy of .341. On the other hand, the cross-domain system achieved 94% performance of the in-domain system. This signals that well-written texts can also be useful to train argument mining system for noisy texts.

pdf bib
What About the Precedent: An Information-Theoretic Analysis of Common Law
Josef Valvoda | Tiago Pimentel | Niklas Stoehr | Ryan Cotterell | Simone Teufel
Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

In common law, the outcome of a new case is determined mostly by precedent cases, rather than by existing statutes. However, how exactly does the precedent influence the outcome of a new case? Answering this question is crucial for guaranteeing fair and consistent judicial decision-making. We are the first to approach this question computationally by comparing two longstanding jurisprudential views; Halsbury’s, who believes that the arguments of the precedent are the main determinant of the outcome, and Goodhart’s, who believes that what matters most is the precedent’s facts. We base our study on the corpus of legal cases from the European Court of Human Rights (ECtHR), which allows us to access not only the case itself, but also cases cited in the judges’ arguments (i.e. the precedent cases). Taking an information-theoretic view, and modelling the question as a case out-come classification task, we find that the precedent’s arguments share 0.38 nats of information with the case’s outcome, whereas precedent’s facts only share 0.18 nats of information (i.e.,58% less); suggesting Halsbury’s view may be more accurate in this specific court. We found however in a qualitative analysis that there are specific statues where Goodhart’s view dominates, and present some evidence these are the ones where the legal concept at hand is less straightforward.

2020

pdf bib
TIARA: A Tool for Annotating Discourse Relations and Sentence Reordering
Jan Wira Gotama Putra | Simone Teufel | Kana Matsumura | Takenobu Tokunaga
Proceedings of the 12th Language Resources and Evaluation Conference

This paper introduces TIARA, a new publicly available web-based annotation tool for discourse relations and sentence reordering. Annotation tasks such as these, which are based on relations between large textual objects, are inherently hard to visualise without either cluttering the display and/or confusing the annotators. TIARA deals with the visual complexity during the annotation process by systematically simplifying the layout, and by offering interactive visualisation, including coloured links, indentation, and dual-view. TIARA’s text view allows annotators to focus on the analysis of logical sequencing between sentences. A separate tree view allows them to review their analysis in terms of the overall discourse structure. The dual-view gives it an edge over other discourse annotation tools and makes it particularly attractive as an educational tool (e.g., for teaching students how to argue more effectively). As it is based on standard web technologies and can be easily customised to other annotation schemes, it can be easily used by anybody. Apart from the project it was originally designed for, in which hundreds of texts were annotated by three annotators, TIARA has already been adopted by a second discourse annotation study, which uses it in the teaching of argumentation.

pdf bib
A Corpus of Very Short Scientific Summaries
Yifan Chen | Tamara Polajnar | Colin Batchelor | Simone Teufel
Proceedings of the 24th Conference on Computational Natural Language Learning

We present a new summarisation task, taking scientific articles and producing journal table-of-contents entries in the chemistry domain. These are one- or two-sentence author-written summaries that present the key findings of a paper. This is a first look at this summarisation task with an open access publication corpus consisting of titles and abstracts, as input texts, and short author-written advertising blurbs, as the ground truth. We introduce the dataset and evaluate it with state-of-the-art summarisation methods.

pdf bib
Metaphor Detection using Context and Concreteness
Rowan Hall Maudslay | Tiago Pimentel | Ryan Cotterell | Simone Teufel
Proceedings of the Second Workshop on Figurative Language Processing

We report the results of our system on the Metaphor Detection Shared Task at the Second Workshop on Figurative Language Processing 2020. Our model is an ensemble, utilising contextualised and static distributional semantic representations, along with word-type concreteness ratings. Using these features, it predicts word metaphoricity with a deep multi-layer perceptron. We are able to best the state-of-the-art from the 2018 Shared Task by an average of 8.0% F1, and finish fourth in both sub-tasks in which we participate.

2019

pdf bib
It’s All in the Name: Mitigating Gender Bias with Name-Based Counterfactual Data Substitution
Rowan Hall Maudslay | Hila Gonen | Ryan Cotterell | Simone Teufel
Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)

This paper treats gender bias latent in word embeddings. Previous mitigation attempts rely on the operationalisation of gender bias as a projection over a linear subspace. An alternative approach is Counterfactual Data Augmentation (CDA), in which a corpus is duplicated and augmented to remove bias, e.g. by swapping all inherently-gendered words in the copy. We perform an empirical comparison of these approaches on the English Gigaword and Wikipedia, and find that whilst both successfully reduce direct bias and perform well in tasks which quantify embedding quality, CDA variants outperform projection-based methods at the task of drawing non-biased gender analogies by an average of 19% across both corpora. We propose two improvements to CDA: Counterfactual Data Substitution (CDS), a variant of CDA in which potentially biased text is randomly substituted to avoid duplication, and the Names Intervention, a novel name-pairing technique that vastly increases the number of words being treated. CDA/S with the Names Intervention is the only approach which is able to mitigate indirect gender bias: following debiasing, previously biased words are significantly less clustered according to gender (cluster purity is reduced by 49%), thus improving on the state-of-the-art for bias mitigation.

2018

pdf bib
Variable Typing: Assigning Meaning to Variables in Mathematical Text
Yiannos Stathopoulos | Simon Baker | Marek Rei | Simone Teufel
Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)

Information about the meaning of mathematical variables in text is useful in NLP/IR tasks such as symbol disambiguation, topic modeling and mathematical information retrieval (MIR). We introduce variable typing, the task of assigning one mathematical type (multi-word technical terms referring to mathematical concepts) to each variable in a sentence of mathematical text. As part of this work, we also introduce a new annotated data set composed of 33,524 data points extracted from scientific documents published on arXiv. Our intrinsic evaluation demonstrates that our data set is sufficient to successfully train and evaluate current classifiers from three different model architectures. The best performing model is evaluated on an extrinsic task: MIR, by producing a typed formula index. Our results show that the best performing MIR models make use of our typed index, compared to a formula index only containing raw symbols, thereby demonstrating the usefulness of variable typing.

2017

pdf bib
Annotation of argument structure in Japanese legal documents
Hiroaki Yamada | Simone Teufel | Takenobu Tokunaga
Proceedings of the 4th Workshop on Argument Mining

We propose a method for the annotation of Japanese civil judgment documents, with the purpose of creating flexible summaries of these. The first step, described in the current paper, concerns content selection, i.e., the question of which material should be extracted initially for the summary. In particular, we utilize the hierarchical argument structure of the judgment documents. Our main contributions are a) the design of an annotation scheme that stresses the connection between legal points (called issue topics) and argument structure, b) an adaptation of rhetorical status to suit the Japanese legal system and c) the definition of a linked argument structure based on legal sub-arguments. In this paper, we report agreement between two annotators on several aspects of the overall task.

2016

pdf bib
Unsupervised Timeline Generation for Wikipedia History Articles
Sandro Bauer | Simone Teufel
Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing

pdf bib
Improving Argument Overlap for Proposition-Based Summarisation
Yimai Fang | Simone Teufel
Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)

pdf bib
Solving the AL Chicken-and-Egg Corpus and Model Problem: Model-free Active Learning for Phenomena-driven Corpus Construction
Dain Kaplan | Neil Rubens | Simone Teufel | Takenobu Tokunaga
Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)

Active learning (AL) is often used in corpus construction (CC) for selecting “informative” documents for annotation. This is ideal for focusing annotation efforts when all documents cannot be annotated, but has the limitation that it is carried out in a closed-loop, selecting points that will improve an existing model. For phenomena-driven and exploratory CC, the lack of existing-models and specific task(s) for using it make traditional AL inapplicable. In this paper we propose a novel method for model-free AL utilising characteristics of phenomena for applying AL to select documents for annotation. The method can also supplement traditional closed-loop AL-based CC to extend the utility of the corpus created beyond a single task. We introduce our tool, MOVE, and show its potential with a real world case-study.

pdf bib
A Proposition-Based Abstractive Summariser
Yimai Fang | Haoyue Zhu | Ewa Muszyńska | Alexander Kuhnle | Simone Teufel
Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers

Abstractive summarisation is not yet common amongst today’s deployed and research systems. Most existing systems either extract sentences or compress individual sentences. In this paper, we present a summariser that works by a different paradigm. It is a further development of an existing summariser that has an incremental, proposition-based content selection process but lacks a natural language (NL) generator for the final output. Using an NL generator, we can now produce the summary text to directly reflect the selected propositions. Our evaluation compares textual quality of our system to the earlier preliminary output method, and also uses ROUGE to compare to various summarisers that use the traditional method of sentence extraction, followed by compression. Our results suggest that cutting out the middle-man of sentence extraction can lead to better abstractive summaries.

pdf bib
Mathematical Information Retrieval based on Type Embeddings and Query Expansion
Yiannos Stathopoulos | Simone Teufel
Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers

We present an approach to mathematical information retrieval (MIR) that exploits a special kind of technical terminology, referred to as a mathematical type. In this paper, we present and evaluate a type detection mechanism and show its positive effect on the retrieval of research-level mathematics. Our best model, which performs query expansion with a type-aware embedding space, strongly outperforms standard IR models with state-of-the-art query expansion (vector space-based and language modelling-based), on a relatively new corpus of research-level queries.

2015

pdf bib
Retrieval of Research-level Mathematical Information Needs: A Test Collection and Technical Terminology Experiment
Yiannos Stathopoulos | Simone Teufel
Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)

pdf bib
A Methodology for Evaluating Timeline Generation Algorithms based on Deep Semantic Units
Sandro Bauer | Simone Teufel
Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)

2014

pdf bib
Topical PageRank: A Model of Scientific Expertise for Bibliographic Search
James Jardine | Simone Teufel
Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics

pdf bib
A Summariser based on Human Memory Limitations and Lexical Competition
Yimai Fang | Simone Teufel
Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics

pdf bib
Resolving Coreferent and Associative Noun Phrases in Scientific Text
Ina Roesiger | Simone Teufel
Proceedings of the Student Research Workshop at the 14th Conference of the European Chapter of the Association for Computational Linguistics

pdf bib
Unsupervised learning of rhetorical structure with un-topic models
Diarmuid Ó Séaghdha | Simone Teufel
Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers

2013

pdf bib
Statistical Metaphor Processing
Ekaterina Shutova | Simone Teufel | Anna Korhonen
Computational Linguistics, Volume 39, Issue 2 - June 2013

2012

pdf bib
Detection of Implicit Citations for Sentiment Detection
Awais Athar | Simone Teufel
Proceedings of the Workshop on Detecting Structure in Scholarly Discourse

pdf bib
Context-Enhanced Citation Sentiment Detection
Awais Athar | Simone Teufel
Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

2010

pdf bib
Metaphor Corpus Annotated for Source - Target Domain Mappings
Ekaterina Shutova | Simone Teufel
Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC'10)

Besides making our thoughts more vivid and filling our communication with richer imagery, metaphor also plays an important structural role in our cognition. Although there is a consensus in the linguistics and NLP research communities that the phenomenon of metaphor is not restricted to similarity-based extensions of meanings of isolated words, but rather involves reconceptualization of a whole area of experience (target domain) in terms of another (source domain), there still has been no proposal for a comprehensive procedure for annotation of cross-domain mappings. However, a corpus annotated for conceptual mappings could provide a new starting point for both linguistic and cognitive experiments. The annotation scheme we present in this paper is a step towards filling this gap. We test our procedure in an experimental setting involving multiple annotators and estimate their agreement on the task. The associated corpus annotated for source ― target domain mappings will be publicly available.

pdf bib
Corpora for the Conceptualisation and Zoning of Scientific Papers
Maria Liakata | Simone Teufel | Advaith Siddharthan | Colin Batchelor
Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC'10)

We present two complementary annotation schemes for sentence based annotation of full scientific papers, CoreSC and AZ-II, applied to primary research articles in chemistry. AZ-II is the extension of AZ for chemistry papers. AZ has been shown to have been reliably annotated by independent human coders and useful for various information access tasks. Like AZ, AZ-II follows the rhetorical structure of a scientific paper and the knowledge claims made by the authors. The CoreSC scheme takes a different view of scientific papers, treating them as the humanly readable representations of scientific investigations. It seeks to retrieve the structure of the investigation from the paper as generic high-level Core Scientific Concepts (CoreSC). CoreSCs have been annotated by 16 chemistry experts over a total of 265 full papers in physical chemistry and biochemistry. We describe the differences and similarities between the two schemes in detail and present the two corpora produced using each scheme. There are 36 shared papers in the corpora, which allows us to quantitatively compare aspects of the annotation schemes. We show the correlation between the two schemes, their strengths and weeknesses and discuss the benefits of combining a rhetorical based analysis of the papers with a content-based one.

2009

pdf bib
Towards Domain-Independent Argumentative Zoning: Evidence from Chemistry and Computational Linguistics
Simone Teufel | Advaith Siddharthan | Colin Batchelor
Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing

pdf bib
Proceedings of the 2009 Workshop on Text and Citation Analysis for Scholarly Digital Libraries (NLPIR4DL)
Min-Yen Kan | Simone Teufel
Proceedings of the 2009 Workshop on Text and Citation Analysis for Scholarly Digital Libraries (NLPIR4DL)

2008

pdf bib
Language Resources and Chemical Informatics
C.J. Rupp | Ann Copestake | Peter Corbett | Peter Murray-Rust | Advaith Siddharthan | Simone Teufel | Benjamin Waldron
Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC'08)

Chemistry research papers are a primary source of information about chemistry, as in any scientific field. The presentation of the data is, predominantly, unstructured information, and so not immediately susceptible to processes developed within chemical informatics for carrying out chemistry research by information processing techniques. At one level, extracting the relevant information from research papers is a text mining task, requiring both extensive language resources and specialised knowledge of the subject domain. However, the papers also encode information about the way the research is conducted and the structure of the field itself. Applying language technology to research papers in chemistry can facilitate eScience on several different levels. The SciBorg project sets out to provide an extensive, analysed corpus of published chemistry research. This relies on the cooperation of several journal publishers to provide papers in an appropriate form. The work is carried out as a collaboration involving the Computer Laboratory, Chemistry Department and eScience Centre at Cambridge University, and is funded under the UK eScience programme.

pdf bib
Proceedings of ACL-08: HLT
Johanna D. Moore | Simone Teufel | James Allan | Sadaoki Furui
Proceedings of ACL-08: HLT

pdf bib
Proceedings of ACL-08: HLT, Short Papers
Johanna D. Moore | Simone Teufel | James Allan | Sadaoki Furui
Proceedings of ACL-08: HLT, Short Papers

2007

pdf bib
Whose Idea Was This, and Why Does it Matter? Attributing Scientific Work to Citations
Advaith Siddharthan | Simone Teufel
Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Conference

pdf bib
Annotation of Chemical Named Entities
Peter Corbett | Colin Batchelor | Simone Teufel
Biological, translational, and clinical language processing

pdf bib
Discourse Annotation Working Group Report
Manfred Stede | Janyce Wiebe | Eva Hajičová | Brian Reese | Simone Teufel | Bonnie Webber | Theresa Wilson
Proceedings of the Linguistic Annotation Workshop

2006

pdf bib
A Bootstrapping Approach to Unsupervised Detection of Cue Phrase Variants
Rashid M. Abdalla | Simone Teufel
Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics

pdf bib
Proceedings of the Workshop on Task-Focused Summarization and Question Answering
Tat-Seng Chua | Jade Goldstein | Simone Teufel | Lucy Vanderwende
Proceedings of the Workshop on Task-Focused Summarization and Question Answering

pdf bib
How to Find Better Index Terms Through Citations
Anna Ritchie | Simone Teufel | Stephen Robertson
Proceedings of the Workshop on How Can Computational Linguistics Improve Information Retrieval?

pdf bib
An annotation scheme for citation function
Simone Teufel | Advaith Siddharthan | Dan Tidhar
Proceedings of the 7th SIGdial Workshop on Discourse and Dialogue

pdf bib
Automatic classification of citation function
Simone Teufel | Advaith Siddharthan | Dan Tidhar
Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing

pdf bib
Creating a Test Collection for Citation-based IR Experiments
Anna Ritchie | Simone Teufel | Stephen Robertson
Proceedings of the Human Language Technology Conference of the NAACL, Main Conference

2004

pdf bib
Evaluating Information Content by Factoid Analysis: Human annotation and stability
Simone Teufel | Hans van Halteren
Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing

pdf bib
Agreement in Human Factoid Annotation for Summarization Evaluation
Simone Teufel | Hans van Halteren
Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC’04)

pdf bib
MEAD - A Platform for Multidocument Multilingual Text Summarization
Dragomir Radev | Timothy Allison | Sasha Blair-Goldensohn | John Blitzer | Arda Çelebi | Stanko Dimitrov | Elliott Drabek | Ali Hakim | Wai Lam | Danyu Liu | Jahna Otterbacher | Hong Qi | Horacio Saggion | Simone Teufel | Michael Topper | Adam Winkel | Zhu Zhang
Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC’04)

2003

pdf bib
Evaluation Challenges in Large-Scale Document Summarization
Dragomir R. Radev | Simone Teufel | Horacio Saggion | Wai Lam | John Blitzer | Hong Qi | Arda Çelebi | Danyu Liu | Elliott Drabek
Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics

pdf bib
Examining the consensus between human summaries: initial experiments with factoid analysis
Hans van Halteren | Simone Teufel
Proceedings of the HLT-NAACL 03 Text Summarization Workshop

2002

pdf bib
Collection and linguistic processing of a large-scale corpus of medical articles
Simone Teufel | Noemie Elhadad
Proceedings of the Third International Conference on Language Resources and Evaluation (LREC’02)

pdf bib
Developing Infrastructure for the Evaluation of Single and Multi-document Summarization Systems in a Cross-lingual Environment
Horacio Saggion | Dragomir Radev | Simone Teufel | Wai Lam | Stephanie M. Strassel
Proceedings of the Third International Conference on Language Resources and Evaluation (LREC’02)

pdf bib
Articles Summarizing Scientific Articles: Experiments with Relevance and Rhetorical Status
Simone Teufel | Marc Moens
Computational Linguistics, Volume 28, Number 4, December 2002

pdf bib
Meta-evaluation of Summaries in a Cross-lingual Environment using Content-based Metrics
Horacio Saggion | Dragomir Radev | Simone Teufel | Wai Lam
COLING 2002: The 19th International Conference on Computational Linguistics

2000

pdf bib
What’s Yours and What’s Mine: Determining Intellectual Attribution in Scientific Text
Simone Teufel | Marc Moens
2000 Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora

1999

pdf bib
Discourse-level argumentation in scientific articles: human and automatic annotation
Simone Teufel | Marc Moens
Towards Standards and Tools for Discourse Tagging

pdf bib
An annotation scheme for discourse-level argumentation in research articles
Simone Teufel | Jean Carletta | Marc Moens
Ninth Conference of the European Chapter of the Association for Computational Linguistics

1998

pdf bib
Meta-discourse markers and problem-structuring in scientific articles
Simone Teufel
Discourse Relations and Discourse Markers

1997

pdf bib
Towards resolution of bridging descriptions
Renata Vieira | Simone Teufel
35th Annual Meeting of the Association for Computational Linguistics and 8th Conference of the European Chapter of the Association for Computational Linguistics

pdf bib
Sentence extraction as a classification task
Simone Teufel
Intelligent Scalable Text Summarization

pdf bib
Resolving bridging references in unrestricted text
Massimo Poesio | Renata Vieira | Simone Teufel
Operational Factors in Practical, Robust Anaphora Resolution for Unrestricted Texts

1995

pdf bib
Corpus-based Method for Automatic Identification of Support Verbs for Nominalizations
Simone Teufel | Gregory Grefenstette
Seventh Conference of the European Chapter of the Association for Computational Linguistics