Halil Kilicoglu


pdf bib
Examining the Causal Impact of First Names on Language Models: The Case of Social Commonsense Reasoning
Sullam Jeoung | Jana Diesner | Halil Kilicoglu
Proceedings of the 3rd Workshop on Trustworthy Natural Language Processing (TrustNLP 2023)

As language models continue to be integrated into applications of personal and societal relevance, ensuring these models’ trustworthiness is crucial, particularly with respect to producing consistent outputs regardless of sensitive attributes. Given that first names may serve as proxies for (intersectional) socio-demographic representations, it is imperative to examine the impact of first names on commonsense reasoning capabilities. In this paper, we study whether a model’s reasoning given a specific input differs based on the first names provided. Our underlying assumption is that the reasoning about Alice should not differ from the reasoning about James. We propose and implement a controlled experimental framework to measure the causal effect of first names on commonsense reasoning, enabling us to distinguish between model predictions due to chance and caused by actual factors of interest. Our results indicate that the frequency of first names has a direct effect on model prediction, with less frequent names yielding divergent predictions compared to more frequent names. To gain insights into the internal mechanisms of models that are contributing to these behaviors, we also conduct an in-depth explainable analysis. Overall, our findings suggest that to ensure model robustness, it is essential to augment datasets with more diverse first names during the configuration stage.


pdf bib
UIUC_BioNLP at SemEval-2021 Task 11: A Cascade of Neural Models for Structuring Scholarly NLP Contributions
Haoyang Liu | M. Janina Sarol | Halil Kilicoglu
Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021)

We propose a cascade of neural models that performs sentence classification, phrase recognition, and triple extraction to automatically structure the scholarly contributions of NLP publications. To identify the most important contribution sentences in a paper, we used a BERT-based classifier with positional features (Subtask 1). A BERT-CRF model was used to recognize and characterize relevant phrases in contribution sentences (Subtask 2). We categorized the triples into several types based on whether and how their elements were expressed in text, and addressed each type using separate BERT-based classifiers as well as rules (Subtask 3). Our system was officially ranked second in Phase 1 evaluation and first in both parts of Phase 2 evaluation. After fixing a submission error in Pharse 1, our approach yields the best results overall. In this paper, in addition to a system description, we also provide further analysis of our results, highlighting its strengths and limitations. We make our code publicly available at https://github.com/Liu-Hy/nlp-contrib-graph.


pdf bib
TextFlow: A Text Similarity Measure based on Continuous Sequences
Yassine Mrabet | Halil Kilicoglu | Dina Demner-Fushman
Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

Text similarity measures are used in multiple tasks such as plagiarism detection, information ranking and recognition of paraphrases and textual entailment. While recent advances in deep learning highlighted the relevance of sequential models in natural language generation, existing similarity measures do not fully exploit the sequential nature of language. Examples of such similarity measures include n-grams and skip-grams overlap which rely on distinct slices of the input texts. In this paper we present a novel text similarity measure inspired from a common representation in DNA sequence alignment algorithms. The new measure, called TextFlow, represents input text pairs as continuous curves and uses both the actual position of the words and sequence matching to compute the similarity value. Our experiments on 8 different datasets show very encouraging results in paraphrase detection, textual entailment recognition and ranking relevance.


pdf bib
Annotating Named Entities in Consumer Health Questions
Halil Kilicoglu | Asma Ben Abacha | Yassine Mrabet | Kirk Roberts | Laritza Rodriguez | Sonya Shooshan | Dina Demner-Fushman
Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)

We describe a corpus of consumer health questions annotated with named entities. The corpus consists of 1548 de-identified questions about diseases and drugs, written in English. We defined 15 broad categories of biomedical named entities for annotation. A pilot annotation phase in which a small portion of the corpus was double-annotated by four annotators was followed by a main phase in which double annotation was carried out by six annotators, and a reconciliation phase in which all annotations were reconciled by an expert. We conducted the annotation in two modes, manual and assisted, to assess the effect of automatic pre-annotation and calculated inter-annotator agreement. We obtained moderate inter-annotator agreement; assisted annotation yielded slightly better agreement and fewer missed annotations than manual annotation. Due to complex nature of biomedical entities, we paid particular attention to nested entities for which we obtained slightly lower inter-annotator agreement, confirming that annotating nested entities is somewhat more challenging. To our knowledge, the corpus is the first of its kind for consumer health text and is publicly available.

pdf bib
Inferring Implicit Causal Relationships in Biomedical Literature
Halil Kilicoglu
Proceedings of the 15th Workshop on Biomedical Natural Language Processing

pdf bib
Aligning Texts and Knowledge Bases with Semantic Sentence Simplification
Yassine Mrabet | Pavlos Vougiouklis | Halil Kilicoglu | Claire Gardent | Dina Demner-Fushman | Jonathon Hare | Elena Simperl
Proceedings of the 2nd International Workshop on Natural Language Generation and the Semantic Web (WebNLG 2016)


pdf bib
A Compositional Interpretation of Biomedical Event Factuality
Halil Kilicoglu | Graciela Rosemblat | Michael Cairelli | Thomas Rindflesch
Proceedings of the Second Workshop on Extra-Propositional Aspects of Meaning in Computational Semantics (ExProM 2015)


pdf bib
Decomposing Consumer Health Questions
Kirk Roberts | Halil Kilicoglu | Marcelo Fiszman | Dina Demner-Fushman
Proceedings of BioNLP 2014

pdf bib
Coreference Resolution for Structured Drug Product Labels
Halil Kilicoglu | Dina Demner-Fushman
Proceedings of BioNLP 2014

pdf bib
Annotating Question Decomposition on Complex Medical Questions
Kirk Roberts | Kate Masterton | Marcelo Fiszman | Halil Kilicoglu | Dina Demner-Fushman
Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)

This paper presents a method for annotating question decomposition on complex medical questions. The annotations cover multiple syntactic ways that questions can be decomposed, including separating independent clauses as well as recognizing coordinations and exemplifications. We annotate a corpus of 1,467 multi-sentence consumer health questions about genetic and rare diseases. Furthermore, we label two additional medical-specific annotations: (1) background sentences are annotated with a number of medical categories such as symptoms, treatments, and family history, and (2) the central focus of the complex question (a disease) is marked. We present simple baseline results for automatic classification of these annotations, demonstrating the challenging but important nature of this task.


pdf bib
Interpreting Consumer Health Questions: The Role of Anaphora and Ellipsis
Halil Kilicoglu | Marcelo Fiszman | Dina Demner-Fushman
Proceedings of the 2013 Workshop on Biomedical Natural Language Processing


pdf bib
Adapting a General Semantic Interpretation Approach to Biological Event Extraction
Halil Kilicoglu | Sabine Bergler
Proceedings of BioNLP Shared Task 2011 Workshop


pdf bib
Arguments of Nominals in Semantic Interpretation of Biomedical Text
Halil Kilicoglu | Marcelo Fiszman | Graciela Rosemblat | Sean Marimpietri | Thomas Rindflesch
Proceedings of the 2010 Workshop on Biomedical Natural Language Processing

pdf bib
A High-Precision Approach to Detecting Hedges and their Scopes
Halil Kilicoglu | Sabine Bergler
Proceedings of the Fourteenth Conference on Computational Natural Language Learning – Shared Task


pdf bib
Syntactic Dependency Based Heuristics for Biological Event Extraction
Halil Kilicoglu | Sabine Bergler
Proceedings of the BioNLP 2009 Workshop Companion Volume for Shared Task


pdf bib
Recognizing Speculative Language in Biomedical Research Articles: A Linguistically Motivated Perspective
Halil Kilicoglu | Sabine Bergler
Proceedings of the Workshop on Current Trends in Biomedical Natural Language Processing


pdf bib
Abstraction Summarization for Managing the Biomedical Research Literature
Marcelo Fiszman | Thomas C. Rindflesch | Halil Kilicoglu
Proceedings of the Computational Lexical Semantics Workshop at HLT-NAACL 2004

pdf bib
Using Natural Language Processing, LocusLink and the Gene Ontology to Compare OMIM to MEDLINE
Bisharah Libbus | Halil Kilicoglu | Thomas C. Rindflesch | James G. Mork | Alan R. Aronson
HLT-NAACL 2004 Workshop: Linking Biological Literature, Ontologies and Databases