Interactive agents increasingly fail on long task processes involving user turns, tool calls, and evolving environment contexts. Existing agent debugging methods provide useful trace-level diagnosis, but generally return explanations for individual failed traces rather than recurring, fine-grained action-context patterns across runs. To offer such corpus-level analyses, we present AgentTAC (Temporal Action-Context), an automated framework for mining erroneous and generalizable temporal sequences of action and context properties from agent traces. AgentTAC first localizes all distinct errors in each involved agent run. Then, it constructs and maintains a dynamic pool of attributes over the involved tasks, actions, and contexts to summarize their intrinsic properties and contextual relationships. Finally, it extracts temporally contiguous attribute patterns anchored at the localized errors. The resulting error patterns describe failures as interpretable sequential combinations of context and actions, such as making decisions under a missing-constraint context or taking an action after an incompatible prior decision. Extensive evaluations of AgentTAC on multiple benchmarks and agents validate its capabilities in error localization and error pattern mining, and provide preliminary evidence for error-pattern-driven test-time repair.

Data annotation plays a crucial role in ensuring your named entity recognition (NER) projects are trained with the right information to learn from. Producing the most accurate labels is a challenge due to the complexity involved with annotation. Label inconsistency between multiple subsets of data annotation (e.g., training set and test set, or multiple training subsets) is an indicator of label mistakes. In this work, we present an empirical method to explore the relationship between label (in-)consistency and NER model performance. It can be used to validate the label consistency (or catches the inconsistency) in multiple sets of NER data annotation. In experiments, our method identified the label inconsistency of test data in SCIERC and CoNLL03 datasets (with 26.7% and 5.4% label mistakes). It validated the consistency in the corrected version of both datasets.

The training process of scientific NER models is commonly performed in two steps: i) Pre-training a language model by self-supervised tasks on huge data and ii) fine-tune training with small labelled data. The success of the strategy depends on the relevance between the data domains and between the tasks. However, gaps are found in practice when the target domains are specific and small. We propose a novel framework to introduce a “pre-fine tuning” step between pre-training and fine-tuning. It constructs a corpus by selecting sentences from unlabeled documents that are the most relevant with the labelled training data. Instead of predicting tokens in random spans, the pre-fine tuning task is to predict tokens in entity candidates identified by text mining methods. Pre-fine tuning is automatic and light-weight because the corpus size can be much smaller than pre-training data to achieve a better performance. Experiments on seven benchmarks demonstrate the effectiveness.

Condition is essential in scientific statement. Without the conditions (e.g., equipment, environment) that were precisely specified, facts (e.g., observations) in the statements may no longer be valid. Existing ScienceIE methods, which aim at extracting factual tuples from scientific text, do not consider the conditions. In this work, we propose a new sequence labeling framework (as well as a new tag schema) to jointly extract the fact and condition tuples from statement sentences. The framework has (1) a multi-output module to generate one or multiple tuples and (2) a multi-input module to feed in multiple types of signals as sequences. It improves F1 score relatively by 4.2% on BioNLP2013 and by 6.2% on a new bio-text dataset for tuple extraction.