Poli Nemkova

Author directory

2026

Self-reflective agents typically bundle monitoring language with a downstream control policy, making it difficult to identify why reflection changes behavior. We present a six-condition ablation that separates evidence re-exposure, diagnostic questions, uncertainty labels, differentiated follow-up actions, and diagnosis–action alignment. Our testbed is an LLM forecasting agent evaluated with Llama-3.3-70B on 310 armed-conflict escalation cases, plus supplementary GPT-4o and 12-country transfer probes. Diagnostic questions produce comparable F1 to free reflection (0.296 vs. 0.297), and adding a seven-label taxonomy while holding the action fixed changes F1 by only +0.008. The closest matched comparison for action differentiation is taxonomy-only versus random action assignment (F vs. D), which holds the typed monitor and taxonomy fixed while replacing the shared-action policy with randomly varying substantive actions: F1 rises from 0.304 to 0.338. Type-matched routing reaches 0.379, but we detect no significant advantage over random assignment (C vs. D, p=0.714). This underpowered contrast does not establish equivalence; it prevents assigning the gain specifically to matching. Brier scores remain flat and transfer is unreliable. Thus, richer reflective traces do not establish better agent control. Our results motivate controls matched for information, vocabulary, and actions when attributing gains in reflection pipelines and identify simple action perturbations as a necessary baseline for type-specific policies.
Agentic AI systems are increasingly proposed for social-good domains, often invoking the United Nations Sustainable Development Goals (SDGs). Yet such claims do not necessarily establish accountability to the communities these systems aim to serve. We present a structured survey of 112 papers on agentic AI for social good published between 2015 and 2026. Each paper is coded across eight dimensions using dual-LLM annotation (GPT-4o and Llama-3.3-70B) with stratified human validation on 50 papers.

We identify a moral-geographic asymmetry: contextual underspecification is concentrated where local context matters most. Overall, 82 of 112 papers (73%) specify no geographic context. Papers aligned with health or physical/ecological SDGs name a geography 37–40% of the time, compared with only 13% for institutional and social-policy SDGs, and 0% under strict coding that requires a concrete country, city, region, or population (𝜒2(2)=8.78, p=.012, Cramér’s V=.283; strict: 𝜒2(2)=7.60, p=.022, V=.263). The pattern is robust across seven of eight sensitivity analyses. SDG 16 is both the most-covered goal and the least geographically specified.

We interpret this pattern as consistent with a form of moral abstraction: institutional good is framed more universally despite its dependence on local political, legal, and cultural conditions. Only 28 papers (25%) report any real-world deployment or small-scale test. We detect no statistically significant differences in deployment rates across architecture, era, or geographic specificity. We conclude with five accountability gaps and a minimal reporting standard for more context-specific, participatory, and accountable research.
Timely and accurate situation awareness underpins effective decision-making in humanitarian response, conflict monitoring, and peacebuilding. Yet, synthesizing heterogeneous and rapidly evolving information from news, conflict databases, and economic indicators remains labor-intensive and delays critical interventions. We present a Retrieval-Augmented Generation (RAG) system that autonomously produces structured, evidence-backed situation awareness reports by integrating real-time data from GDELT, ACLED, ReliefWeb, and World Bank APIs. To rigorously assess report quality without ground-truth references, we introduce a three-level reference-free evaluation framework combining automated NLP metrics, expert review by United Nations crisis analysts, and scalable LLM-as-a-Judge assessment. In a multi-country study across 15 conflict-prone regions, our system generated coherent, relevant, and actionable reports, with UNDP analysts estimating a ~50% reduction in report preparation time. These findings demonstrate the feasibility of deploying RAG-based systems for peacebuilding and humanitarian operations and provide a reproducible framework for generating and evaluating AI-assisted situational intelligence at scale.
We investigate whether hate-speech intensity on public Telegram channels serves as a conflict signal, using 1,514 days of Arabic-language posts from Sudan aligned with ACLED conflict-event records. Applying a pretrained multilingual hate-speech classifier, we construct a daily intensity index and analyze its relationship to political violence using within-regime analysis and autocorrelation-robust moving-block bootstrap inference. Hate-speech intensity increases approximately fourfold over the months following war onset and remains elevated, but day-to-day fluctuations do not predict short-term changes in violence. Within the post-war period, the index is positively associated with explosions and remote violence (r=+0.37) and negatively associated with protests (r=-0.25) and riots (r=-0.30), with all three associations surviving multiple-comparison correction. These patterns also hold for continuous classifier-score measures, although the regime-level increase is closely intertwined with growth in posting volume and active-channel participation. We further show that pooling across conflict regimes inflates effect sizes, while naive inference on serially dependent time series overstates statistical significance. Hate-speech intensity should therefore be interpreted as a descriptive marker of the conflict-related Telegram environment rather than as an early-warning predictor of imminent violence.
Humanitarian organizations face a critical choice: invest in costly commercial APIs or rely on free open-weight models for multilingual human rights monitoring. While commercial systems offer reliability, open-weight alternatives lack empirical validation - especially for low-resource languages common in conflict zones. This paper presents the first systematic comparison of commercial and open-weight large language models (LLMs) for human-rights-violation detection across seven languages, quantifying the cost-reliability trade-off facing resource-constrained organizations. Across 78,000 multilingual inferences, we evaluate six models - four instruction-aligned (Claude-Sonnet-4, DeepSeek-V3, Gemini-Flash-2.0, GPT-4.1-mini) and two open-weight (LLaMA-3-8B, Mistral-7B) - using both standard classification metrics and new measures of cross-lingual reliability: Calibration Deviation (CD), Decision Bias (ΔBias), Language Robustness Score (LRS), and Language Stability Score (LSS). Results show that alignment, not scale, determines stability: aligned models maintain near-invariant accuracy and balanced calibration across typologically distant and low-resource languages (e.g., Lingala, Burmese), while open-weight models exhibit significant prompt-language sensitivity and calibration drift. These findings demonstrate that multilingual alignment enables language-agnostic reasoning and provide practical guidance for humanitarian organizations balancing budget constraints with reliability in multilingual deployment.
Natural language processing (NLP) now shapes many aspects of our world, yet its potential for positive social impact is underexplored. This paper surveys work in “NLP for Social Good" (NLP4SG) across nine domains relevant to global development and risk agendas, summarizing principal tasks and challenges. We analyze ACL Anthology trends, finding that inclusion and AI harms attract the most research, while domains such as poverty, peacebuilding, and environmental protection remain underexplored. Guided by our review, we outline opportunities for responsible and equitable NLP and conclude with a call for cross-disciplinary partnerships and human-centered approaches to ensure that future NLP technologies advance the public good.