Raymond Alavo
Author directory2026
CALIBER: Calibrated Assessment via LLM for Interview Behavioral Evaluation and Rating
Katherine Kappaz | Raymond Alavo | Sofia Calderon | Ahmed Abbasi
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: System Demonstrations
Katherine Kappaz | Raymond Alavo | Sofia Calderon | Ahmed Abbasi
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: System Demonstrations
We present **CALIBER**, an interactive dashboard for calibrated, LLM-assisted behavioral interview evaluation. The system is built around three core design principles. First, a structured **Annotate–Rubric–Rate (ARR)** workflow sequences each evaluation into span annotation, rubric consultation, and scoring, decomposing a holistic judgment into auditable steps. Second, LLM-generated STAR span highlights serve as **attention scaffolds** rather than scores: the model surfaces structural cues in the transcript while all final ratings remain the annotator’s responsibility. Third, a 21-step guided training tour with Behaviorally Anchored Rating Scales (BARS) brings every evaluator to a shared baseline before live annotation begins. On the backend, a shrinking-pool randomizer with atomic DynamoDB counters guarantees uniform item coverage across crowdworkers, and a rich telemetry stream captures click, mouse-movement, and timing data alongside every annotation. A user study (N=24) confirms that the ARR design achieves its goals: both AI-friendly and AI-averse annotators complete tasks with less effort under AI assistance, and alignment with a held-out LLM reference improves for all groups. We release CALIBER as an open-source toolkit together with a live demo and code.
Dash-M5H: An Interactive Dashboard for Multi-Modal, Multi-Model Mental Health Assessment
Raymond Alavo | Xinyuan Zhang | Gemza Ademaj | Junhui Cai | Hyeokhyen Kwon | Robert Cotes | Gari D. Clifford | Ahmed Abbasi
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations)
Raymond Alavo | Xinyuan Zhang | Gemza Ademaj | Junhui Cai | Hyeokhyen Kwon | Robert Cotes | Gari D. Clifford | Ahmed Abbasi
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations)
We present Dash-M5H, an interactive dashboard for multi-modal, multi-model mental health assessment that helps clinicians and researchers jointly inspect multimodal behavioral data with multi-model signal outputs of recorded clinical interviews. Guided by signal detection and integrated sensemaking theories, Dash-M5H synchronizes transcript text, audio, and facial behavior (action units and gaze) to support overview-to-detail evidence tracing; and it integrates extracted signals (e.g., sentiment and facial activity) with a clinically grounded VLM prediction pipeline that produces DSM-5-aligned depression predictions. Dash-M5H (https://dash-m5h.io) is implemented in a lightweight, browser-based stack (Quarto + Observable JS + D3), supports local data import and time-synced clinical annotation with export. We demonstrate Dash-M5H through a depression screening scenario, evaluate its note-taking and screening capabilities through a user experiment, and release a live demo (https://youtu.be/w3qCJ02k6bw) and code (https://github.com/nd-hal/M5H-Dashboard-VLM) to facilitate reproducible evaluation.