Joshua Reiss
Author directory2026
SPHERE: Automatic Music Upmixing via Audio Language Model Post-Training with Spatial Heuristics Rewards
Zixun Guo | Calvin Murdock | Sanjeel Parekh | W. Owen Brimijoin | Simon Dixon | Joshua Reiss | Ishwarya Ananthabhotla
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Zixun Guo | Calvin Murdock | Sanjeel Parekh | W. Owen Brimijoin | Simon Dixon | Joshua Reiss | Ishwarya Ananthabhotla
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
In this paper, we study the task of automatic music upmixing, wherein a system predicts spatial mixing parameters from a multi-stem recording. Different from existing methods that rely on task-specific music encoders, we approach this task via audio language model (ALM) post-training, leveraging rich representations from existing ALMs, which encode both music semantics and mixing knowledge. Specifically, we propose a post-training recipe that first employs rejection sampling SFT, followed by reinforcement learning (RL) with verifiable rewards (RLVR) via GRPO. We propose Sphere (Spatial Heuristic Rewards), a deterministic reward suite inspired by music mixing conventions, to guide our post-training. It consists of 6 perceptually-motivated sub-rewards and encourages the output mix to be centered, balanced and spacious. More broadly, our results suggest that expert domain knowledge can be encoded as verifiable rewards and distilled into language models, without task-specific architectures.
2023
Identifying Visual Depictions of Animate Entities in Narrative Comics: An Annotation Study
Lauren Edlin | Joshua Reiss
Proceedings of the 5th Workshop on Narrative Understanding
Lauren Edlin | Joshua Reiss
Proceedings of the 5th Workshop on Narrative Understanding
Animate entities in narrative comics stories are expressed through a number of visual representations across panels. Identifying these entities is necessary for recognizing characters and analysing narrative affordances unique to comics, and integrating these with linguistic reference annotation, however an annotation process for animate entity identification has not received adequate attention. This research explores methods for identifying animate entities visually in comics using annotation experiments. Two rounds of inter-annotator agreement experiments are run: the first asks annotators to outline areas on comic pages using a Polygon segmentation tool, and the second prompts annotators to assign each outlined entity’s animacy type to derive a quantitative measure of agreement. The first experiment results show that Polygon-based outlines successfully produce a qualitative measure of agreement; the second experiment supports that animacy status is best conceptualised as a graded, rather than binary, concept.