In Japanese–English machine-translation,especially for creative and localized content, standard MT metrics typically summarize translation quality as scalar scores, but provide little diagnostic information about how style changes in translation. This is limiting because politeness, directness, interpersonal stance, and character voice are encoded differently across Japanese and English, and stylistic change may reflect localization-oriented adaptation rather than error. We propose the Style Diagnostic Profile (SDP) framework: SDP assigns each utterance a seven-dimensional bipolar style vector, and 𝛥sdp, the difference between target and source SDP vectors, represents translation-induced stylistic change as a multi-axis shift vector. To evaluate the framework, we first validate an LLM-based SDP scorer against bilingual human annotations on professional JA→EN subtitles and show that it approximates human style judgments. Using Style25, a controlled JA↔EN style-perturbation benchmark, we demonstrate that different perturbation categories occupy distinct regions in 𝛥sdp space. As a case study, we analyze professional subtitles and observe stylistic adaptation rather than simple style copying. These results position the SDP framework as a diagnostic layer complementary to adequacy-oriented MT evaluation.

Over the years, scalar MT metrics have advanced rapidly on benchmarks. Yet they remain black boxes, offering little insight into their decisions and sometimes degrading under out-of-distribution inputs. We introduce Remedy-R, a reasoning-driven generative MT metric trained with reinforcement learning from pairwise translation preferences, without requiring error-span annotations or distillation from closed LLMs. Unlike scalar MT metrics that only outputs translation quality scores, Remedy-R produces step-by-step analyses of accuracy, fluency, and completeness, enabling more interpretable assessments. With only 60K pairwise training samples across two language pairs, Remedy-R remains competitive with top scalar metrics and GPT-4-based judges on WMT22–24 metric benchmarks, generalizes to other languages, and shows strong robustness on OOD stress tests. Moreover, Remedy-R generates self-reflective feedback that can be reused for translation refinement. We validate the faithfulness of such feedback with GPT-4 and show that a simple evaluate–revise pipeline leveraging Remedy-R’s analyses consistently improves translation quality across diverse models without any task-specific tuning.

LLM-based machine translation systems can now do more than translate a source segment in isolation. At test time, they can translate directly, prepend previous context, or refine a context-aware draft. However, these interventions are not uniformly beneficial. Context can resolve discourse-sensitive choices, but irrelevant context and extra refinement increases cost and may perturb adequate translations. We introduce ContextRouter, a lightweight policy router that learns which intervention to apply for each source segment. During training, we first generate candidates for all contextual interventions and score them with a routing objective combining overall translation quality with incurred token costs. We then train our ContextRouter with group-relative supervision, so it learns within-example preferences rather than absolute global action scores. At inference, our router uses only test-time-available signals from the source, previous source-side context, and a no-context draft to determine and execute only the selected context intervention. On a filtered context-aware subset of TED, QED, and OpenSubtitles covering multiple languages and domains, our ContextRouter improves the quality-cost tradeoff over fixed no-context, always-context, always-refinement, QE-reranking, and score-regression policies, and human evaluation confirms that annotators prefer its chosen interventions. These results support treating context and refinement actions in LLM-MT not as always present defaults but conditional test-time interventions instead.

While large language models (LLMs) excel at machine translation (MT), the impact of how LLMs utilize different forms of contextual information on discourse-level phenomena remains underexplored. We systematically investigate how different forms of context such as prior source sentences, models’ generated hypotheses, and reference translations influence standard MT metrics and specific discourse phenomena (formality, pronoun selection, and lexical cohesion). Evaluating multiple LLMs across multiple domains and language pairs, our findings consistently show that context boosts both translation and discourse-specific performance. Notably, the context strategy of combining source text with the model’s own prior hypotheses effectively improves discourse consistency without gold references, demonstrating effective use of model’s own imperfect generations as diverse contextual cues.

Building open-domain dialogue systems capable of rich human-like conversational ability is one of the fundamental challenges in language generation. However, even with recent advancements in the field, existing open-domain generative models fail to capture and utilize external knowledge, leading to repetitive or generic responses to unseen utterances. Current work on knowledge-grounded dialogue generation primarily focuses on persona incorporation or searching a fact-based structured knowledge source such as Wikipedia. Our method takes a broader and simpler approach, which aims to improve the raw conversation ability of the system by mimicking the human response behavior through casual interactions found on social media. Utilizing a joint retriever-generator setup, the model queries a large set of filtered comment data from Reddit to act as additional context for the seq2seq generator. Automatic and human evaluations on open-domain dialogue datasets demonstrate the effectiveness of our approach.