Video-to-audio (V2A) generation aims to produce a single soundtrack that is simultaneously semantically faithful, temporally synchronized, aesthetically natural, and—for stereo output—spatially accurate. These perceptual axes are judged on the same waveform yet often pull generation in locally conflicting directions, while the dominant post-training recipe—reinforcement learning (RL) from an aggregated scalar reward—supervises them only at the final waveform with one entangled signal that gives little indication of which axis failed or where, so competing axes interfere through a single advantage value. We present MosaicAudio, a V2A framework that replaces terminal scalar rewards with dense, per-axis supervision distilled on-policy. MosaicAudio first trains four specialist flow-matching teachers, one per perceptual axis, then consolidates them into a single student: the student samples its own denoising trajectories and, at each visited state, receives a full teacher velocity field routed to one axis at a time, so that incompatible corrections are not averaged together. A perceptual manifold anchor keeps the student on a high-fidelity audio prior and curbs reward hacking during alignment. On the in-domain VGGSound test set and the out-of-domain AudioCanvas benchmark, MosaicAudio achieves state-of-the-art performance among compared V2A systems across objective metrics and human ratings, with especially consistent gains in complex multi-event scenes.

Transformer models have been used in automatic speech recognition (ASR) successfully and yields state-of-the-art results. However, its performance is still affected by speaker mismatch between training and test data. Further finetuning a trained model with target speaker data is the most natural approach for adaptation, but it takes a lot of compute and may cause catastrophic forgetting to the existing speakers. In this work, we propose a unified speaker adaptation approach consisting of feature adaptation and model adaptation. For feature adaptation, we employ a speaker-aware persistent memory model which generalizes better to unseen test speakers by making use of speaker i-vectors to form a persistent memory. For model adaptation, we use a novel gradual pruning method to adapt to target speakers without changing the model architecture, which to the best of our knowledge, has never been explored in ASR. Specifically, we gradually prune less contributing parameters on model encoder to a certain sparsity level, and use the pruned parameters for adaptation, while freezing the unpruned parameters to keep the original model performance. We conduct experiments on the Librispeech dataset. Our proposed approach brings relative 2.74-6.52% word error rate (WER) reduction on general speaker adaptation. On target speaker adaptation, our method outperforms the baseline with up to 20.58% relative WER reduction, and surpasses the finetuning method by up to relative 2.54%. Besides, with extremely low-resource adaptation data (e.g., 1 utterance), our method could improve the WER by relative 6.53% with only a few epochs of training.

This work describes the En→De Alibaba speech translation system developed for the evaluation campaign of the International Workshop on Spoken Language Translation (IWSLT) 2018. In order to improve ASR performance, multiple ASR models including conventional and end-to-end models are built, then we apply model fusion in the final step. ASR pre and post-processing techniques such as speech segmentation, punctuation insertion, and sentence splitting are found to be very useful for MT. We also employed most techniques that have proven effective during the WMT 2018 evaluation, such as BPE, back translation, data selection, model ensembling and reranking. These ASR and MT techniques, combined, improve the speech translation quality significantly.