Multimodal behavioral analysis is typically studied using separate task-specific corpora. Practical applications, however, would benefit from unified models that can support several behavioral predictions despite fragmented supervision. We study this problem as disjoint-corpus multimodal multi-task learning across emotion recognition on CMU-MOSEI, apparent Big Five personality assessment on ChaLearn FIv2, and ambivalence recognition on BAH. We propose Task-Aware Cross-Modal Experts (TACME), a multimodal fusion model with a shared bank of directed cross-modal Transformer experts and task-specific expert gates. Each expert models one ordered modality interaction, while task-specific gates select different expert mixtures for different behavioral tasks. We further adapt feature- and prediction-level flow-matching regularization to this disjoint-corpus behavioral setting and propose a Progress-Aware Gradient Balancing (PAGB) multi-task learning strategy for heterogeneous task optimization. Across three independent corpora and six task-specific metrics, TACME with PAGB and feature- and prediction-level flow-matching regularization achieves the best average rank of 1.00 among the compared multimodal fusion models. Although single-task TACME is stronger on the source-corpus metrics, the multi-task TACME model improves zero-shot unweighted average recall (UAR) on the unseen MELD test set from 28.9% to 35.9%, demonstrating improved generalization to unseen data.

We present a new audio-visual speech corpus (RUSAVIC) recorded in a car environment and designed for noise-robust speech recognition. Our goal was to produce a speech corpus which is natural (recorded in real driving conditions), controlled (providing different SNR levels by windows open/closed, moving/parked vehicle, etc.), and adequate size (the amount of data is enough to train state-of-the-art NN approaches). We focus on the problem of audio-visual speech recognition: with the use of automated lip-reading to improve the performance of audio-based speech recognition in the presence of severe acoustic noise caused by road traffic. We also describe the equipment and procedures used to create RUSAVIC corpus. Data are collected in a synchronous way through several smartphones located at different angles and equipped with FullHD video camera and microphone. The corpus includes the recordings of 20 drivers with minimum of 10 recording sessions for each. Besides providing a detailed description of the dataset and its collection pipeline, we evaluate several popular audio and visual speech recognition methods and present a set of baseline recognition results. At the moment RUSAVIC is a unique audio-visual corpus for the Russian language that is recorded in-the-wild condition and we make it publicly available.