Multimodal behavioral analysis is typically studied using separate task-specific corpora. Practical applications, however, would benefit from unified models that can support several behavioral predictions despite fragmented supervision. We study this problem as disjoint-corpus multimodal multi-task learning across emotion recognition on CMU-MOSEI, apparent Big Five personality assessment on ChaLearn FIv2, and ambivalence recognition on BAH. We propose Task-Aware Cross-Modal Experts (TACME), a multimodal fusion model with a shared bank of directed cross-modal Transformer experts and task-specific expert gates. Each expert models one ordered modality interaction, while task-specific gates select different expert mixtures for different behavioral tasks. We further adapt feature- and prediction-level flow-matching regularization to this disjoint-corpus behavioral setting and propose a Progress-Aware Gradient Balancing (PAGB) multi-task learning strategy for heterogeneous task optimization. Across three independent corpora and six task-specific metrics, TACME with PAGB and feature- and prediction-level flow-matching regularization achieves the best average rank of 1.00 among the compared multimodal fusion models. Although single-task TACME is stronger on the source-corpus metrics, the multi-task TACME model improves zero-shot unweighted average recall (UAR) on the unseen MELD test set from 28.9% to 35.9%, demonstrating improved generalization to unseen data.