Embodied agents must reason and act from experience accumulated before future questions or goals are known, yet existing memory systems often treat such experience as flat text, retrieved clips, or static graphs. We introduce MEMORA, an embodied memory framework that forms persistent participant-specific world memory from continuous egocentric video. MEMORA instantiates a formation–consolidation lifecycle: online memory editing maintains typed stores for spatial context, object identity and state, and temporal action evidence, while offline consolidation abstracts repeated experience into reusable routines, habits, and preferences. We evaluate MEMORA with MEMORA-Bench, a 45-hour egocentric-video suite for retrospective memory faithfulness and prospective memory-grounded planning. Across four open-weight answer models, MEMORA improves over no-memory, flat-text, and graph-structured memory baselines under matched perception and retrieval. The gains are largest on out-of-distribution planning, where MEMORA improves robot-grounded planning score by up to , showing that formed memory can compose new goals from remembered procedures, objects, and preferences. A physical-robot demonstration further suggests that memory formed from human egocentric video can guide participant-grounded action without in-domain robot data.

We present a novel visual instruction tuning strategy to improve the zero-shot task generalization of multimodal large language models by building a firm text-only knowledge base. Existing work lacks sufficient experimentation on the importance of each modality in the instruction tuning stage, often using a majority of vision-language data while keeping text-only data limited and fixing mixtures of modalities. By incorporating diverse text-only data in the visual instruction tuning stage, we vary vision-language data in various controlled experiments to investigate the importance of modality in visual instruction tuning. Our comprehensive evaluation shows that the text-heavy instruction tuning approach is able to perform on par with traditional vision-heavy mixtures on both modalities across 12 general datasets while using as low as half the total training tokens. We find that simply increasing sufficiently diverse text-only data enables transfer of instruction following ability and domain knowledge across modalities while being more efficient than the vision-language approach.

Large Language Models encode behaviors like refusal within their activation space, but identifying these behaviors remains challenging. Existing methods depend on predefined refusal templates detectable in output tokens or manual review. We introduce COSMIC (Cosine Similarity Metrics for Inversion of Concepts), an automated framework for direction selection that optimally identifies steering directions and target layers using cosine similarity, entirely independent of output text. COSMIC achieves steering effectiveness comparable to prior work without any prior knowledge or assumptions of a model’s refusal behavior such as the use of certain refusal tokens. Additionally, COSMIC successfully identifies refusal directions in adversarial scenarios and models with weak safety alignment, demonstrating its robustness across diverse settings.