Shuaishuai Guo

Author directory

2026

Artificial intelligence (AI) assistants for blind and low-vision (BLV) users deliver scene descriptions through a serial auditory channel, often followed by spoken user queries. A central challenge is that scene-description pipelines typically prioritize visually or corpus-salient content, whereas users’ follow-up questions often depend on low-salience attributes that are omitted during generation or compression. To diagnose where this mismatch arises, we introduce an encoder-agnostic framework that decomposes scene-description systems into four stages: visual encoding, length control, content selection, and audio scheduling. Applying this framework to 200 blind-user-captured VizWiz images and 500 questions reveals a salience–relevance mismatch: salience-based compression preserves identity information but discards low-salience attributes needed for attribute-bound questions. Improving the visual-encoding stage by replacing BLIP-base with stronger open-weight MLLMs raises semantic retention but does not eliminate this mismatch. As a lightweight repair, we propose CA-Keyword, which explicitly preserves attribute terms and recovers color-question answerability by 0.23 while producing descriptions 37% shorter than the sentence/
To mitigate catastrophic forgetting in downstream continual learning (CL) for large language models (LLMs), existing methods typically constrain parameter updates or introduce task-specific adaptation modules. However, these methods often rely on explicit task boundaries during training, limiting their applicability to realistic task-free scenarios. In this paper, we propose a Fisher-guided unified (FiUni) framework for batch-level task detection and parameter-efficient continual adaptation. FiUni is motivated by a key observation about the Fisher information matrix (FIM) of pre-trained models: the orthogonality among the principal subspaces of its Kronecker-Factored Approximate Curvature (K-FAC) approximation, estimated from a small number of downstream task samples, can reflect the similarity between different tasks. Based on this observation, FiUni constructs FIM-derived frozen subspaces to guide low-rank adaptation (LoRA), while matching the Fisher principal subspace of each incoming batch window with historical subspaces. This enables FiUni to adaptively determine whether to reuse existing knowledge, expand a related subspace, or create a new subspace, dynamically balancing knowledge sharing and task isolation. Experiments show that FiUni can effectively infer latent batch-level task affiliations and achieve competitive performance against advanced task-aware CL methods with fewer trainable parameters.