Adaptation of black-box large language models (LLMs) at test time seeks to adjust LLMs to target domains in order to handle distribution shifts during testing, without access to model parameters. Most existing approaches rely on adapters trained with substantial target-domain data, which is often scarce in practical scenarios, and these adapters are tightly coupled to their training distribution, leading to degraded performance in dynamic scenarios with changing domains. To solve this, we propose LADA, a framework leveraging meta-training for continuous and rapid adaptation of black-box LLMs to unseen and changing domains. Specifically, LADA meta-trains an adapter on diverse tasks that cover multiple datasets and error combinations to learn transferable adaptation skills. At test time, the adapter needs only a few target-domain examples for lightweight adaptation and guides the LLM stepwise toward domain-appropriate reasoning via adaptive selection of reliable steps, without any updates to the LLM parameters. Experiments on datasets spanning multiple domains validate the effectiveness of the proposed approach.