Large language model (LLM) agents are increasingly used for infrastructure operations on Kubernetes, yet how to test them efficiently and accurately remains unresolved. Realistic long-horizon evaluations are time-consuming and sometimes error-prone to build, since test cases are usually written from scratch even though components such as setup, state management, and verification logic are duplicated across them. We present KARMA, an open-source framework that composes reusable, parameterized operational cases into stateful, multi-stage Kubernetes workflows. Each case textually specifies a task and a required cluster state, and contains a deterministic oracle. The cluster state is preserved across stages, so long-horizon tasks can be constructed with little additional effort. KARMA also provides controls that add further complexity, namely alternating prompt modes and adversary injection. For cross-stage testing, a regression sweep reruns all previous cases and an LLM judge adjudicates the results, detecting execution inconsistencies or conflicts introduced by agents. Alongside the framework, we release a corpus of 89 cases, 38 adversaries, and 327 workflows across eight systems.