Membership inference attacks (MIAs) assess whether a sample was used to train a model, posing privacy risks for large language models (LLMs). Existing MIAs typically rely on single loss or confidence signals, which are often unstable and easily confounded by noise, leading to limited robustness and generalization. In this paper, we propose Group-Invariant Membership Inference Attack (GI-MIA) that explicitly exploits semantic-preserving transformations to capture stability differences between members and non-members. Our method constructs an orbit of paraphrased variants for each input using LLM-driven transformations, and extracts invariant statistics (mean, maximum, and variance of normalized losses) across the orbit. A simple decision rule based on these statistics effectively discriminates members from non-members by leveraging the model’s inherent invariance on training data. We conducted extensive experiments on the Pythia family of LLMs (different model parameter sizes) across six datasets. Results show that GI-MIA consistently outperforms strong baselines, achieving state-of-the-art performance while maintaining robustness against semantic-preserving transformations. Our study highlights new directions for MIAs and calls for further research into LLM privacy protection, emphasizing the need for developing reliable models with better privacy resilience.