Native Language Identification (NLI) consists in predicting a person’s native language based on their second language production. In the field of text-based NLI, approaches based on Large Language Models (LLM) outperform previous approaches based on statistical classifiers trained on linguistic features. However, the unstructured nature of LLM textual output does not support the direct application of usual classifiers. To address this limitation, we explore the use of LLM to generate effective latent vector representations (embeddings) for NLI. Our experiments, across ten models ranging from 0.1B to 405B parameters, show that embeddings gathered from a frozen LLM coupled with a linear classifier match the performance of fine-tuned generative LLM. Furthermore, layer-depth analysis reveals that the first 40% of an LLM’s layers are sufficient to achieve full representational utility for NLI.