This study investigates Large Language Model (LLM) surprisal as an indicator of global linguistic naturalness in L2 Japanese automatic essay scoring. Results demonstrate that surprisal effectively distinguishes learner proficiency levels. Combining surprisal with feature-based indices achieves the highest classification accuracy, validating surprisal as a valuable metric for automated writing evaluation.