As large language models (LLMs) are increasingly deployed globally, aligning them with pluralistic human values remains a challenge, as current efforts are predominantly Western-focused. To bridge this gap, we introduce SEAPlur, a dataset designed to benchmark and improve LLM alignment with the pluralistic values of four Southeast Asian (SEA) countries: Indonesia, Laos, Myanmar, and Vietnam. SEAPlur consists of independent datasets for each country and is constructed in their respective native, non-English languages. Sourced from local news and structured around the Schwartz’s theory of basic human values, SEAPlur captures local human agreement distributions across complex social dilemmas. In total, the dataset comprises 4,000 curated scenarios. For evaluation, we split 250 scenarios for each country resulting in SEAPlur-Test. The remaining data with "Agree" and "Disagree" annotations is transformed into a SEAPlur-Train 7,460 instruction-response pairs. Our evaluations using SEAPlur-Test reveal that general API LLMs struggle with SEA values, achieving peak overall accuracy (OA) scores of only 0.50. In contrast, region-specific LLMs perform better, reaching OA scores up to 0.58. Furthermore, supervised fine-tuning (SFT) using SEAPlur-Train significantly improves alignment. However, this process reveals a critical safety tension: enhancing regional alignment in certain LLMs may trigger unsafe behaviors. The dataset is publicly available at https://github.com/tjunlp-lab/SEAPlur.