LLM-based assistants have become ubiquitous, yet information security awareness (ISA) remains an underexplored aspect of LLM safety. ISA encompasses not only security knowledge, but also the attitudes and behaviors required to recognize implicit security risks and avoid prioritizing user satisfaction over secure guidance. We introduce LISAA, a framework for evaluating LLM ISA under realistic end-user interactions. The framework includes a dataset of 100 realistic scenarios spanning all criteria of an ISA taxonomy, each with a tension between implicit security implications and user satisfaction. We further introduce methods for systematically refining realistic ISA scenarios from authentic user queries and for selecting reliable LLM judges without requiring large annotated datasets. Robustness analyses show that LISAA rankings remain stable under scenario resampling and stochastic generation, while construct-validity experiments demonstrate sensitivity to changes in security behavior. Applying LISAA to leading LLMs reveals that many widely deployed models still exhibit medium-to-low ISA levels, and that strong performance on traditional cybersecurity knowledge benchmarks does not necessarily translate to high ISA. In addition, the framework enables comparative analyses across model families, revealing consistent trends such as higher ISA in newer and larger variants, although meaningful gaps persist even among recent models. We release the dataset, evaluation pipeline, source code, and an online tool for evaluating new models using the LISAA framework.