We discuss how human-led Evidence-Centered Design (ECD) can inform the development of AI-supported automated item generation (AIG), as well as how it can be used as a quality-control mechanism. In this particular application, we describe how we collaborated with Claude Sonnet 5 to produce kinematics items together with interactive tools through an AIG pipeline. A preliminary qualitative analysis of a small number of generated items suggested they have identifiable strengths and weaknesses, which we discuss in depth per item.