Tabular data derives its value from inter-feature dependencies, yet preserving them during synthesis is fragile when samples are scarce. Existing approaches either learn dependencies implicitly through distribution fitting, rely on statistical graph learning that becomes unstable with few samples, or encode structure through flat text serialization. Recent graph-aware methods use dependency graphs as attention biases or as backbones for non-LLM samplers, but they do not use the graph as a prompt-level plan for organizing the LLM’s own generation process. We introduce StructSynth, a framework that treats a dependency graph as a generation plan—determining the generation order, conditioning context, and scope of each black-box LLM call. In Evidence-Grounded Graph Induction, LLM reasoning and statistical association cues jointly construct a Directed Acyclic Graph (DAG) from limited samples. In Graph-Planned Conditional Synthesis, this DAG drives autoregressive synthesis in topological order, conditioning each feature on previously generated values with graph-specified parent structure guiding each step, making the conditioning schedule explicit throughout synthesis. Experiments show that StructSynth achieves state-of-the-art downstream utility and the best privacy-risk ranking among fourteen compared generators in low-data settings.

Sparse Autoencoders (SAEs) are a prominent tool in mechanistic interpretability (MI) for decomposing neural network activations into interpretable features. However, the aspiration to identify a canonical set of features is challenged by the observed inconsistency of learned SAE features across different training runs, undermining reproducibility and complicating model comparison. We study run-to-run feature consistency in SAEs and argue that it should be reported as a standard evaluation axis alongside reconstruction and sparsity. We propose the Pairwise Dictionary Mean Correlation Coefficient (PW-MCC) as an assignment-based metric to quantify consistency and demonstrate that high levels are achievable (PW-MCC ≈ 0.80 for TopK SAEs on LLM activations) with appropriate architectural choices.Our contributions include: (i) theoretical grounding for strong consistency in the idealized setting of TopK SAEs; (ii) synthetic validation using a model organism, which verifies PW-MCC as a reliable proxy for ground-truth recovery; and (iii) empirical analysis on LLM activations, where PW-MCC correlates with the similarity of automatically generated natural-language feature explanations.