Structure-Guided Approaches for Robust Language Model Reasoning

Abstract

Large Language Models exhibit increasingly sophisticated behaviors, yet we lack systematic methods to evaluate their true reasoning capabilities. Current approaches—such as collecting human-generated data—cannot reliably predict model failure modes, distinguish genuine reasoning from complex memorization patterns, nor guarantee synthetic data quality.This thesis explores the idea that imbuing tasks with explicit structure provides the control and verification mechanisms necessary for more principled evaluation and reliable data generation. Structure unlocks optimization and search algorithms that can systematically discover model flaws while supporting both higher-quality training data generation and targeted inference-time improvements through more interpretable methods. This creates a virtuous cycle where better evaluation can lead to stronger modeling. Throughout four works, I demonstrate this approach across degrees of structural abstraction, ranging from surface-level to semantic-level control. Applications span from prompt formatting sensitivity and compositional reasoning analysis, to theory of mind reasoning improvement and evaluation. Crucially, it is this explicit structure that unlocks diverse techniques including multi-armed bandits for exploration, A* search for adversarial discovery of challenging data, and custom graph algorithms for reasoning representation and mental state modeling. By simultaneously showing successful applications for improving evaluation rigor, data quality, and overall model performance, structure-guided approaches offer a path toward more reliable and capable language modeling.