2026-09-18T16:29:12Zhttps://digital.lib.washington.edu/server/oai/request

oai:digital.lib.washington.edu:1773/573222026-08-12T10:00:40Zcom_1773_4888col_1773_49166

Shlizerman, Eli Zheng, Caleb 2026-08-11T19:28:43Z 2026-08-11 2026 Zheng_washington_0250E_29815.pdf https://hdl.handle.net/1773/57322 Thesis (Ph.D.)--University of Washington, 2026 Neural networks are powerful but difficult to reason about. Understanding why a model is stable, why it is trained effectively, or how it is being compressed without performance degradation requires tools that go beyond inspecting weights or monitoring loss curves. Linearization, which characterizes how a nonlinear system responds to small perturbations, offers a principled foundation for these questions. While a well known tool in fields such as control and dynamical systems, its potential as a unifying design and optimization language for neural networks has remained largely unexplored. This dissertation develops linearization as a dual framework for networks of neurons, addressing two complementary challenges: 1) Stability and hyperparameter sensitivity of Recurrent Neural Networks (RNNs), and 2) Performance recovery of compressed generative diffusion models. My thesis introduces the following contributions to the above goals. For RNNs, I use linearization as a design target. By shaping the eigenvalue spectrum of the recurrent Jacobian at initialization, I proposed to obtain networks whose dynamics remain stable across a broad range of regimes, making FORCE-based training robust to network configuration rather than confined to a narrow edge-of-chaos band. Turning from design to analysis, I then show that the Lyapunov Exponent spectrum, a signature of a network's long-run dynamical behavior derived from its linearization, encodes rich information about model performance. I introduce a representation learning approach that maps these spectra into a compact latent space, enabling reliable cross-architecture performance prediction from early in training and substantially reducing the cost of hyperparameter search. Building on this, I further leverage the Lyapunov Spectrum distance as a more reliable proxy than loss-based metrics for guiding automated search over pruned RNN variants, consistently identifying better-performing configurations at lower computational cost. For diffusion models, I demonstrate that Jacobian matching, aligning the input-output derivatives between dense teacher and compressed student, provides an effective finetuning signal that standard training objectives do not exploit. Since na\"{i}ve Jacobian matching is incompatible with diffusion training's inherent noise injection, I develop a second-order extension inspired by finite-time Lyapunov exponents that resolves this incompatibility and recovers generation quality after pruning. Together, these contributions establish linearization as a versatile theoretical language for both analysis and deployment of neural networks. application/pdf en_US none Deep Learning Diffusion Model Generative Model Linearization Lyapunox Exponents Recurrect Neural Network Artificial intelligence Computer science Engineering Electrical and computer engineering Linearization of Neural Networks as a Dual Framework for Stability and Sparsity Thesis Open Access open.access