Advancing Time Series Forecasting: Insights from Deep Learning and Dynamic Mode Decomposition

Abstract

Time series forecasting presents significant challenges across engineering and scientific disciplines, particularly in handling non-stationary real-world data and providing real-time predictions from streaming sources. Deep learning approaches, including Recurrent Neural Networks (RNNs), Convolutional Neural Networks (CNNs), and Transformer-based models, have advanced the field but often fall short in interpretability, computational efficiency, and real-time adaptability. Despite their capacity for modeling complex non-linear dynamics, these models require extensive hyperparameter tuning and lack robust mechanisms for incremental updates. They also suffer from catastrophic forgetting in streaming scenarios, limiting their deployment in dynamic and resource-constrained environments. This dissertation addresses these limitations through two complementary research directions: enhancing deep learning interpretability through distance correlation analysis and developing efficient Dynamic Mode Decomposition (DMD) methods for batch and streaming forecasting. First, this work introduces a distance correlation-based framework to examine the internal mechanics of RNNs in time series forecasting. This versatile metric enables systematic analysis of information flow through RNN activation layers, revealing how these networks process temporal dependencies. Empirical analysis demonstrates that RNN activation layers effectively learn lag structures in early layers but progressively lose this temporal information in deeper layers, degrading forecast quality for series with large lag dependencies. The study further reveals fundamental limitations in RNN capabilities for modeling moving average and heteroskedastic processes. Distance correlation heatmaps provide visual comparisons across architectures and hyperparameters, demonstrating that input window size influences model behavior far more than conventional hyperparameters such as hidden units or activation functions. These findings enable practitioners to assess RNN suitability for specific time series characteristics without extensive trial-and-error experimentation. The second direction introduces novel DMD-based forecasting methods that address deep learning limitations. For batch scenarios, Incremental Kernel Dynamic Mode Decomposition (IKDMD) enhances adaptability and efficiency by integrating incremental kernel singular value decomposition and randomized linear algebra into the kernel DMD framework. Comparative analysis across real-world datasets demonstrates that IKDMD outperforms state-of-the-art deep learning methods, particularly for highly non-stationary and volatile data, while providing interpretable eigenvalue diagnostics unavailable in black-box neural networks. For streaming applications, this dissertation presents Windowed Online Random Kernel Dynamic Mode Decomposition (WORK-DMD), which integrates Random Fourier Features with online DMD to enable real-time forecasting from continuously arriving data. By employing explicit feature mappings rather than implicit kernel methods, WORK-DMD achieves fixed computational complexity per update while capturing nonlinear dynamics. Its adaptive windowing mechanism naturally handles non-stationary dynamics without catastrophic forgetting. Experimental evaluation across benchmark datasets demonstrates remarkable sample efficiency, requiring only single-pass learning while achieving competitive or superior accuracy compared to deep learning methods that demand multiple training epochs and extensive sample exposures. This efficiency translates to reduced computational costs, faster deployment, and viability for resource-constrained edge devices. Together, these contributions advance time series forecasting by providing both diagnostic tools for understanding deep learning limitations and computationally efficient alternatives that balance accuracy, interpretability, and real-time adaptability. The methods presented enable practical deployment in scenarios where traditional deep learning approaches struggle with sample efficiency, computational constraints, and evolving data dynamics.