Geometry-Aware Statistical Learning and Causal Inference for Complex Observational Data

Abstract

Motivated by cosmic web detection and related problems in astronomy, this dissertation develops statistically principled and computationally efficient methods for analyzing complex observational data. Three major challenges arise from such data: (i) nonlinear geometric structure in the observational domain, (ii) high dimensionality and missingness in measured features, and (iii) confounding and other causal assumption violations when interpreting scientific relationships from observational studies. The first part of the dissertation addresses the geometric challenge in (i). In Chapter 2, we introduce directional density ridges as statistical surrogates for representing low-dimensional high-density structures on the sphere. These surrogates provide geometry-aware models for cosmic nodes and filaments while respecting the spherical geometry of sky observations. We establish stability and statistical consistency guarantees for directional density ridge estimation based on kernel smoothing. In Chapter 3, we derive practical algorithms for recovering directional density ridges from data and study their linear convergence properties. In Chapter 4, we apply the statistical and computational framework of directional density ridges to galaxy and quasar observations from the Sloan Digital Sky Survey IV, producing a tomographic cosmic web catalog. The second part of the dissertation develops statistical inference methods for the high-dimensional, incomplete, and confounded observational settings described in (ii) and (iii). In Chapter 5, we study high-dimensional linear regression with missing outcomes and propose an efficient debiased inference procedure for low-dimensional functionals of the regression function. When the missingness probabilities are consistently estimated at the observed data points, our proposed estimator is asymptotically normal and semiparametrically efficient among all asymptotically linear estimators. We apply this method to study associations between galactic stellar mass and nearby cosmic web structures. In Chapter 6, we turn from association analysis to causal inference for continuous treatments in observational studies, focusing on settings where the usual positivity condition may fail. We propose novel identification and estimation strategies for the dose-response curve and its derivative under an additive structural model for the counterfactual outcome. Using tools from nonparametric set estimation, we construct inverse probability weighted and doubly robust estimators that remain unbiased under certain positivity violations. We further introduce an extension of the classical dose-response curve based on the nearest feasible treatment policy regime, which adapts to the geometry of the conditional treatment support and accounts for treatment-level heterogeneity. Across these causal settings, we establish asymptotic normality at standard nonparametric rates of convergence, enabling valid statistical inference. We illustrate the proposed causal framework with an application to the IllustrisTNG simulation data, studying the effect of local environment on galactic star formation rate and providing a new causal perspective on the "Nature versus Nurture'' debate in galaxy evolution.