Stochastic Analysis on Graphons

Abstract

We say that a function on the space of symmetric matrices is \emph{invariant} if it is invariant under the simultaneous permutation of rows and columns of the input matrix by the same permutation. Homomorphism densities of finite simple graphs offer a rich class of examples of invariant functions that prominently feature in several areas of mathematics. In this thesis, we study two natural classes of interrelated problems. The first problem concerns the optimization of invariant functions in large dimensions. The second problem concerns the analysis of the dynamics on graphs/matrices where the evolution of coordinates depends on the full graph/matrix via an invariant function as the dimension goes to infinity. An important theme of the present thesis is that due to the symmetry of invariant functions, their optimization and dynamics can be reduced to optimization and dynamics on the space of graphons as the dimension of the underlying space goes to infinity. The rich geometry and the analytical properties of the space of graphons make the problems on the space of graphons more tractable. We develop a notion of gradient flow on the space of graphons following the general theory of gradient flows in metric spaces. We show that under mild differentiability assumption, any invariant function on the space of graphons admits a gradient flow which is an absolutely continuous curve with respect to the invariant $L^2$ metric. Furthermore, under appropriate convexity and differentiability assumptions, we show that the Euclidean gradient flows of invariant functions converge to the gradient flow of a suitable function on the space of graphons. We then consider a class of symmetric $n\times n$ matrix-valued diffusions where the drift is given by an invariant function. Such diffusions arise, for example, as the scaling limits of stochastic gradient descent of an invariant function. We establish a propagation of chaos phenomenon for such matrix-valued processes. That is, we show that any finite collection of coordinates of such processes becomes conditionally independent as $n\to \infty$ and that a uniformly random coordinate of such processes satisfies a novel graphon McKean-Vlasov SDE, in $n\to \infty$ limit. As a consequence of this, we obtain that these matrix-valued processes converge to a deterministic curve on the space of graphons. We also construct a Metropolis chain, with a novel relaxation step, whose state space is the stochastic block model with $r$ communities and $n$ individuals in each community. We show that fixed $r$, under appropriate scaling of parameters, the $r\times r$ matrix of connection probabilities between communities converges to a diffusion of the previous type. In particular, as $r\to \infty$, the connection probability between communities becomes conditionally independent. This allows us to prove that the trajectory of this Metropolis chain is concentrated near a deterministic curve of graphons. This allows us to approximate the gradient flow of function on graphons by suitable Markov chains on stochastic block models. Towards the end of the thesis, we also consider the scaling limit of the iterated product of matrices that are small perturbations of the identity matrix as the dimension of these matrices goes to infinity. In the fixed dimension, the scaling limit of the iterated product of such matrices is described by a non-commutative exponential of a matrix-valued semimartingale. Suppose that the bounded variation part of these semimartingales converges to some graphon as the dimension of these matrices goes to infinity. Then, we show that non-commutative exponentials converge to an infinite exchangeable array whose coordinates are Gaussians and whose mean and covariances can be described explicitly in terms of the limiting graphon.