Algebraic and Geometric Structures in Machine Learning

Abstract

We explore four examples of algebraic and geometric structures motivated by problems in machine learning. First, we consider the problem of learning under symmetry with kernel methods. In particular, we demonstrate that invariances have a pronounced effect on the rank of kernel matrices, with this rank becoming independent of the data's dimension in key examples. Relating this to the phenomenon of representation stability, we show that a simple regression procedure is minimax optimal for learning invariant functions that are well-defined across different dimensions. Next, we consider the problem of density estimation on non-Euclidean spaces. Motivated by results for kernel density estimation on manifolds, we consider density estimation on d-rectifiable sets, which include smooth manifolds and algebraic varieties as special cases. In this context, we give a suitably modified kernel density estimator and prove convergence rates that are analogous to previous results. We then consider two families of ideals, multiview and universal multiview ideals, that arise in the field of computer vision and show that a well-known family of polynomials form a universal Gröbner basis for any ideal in either family. This recovers a known result and proves a conjecture from the literature. While we only discuss multiview and universal multiview ideals, our proof strategy, which relies on a recent criterion from the commutative algebra literature, symmetry reduction, and induction, is general and may be applied to other infinite families of ideals. Finally, we discuss tree-SNE, a data visualization technique based on the popular t-distributed stochastic neighbor embedding (t-SNE) method. A tree-SNE embedding consists of many layers, each of which is a t-SNE embedding corresponding to a different In particular, we show that tree-SNE embeddings have a continuous structure and illustrate that the embeddings provide meaningful insights through three examples.