Using Machine Learning to Identify Functional Properties of Complex Systems: from Neurons to Networks

Abstract

The human brain may be the most complicated thing in the discovered universe. Now is an exciting time, when the capabilities of computing and neural recording technologies provide unprecedented opportunity to use advanced machine learning models to begin understanding it. The two main goals of such models are to successfully predict neural data withheld from training and to provide scientific insight into the functioning of the neural system. To this end, I introduce two model designs, each of which achieves these goals by incorporating a scientific hypothesis about the brain into the structure of the model. First, in Chapter 2, I train models of individual neurons with the assumption that these models are well described by a finite number of cell-types. A hierarchical model that uses the expectation-maximization algorithm to simultaneously learn parameters associated with individual neurons and a description of the cell-types allows parameters associated with each neuron to borrow strength from other neurons' data. I use simulated data to show that this allows the hierarchical model to recover true model parameters and cell type identities better than single-cell models that make no assumption about cell types and are clustered after being fit independently. I apply this hierarchical model to recordings from $634$ neurons and show that, compared to the naive approach described above, it yields better predictions of held out data for the overwhelming majority of neurons and discovers cell types that are more robust to the exclusion of different neurons from the training data. These discovered cell types also relate to available information about the gene-expression, morphology, and location of each neuron. Then, in Chapter 3, I ``joint-train'' neural networks to simultaneously predict the activity of a neural population and perform an auxiliary task, hypothesized to be related the the function of those neurons. I find analytic conditions for when the inclusion of an auxiliary task can improve prediction of neural responses in a linear network, and derive an expression for the degree of improvement, which reveals that the most useful auxiliary tasks are those most predictable from the neural data. I then show that this result is born out in deep, nonlinear, structured networks via numerical analyses with simulated data. These analyses also reveal that joint-training allows the model to precisely and accurately associate simulated neurons with a subset of model units. Finally, I use further theory and numerical analysis with the linear model to show how a derived condition on network structure necessary for an auxiliary task to yield improved neural response prediction is modified by nonlinearities, constraints on network parameters, and finite training time, providing insight into how the numerical results differ from those predicted by our theory. Overall, this work provides insights into how we might incorporate specific hypotheses about brains into statistical models in order to best explain neural data and gain insight into the functioning of neural systems.