Artificial Neural Networks with Dynamic Connections

Abstract

While highly successful in many different domains, most artificial neural networks suffer from a severe limitation ; they use the same parameters for different inputs. Different examples can have significantly different characteristics and can require different treatment from the model. This work investigates how to alleviate this issue using the recently introduced concept of "hypernetworks", neural networks that generate other neural networks. The first part of this thesis discusses how these models can be used to learn dynamic features from unlabeled image and video data. The second part introduces Active Predictive Coding Networks, models that hierarchically reconstruct images using neural sub-programs. The final part is dedicated to applications of hypernetworks to generating custom policies from contextual inputs in reinforcement learning settings.