Towards Efficient, Customizable, and Communal Natural Language Processing

Abstract

I advocate for efficient, customizable, and communal approaches to natural language pro- cessing (NLP) and artificial intelligence (AI), where people with diverse skill levels and research backgrounds can: build, use, analyze, and evaluate models; collaborate to solve research prob- lems; and accelerate advances in NLP and AI. AI and NLP have made remarkable progress from recent, large-scale training on massive datasets. These technologies are being developed and used by many cross-disciplinary researchers and practitioners. People with scant computer science training—including physicians, translators, and historians—now rely on AI models for work problems that can be solved by using massive amounts of data. This thesis discusses my key contributions to ways to make AI and NLP more accessible to researchers, practitioners, and users. How can we encourage model builders and practitioners to work as a community to broaden the appeal and utility of NLP and AI models across disciplines? How can we make it easier for them to formulate and answer complex real-world questions using these technologies and ensure these models are robustly evaluated? I first introduce and empirically demonstrate efficient architectures and learning paradigms for state-of-the-art NLP models. More efficient methods will lower the cost of developing and using these models, making them deployable to less-well-funded fields or institutions. I then present an algorithm for flexible and customizable language generation in the areas of collaborative inference between diverse models. This inference method avoids the computationally (and thus financially and environmentally) expensive training process of large models. Lastly, I pro- pose methodologies and interfaces to make model evaluations more transparent, consistent, and reliable. I present a collaborative platform that bridges the modeling and evaluation research communities to enable robust evaluation of AI models.