Annotating and Modeling Shallow Semantics Directly from Text

Abstract

One key challenge to understanding human language is to find out the word to word semantic relations, such as ``who does what to whom”, “when”, and “where”. Semantic role labeling (SRL) is the widely studied challenge of recovering such predicate-argument structure. SRL is designed to be consistent across syntactic alternations, which can potentially benefit downstream applications such as information extraction, machine translation, and summarization. However, the performance of SRL system is limited by the amount of training data and its dependence on the intermediate syntactic representation. %further impeding its usage in downstream applications. In this thesis, our goal is to develop annotation frameworks and learning models for recovering semantic structures directly from text, in an end-to-end manner. We first introduce question-answer driven semantic role labeling (QA-SRL), an annotation framework that allows us to gather SRL information from non-expert annotators. Different from the traditional SRL formalisms (e.g. PropBank), this new task does not depend on predefined syntactic structure or frame ontology. It is simple and intuitive enough that we can train any native speaker to provide annotation, as long as they can understand the meanings of sentences. We also develop two general-purpose, syntax-independent neural models that lead to significant performance gains, including an over 40\% error reduction over long-standing pre-neural performance levels on PropBank. Our first model, DeepSRL, uses highway BiLSTMs to make local BIO-tagging decisions for each token. While significantly out-performing previous systems, DeepSRL cannot jointly process multiple predicates or incorporate span-level features. To address these limitations, we further introduce a span-based neural model called the Labeled Span Graph Networks (LSGNs). Inspired by a recent state-of-the-art coreference resolution model, LSGNs build contextualized representations for all spans in the input text, and use lightweight classifiers to make independent edge labeling decisions. With LSGNs, we are able to model all predicate words and argument spans jointly, the first end-to-end result we know of. LSGNs also lead to a unified view for many NLP structures involving span-labeling or span-span relations. In addition to SRL and coreference resolution, LSGNs also achieve state-of-the-art performance when applied to named entity recognition (NER) without any feature engineering. This opens up exciting future directions to build a single, unified model for end-to-end, document-level semantic analysis.