Grammar-Engineered Synthetic Languages for Studying Typological Biases in Language Models: A Case Study of Action Nominal Constructions

Abstract

This thesis proposes a grammar-engineered method for constructing synthetic corpora to study language models' learning preferences in acquiring action nominal constructions (ANCs) and related phenomena. The method uses the Grammar Matrix to construct Head-driven Phrase Structure Grammar (HPSG) grammars varying in typological parameters, with Minimal Recursion Semantics (MRS) as a shared semantic interface across grammars. This design makes it possible to control fine-grained grammatical parameters while preserving part of the semantic and lexical distribution of natural data. Using this method, I construct 96 parallel languages varying in clause word order, NP word order, alignment, complement system, and ANC strategy. For each language, I train GPT-2-small models from scratch and evaluate them using targeted minimal pairs covering 34 grammatical phenomena. The results show that models more easily learn systems that maintain consistent linear organization across related grammatical structures, which aligns with preferences in human language typology. At the same time, additional morphological cues can help models distinguish similar structures, but human languages do not always favor more such cues. As a result, some parameter combinations that are relatively easy for models to learn remain rare in human languages. Overall, these results highlight both the typological relevance of model learning preferences and the limits of learnability as an explanation for human language patterns.