On Internal Interfaces in Machine Learning Systems

Abstract

Machine learning systems are often studied through the standard interfaces they expose, such as image classification or text generation. While these interfaces are central to evaluation and deployment, they can obscure internal structure that is essential for understanding how models are trained, how they fail, and how their behavior can be controlled. In this thesis, I study such hidden structure in two settings: backdoor attacks in supervised learning, and tokenization in large language models. In the first part, I study how small adversarial changes to training data can induce targeted changes in model behavior. I first present the detectable signature poisoned training examples leave in the trained model's hidden representations, and develop a defense based on robust covariance estimation that amplifies these traces, removing poisoned examples even when previous spectral methods fail. I then switch roles and approach the same phenomenon from the attacker's perspective, showing that neural tangent kernels can be used to approximate training dynamics, enabling the construction of substantially stronger few-shot backdoor attacks. In the second part, I study the tokenizer—sequence-model decomposition of language models. I first show that the merge lists of byte-pair encoding tokenizers reveal systematic information about the data on which they were trained, and formulate this as a new inference problem called data mixture inference. I then examine how tokenization shapes model behavior at inference time, giving rise to the prompt boundary problem, in which the standard prompting interface imposes token boundaries that distort the model's completions. To address this, I introduce ByteSampler, an inference-time method that converts a BPE-based autoregressive language model into an equivalent byte-level model, eliminating the prompt boundary problem and enabling cross-tokenizer ensembling and behavior transfer. Together, these results show that internal structure beyond a model’s standard interface can be exploited for defense and attack analysis, inference about training, and improved control over model behavior.