What we did: We designed a simple and efficient method, called Auxiliary Tuning, for adapting a pre-trained Language Model (LM) to a novel task; we demonstrate the approach on the task of conditional text generation. Our approach supplements the original pre-trained model with an auxiliary model that shifts the output distribution according to the target task.
Why it matters: Achieving state-of-the-art fluency in language tasks such as text generation entails costly training of large LMs [1]. Auxiliary Tuning allows practitioners to amortize this cost across target tasks by leveraging existing pre-trained LMs. This is done without modifying the pre-trained weights, avoiding the risks of rigidity and catastrophic forgetting, and allowing natural scaling to multiple target tasks.
How it works: The auxiliary model is trained by adding its logits to the pre-trained model logits and maximizing the likelihood of the target task output. Our method imposes no constraints on the auxiliary architecture. In particular, the auxiliary model can ingest additional input relevant to the target task, independently from the pre-trained model’s input. Furthermore, mixing the models at the logits level provides a natural probabilistic interpretation of the method.
Results: Our method achieved similar results to training from scratch for a number of different tasks, while using significantly less compute for training; we share a specific example of text generation conditioned on keywords.
