Last-Layer Gradient Features for Targeted Data Selection in Instruction Tuning

Instruction-tuning data vary widely in value, and carefully curating a small, target-matched subset from a candidate pool can match or even exceed the performance of training on the entire pool. Gradient-based data selection algorithms are popular for this, but they typically assume access to full gradients, which require an expensive backward pass over every sample of the pool. Interestingly, we find that replacing full gradients with last-layer gradients, which require only a forward pass, can often maintain performance at a fraction of the cost. Moreover, this substitution also makes selection more robust to perturbations in the model weights. We demonstrate the performance and efficiency of our data selection method, PROD, across three language models and five fine-tuning tasks.