Improving Influence-based Instruction Tuning Data Selection for Balanced
Learning of Diverse Capabilities
Improving Influence-based Instruction Tuning Data Selection for Balanced
Learning of Diverse Capabilities
Selecting appropriate training data is crucial for effective instruction fine-tuning of large language models (LLMs), which aims to (1) elicit strong capabilities, and (2) achieve balanced performance across a diverse range of tasks. Influence-based methods show promise in achieving (1) by estimating the contribution of each training example to the …