New Paradigm for Editing Machine Learning Models Based on Arithmetic Operations Over Task Vectors

When it comes to machine learning, pre-training models and fine-tuning them to fit specific tasks are becoming increasingly common. But what if you could change and update these models after they have been pre-trained? In this blog post, we explore a clever method developed by a team of researchers from the University of Washington, Microsoft Research, and Allen Institute for AI, which allows you to stir the behavior of pre-trained models based on task vectors. Read on to learn more about this fascinating research and the potential implications it has for the future of AI.

What are Task Vectors?
Task vectors are defined as the element-wise difference between the weights of pre-trained and fine-tuned models. They can be applied to any model parameters using element-wise addition and an optional scaling term, which is determined using held-out validation sets.

Negating Task Vectors to Remove Unwanted Behavior
The authors show that users can perform simple arithmetic operations on these task vectors to change models. Negating the vector can be used to remove undesirable behaviors or unlearn tasks. This method is reliable in forgetting unwanted behavior both in the vision and text domains, and can decrease the performance on the target task by up to 45.8 percentage points.

Adding Task Vectors to Improve Multi-Task Models
Adding task vectors can also be used to improve multi-task models or performance on a single task. When tasks form an analogy relationship, task vectors can be combined to improve performance on tasks where data is scarce. In the case of image classification, adding task vectors from two tasks improves accuracy on both, resulting in a single model that is competitive with using two specialized fine-tuned models.

Performing Task Analogies with Task Vectors
Finally, performing task analogies with task vectors allow both to improve performance on domain generalization tasks and subpopulations with little data. For instance, to obtain better performance on specific rare images (e.g., lions indoors), one can build a task vector by adding to the lion-outdoor task vector the difference between task vectors of dogs indoors and outdoors.

In conclusion, this research introduces a new approach for editing models by performing arithmetic operations on task vectors. The method is efficient, and users can easily experiment with various model edits by recycling and transferring knowledge from extensive collections of publicly available fine-tuned models. To learn more about this research, check out the paper and Github.

Leave a comment

Your email address will not be published. Required fields are marked *