Publication | Conference on Neural Information Processing Systems 2022

MaskTune

Mitigating Spurious Correlations by Forcing to Explore

Activation visualizations of ERM (middle) and MaskTune (right) for Waterbirds samples, in which MaskTune enforces exploring new features. After applying MaskTune, the task-relevant input signals (bird features) are emphasized.

Learning the right features during training is a significant challenge for deep neural networks (DNNs). DNNs might instead pick up spurious features. This work investigates a novel solution to this problem.

Code for MaskTune is available at https://github.com/aliasgharkhani/Masktune.

Download

Abstract

MaskTune: Mitigating Spurious Correlations by Forcing to Explore

Saeid Asgari, Aliasghar Khani, Fereshte Khani, Ali Gholami, Linh Tran, Ali Mahdavi-Amiri, Ghassan Hamarneh

Conference on Neural Information Processing Systems 2022

A fundamental challenge of over-parameterized deep learning models is learning meaningful data representations that yield good performance on a downstream task without over-fitting spurious input features. This work proposes MaskTune, a masking strategy that prevents over-reliance on spurious (or a limited number of) features. MaskTune forces the trained model to explore new features during a single epoch finetuning by masking previously discovered features. MaskTune, unlike earlier approaches for mitigating shortcut learning, does not require any supervision, such as annotating spurious features or labels for subgroup samples in a dataset. Our empirical results on biased MNIST, CelebA, Waterbirds, and ImagenNet-9L datasets show that MaskTune is effective on tasks that often suffer from the existence of spurious correlations. Finally, we show that \method{} outperforms or achieves similar performance to the competing methods when applied to the selective classification (classification with rejection option) task.

Associated Researchers

Aliasghar Khani

School of Computing Science, Simon Fraser University

Fereshte Khani

Stanford University

Linh Tran

Autodesk AI Lab

Ghassan Hamarneh

School of Computing Science, Simon Fraser University

Ali Mahdavi-Amiri

School of Computing Science, Simon Fraser University

View all researchers

Related Resources

Publication

2023

SLiMe: Segment Like Me

We explore leveraging extensive vision-language models for segmenting…

Project

2022

Leveraging Robotics for Cleaner Construction Jobsites

The value in Spot’s ability to execute repeatable, autonomous missions…

Project

2021

Meshmixer: Mesh Technology for Interactive Design and Fabrication

Meshmixer is a prototype design tool based on high-resolution dynamic…

Article

2023

Recently Published by Autodesk Researchers

A round up of recent publications from scientific journals and…

Get in touch

Something pique your interest? Get in touch if you’d like to learn more about Autodesk Research, our projects, people, and potential collaboration opportunities.

Contact us