Do tabular foundation models repair themselves?

TL;DR Ablation is a widely adopted technique in mechanistic interpretability research. Self-repair invalidates conclusions drawn from ablation experiments. While self-repair is prevalent in language models, we ask whether it exists in tabular foundations models (TFMs). Our experiments on 4 SOTA tabular foundation models and 15 binary classification tasks suggest a negative result. Redundancy is more likely instead. The full version of our work is available at https://xiaohan2012.github.io/articl...