Large to Small Model Stitching Destroys What You Built It to Carry Along the Fit

Independent research; early stage. Posting for feedback, particularly on whether the effect survives at realistic scale, and on prior work I've missed.Model stitching is extensively used in AI Safety infrastructure in reusing SAE & probes to make interpretability cheaper; transferring refusal, steering vectors so that small model can help align larger ones, cross architecture model diffing (1, 2).In this post, I expand on Chen's et al. work on Model Stitching to transfer linear features across l...