SONI: Selective Orthogonalisation via Noise Injection

This project was completed as a capstone for TARA. All code is available in github.TL;DRThe Problem: Neural networks use superposition to pack many concepts into small latent spaces by making feature vectors almost-orthogonal. This entanglement makes models opaque and breaks safety interventions (e.g. concept erasure, activation steering) which rely on clean, isolated concept directions.The Gap: Full orthogonalisation (via sparsity penalties) destroys model capacity, while Sparse Autoencoders (S...