R-lens: Making J-lens More Faithful on Early Layers

TL;DR:We introduce the R-lens: a drop-in replacement for J-lens that produces clearer readouts on earlier layers. We fit the R-lens on Jacobians computed through an LRP-modified backward pass, using the gradient-propagation rules from relevance patching (RelP) rather than standard gradients, which allows us to reduce the propagation of errors throughout the backwards pass. This method allows us to surface important intermediate variables more consistently and more saliently, reduce the frequency...