LLM self-decompilation to turn weights into conventional code

The fact that current LLMs are effectively black boxes is arguably the biggest obstacle on the road to alignment. There have been many proposals for mechanistic interpretability and other ideas to pull back the veil on the internal workings of LLMs, but not enough attention has been given to the possibility of translating weights into actual code.The gargantuan nature of the task puts it obviously beyond the reach of unassisted humans, but it seems plausible to me that a sufficiently capable LLM...