Aquarium Security and Other Organisational Priors

TL;DR a few model providers are learning from private conversations across many organisations, giving their models better representations of how those organisations secure their systems and plausibly making semi-autonomous attacks easier. Private credentials are given directly to LLMs at scale. Beyond the same WiFi password being reused across office-wifis or stored in a Google doc, people post API keys straight into prompts, hardcode them in files, copy them into chats on personal accounts, etc...