Some Thoughts on The Environment Problem in Agent Training

As Large Language Models move away from being chat interfaces and become increasingly autonomous actors in the real world, a few insights about evaluation and training of these systems emerge, and I'd like to discuss them.Context:I've gained the insights and ideas laid out below through ongoing work I'm doing. This post serves to outline my working model in pursuing it, and constraints and lessons I learned along the way. Some of these are offered as learned lessons, others as assumptions, and s...