An agent’s account of its own work is not evidence of what it did by KrishanKVerma

I ran a browser agent 160 times, four tasks, two mod­els, twenty runs each. Not once did an agent re­port that it couldn’t do the job. That needs un­pack­ing, be­cause “zero failures” is not what hap­pened. There were plenty of failures.The har­ness sorts ev­ery run into three buck­ets: the agent did the task, the agent didn’t do the task and said so, or the agent didn’t do the task and re­ported suc­cess any­way. The mid­dle bucket, the hon­est failure, stayed empty across all 160 runs. Every f...