Measure Optimal AI Share of Labor Costs Instead of AI or Human-Only Performance
In a research post, Cunningham et al. of METR propose the “‘expenditure horizon’ over an optimization problem as the dollar value at which the improvement to the goal metric is equal to the improvement by a human with the same budget.” Though their work focuses on individual optimization problems (with NanoGPT as the main subject), the expenditure horizon can be meaningful for any task distribution: it’s the budget at which human-only and AI-only work reach the same performance.Though I like exp...
Read full article →