General capability—and capabilities generally—have no good y-axis by Gregory Lewis🔸

BLUF:To de­ter­mine whether AI is ‘im­prov­ing ex­po­nen­tially’, ‘hit­ting the wall’, or any other claim which in­volves a quan­tity or mag­ni­tude (e.g. ‘This model was a big leap/​small in­cre­ment’). We need a good y-axis: an in­ter­val scale of AI ca­pa­bil­ity which means +1 unit always rep­re­sents the same de­gree of ‘how much bet­ter’, in the same way +1 de­gree Cel­sius is always the same amount of ‘how much hot­ter’.Yet there is no good y-axis for AI ca­pa­bil­ity. All our mea­sures a...