Failing to Understand the Exponential, Again
It may sound overly simplistic, but making predictions by extrapolating straight lines on graphs is likely to give you a better model of the future than most “experts” - even better than most actual domain experts!
Reasoning about exponential progress is extremely hard to do for humans. As Schrittwieser points out, we made the same cognitive errors at the beginning of Covid.
His conclusion: using the METR evals as one measure, we’re right on track and models continue to make exponential progress.