We live in an era of charts that are going up and to the right. This image obviously describes the stock market, particularly any company whose business is adjacent to artificial intelligence. But beyond stocks, another sort of chart we keep seeing is of AI capabilities also going up and to the right. The most famous and viral of these comes from an organization called METR, which stands for Model Evaluation and Threat Research. The organization is focused on understanding the degree to which AI models can engage in autonomous, complex tasks. METR see this is as a particularly important benchmark, given the risk that AI could one day be engaged in recursive self improvement, taking humans out of the loop. But how do you really gauge a model's ability to do complex problems. And what is being measured for exactly? On this episode, we speak with METR's President Chris Painter as well as Joel Becker, a member of the technical staff who works on evaluation methods for the organization. We discuss both the mechanics and the philosophy of METR's work, and what it means when we see a a chart showing that Clause Opus 4.6 can do a task that would take a human nearly 12 hours. Read more: <a href="https://www.bloomberg.com/news/articles/2026-04-24/deepseek-unveils-newest-flagship-a-year-after-ai-breakthrough?utm_medium=referral&utm_source=podcast&utm_campaign=odd_lots&utm_content=article" data-prosemirror-content-type="mark" data-prosemirror-mark-name="link">DeepSeek Unveils Flagship AI Model a Year After Breakthrough</a> <a href="https://www.bloomberg.com/news/articles/2026-04-24/meta-inks-multibillion-dollar-deal-to-use-amazon-chips-for-ai?utm_medium=referral&utm_source=podcast&utm_campaign=odd_lots&utm_content=article" data-prosemirror-content-type="mark" data-prosemirror-mark-name="link">Meta Inks Deal to Use Amazon&rsquo;s Graviton Processors for AI</a> Only <a href="http://Bloomberg.com" data-inline-card="" data-card-data="">http://Bloomberg.com</a> subscribers can get the Odd Lots newsletter in their inbox each week, plus unlimited access to the site and app. Subscribe at&nbsp; <a href="https://www.bloomberg.com/subscriptions/oddlots?in_source=oddlotspodcast" data-prosemirror-content-type="mark" data-prosemirror-mark-name="link">bloomberg.com/subscriptions/oddlots</a> <a href="http://bloomberg.com/subscriptions/oddlots" data-prosemirror-content-type="mark" data-prosemirror-mark-name="link">Subscribe to the Odd Lots Newsletter</a> Join the conversation: <a href="https://discord.gg/oddlots" data-prosemirror-content-type="mark" data-prosemirror-mark-name="link">discord.gg/oddlots</a>See <a href="https://omnystudio.com/listener">omnystudio.com/listener</a> for privacy information.

Podurama

Home

Trending

Personal

Your Podcasts

New Episodes

My Files

Favourite Episodes

Tags (Folders)

Recently Played

Bookmarks

Followed Playlists

Understanding the Most Viral Chart in Artificial Intelligence

Odd Lots

AI Chapters

AI Insights

Reviews