How many files does your query read before it reads any data? On some data lakes, you go through JSON and metadata files first, just to learn which Parquet files matter. DuckLake asks one SQL question instead. The metadata lives in a real database. The data stays in plain Parquet. That's the entire format.
 

 
Pedro Holanda joined DuckDB in 2018, when it was still a research prototype at CWI. He's the lead DuckLake developer. Guillermo Sanchez Dionis works on DuckLake and the new Quack protocol.
 

 
With Quack as the catalog, DuckLake handles 200 transactions a second under heavy contention. No other open table format comes close. 
 
Episode sponsors 
 
<a href='https://talkpython.fm/sixfeetup'>Six Feet Up</a>
<a href='https://talkpython.fm/training'>Talk Python Courses</a> 
 
<h2 class="links-heading mb-4">Links from the show</h2>
<div>Guests 
Pedro Holanda: <a href="https://pedroholanda.org?featured_on=talkpython" target="_blank" >pedroholanda.org</a> 
Guillermo Sanchez: <a href="https://www.linkedin.com/in/guillermo-s%C3%A1nchez-dionis/?featured_on=talkpython" target="_blank" >linkedin.com</a> 
 
PhD on progressive indexes: <a href="https://ir.cwi.nl/pub/31048?featured_on=talkpython" target="_blank" >ir.cwi.nl</a> 
SQLite: <a href="https://www.sqlite.org?featured_on=talkpython" target="_blank" >www.sqlite.org</a> 
Litestream: <a href="https://litestream.io?featured_on=talkpython" target="_blank" >litestream.io</a> 
boring hardware: <a href="https://talkpython.fm/episodes/show/531/talk-python-in-production" target="_blank" >talkpython.fm</a> 
DuckDB: <a href="https://duckdb.org?featured_on=talkpython" target="_blank" >duckdb.org</a> 
episode 491: <a href="https://talkpython.fm/episodes/show/491/duckdb-and-python-ducks-and-snakes-living-together" target="_blank" >talkpython.fm</a> 
Iceberg: <a href="https://iceberg.apache.org?featured_on=talkpython" target="_blank" >iceberg.apache.org</a> 
manifesto: <a href="https://ducklake.select/manifesto/?featured_on=talkpython" target="_blank" >ducklake.select</a> 
DuckLake: <a href="https://ducklake.select?featured_on=talkpython" target="_blank" >ducklake.select</a> 
spec: <a href="https://ducklake.select/docs/stable/specification/introduction?featured_on=talkpython" target="_blank" >ducklake.select</a> 
this diagram: <a href="https://blobs.talkpython.fm/ducklake-architecture.png?cache_id=13839f" target="_blank" >blobs.talkpython.fm</a> 
Data inlining: <a href="https://ducklake.select/2026/04/02/data-inlining-in-ducklake/?featured_on=talkpython" target="_blank" >ducklake.select</a> 
ducklake-dataframe: <a href="https://github.com/pdet/ducklake-dataframe?featured_on=talkpython" target="_blank" >github.com</a> 
Polars course: <a href="https://training.talkpython.fm/courses/polars-for-power-users" target="_blank" >training.talkpython.fm</a> 
CSV parser: <a href="https://duckdb.org/2025/04/16/duckdb-csv-pollock-benchmark.html?featured_on=talkpython" target="_blank" >duckdb.org</a> 
Zero-copy Arrow: <a href="https://duckdb.org/2021/12/03/duck-arrow.html?featured_on=talkpython" target="_blank" >duckdb.org</a> 
ART index: <a href="https://duckdb.org/2022/07/27/art-storage.html?featured_on=talkpython" target="_blank" >duckdb.org</a> 
async I/O: <a href="https://duckdb.org/2026/07/31/asynchronous-io?featured_on=talkpython" target="_blank" >duckdb.org</a> 
v1.0: <a href="https://ducklake.select/2026/04/13/ducklake-10/?featured_on=talkpython" target="_blank" >ducklake.select</a> 
Git-like branching: <a href="https://ducklake.select/roadmap.html?featured_on=talkpython" target="_blank" >ducklake.select</a> 
 
Watch this episode on YouTube: <a href="https://www.youtube.com/watch?v=wEKbbY5tPtI" target="_blank" >youtube.com</a> 
Episode #562 deep-dive: <a href="https://talkpython.fm/episodes/show/562/ducklake-the-lakehouse-thats-just-sql-and-parquet#takeaways-anchor" target="_blank" >talkpython.fm/562</a> 
Episode transcripts: <a href="https://talkpython.fm/episodes/transcript/562/ducklake-the-lakehouse-thats-just-sql-and-parquet" target="_blank" >talkpython.fm</a> 
 
Theme Song: Developer Rap 
🥁 Served in a Flask 🎸: <a href="https://talkpython.fm/flasksong" target="_blank" >talkpython.fm/flasksong</a> 
 
---== Don't be a stranger ==--- 
YouTube: <a href="https://talkpython.fm/youtube" target="_blank" > youtube.com/@talkpython</a> 
 
Bluesky: <a href="https://bsky.app/profile/talkpython.fm" target="_blank" >@talkpython.fm</a> 
Mastodon: <a href="https://fosstodon.org/web/@talkpython" target="_blank" > @talkpython@fosstodon.org</a> 
X.com: <a href="https://x.com/talkpython" target="_blank" > @talkpython</a> 
 
Michael on Bluesky: <a href="https://bsky.app/profile/mkennedy.codes?featured_on=talkpython" target="_blank" >@mkennedy.codes</a> 
Michael on Mastodon: <a href="https://fosstodon.org/web/@mkennedy" target="_blank" > @mkennedy@fosstodon.org</a> 
Michael on X.com: <a href="https://x.com/mkennedy?featured_on=talkpython" target="_blank" > @mkennedy</a> </div>

Podurama

Home

Trending

Personal

Your Podcasts

New Episodes

My Files

Favourite Episodes

Tags (Folders)

Recently Played

Bookmarks

Followed Playlists

#562: DuckLake: The Lakehouse That's Just SQL and Parquet

Talk Python To Me

AI Chapters

AI Insights

Reviews