The shift from monolithic to decentralized data workflows changes how teams build, connect and scale pipelines.In this episode, we feature <a href="https://www.linkedin.com/in/oscar-ligthart/" target="_blank" style="background-color: transparent; color: rgb(17, 85, 204);">Oscar Ligthart</a>, Lead Data Engineer, and <a href="https://www.linkedin.com/in/rodrigo-loredo-410a16134/" target="_blank" style="background-color: transparent; color: rgb(17, 85, 204);">Rodrigo Loredo</a>, Lead Analytics Engineer, both at <a href="https://www.linkedin.com/company/vinted/" target="_blank" style="background-color: transparent; color: rgb(17, 85, 204);">Vinted</a>, as we unpack their YAML-driven abstraction that generates Airflow DAGs and standardizes cross-team orchestration.Key Takeaways:00:00 Introduction.05:28 Challenges of decentralization.06:45 YAML-based generator standardizes pipelines and dependencies.12:28 Declarative assets and sensors align cross-DAG dependencies.17:29 Task-level callbacks enable auto-recovery and clear ownership.21:39 Standardized building blocks simplify upgrades and maintenance.24:52 Platform focus frees domain work.26:49 Container-only standardization prevents sprawl.Resources Mentioned:<a href="https://www.linkedin.com/in/oscar-ligthart/" target="_blank" style="background-color: transparent; color: rgb(17, 85, 204);">Oscar Ligthart</a>https://www.linkedin.com/in/oscar-ligthart/<a href="https://www.linkedin.com/in/rodrigo-loredo-410a16134/" target="_blank" style="background-color: transparent; color: rgb(17, 85, 204);">Rodrigo Loredo</a>https://www.linkedin.com/in/rodrigo-loredo-410a16134/<a href="https://www.linkedin.com/company/vinted/" target="_blank" style="background-color: transparent; color: rgb(17, 85, 204);">Vinted</a> | LinkedInhttps://www.linkedin.com/company/vinted/<a href="https://www.vinted.com/?srsltid=AfmBOor87MGR_eLOauCO93V9A-aLDaAhGYx9cnu_oN8s1SAXMlCRuhW7" target="_blank" style="background-color: transparent; color: rgb(17, 85, 204);">Vinted</a> | Websitehttps://www.vinted.com/?srsltid=AfmBOor87MGR_eLOauCO93V9A-aLDaAhGYx9cnu_oN8s1SAXMlCRuhW7<a href="https://airflow.apache.org/" target="_blank" style="background-color: transparent; color: rgb(17, 85, 204);">Apache Airflow</a>https://airflow.apache.org/<a href="https://kubernetes.io/" target="_blank" style="background-color: transparent; color: rgb(17, 85, 204);">Kubernetes</a>https://kubernetes.io/<a href="https://www.getdbt.com/" target="_blank" style="background-color: transparent; color: rgb(17, 85, 204);">dbt</a>https://www.getdbt.com/<a href="https://cloud.google.com/vertex-ai" target="_blank" style="background-color: transparent; color: rgb(17, 85, 204);">Google Cloud Vertex AI</a>https://cloud.google.com/vertex-ai<a href="https://www.astronomer.io/docs/learn/airflow-datasets" target="_blank" style="background-color: transparent; color: rgb(17, 85, 204);">Airflow Datasets & Assets (concepts)</a>https://www.astronomer.io/docs/learn/airflow-datasets<a href="https://airflowsummit.org/" target="_blank" style="background-color: transparent; color: rgb(17, 85, 204);">Airflow Summit</a>https://airflowsummit.org/Thanks for listening to “The Data Flowcast: Mastering Apache Airflow® for Data Engineering and AI.” If you enjoyed this episode, please leave a 5-star review to help get the word out about the show. And be sure to subscribe so you never miss any of the insightful conversations.#AI #Automation #Airflow #MachineLearning

Podurama

Home

Trending

Personal

Your Podcasts

New Episodes

My Files

Favourite Episodes

Tags (Folders)

Recently Played

Bookmarks

Followed Playlists

Inside Vinted’s Code-Generated Airflow Pipelines with Oscar Ligthart and Rodrigo Loredo

The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI

AI Chapters

AI Insights

Reviews