The evolution of Intercom’s data infrastructure reveals how a well-built orchestration system can scale to serve global needs. With thousands of DAGs powering analytics, AI and customer operations, the team’s approach combines technical depth with organizational insight.In this episode, <a href="https://www.linkedin.com/in/andrasgombosi/" target="_blank" style="background-color: transparent; color: rgb(17, 85, 204);">András Gombosi</a>, Senior Engineering Manager of Data Infra and Analytics Engineering, and <a href="https://www.linkedin.com/in/paul-vickers-a22b76a3/" target="_blank" style="background-color: transparent; color: rgb(17, 85, 204);">Paul Vickers</a>, Principal Engineer, both at <a href="https://www.linkedin.com/company/intercom/" target="_blank" style="background-color: transparent; color: rgb(17, 85, 204);">Intercom</a>, share how they built one of the largest Airflow deployments in production and enabled self-serve data platforms across teams.Key Takeaways:00:00 Introduction.04:24 Community input encourages confident adoption of a common platform.08:50 Self-serve workflows require consistent guardrails and review.09:25 Internal infrastructure support accelerates scalable deployments.13:26 Batch LLM processing benefits from a configuration-driven design.15:20 Standardized development environments enable effective AI-assisted work.19:58 Applied AI enhances internal analysis and operational enablement.27:27 Strong test coverage and staged upgrades protect stability.30:36 Proactive observability and on-call ownership improve outcomes.Resources Mentioned:<a href="https://www.linkedin.com/in/andrasgombosi/" target="_blank" style="background-color: transparent; color: rgb(17, 85, 204);">András Gombosi</a>https://www.linkedin.com/in/andrasgombosi/<a href="https://www.linkedin.com/in/paul-vickers-a22b76a3/" target="_blank" style="background-color: transparent; color: rgb(17, 85, 204);">Paul Vickers</a>https://www.linkedin.com/in/paul-vickers-a22b76a3/<a href="https://www.linkedin.com/company/intercom/" target="_blank" style="background-color: transparent; color: rgb(17, 85, 204);">Intercom</a> | LinkedInhttps://www.linkedin.com/company/intercom/<a href="https://www.intercom.com" target="_blank" style="background-color: transparent; color: rgb(17, 85, 204);">Intercom</a> | Websitehttps://www.intercom.com<a href="https://airflow.apache.org/" target="_blank" style="background-color: transparent; color: rgb(17, 85, 204);">Apache Airflow</a>https://airflow.apache.org/<a href="https://www.getdbt.com/" target="_blank" style="background-color: transparent; color: rgb(17, 85, 204);">dbtLabs</a>https://www.getdbt.com/<a href="https://www.snowflake.com/en/product/features/cortex/" target="_blank" style="background-color: transparent; color: rgb(17, 85, 204);">Snowflake Cortex AI</a>https://www.snowflake.com/en/product/features/cortex/<a href="https://www.datadoghq.com/" target="_blank" style="background-color: transparent; color: rgb(17, 85, 204);">Datadog</a>https://www.datadoghq.com/﻿﻿Thanks for listening to “The Data Flowcast: Mastering Apache Airflow® for Data Engineering and AI.” If you enjoyed this episode, please leave a 5-star review to help get the word out about the show. And be sure to subscribe so you never miss any of the insightful conversations.#AI #Automation #Airflow

Podurama

Home

Trending

Personal

Your Podcasts

New Episodes

My Files

Favourite Episodes

Tags (Folders)

Recently Played

Bookmarks

Followed Playlists

Scaling Airflow to 11,000 DAGs Across Three Regions at Intercom with András Gombosi and Paul Vickers

The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI

AI Chapters

AI Insights

Reviews