<div><br>In this special roundup episode of <em>The Data Engineering Show</em>, the Bros revisits some of the best bits from episodes with data thought leaders Zach Wilson, Matthew Housley, Joe Reis, and Krishnan Viswanathan, spotlighting essential trends and lessons learned across the evolving data engineering landscape. From data observability to bridging academia with real-world practice, this episode covers perspectives on where data engineering is heading and why certain challenges persist.<br><br></div><div><br>Topics include:<br><br></div><ul><li><strong>Foundations of Data Engineering</strong>: Zach Wilson emphasizes the importance of core, tech-agnostic skills in data modeling, quality assurance, and storytelling. By sharing his experiences at Airbnb and in education, he reveals that effective data engineering hinges on creating robust data models, quality controls, and persuasive narratives rather than expertise in any single tool or language.</li><li><strong>Bridging Academia and Practice:</strong> Matthew Housley and Joe Reis delve into the need for better data education, emphasizing hands-on experience and data fundamentals over tool-specific training, and advocate for apprenticeships and real-world collaborations in educational settings.</li><li><strong>Legacy Meets Modern in Data Engineering:</strong> Krishnan Viswanathan reflects on recurring themes in data engineering and the importance of adapting legacy approaches to new data needs, underscoring the challenges and benefits of vendor-built versus in-house solutions.<br><br></li></ul><div><br>Join the Bros for a well-rounded exploration of current themes in data engineering, filled with practical advice for data professionals at any stage of their journey.<br><br></div><div><br><br></div><div>The Data Engineering Show is handcrafted by our friends over at: <a href="https://www.fame.so/?utm_medium=podcast&amp;utm_source=bcast&amp;utm_campaign=masters-of-community-with-david-spinks?utm_medium=podcast&amp;utm_source=bcast&amp;utm_campaign=confessions-of-a-b2b-marketer">fame.so</a><br><br>Previous guests include: Joseph Machado of Linkedin, Metthew Weingarten of Disney, Joe Reis and Matt Housely, authors of <em>The Fundamentals of Data Engineering,&nbsp;</em>Zach Wilson of Eczachly Inc, Megan Lieu of Deepnote, Erik Heintare of Bolt, Lior Solomon of Vimeo, Krishna Naidu of Canva, Mike Cohen of Substack, Jens Larsson of Ark, Gunnar Tangring of Klarna, Yoav Shmaria of Similarweb and Xiaoxu Gao of Adyen.<br><br>Check out our three most downloaded episodes:</div><ul><li><a href="https://www.dataengineeringshow.com/e/6nrrrlzn">Zach Wilson on What Makes a Great Data Engineer</a></li><li><a href="https://www.dataengineeringshow.com/e/v8w441mn">Joe Reis and Matt Housley on The Fundamentals of Data Engineering</a></li><li><a href="https://www.dataengineeringshow.com/e/p8lxxy18">Bill Inmon, The Godfather of Data Warehousing</a></li></ul>

<description>This is a special episode of The Data Engineering Show, and joining the Bros is not one guest, nor even two – instead they’re revisiting the best bits from three different fascinating episodes. In each, they spotlight essential trends and lessons learned across the evolving data engineering landscape. From data observability to bridging academia with real-world practice, this episode covers perspectives on where data engineering is heading and why certain challenges persist.</description>

This is a special episode of The Data Engineering Show, and joining the Bros is not one guest, nor even two – instead they’re revisiting the best bits from three different fascinating episodes. In each, they spotlight essential trends and lessons learned across the evolving data engineering landscape. From data observability to bridging academia with real-world practice, this episode covers perspectives on where data engineering is heading and why certain challenges persist.

Data Rewind: Conversation Highlights from Zach Wilson, Matthew Housley, Joe Reis, and Krishnan Viswanathan

<div><br>In this episode of <em>The Data Engineering Show</em>, the bros, Eldad and Benjamin are joined by Ryanne Dolan from LinkedIn to discuss the innovative <a href="https://github.com/linkedin/Hoptimator">Hoptimator</a> (H2) project. This conversation reveals how LinkedIn has improved its data pipelines by automating the setup and management of complex workflows.<br><br></div><div><br>Together they cover:<br><br></div><ul><li><strong>Automated Data Pipelines:</strong> Ryanne explains how Hoptimator allows users to create and manage data pipelines using just a simple SQL SELECT query, streamlining the process of setting up Kafka topics, Flink jobs, and schemas.</li><li><strong>Integration with Kubernetes:</strong> The project utilizes Kubernetes to handle infrastructure tasks, treating Kubernetes as a database for managing state. This integration simplifies the orchestration of data workflows and automates routine tasks.</li><li><strong>Consumer-Driven Model:</strong> Ryanne discusses the shift from a producer-driven to a consumer-driven data model, emphasizing the importance of understanding and addressing consumer needs to reduce engineering complexity and optimize data systems.</li><li><strong>Future of Data Engineering:</strong> The conversation touches on the ongoing experimental nature of Hoptimator and its potential to transform data engineering practices, highlighting its impact on LinkedIn's data infrastructure.</li></ul><div><br><br></div><div>The Data Engineering Show is handcrafted by our friends over at: <a href="https://www.fame.so/?utm_medium=podcast&amp;utm_source=bcast&amp;utm_campaign=masters-of-community-with-david-spinks?utm_medium=podcast&amp;utm_source=bcast&amp;utm_campaign=confessions-of-a-b2b-marketer">fame.so</a><br><br>Previous guests include: Joseph Machado of Linkedin, Metthew Weingarten of Disney, Joe Reis and Matt Housely, authors of <em>The Fundamentals of Data Engineering,&nbsp;</em>Zach Wilson of Eczachly Inc, Megan Lieu of Deepnote, Erik Heintare of Bolt, Lior Solomon of Vimeo, Krishna Naidu of Canva, Mike Cohen of Substack, Jens Larsson of Ark, Gunnar Tangring of Klarna, Yoav Shmaria of Similarweb and Xiaoxu Gao of Adyen.<br><br>Check out our three most downloaded episodes:</div><ul><li><a href="https://www.dataengineeringshow.com/e/6nrrrlzn">Zach Wilson on What Makes a Great Data Engineer</a></li><li><a href="https://www.dataengineeringshow.com/e/v8w441mn">Joe Reis and Matt Housley on The Fundamentals of Data Engineering</a></li><li><a href="https://www.dataengineeringshow.com/e/p8lxxy18">Bill Inmon, The Godfather of Data Warehousing</a></li></ul>

<description>In this episode of The Data Engineering Show, Ryanne Dolan from LinkedIn joins the Bros to discuss LinkedIn's Hoptimator project. Ryanne explains how they’re simplifying complex data workflows by automating them through SQL queries, integrating Kubernetes, Kafka, and Flink. The conversation highlights the shift towards a consumer-driven data model and the future of data engineering.</description>

In this episode of The Data Engineering Show, Ryanne Dolan from LinkedIn joins the Bros to discuss LinkedIn's Hoptimator project. Ryanne explains how they’re simplifying complex data workflows by automating them through SQL queries, integrating Kubernetes, Kafka, and Flink. The conversation highlights the shift towards a consumer-driven data model and the future of data engineering.

The Resurgence of SQL: Insights from Ryanne Dolan from LinkedIn


        <p>SQL’s slow. SQL’s stupid. We hear these claims every time a new shiny tool enters the market, only to realize five years later when the hype dies down that SQL is actually a good idea. </p><p>In this super techie episode of the Data Engineering Show, Andy Pavlo, Associate Professor at Carnegie Mellon University, joins the bros to delve into database internals and optimization. </p><p>Andy discusses leveraging ML for autonomous database optimization, using Postgres for practical applications, tuning production databases safely, and why SQL is here to stay.</p>
      

<description>
        &lt;p&gt;SQL’s slow. SQL’s stupid. We hear these claims every time a new shiny tool enters the market, only to realize five years later when the hype dies down that SQL is actually a good idea. &lt;/p&gt;&lt;p&gt;In this super techie episode of the Data Engineering Show, Andy Pavlo, Associate Professor at Carnegie Mellon University, joins the bros to delve into database internals and optimization. &lt;/p&gt;&lt;p&gt;Andy discusses leveraging ML for autonomous database optimization, using Postgres for practical applications, tuning production databases safely, and why SQL is here to stay.&lt;/p&gt;
      </description>

Vector Databases Won’t Replace SQL - Andy Pavlo

<p>SQL’s slow. SQL’s stupid. We hear these claims every time a new shiny tool enters the market, only to realize five years later when the hype dies down that SQL is actually a good idea. </p><p>In this super techie episode of the Data Engineering Show, Andy Pavlo, Associate Professor at Carnegie Mellon University, joins the bros to delve into database internals and optimization. </p><p>Andy discusses leveraging ML for autonomous database optimization, using Postgres for practical applications, tuning production databases safely, and why SQL is here to stay.</p><div>The Data Engineering Show is handcrafted by our friends over at: <a href="https://www.fame.so/?utm_medium=podcast&amp;utm_source=bcast&amp;utm_campaign=masters-of-community-with-david-spinks?utm_medium=podcast&amp;utm_source=bcast&amp;utm_campaign=confessions-of-a-b2b-marketer">fame.so</a><br><br>Previous guests include: Joseph Machado of Linkedin, Metthew Weingarten of Disney, Joe Reis and Matt Housely, authors of <em>The Fundamentals of Data Engineering,&nbsp;</em>Zach Wilson of Eczachly Inc, Megan Lieu of Deepnote, Erik Heintare of Bolt, Lior Solomon of Vimeo, Krishna Naidu of Canva, Mike Cohen of Substack, Jens Larsson of Ark, Gunnar Tangring of Klarna, Yoav Shmaria of Similarweb and Xiaoxu Gao of Adyen.<br><br>Check out our three most downloaded episodes:</div><ul><li><a href="https://www.dataengineeringshow.com/e/6nrrrlzn">Zach Wilson on What Makes a Great Data Engineer</a></li><li><a href="https://www.dataengineeringshow.com/e/v8w441mn">Joe Reis and Matt Housley on The Fundamentals of Data Engineering</a></li><li><a href="https://www.dataengineeringshow.com/e/p8lxxy18">Bill Inmon, The Godfather of Data Warehousing</a></li></ul>

<description>SQL’s slow. SQL’s stupid. We hear these claims every time a new shiny tool enters the market, only to realize five years later when the hype dies down that SQL is actually a good idea. In this super techie episode of the Data Engineering Show, Andy Pavlo, Associate Professor at Carnegie Mellon University, joins the bros to delve into database internals and optimization. Andy discusses leveraging ML for autonomous database optimization, using Postgres for practical applications, tuning production databases safely, and why SQL is here to stay.</description>

SQL’s slow. SQL’s stupid. We hear these claims every time a new shiny tool enters the market, only to realize five years later when the hype dies down that SQL is actually a good idea. In this super techie episode of the Data Engineering Show, Andy Pavlo, Associate Professor at Carnegie Mellon University, joins the bros to delve into database internals and optimization. Andy discusses leveraging ML for autonomous database optimization, using Postgres for practical applications, tuning production databases safely, and why SQL is here to stay.


        <p>Too often expensive resources and manhours are spent on dashboards no one uses, resulting in zero ROI. Philip Philip Zelitchenko, VP of Data &amp; Analytics at ZoomInfo met the bros to talk about adopting product management principles to ensure data projects have value, and provide an unfiltered peak into ZoomInfo’s data stack and unique tech culture. </p>
      

<description>
        &lt;p&gt;Too often expensive resources and manhours are spent on dashboards no one uses, resulting in zero ROI. Philip Philip Zelitchenko, VP of Data &amp;amp; Analytics at ZoomInfo met the bros to talk about adopting product management principles to ensure data projects have value, and provide an unfiltered peak into ZoomInfo’s data stack and unique tech culture. &lt;/p&gt;
      </description>

How ZoomInfo transitioned from data graveyards to ROI-driven data projects

The Data Engineering Show is a podcast for data engineering and BI practitioners to go beyond theory. Learn from the biggest influencers in tech about their practical day-to-day data challenges and solutions in a casual and fun setting.

SEASON 1 DATA BROS
Eldad and Boaz Farkash shared the same stuffed toys growing up as well as a big passion for data. After founding Sisense and building it to become a high-growth analytics unicorn, they moved on to their next venture, Firebolt, a leading high-performance cloud data warehouse.

SEASON 2 DATA BROS
In season 2 Eldad adopted a brilliant new little brother, and with their shared love for query processing, the connection was immediate. After excelling in his MS, Computer Science degree, Benjamin Wagner joined Firebolt to lead its query processing team and is a rising star in the data space. 

For inquiries contact tamar@firebolt.io

The Data Engineering Show

Details

Recent Episodes