<description>Summary&lt;br /&gt;In this episode of the AI Engineering podcast, Philip Rathle, CTO of Neo4J, talks about the intersection of knowledge graphs and AI retrieval systems, specifically Retrieval Augmented Generation (RAG). He delves into GraphRAG, a novel approach that combines knowledge graphs with vector-based similarity search to enhance generative AI models. Philip explains how GraphRAG works by integrating a graph database for structured data storage, providing more accurate and explainable AI responses, and addressing limitations of traditional retrieval systems. The conversation covers technical aspects such as data modeling, entity extraction, and ontology use cases, as well as the infrastructure and workflow required to support GraphRAG, setting the stage for innovative applications across various industries.&lt;br /&gt;&lt;br /&gt;Announcements&lt;br /&gt;&lt;ul&gt;&lt;li&gt;Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems&lt;/li&gt;&lt;li&gt;Your host is Tobias Macey and today I'm interviewing Philip Rathle about the application of knowledge graphs in AI retrieval systems&lt;/li&gt;&lt;/ul&gt;Interview&lt;br /&gt;&lt;ul&gt;&lt;li&gt;Introduction&lt;/li&gt;&lt;li&gt;How did you get involved in machine learning?&lt;/li&gt;&lt;li&gt;Can you describe what GraphRAG is?&lt;ul&gt;&lt;li&gt;What are the capabilities that graph structures offer beyond vector/similarity-based retrieval methods of prompting?&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;What are some examples of the ways that semantic limitations of nearest-neighbor vector retrieval fail to provide relevant results?&lt;/li&gt;&lt;li&gt;What are the technical requirements to implement graph-augmented retrieval?&lt;ul&gt;&lt;li&gt;What are the concrete ways in which the embedding and retrieval steps of a typical RAG pipeline need to be modified to account for the addition of the graph?&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;Many tutorials for building vector-based knowledge repositories skip over considerations around data modeling. For building a graph-based knowledge repository there obviously needs to be a bit more work put in. What are the key design choices that need to be made for implementing the graph for an AI application?&lt;ul&gt;&lt;li&gt;How does the selection of the ontology/taxonomy impact the performance and capabilities of the resulting application?&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;Building a fully functional knowledge graph can be a significant undertaking on its own. How can LLMs and AI models help with the construction and maintenance of that knowledge repository?&lt;ul&gt;&lt;li&gt;What are some of the validation methods that should be brought to bear to ensure that the resulting graph properly represents the knowledge domain that you are trying to model?&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;Vector embedding and retrieval are a core building block for a majority of AI application frameworks. How much support do you see for GraphRAG in the ecosystem?&lt;ul&gt;&lt;li&gt;For the case where someone is using a framework that does not explicitly implement GraphRAG techniques, what are some of the implementation strategies that you have seen be most effective for adding that functionality?&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;What are some of the ways that the combination of vector search and knowledge graphs are useful independent of their combination with language models?&lt;/li&gt;&lt;li&gt;What are the most interesting, innovative, or unexpected ways that you have seen GraphRAG used?&lt;/li&gt;&lt;li&gt;What are the most interesting, unexpected, or challenging lessons that you have learned while working on GraphRAG applications?&lt;/li&gt;&lt;li&gt;When is GraphRAG the wrong choice?&lt;/li&gt;&lt;li&gt;What are the opportunities for improvement in the design and implementation of graph-based retrieval systems?&lt;/li&gt;&lt;/ul&gt;Contact Info&lt;br /&gt;&lt;ul&gt;&lt;li&gt;&lt;a href="https://www.linkedin.com/in/prathle/" target="_blank"&gt;LinkedIn&lt;/a&gt;&lt;/li&gt;&lt;/ul&gt;Parting Question&lt;br /&gt;&lt;ul&gt;&lt;li&gt;From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today?&lt;/li&gt;&lt;/ul&gt;Closing Announcements&lt;br /&gt;&lt;ul&gt;&lt;li&gt;Thank you for listening! Don't forget to check out our other shows. The &lt;a href="https://www.dataengineeringpodcast.com" target="_blank"&gt;Data Engineering Podcast&lt;/a&gt; covers the latest on modern data management. &lt;a href="https://www.pythonpodcast.com" target="_blank"&gt;Podcast.__init__&lt;/a&gt; covers the Python language, its community, and the innovative ways it is being used.&lt;/li&gt;&lt;li&gt;Visit the &lt;a href="https://www.aiengineeringpodcast.com" target="_blank"&gt;site&lt;/a&gt; to subscribe to the show, sign up for the mailing list, and read the show notes.&lt;/li&gt;&lt;li&gt;If you've learned something or tried out a project from the show then tell us about it! Email hosts@aiengineeringpodcast.com with your story.&lt;/li&gt;&lt;li&gt;To help other people find the show please leave a review on &lt;a href="https://podcasts.apple.com/us/podcast/the-machine-learning-podcast/id1626358243" target="_blank"&gt;iTunes&lt;/a&gt; and tell your friends and co-workers.&lt;/li&gt;&lt;/ul&gt;Links&lt;br /&gt;&lt;ul&gt;&lt;li&gt;&lt;a href="https://neo4j.com/" target="_blank"&gt;Neo4J&lt;/a&gt;&lt;/li&gt;&lt;li&gt;&lt;a href="https://neo4j.com/blog/graphrag-manifesto/" target="_blank"&gt;GraphRAG Manifesto&lt;/a&gt;&lt;/li&gt;&lt;li&gt;&lt;a href="https://github.blog/ai-and-ml/generative-ai/what-is-retrieval-augmented-generation-and-what-does-it-do-for-generative-ai/" target="_blank"&gt;RAG == Retrieval Augmented Generation&lt;/a&gt;&lt;ul&gt;&lt;li&gt;&lt;a href="https://www.aiengineeringpodcast.com/retrieval-augmented-generation-implementation-episode-34" target="_blank"&gt;Podcast Episode&lt;/a&gt;&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;a href="https://en.wikipedia.org/wiki/Very_large_database" target="_blank"&gt;VLDB == Very Large DataBases&lt;/a&gt;&lt;/li&gt;&lt;li&gt;&lt;a href="https://en.wikipedia.org/wiki/Knowledge_graph" target="_blank"&gt;Knowledge Graph&lt;/a&gt;&lt;/li&gt;&lt;li&gt;&lt;a href="https://en.wikipedia.org/wiki/Nearest_neighbor_search" target="_blank"&gt;Nearest Neighbor Search&lt;/a&gt;&lt;/li&gt;&lt;li&gt;&lt;a href="https://en.wikipedia.org/wiki/PageRank" target="_blank"&gt;PageRank&lt;/a&gt;&lt;/li&gt;&lt;li&gt;&lt;a href="https://blog.google/products/search/introducing-knowledge-graph-things-not/" target="_blank"&gt;Things Not Strings&lt;/a&gt;) Google Knowledge Graph Paper&lt;/li&gt;&lt;li&gt;&lt;a href="https://github.com/pgvector/pgvector" target="_blank"&gt;pgvector&lt;/a&gt;&lt;/li&gt;&lt;li&gt;&lt;a href="https://www.pinecone.io/" target="_blank"&gt;Pinecone&lt;/a&gt;&lt;ul&gt;&lt;li&gt;&lt;a href="https://www.dataengineeringpodcast.com/pinecone-vector-database-similarity-search-episode-189/" target="_blank"&gt;Data Engineering Podcast Episode&lt;/a&gt;&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;a href="https://neo4j.com/docs/getting-started/data-modeling/relational-to-graph-modeling/" target="_blank"&gt;Tables To Labels&lt;/a&gt;&lt;/li&gt;&lt;li&gt;&lt;a href="https://en.wikipedia.org/wiki/Natural_language_processing" target="_blank"&gt;NLP == Natural Language Processing&lt;/a&gt;&lt;/li&gt;&lt;li&gt;&lt;a href="https://graph.build/resources/ontology" target="_blank"&gt;Ontology&lt;/a&gt;&lt;/li&gt;&lt;li&gt;&lt;a href="https://www.langchain.com/" target="_blank"&gt;LangChain&lt;/a&gt;&lt;/li&gt;&lt;li&gt;&lt;a href="https://www.llamaindex.ai/" target="_blank"&gt;LlamaIndex&lt;/a&gt;&lt;/li&gt;&lt;li&gt;&lt;a href="https://en.wikipedia.org/wiki/Reinforcement_learning_from_human_feedback" target="_blank"&gt;RLHF == Reinforcement Learning with Human Feedback&lt;/a&gt;&lt;/li&gt;&lt;li&gt;&lt;a href="https://senzing.com/" target="_blank"&gt;Senzing&lt;/a&gt;&lt;/li&gt;&lt;li&gt;&lt;a href="https://neo4j.com/labs/genai-ecosystem/neoconverse/" target="_blank"&gt;NeoConverse&lt;/a&gt;&lt;/li&gt;&lt;li&gt;&lt;a href="https://en.wikipedia.org/wiki/Cypher_(query_language" target="_blank"&gt;Cypher&lt;/a&gt; query language&lt;/li&gt;&lt;li&gt;&lt;a href="https://www.gqlstandards.org/" target="_blank"&gt;GQL&lt;/a&gt; query standard&lt;/li&gt;&lt;li&gt;&lt;a href="https://aws.amazon.com/bedrock/" target="_blank"&gt;AWS Bedrock&lt;/a&gt;&lt;/li&gt;&lt;li&gt;&lt;a href="https://cloud.google.com/vertex-ai" target="_blank"&gt;Vertex AI&lt;/a&gt;&lt;/li&gt;&lt;li&gt;&lt;a href="https://www.sequoiacap.com/podcast/training-data-sebastian-siemiatkowski/" target="_blank"&gt;Sequoia Training Data - Klarna episode&lt;/a&gt;&lt;/li&gt;&lt;li&gt;&lt;a href="https://en.wikipedia.org/wiki/Ouroboros" target="_blank"&gt;Ouroboros&lt;/a&gt;&lt;/li&gt;&lt;/ul&gt;The intro and outro music is from &lt;a href="https://freemusicarchive.org/music/The_Freak_Fandango_Orchestra/Tales_Of_A_Dead_Fish/Hitmans_Lovesong/" target="_blank"&gt;Hitman's Lovesong feat. Paola Graziano&lt;/a&gt; by &lt;a href="http://freemusicarchive.org/music/The_Freak_Fandango_Orchestra/" target="_blank"&gt;The Freak Fandango Orchestra&lt;/a&gt;/&lt;a href="https://creativecommons.org/licenses/by-sa/3.0/" target="_blank"&gt;CC BY-SA 3.0&lt;/a&gt;</description>

AI Engineering Podcast

Tobias Macey

Enhancing AI Retrieval with Knowledge Graphs: A Deep Dive into GraphRAG

SEP 10, 202459 MIN
AI Engineering Podcast

Enhancing AI Retrieval with Knowledge Graphs: A Deep Dive into GraphRAG

SEP 10, 202459 MIN

Description

Summary<br />In this episode of the AI Engineering podcast, Philip Rathle, CTO of Neo4J, talks about the intersection of knowledge graphs and AI retrieval systems, specifically Retrieval Augmented Generation (RAG). He delves into GraphRAG, a novel approach that combines knowledge graphs with vector-based similarity search to enhance generative AI models. Philip explains how GraphRAG works by integrating a graph database for structured data storage, providing more accurate and explainable AI responses, and addressing limitations of traditional retrieval systems. The conversation covers technical aspects such as data modeling, entity extraction, and ontology use cases, as well as the infrastructure and workflow required to support GraphRAG, setting the stage for innovative applications across various industries.<br /><br />Announcements<br /><ul><li>Hello and welcome to the AI Engineering Podcast, your guide to the fast-moving world of building scalable and maintainable AI systems</li><li>Your host is Tobias Macey and today I'm interviewing Philip Rathle about the application of knowledge graphs in AI retrieval systems</li></ul>Interview<br /><ul><li>Introduction</li><li>How did you get involved in machine learning?</li><li>Can you describe what GraphRAG is?<ul><li>What are the capabilities that graph structures offer beyond vector/similarity-based retrieval methods of prompting?</li></ul></li><li>What are some examples of the ways that semantic limitations of nearest-neighbor vector retrieval fail to provide relevant results?</li><li>What are the technical requirements to implement graph-augmented retrieval?<ul><li>What are the concrete ways in which the embedding and retrieval steps of a typical RAG pipeline need to be modified to account for the addition of the graph?</li></ul></li><li>Many tutorials for building vector-based knowledge repositories skip over considerations around data modeling. For building a graph-based knowledge repository there obviously needs to be a bit more work put in. What are the key design choices that need to be made for implementing the graph for an AI application?<ul><li>How does the selection of the ontology/taxonomy impact the performance and capabilities of the resulting application?</li></ul></li><li>Building a fully functional knowledge graph can be a significant undertaking on its own. How can LLMs and AI models help with the construction and maintenance of that knowledge repository?<ul><li>What are some of the validation methods that should be brought to bear to ensure that the resulting graph properly represents the knowledge domain that you are trying to model?</li></ul></li><li>Vector embedding and retrieval are a core building block for a majority of AI application frameworks. How much support do you see for GraphRAG in the ecosystem?<ul><li>For the case where someone is using a framework that does not explicitly implement GraphRAG techniques, what are some of the implementation strategies that you have seen be most effective for adding that functionality?</li></ul></li><li>What are some of the ways that the combination of vector search and knowledge graphs are useful independent of their combination with language models?</li><li>What are the most interesting, innovative, or unexpected ways that you have seen GraphRAG used?</li><li>What are the most interesting, unexpected, or challenging lessons that you have learned while working on GraphRAG applications?</li><li>When is GraphRAG the wrong choice?</li><li>What are the opportunities for improvement in the design and implementation of graph-based retrieval systems?</li></ul>Contact Info<br /><ul><li><a href="https://www.linkedin.com/in/prathle/" target="_blank">LinkedIn</a></li></ul>Parting Question<br /><ul><li>From your perspective, what are the biggest gaps in tooling, technology, or training for AI systems today?</li></ul>Closing Announcements<br /><ul><li>Thank you for listening! Don't forget to check out our other shows. The <a href="https://www.dataengineeringpodcast.com" target="_blank">Data Engineering Podcast</a> covers the latest on modern data management. <a href="https://www.pythonpodcast.com" target="_blank">Podcast.__init__</a> covers the Python language, its community, and the innovative ways it is being used.</li><li>Visit the <a href="https://www.aiengineeringpodcast.com" target="_blank">site</a> to subscribe to the show, sign up for the mailing list, and read the show notes.</li><li>If you've learned something or tried out a project from the show then tell us about it! Email [email protected] with your story.</li><li>To help other people find the show please leave a review on <a href="https://podcasts.apple.com/us/podcast/the-machine-learning-podcast/id1626358243" target="_blank">iTunes</a> and tell your friends and co-workers.</li></ul>Links<br /><ul><li><a href="https://neo4j.com/" target="_blank">Neo4J</a></li><li><a href="https://neo4j.com/blog/graphrag-manifesto/" target="_blank">GraphRAG Manifesto</a></li><li><a href="https://github.blog/ai-and-ml/generative-ai/what-is-retrieval-augmented-generation-and-what-does-it-do-for-generative-ai/" target="_blank">RAG == Retrieval Augmented Generation</a><ul><li><a href="https://www.aiengineeringpodcast.com/retrieval-augmented-generation-implementation-episode-34" target="_blank">Podcast Episode</a></li></ul></li><li><a href="https://en.wikipedia.org/wiki/Very_large_database" target="_blank">VLDB == Very Large DataBases</a></li><li><a href="https://en.wikipedia.org/wiki/Knowledge_graph" target="_blank">Knowledge Graph</a></li><li><a href="https://en.wikipedia.org/wiki/Nearest_neighbor_search" target="_blank">Nearest Neighbor Search</a></li><li><a href="https://en.wikipedia.org/wiki/PageRank" target="_blank">PageRank</a></li><li><a href="https://blog.google/products/search/introducing-knowledge-graph-things-not/" target="_blank">Things Not Strings</a>) Google Knowledge Graph Paper</li><li><a href="https://github.com/pgvector/pgvector" target="_blank">pgvector</a></li><li><a href="https://www.pinecone.io/" target="_blank">Pinecone</a><ul><li><a href="https://www.dataengineeringpodcast.com/pinecone-vector-database-similarity-search-episode-189/" target="_blank">Data Engineering Podcast Episode</a></li></ul></li><li><a href="https://neo4j.com/docs/getting-started/data-modeling/relational-to-graph-modeling/" target="_blank">Tables To Labels</a></li><li><a href="https://en.wikipedia.org/wiki/Natural_language_processing" target="_blank">NLP == Natural Language Processing</a></li><li><a href="https://graph.build/resources/ontology" target="_blank">Ontology</a></li><li><a href="https://www.langchain.com/" target="_blank">LangChain</a></li><li><a href="https://www.llamaindex.ai/" target="_blank">LlamaIndex</a></li><li><a href="https://en.wikipedia.org/wiki/Reinforcement_learning_from_human_feedback" target="_blank">RLHF == Reinforcement Learning with Human Feedback</a></li><li><a href="https://senzing.com/" target="_blank">Senzing</a></li><li><a href="https://neo4j.com/labs/genai-ecosystem/neoconverse/" target="_blank">NeoConverse</a></li><li><a href="https://en.wikipedia.org/wiki/Cypher_(query_language" target="_blank">Cypher</a> query language</li><li><a href="https://www.gqlstandards.org/" target="_blank">GQL</a> query standard</li><li><a href="https://aws.amazon.com/bedrock/" target="_blank">AWS Bedrock</a></li><li><a href="https://cloud.google.com/vertex-ai" target="_blank">Vertex AI</a></li><li><a href="https://www.sequoiacap.com/podcast/training-data-sebastian-siemiatkowski/" target="_blank">Sequoia Training Data - Klarna episode</a></li><li><a href="https://en.wikipedia.org/wiki/Ouroboros" target="_blank">Ouroboros</a></li></ul>The intro and outro music is from <a href="https://freemusicarchive.org/music/The_Freak_Fandango_Orchestra/Tales_Of_A_Dead_Fish/Hitmans_Lovesong/" target="_blank">Hitman's Lovesong feat. Paola Graziano</a> by <a href="http://freemusicarchive.org/music/The_Freak_Fandango_Orchestra/" target="_blank">The Freak Fandango Orchestra</a>/<a href="https://creativecommons.org/licenses/by-sa/3.0/" target="_blank">CC BY-SA 3.0</a>