The journey towards a knowledge graph for generative AI

How does the journey to a knowledge graph start with unstructured data—such as text, images, and other media? The evolution of web search engines offers an instructive example, showing how knowledge can be extracted from unstructured sources and refined over time into a structured, interconnected graph. As we will show in this post, this process underpins the journey from retrieval-augmented generation (RAG) systems to more sophisticated approaches like GraphRAG and Knowledge-GraphRAG.

From isolated nodes to graph of knowledge and knowledge graph 

Early search engines like AltaVista relied on simple keyword matching, treating web pages as isolated entities. However, web pages are interconnected through hyperlinks. Google transformed search by recognizing that the world wide web is not merely a collection of standalone pages but a vast network of interconnected knowledge—what we refer to as a graph of knowledge. 

From AltaVista to Google and from strings to things.
RelationalAI

While this approach significantly enhanced search capabilities, it soon became apparent that to support more advanced functions like reasoning, a more robust solution was necessary: a structured, machine-readable framework. This shift in perspective culminated in Google’s 2012 introduction of the knowledge graph, encapsulated by the phrase “things not strings,” which aimed to connect entities rather than just words.

The graph of knowledge vs. the knowledge graph: GoK is a broader, more conceptual idea focusing on interconnected information, without necessarily being highly structured. KG refers to a formal, structured, machine-readable network of entities and relationships, designed for advanced reasoning and AI tasks.
RelationalAI

Whereas the graph of knowledge (GoK) is a broader, more conceptual idea focusing on interconnected information, without necessarily being highly structured, the knowledge graph (KG) refers to a formal, structured, machine-readable network of entities and relationships, designed for advanced reasoning and AI tasks.

Tim Berners-Lee, the inventor of the web, had long foreseen this need for a structured way to organize information, coining the term “semantic web” in his book Weaving the Web. While this vision of the semantic web took time to materialize, Google’s knowledge graph made it practical, setting the stage for the development of sophisticated AI systems that could reason over these knowledge networks. Similarly, companies like Amazon created a product graph, and the open-source community worked on initiatives like Wikidata, which organized Wikipedia into a massive, public knowledge graph.

From knowledge graphs to question answering 

The creation of knowledge graphs transformed how information was retrieved, organized, and connected, moving from simple keyword matching to sophisticated entity recognition. But this advancement didn’t stop at improving web search. It became a cornerstone in solving more complex problems in AI, particularly in the realm of question answering (QA) systems.

QA systems are one of the most powerful applications in the generative AI space, requiring the ability to extract precise information from both structured and unstructured data. As the complexity of questions increases, so does the need for more structured, interconnected knowledge—just as the development of knowledge graphs addressed the need for a deeper, more context-aware web search.

There are three common types of questions, each with varying levels of complexity and requirements for structured data:

Single-point access questions: Simple fact-based queries that can be answered by retrieving a single text snippet.

Multi-point access questions: Questions requiring multiple text snippets that must be retrieved and presented together for comprehensive answers.

Advanced reasoning questions: More complex queries that necessitate integrating multiple pieces of information, often requiring symbolic reasoning that goes beyond the capabilities of standard language models.

While the first two question types can often be answered using a graph of knowledge, the third type (advanced reasoning questions) demands a more structured approach—a true knowledge graph.

Examples of different types of questions. 
RelationalAI

From RAG to GraphRAG: Answering questions over a GoK

Retrieval-augmented generation has emerged as the state-of-the-art approach for question answering in the generative AI era. Like the early keyword-based search engines, RAG treats documents as independent entities, indexing each document segment separately. So while RAG is effective for simpler queries, the approach doesn’t leverage the deeper connections between information that exist across documents.

From RAG to GraphRAG. 
RelationalAI

To address this limitation, Microsoft introduced the concept of GraphRAG in early 2024. GraphRAG organizes information into a graph of knowledge, enabling it to leverage relationships between pieces of information, much like how Google revolutionized web search by treating pages as part of an interconnected web. Large language models (LLMs) play a crucial role in this process. When presented with a set of documents, LLMs generate entities and relationships in the form of triplets. Although these triplets may contain noise or redundancy, they offer a robust method for organizing information effectively. 

By treating text passages as nodes in a graph, GraphRAG enables graph operations like community detection, pattern extraction, and graph traversal. These operations allow for the synthesis of multiple pieces of information, which can then be fed into RAG models to generate richer, more accurate answers to multi-point questions.

In short, GraphRAG helps build a graph of knowlege by connecting fragmented text into a graph-like structure, providing LLMs with more relevant, interconnected input to improve question answering performance.

In the GraphRAG pipeline, documents are linked together (graph of knowledge) through entities and relations among them, followed by community detection, which results in a summary for each community of documents. 
RelationalAI

From GraphRAG to Knowledge-GraphRAG: Answering questions over a KG

While GraphRAG relies on the reasoning capabilities of LLMs to connect text-based data in a graph of knowledge, the third type of question—those requiring deep reasoning—need more than a GoK. They require a fully structured knowledge graph, where facts, entities, and relationships are organized into a formal ontology. Knowledge graphs not only store factual information but also capture complex relationships and rules that are essential for advanced reasoning.

In these scenarios, LLMs are still important, but their role shifts from generating or synthesizing content to querying the structured KG. The LLM retrieves entities and relations from the knowledge graph and formulates a query based on the structured knowledge within it. A specialized knowledge graph engine then executes the query, returning a precise and logical answer.

This process is detailed in our recent  publication, QirK: Question Answering via Intermediate Representation on Knowledge Graphs, which outlines a framework for combining LLM capabilities with the logical power of knowledge graphs to answer complex queries. The framework supports question answering on top of the popular Wikidata knowledge graph. Example questions that can be answered are “Name a movie directed by Quentin Tarantino or Martin Scorsese that has Robert De Niro as a cast member”, “Which movie’s director is married to a cast member?”, and “List the movies in which both Robert De Niro and Al Pacino were casted”. 

Qirk architecture. 
RelationalAI

The road to advanced AI reasoning

Generative AI provides an unprecedented opportunity to reshape the way we organize and retrieve knowledge. The journey from unstructured data (texts, images, etc.) to a fully structured knowledge graph—rich in facts, logical constraints, and recursive rules—is complex and challenging, but the rewards are immense. Intermediate steps, such as the graph of knowledge, offer practical solutions that advance AI applications like question answering, even as we work toward the more ambitious goal of fully realized knowledge graphs.

While the path to creating high-quality knowledge graphs may be long, tools like GraphRAG represent important milestones on this journey. By bridging the gap between unstructured text and structured knowledge, they pave the way for AI systems capable of answering increasingly complex questions with greater accuracy, making the vision of advanced, reasoning-powered QA systems a reality.Nikolaos Vasiloglou is VP of Research-ML for RelationalAI, the industry’s first knowledge graph coprocessor for the data cloud. Nikolaos has over 20 years of experience implementing high-value machine learning and AI solutions across various industries.  

—

Generative AI Insights provides a venue for technology leaders—including vendors and other outside contributors—to explore and discuss the challenges and opportunities of generative artificial intelligence. The selection is wide-ranging, from technology deep dives to case studies to expert opinion, but also subjective, based on our judgment of which topics and treatments will best serve InfoWorld’s technically sophisticated audience. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Contact doug_dineley@foundryco.com.