A knowledge graph stores knowledge as (subject, predicate, object) triples: (Albert Einstein, born_in, Ulm), (Ulm, located_in, Germany). These triples form a graph where entities are nodes and relationships are edges. You can traverse the graph to answer multi-hop questions: "Where was the birthplace of the person who developed general relativity?" follows Einstein → born_in → Ulm → located_in → Germany.
The integration of knowledge graphs with LLMs takes several forms: using KGs as a source for RAG (retrieve relevant subgraphs for a query), using LLMs to populate KGs (extract entities and relationships from text), and using KGs to verify LLM outputs (check stated facts against the graph). GraphRAG (Microsoft) uses LLMs to build a knowledge graph from documents, then queries that graph for more structured retrieval than pure vector search.
Constructing a knowledge graph requires: entity extraction (identifying people, places, concepts in text), relation extraction (identifying how entities relate), entity resolution (recognizing that "NYC," "New York City," and "The Big Apple" are the same entity), and schema design (defining what types of entities and relationships exist). LLMs have made each of these steps cheaper and more accurate, democratizing KG construction for organizations that previously couldn't afford the manual effort.