AI-Driven Knowledge Graph Construction

Abstract

Knowledge graphs -- structured representations of entities and their relationships -- underpin applications ranging from search engines and recommendation systems to drug discovery and enterprise data integration. Traditional knowledge graph construction requires extensive manual curation that cannot scale to the volume of knowledge embedded in scientific literature, enterprise documents, and the open web. AI-driven knowledge graph construction -- using information extraction, entity linking, relation classification, and graph completion to automatically populate and extend knowledge graphs -- has advanced substantially but remains challenged by ambiguity, domain specificity, and the hallucination risks of LLM-based extraction. This study evaluates nine AI systems for knowledge graph construction across four domains: biomedical (PubMed literature), legal (EU legislation and case law), financial (company filings and news), and general web (WikiData extension). Systems include rule-based baselines, BERT-based relation extractors (REBEL, UniRel), LLM extractors (GPT-4-KG, Claude-KG), graph neural network completion (RotatE, ComplEx), a proposed LLM-GNN hybrid (HybridKGC), and a human expert baseline. Evaluation covers precision, recall, F1 for entity and relation extraction, graph completion accuracy (MRR, Hits@10), and hallucination rate for LLM-generated triples. HybridKGC achieves the highest F1 for relation extraction across all domains (mean F1 0.784), combining LLM extraction coverage with GNN consistency checking. GPT-4-KG achieves 84.2% recall but 22.4% hallucination rate for biomedical triples. RotatE achieves MRR 0.482 for biomedical graph completion. A domain-specific knowledge graph construction pipeline and quality assurance framework are proposed.