TopoGuard: Graph-Theory Defense Blocks Split-Knowledge Attacks on RAG Systems

Researchers have unveiled TopoGuard, a new defense system designed to protect Retrieval-Augmented Generation (RAG) systems from 'split-knowledge' attacks. RAG systems gather information from multiple external documents to answer complex queries. In a split-knowledge attack, adversaries inject documents that appear harmless individually but create false associations when combined and fed to a language model, misleading the AI.

This type of attack is particularly dangerous because it is structurally invisible to existing per-document filters like LlamaGuard, which check each document in isolation. TopoGuard uses graph theory—a mathematical framework for analyzing relationships—to detect these hidden patterns and block the attacks. The approach models the relationships between documents as a graph and identifies suspicious structural patterns that indicate a coordinated attack.

The research, published on arXiv, demonstrates that TopoGuard can effectively defend against split-knowledge attacks without requiring changes to the underlying RAG pipeline. This innovation could make AI systems more reliable and secure for everyday users, ensuring that the information they receive is accurate and trustworthy.