Arkency: Using LLMs and Event Sourcing to Maintain Organizational Knowledge Graphs

Arkency published an article detailing how large language models (LLMs) and event sourcing can be used to maintain an organizational knowledge graph. A knowledge graph is a structured way of representing information, making it easier to search and connect different pieces of data. Event sourcing is a technique where changes to data are stored as a sequence of events, allowing the system to reconstruct the current state at any point in time.

## How Knowledge Graphs and Event Sourcing Work A knowledge graph is a system that organizes information into a network of entities and their relationships. For example, in an organization, a knowledge graph could connect employees, projects, and departments, showing how they are related. Event sourcing is a method of storing data by keeping a log of all changes made to it. Instead of just saving the current state, every change is recorded as an event, which can be replayed to understand how the data evolved.

## Processing Unstructured Data with LLMs Arkency explains that LLMs can be used to process and extract meaningful information from unstructured data, such as emails, documents, and meeting notes. This information can then be converted into events that are stored in an event-sourced system. By doing this, the system can keep the knowledge graph up-to-date with the latest information. For example, if a new project is mentioned in an email, the LLM can identify this and create an event that updates the knowledge graph to include the new project and its relationships.

## Benefits for Organizations Maintaining a knowledge graph with LLMs and event sourcing has several advantages. It ensures that the knowledge graph is always current, as every change is captured and processed. This makes it easier for employees to find the information they need, improving productivity and decision-making. Additionally, the event log provides a complete history of changes, which can be useful for auditing and understanding how the organization's knowledge has evolved over time.

## Steps to Implement This Approach To start using LLMs and event sourcing for your organizational knowledge graph, you can follow these steps: 1. Identify the sources of unstructured data in your organization, such as emails, documents, and meeting notes. 2. Use an LLM to process this data and extract relevant information. 3. Convert the extracted information into events and store them in an event-sourced system. 4. Update the knowledge graph based on these events. 5. Regularly review and refine the process to ensure accuracy and completeness.

For more detailed guidance, you can read the full article on Arkency's blog.