[REQUEST] Integration of Graph Database (e.g., Neo4j) with Kotaemon

[EvelynBai]

Reference Issues

No response

Summary

Hi there! I'd like to say that this is a wonderful project! Great job on all the work you've put into it.

I was wondering if it would be possible to integrate a graph database, such as Neo4j, with Kotaemon? This would be a great addition for projects that require more advanced data relationships and graph-based queries.

Basic Example

Add support for a graph database like Neo4j to Kotaemon to manage and visualize both structured and unstructured data. Structured data could be inserted directly into the graph DB, while unstructured data could first be extracted using LLMs and then stored in the graph DB.

Drawbacks

Potentially heavy workload

Additional information

No response

[taprosoft]

@EvelynBai Can you describe further about the use-case of a GraphDB. Currently GraphDB is utilized internally in GraphRAG or alternatives such as LightRAG implementation. However, we haven't found a compelling reason to manage them from Kotaemon side. May be your insights can further help here.

[MattLeho]

I was going to ask the same thing right now, coincidence?
I found it easier to set up with Neo4j as the data is also accessible through their prebuilt apps, which is great- but it would be nice to have a better way to break down the information in a more finetuned way which I can achieve with Kotaemon.
The information I am dealing with- being an Urban Planner, ranges from 3d maps, GIS data and statistics, theory, and a lot of opinions to balance and consider! I've been using Neo4j to collate all this data together and help inform my work and not only that, the documentation is very detailed which allowed me to learn very quick.

[EvelynBai]

@EvelynBai Can you describe further about the use-case of a GraphDB. Currently GraphDB is utilized internally in GraphRAG or alternatives such as LightRAG implementation. However, we haven't found a compelling reason to manage them from Kotaemon side. May be your insights can further help here.

@taprosoft

As @MattLeho mentioned, if Kotaemon supports a graph database, it could enable efficient management and access to data from multiple sources.

Besides, I’m wondering if Kotaemon could support such use cases, allowing LLMs to answer queries based on data manually created in Neo4j? For example, I’m working with demo transaction data (CSV format with multiple columns like account, transaction time, etc.), but LLMs struggle to extract entities directly from the CSV. To work around this, I convert the data into natural language text, but the entity and relation extraction process is slow. Constructing a graph in Neo4j is much more efficient for managing structured data.

[taprosoft]

@MattLeho @EvelynBai thanks for your feedback, let me take a looks deeper at Neo4J features that might be suitable within Kotaemon.

By the way, could you help by pointing our the document regarding graph building feature (non-LLM) in Neo4J. I am also surveying methods to construct Knowledge Graph without reliance on LLM which is a slow and costly process.

Personally I am waiting for this https://www.microsoft.com/en-us/research/blog/lazygraphrag-setting-a-new-standard-for-quality-and-cost/ before considering official graphDB integration.

[MattLeho]

@taprosoft
LazyGraphRAG seems very promising indeed, i'd not heard of it yet.
I believe the documentation you are looking is here [https://neo4j.com/docs/getting-started/data-modeling/guide-data-modeling/]