feat: support omop emb 1.0.0

[nicoloesch]

Motivation

omop-emb introduces a new storage and DB concept with significant and breaking changes to support a local-first and backend-agnostic storage solution for the embeddings. To include these changes and fixes that come with a successful PR, we need to prepare omop-graph for the new incoming interfaces etc.

Closes #8

To prevent additional versioning updates, this PR also absorbed the following issues:

• Fixes Support orm-loader >0.4.0 #13
• Fixes Remove application-level lru_cache from KnowledgeGraph #12

[gkennos]

in README it mentions ClassIDEnum.HIERARCHICAL, but it should be ClassIDEnum.HIERARCHY

also stale refs: rank_paths, kg.find_shortest_paths, and kg.rank_paths

[gkennos]

in paths.py:

reconstruct_paths makes all nodes standard=False --> path metadata is wrong

[gkennos]

EdgeView.from_query() depends on positional column order rather than row names

[gkennos]

if you add synonym (bool) field to LabelMatch and populate from KnowledgeGraph.concept_lookup(), then LabelMatchGroupView can faithfully return direct/synonym --> at the moment it's only returning exact/partial/fulltext/embedding

[gkennos]

LabelMatchGroupView.from_matches assumes pre-sorting rather than explicit rule (or - stop using ms[0] as 'best' by definition)

[gkennos]

in paths.py

should we rename 'num_hops' in find_standard_paths to max_standard_hops or something? this is very specifically only traversing standard-standard relationships

alternatively: could add in param allow_non_standard_intermediates: bool = False

reason is that for MCP exploration/view use case, users will likely expect broader graph navigation

[gkennos]

find_standard_paths drops edges when multiple outgoing edges point to the same ID

this is an issue for concepts joined by >1 valid relationship

select * from (select concept_id_1, concept_id_2, count(distinct(relationship_id)) as c fr
om concept_relationship where concept_id_1 != concept_id_2 group by concept_id_1, concept_id_2) as d w
here c > 1 limit 10;
concept_id_1 | concept_id_2 | c
--------------+--------------+---
327 | 732315 | 2
8481 | 8600 | 2
8482 | 8569 | 2
8486 | 9222 | 2
8487 | 8545 | 2
8495 | 8548 | 2
8500 | 8609 | 2
8501 | 8611 | 2
8502 | 8610 | 2
8519 | 9540 | 2

  concept_views() returns one row per concept, while edges can contain multiple rows for the same object_id
  
  Build a lookup map by concept_id, then iterate all edges independently?

[gkennos]

PathProfile.from_path() cannot handle valid empty-path case returned when source == target

Profiling or explaining a zero-hop path will fail at runtime instead of returning a sensible self-match profile

[gkennos]

ranked_concepts = [
    _score_standard_concept(
        text=text,
        kg=kg,
        standard_concept=sc,
        num_ancestors=num_ancestors.get(sc.concept_id, 0),
        similarity_score=nearest_concept_matches_dict_for_single_query.get(sc.concept_id, None),
    )
    for sc in standard_concepts
]

_score_standard_concept just scores and does not rank --> confusing to downstream consumers

[gkennos]

traverse()

• During expansion, every discovered outgoing edge is appended to edges_out immediately.
• Neighbor nodes are only added to visited later when popped from the queue.
• If traversal terminates early because max_nodes is reached, some queued neighbor nodes may never become visited.
• This breaks the usual expectation that a subgraph’s edge set is closed over its node set.

[gkennos]

Scoring docs say relevance is composite, but _score_standard_concept() uses embedding similarity alone whenever it is available

I think this is intentional but perhaps having some kind of parameter to override this is worthwhile?

[gkennos]

Great! Confirmed all identified issues now closed with very minor outstanding issues - I am commenting rather than approving only because I think we need to release omop-alchemy 0.6.3 before we merge and release this update 👍