Skip to content

feat: add documentation-aware, opt-in embedding refresh#653

Merged
tirth8205 merged 1 commit into
mainfrom
codex/reconcile-embeddings
Jul 17, 2026
Merged

feat: add documentation-aware, opt-in embedding refresh#653
tirth8205 merged 1 commit into
mainfrom
codex/reconcile-embeddings

Conversation

@tirth8205

@tirth8205 tirth8205 commented Jul 17, 2026

Copy link
Copy Markdown
Owner

Summary

This is the current-main replacement for the compatible parts of #602 and #599. It preserves Stefan Hudici's authorship with a Co-authored-by trailer.

Adopted from #602

  • store first-paragraph function/class documentation in the existing extra JSON field (no schema migration)
  • include the summary deterministically in semantic embedding text
  • support Python docstrings, JSDoc/Javadoc/Doxygen, C# XML docs, Rust outer docs, and Go doc comments

The port additionally fixes Python escape/concatenation semantics with ast.literal_eval, strips documentation markup, handles exported/template-wrapped declarations and Go directives, enforces a 400-character bound both at parse and provider-input time, and verifies full-build plus incremental persistence/removal.

Adopted and safety-adjusted from #599

  • purge embeddings for deleted or renamed graph nodes
  • keep hash-incremental refresh and exact provider identity checks
  • surface refresh/purge counts without failing graph builds

The source branch's default refresh wiring was intentionally not carried over. Build, update, postprocess, and watch remain default-off. Refresh occurs only when that specific invocation supplies both an exact provider and model. A never-embedded graph returns before provider resolution; an existing index must match the resolved provider/model/endpoint exactly; mixed, legacy, or mismatched identities are refused; missing credentials, offline model availability, and transport errors become build warnings. Manual embed also purges orphans, even when the local provider is unavailable.

Verification

Exact base: 4f9f035b49515fa79e4c084aafb10828d607ba7b

  • focused parser/embedding/tool/CLI/MCP tests: 411 passed, 1 skipped
  • complete non-daemon suite after final rebase: 1684 passed, 1 skipped, 2 xpassed
  • Ruff: clean
  • mypy (63 source files): clean
  • Bandit: zero findings
  • git diff --check: clean

The native macOS daemon/watcher test file remains excluded locally because the unchanged baseline crashes in the platform FSEvents layer; GitHub CI will run the repository's normal cross-platform matrix.

@github-actions

github-actions Bot commented Jul 17, 2026

Copy link
Copy Markdown

code-review-graph review

Overall risk: 0.80 (HIGH) — 69 changed function(s)/class(es), 0 affected flow(s), 38 test gap(s)

Risk-scored changes

Risk Level Symbol Location Tested
0.80 high code_review_graph/embeddings.py::_node_to_text code_review_graph/embeddings.py:802 no
0.80 high code_review_graph/embeddings.py::EmbeddingStore code_review_graph/embeddings.py:865 no
0.70 high code_review_graph/tools/build.py::_run_postprocess code_review_graph/tools/build.py:46 no
0.65 medium code_review_graph/embeddings.py::EmbeddingStore.purge_orphans code_review_graph/embeddings.py:981 no
0.65 medium code_review_graph/parser.py::CodeParser._get_docstring_summary code_review_graph/parser.py:5753 no
0.65 medium code_review_graph/tools/build.py::_run_embedding_refresh code_review_graph/tools/build.py:16 no
0.60 medium code_review_graph/parser.py::_clean_docstring_summary code_review_graph/parser.py:917 no
0.60 medium code_review_graph/parser.py::_strip_block_doc_comment code_review_graph/parser.py:944 no
0.60 medium code_review_graph/parser.py::CodeParser._python_docstring_value code_review_graph/parser.py:5764 no
0.60 medium code_review_graph/parser.py::CodeParser._line_doc_payload code_review_graph/parser.py:5802 no

Test gaps

  • code_review_graph/cli.py::_EmbeddingRefreshKwargs (code_review_graph/cli.py:60)
  • code_review_graph/cli.py::_add_embedding_refresh_args (code_review_graph/cli.py:372)
  • code_review_graph/cli.py::_embedding_refresh_kwargs (code_review_graph/cli.py:393)
  • code_review_graph/cli.py::main (code_review_graph/cli.py:409)
  • code_review_graph/embeddings.py::_node_to_text (code_review_graph/embeddings.py:802)
  • ...and 33 more without direct tests

Token savings: this graph-backed report used ~168,683 fewer tokens (~94%) than reading every changed file in full (estimated, chars/4 approximation).


Powered by code-review-graph — local-first analysis; no code leaves the CI runner.

@tirth8205
tirth8205 force-pushed the codex/reconcile-embeddings branch from cff9c84 to c2063f2 Compare July 17, 2026 17:28
Extract bounded language-aware documentation summaries, include them in semantic text, purge orphan vectors, and expose provider/model-scoped refresh as an explicit default-off operation.

Co-authored-by: Stefan Hudici <shudici@gmail.com>
@tirth8205
tirth8205 force-pushed the codex/reconcile-embeddings branch from c2063f2 to 9bccae6 Compare July 17, 2026 17:38
@tirth8205
tirth8205 merged commit adf8604 into main Jul 17, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant