Skip to content

[recipes] Thought enrichment pipeline#9

Open
alanshurafa wants to merge 6 commits intomainfrom
contrib/alanshurafa/thought-enrichment
Open

[recipes] Thought enrichment pipeline#9
alanshurafa wants to merge 6 commits intomainfrom
contrib/alanshurafa/thought-enrichment

Conversation

@alanshurafa
Copy link
Copy Markdown
Owner

Summary

  • LLM-powered enrichment that classifies existing thoughts with type, importance, quality, sensitivity, topics, tags, people, action items
  • Supports OpenRouter (default) and Anthropic API providers
  • Includes companion scripts: backfill-type.mjs and backfill-sensitivity.mjs
  • Batching, retry, checkpoint/resume
  • Part 2 of 12 in OB1 Alpha Milestone

Test plan

  • Run enrich-thoughts.mjs --dry-run --batch-size 5
  • Run enrich-thoughts.mjs --apply --provider openrouter --batch-size 10
  • Verify enriched thoughts have populated columns
  • Confirm OB1 PR Gate passes

Add LLM-powered enrichment recipe that retroactively classifies existing
thoughts with type, importance, quality score, sensitivity tier, and
metadata (topics, tags, people, action items). Supports OpenRouter and
Anthropic providers with batching, retry, and checkpoint/resume.
Part of the OB1 alpha milestone.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@github-actions github-actions bot added the recipe label Apr 6, 2026
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3c1afad96a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

totalUpdated += updates.length;
}

offset += rows.length;
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Paginate type backfill by stable cursor

Advancing offset after mutating rows causes this backfill to skip records in --apply mode. fetchBatch() reads only type=eq.reference, but each successful update changes type away from reference, shrinking the result set before the next page; then offset += rows.length jumps past still-unprocessed rows. This means some eligible thoughts never get visited whenever a batch performs updates.

Useful? React with 👍 / 👎.

}
}

offset += data.length;
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Page sensitivity backfill with id cursor

This loop has the same shrinking-set pagination bug: it queries only rows with null/standard/empty sensitivity_tier, updates many of them to personal/restricted, and then increments offset. Because updated rows drop out of the filtered set, subsequent pages skip remaining candidates, so the script can finish with a large subset never scanned in --apply mode.

Useful? React with 👍 / 👎.

alanshurafa and others added 2 commits April 6, 2026 13:38
Add blank lines around headings (MD022), fenced code blocks (MD031),
and between adjacent blockquotes (MD028). Fix broken link fragment
(MD051) and remove extra blank line (MD012). No content changes.

These errors were blocking CI on all open PRs since the lint check
runs repo-wide.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Each section's numbered list now restarts at 1 instead of continuing
the global count (3-14), satisfying markdownlint MD029/ol-prefix rule.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@github-actions github-actions bot added documentation Improvements or additions to documentation integration labels Apr 6, 2026
alanshurafa and others added 3 commits April 6, 2026 13:53
OFFSET pagination skips rows when the query filters on columns being
mutated. Switching to id-based cursors (id=gt.lastId&order=id.asc)
ensures every row is visited exactly once regardless of mutations.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation integration recipe

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant