Skip to content

recipes: add adaptive capture classification with confidence gating#139

Open
mjh756 wants to merge 4 commits intoNateBJones-Projects:mainfrom
mjh756:recipes/adaptive-capture-classification
Open

recipes: add adaptive capture classification with confidence gating#139
mjh756 wants to merge 4 commits intoNateBJones-Projects:mainfrom
mjh756:recipes/adaptive-capture-classification

Conversation

@mjh756
Copy link
Copy Markdown

@mjh756 mjh756 commented Mar 31, 2026

Contribution Type

  • Recipe (/recipes)

What does this do?

Adds a confidence gating and learning loop on top of OB1's existing capture flow. The classifier reports a confidence score (0–10); a per-type threshold gates whether to auto-classify or ask the user for confirmation; every user response nudges that threshold up or down. Over time the system learns which capture types it can reliably auto-classify and becomes progressively less intrusive. Includes optional A/B model comparison tables for empirically choosing between classifier models.

Requirements

  • Open Brain setup complete
  • Any OpenAI-compatible LLM gateway (OpenRouter, Ollama, etc.)
  • Supabase CLI or Supabase Studio (to run the SQL migration)

Checklist

  • I've read CONTRIBUTING.md
  • My contribution has a README.md with prerequisites, step-by-step instructions, and expected outcome
  • My metadata.json has all required fields
  • I tested this on my own Open Brain instance
  • No credentials, API keys, or secrets are included

@github-actions github-actions bot added the recipe Contribution: step-by-step recipe label Mar 31, 2026
@github-actions
Copy link
Copy Markdown

github-actions bot commented Mar 31, 2026

Hey @mjh756 — welcome to Open Brain Source! 👋

Thanks for submitting your first PR. The automated review will run shortly and check things like metadata, folder structure, and README completeness. If anything needs fixing, the review comment will tell you exactly what.

Once the automated checks pass, a human admin will review for quality and clarity. Expect a response within a few days.

If you have questions, check out CONTRIBUTING.md or open an issue.

matthallett1
matthallett1 requested changes Mar 31, 2026
View reviewed changes
Copy link
Copy Markdown
Collaborator

@matthallett1 matthallett1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review — Adaptive Capture Classification

Thoughtful recipe. The per-type threshold learning loop is a genuinely useful pattern, and the A/B model comparison is a nice bonus. Schema has proper RLS. Well-documented.

Issues to fix

1. metadata.json author mismatch
Author is listed as "Matt Hardy" / "matthardy" but the PR is from mjh756. Fix to match your GitHub account.

2. Classifier prompt uses non-OB1 types
The default types in classifier_prompt.md are:

["thought", "task", "note", "decision", "question", "event"]

OB1's canonical types are: idea, task, person_note, reference, decision, lesson, meeting, journal. The schema.sql and README should reference the standard types, even if the recipe allows customization.

3. No implementation code
This is pseudocode + schema only. That's fine for a recipe, but worth noting in the README that this requires the user to implement the gating logic in their capture interface. The pseudocode is clear enough to translate, but a TypeScript reference implementation (even a minimal one) would make this much more actionable.

What's good

  • RLS on all 4 tables
  • ab_comparisons CHECK constraint on winner field
  • Threshold floor (0.50) and ceiling (0.95) prevent the system from becoming fully autonomous or fully manual
  • Consistency check (run classifier twice when confidence is low) is a smart pattern
  • Schema is fully additive, clean rollback story

Fix the author and types, then this is ready.

@mjh756
Copy link
Copy Markdown
Author

mjh756 commented Mar 31, 2026

Thanks, my first git PR. The system is working well for me, Thanks to Nate, yourself and the community.

@matthallett1
Copy link
Copy Markdown
Collaborator

matthallett1 commented Mar 31, 2026

Thank you mjh! It is a fun ride. Glad to have you part of it!

   !
   !
   ^
  / \
 /___\
|=   =|
|     |
|     |
|     |
|     |
|     |
|     |
|     |
|     |
|     |

/|##!##|
/ |##!##|
/ |##!##|
| / ^ | ^ \ |
| / ( | ) \ |
|/ ( | ) |
(( ))
(( : ))
(( : ))
(( ))
(( ))
( )
.
.
.

@matthallett1
Copy link
Copy Markdown
Collaborator

matthallett1 commented Mar 31, 2026

That is meant to be a rocket...

@mjh756
Copy link
Copy Markdown
Author

mjh756 commented Mar 31, 2026 via email

@matthallett1
Copy link
Copy Markdown
Collaborator

matthallett1 commented Mar 31, 2026

Ha! BASIC on a C64? That’s legend. You were basically shipping before indie games existed. I bet you had PETSCII art that put my rocket ship to shame 😂 were you a Zork fan? I was a "ladder" freak on Kapro. :-)

@mjh756
Copy link
Copy Markdown
Author

mjh756 commented Apr 1, 2026 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@matthallett1 matthallett1 matthallett1 requested changes

Requested changes must be addressed to merge this pull request.

Assignees

No one assigned

Labels

recipe Contribution: step-by-step recipe

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@mjh756 @matthallett1