docs: clarify self-hosted provider alternatives in README and Concepts - #15
Conversation
f1d7a36 to
7147b72
Compare
adelinamart
left a comment
There was a problem hiding this comment.
Thanks Radu — good change. The provider table is much clearer than the single
OpenAI-only paragraph it replaces, and splitting the four paths out is the right
call.
I checked every env var in the new Concepts section against the code and they
all line up: LLM_PROVIDER, OPENAI_BASE_URL, OPENAI_LLM_MODEL,
OPENAI_EMBEDDING_MODEL, OPENAI_EMBEDDING_DIMENSIONS, EMBEDDING_PROVIDER
with the Voyage/Cohere keys. The 1536 advice is a good catch specifically —
embed() truncates anything longer, so asking Gemini for 1536 natively avoids
throwing away half the vector.
Four things before this goes in:
-
This is open twice. #15 and #16 are identical (same tree hash). Could you
close one? -
gemini-3.6-flash— did you actually run the Gemini path end to end, or
is this written from the pattern? I'd rather not publish a model name we
haven't tried. -
Related: our OpenAI path sends
response_format: {type: "json_object"}
andmax_tokens(packages/shared/src/llm.ts). Extraction needs strict JSON
back. If Google's compatibility layer handles those differently, the path
would look fine and quietly produce nothing. One real extraction run against
Gemini would settle it — happy to merge once you've confirmed. -
Small thing: moving the upgrade note into
<details>dropped
"(pulls the new Perception image and applies startup DB migrations)". That
parenthetical is the reason people re-runupat all — can you put it back?
One note for next time on index.html: it's a built file, so a five-word edit
shows up as the whole page being rewritten. Nothing wrong with what you did —
both copies inside it (the HTML and the RSC payload) were updated consistently,
which is what I'd check for. Just flagging it so we both know the diff size is
noise, not scope.
Default Anthropic+OpenAI is one combo, not the only path. Document OpenAI-only, Gemini (same OPENAI_* vars), other embeddings, and fully local under Concepts; keep the README install path light and keep cloud Option 2 free of self-hosted key/upgrade notes. Mirror the note on the site.
Switch Concepts to gemini-3.5-flash-lite (works with json_object at max_tokens=300) and restore the Perception image/migrations note.
|
Hey @adelinamart, fixed the issues, PTAL. If you'd like I can also merge the 2 commits because they solve the same issue.
2 and 3. Gemini model + json_object / max_tokens
For the embeddings gemini-embedding-001 at 1536 works ok.
|
Summary
Fixes issue #14
Also clarified alternatives in robrain.dev docs