fix(rag): initialize the LLM service a RAG session creates

[ayaangazali]

What is wrong

#735 fixed this in HttpServer::loadModel and stated the rule in its comment: rac_llm_create() only routes to the plugin's create op, and a backend that defers the weight load to initialize (MLX) is not loaded when it returns. llama.cpp loads synchronously inside create, which is why the omission stays invisible until another backend reaches generate.

rac_rag_session_create_proto has the same shape. It creates the session's generation model:

rc = rac_llm_create(llm_path.c_str(), &llm_handle);

and hands that handle straight to RAGBackend:

session->backend = std::make_unique<RAGBackend>(backend_config, llm_handle, embed_handle, ...);

rac_llm_initialize does not appear anywhere in rac_rag_proto_abi.cpp. The handle is then driven directly through:

• rag_pipeline_graph.cpp:85 (rac_llm_generate)
• rag_pipeline_graph.cpp:357 (rac_llm_generate_stream)
• rag_rerank.cpp:121 (rac_llm_generate)

So a RAG session backed by MLX fails at generation with "model is not loaded", for exactly the reason the server did.

Why I am confident this is the same defect and not a deliberate difference

I checked every rac_llm_create call site in the tree. There are four, and the other three settle the question:

call site	calls rac_llm_initialize?
llm_module.cpp:335 (llm_create_service)	yes, and always has
http_server.cpp:309	yes, as of #735
rac_rag_proto_abi.cpp:781	no
core/tests/test_advanced_modality_proto_abi.cpp:1149	test

What this does

Adds the initialize call, with the same failure handling as the sibling early-out just above it: destroy the LLM handle, release the embedding service created a few lines earlier, and publish the same rag.sessionCreate failure the other early-outs publish. 11 lines.

Verification

Built and ran the full commons suite on this branch:

cmake -B build -DRAC_BUILD_TESTS=ON -DCMAKE_BUILD_TYPE=Debug
cmake --build build -j$(sysctl -n hw.logicalcpu)   # clean, 0 errors
ctest --test-dir build -j4                          # 100% tests passed out of 100

No test added, and I want to be straight about why: reproducing the failure needs a backend whose create defers the load, which in practice means MLX. The commons suite has no such backend, so a test here would either need a fake plugin built for this one case or would pass on llamacpp whether or not the fix is present. #735 added no test for the same reason. What I did instead was enumerate the call sites above, so the claim rests on the invariant the other three already follow rather than on a test that cannot distinguish the two states.

Summary by CodeRabbit

• Bug Fixes
  • RAG sessions now correctly initialize the language model service with the resolved model path.
  • Session creation now stops cleanly when model initialization fails.
  • Failed session creation also properly releases associated language model and embedding resources.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: babdec09-bdf1-4920-af22-e0d9ec9ea376

📥 Commits

Reviewing files that changed from the base of the PR and between 101bf2f and c3c69bd.

📒 Files selected for processing (1)

• core/src/features/rag/rac_rag_proto_abi.cpp

Included review availability: Your plan includes up to 10 reviews per rolling hour; 9 remain after this review.

---

📝 Walkthrough

Walkthrough

RAG session creation now initializes the optional LLM service with its resolved model path. Initialization failures clean up both service handles, publish the error, and stop session creation.

Changes

RAG session creation

Layer / File(s)	Summary
LLM initialization and failure cleanup core/src/features/rag/rac_rag_proto_abi.cpp	The session creation path initializes the LLM after creation. It releases the LLM and embedding services, publishes the failure, and returns when initialization fails.

Estimated code review effort: 2 (Simple) | ~10 minutes

Merge Risk: ⚪ Minimal · up to c3c69

The change initializes LLM services created for RAG sessions, preventing deferred-load backends from failing at generation; no actionable merge-blocking risk remains after normal checks and review.

Suggested reviewers: sanchitmonga22

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely identifies the main change: initializing the LLM service created for a RAG session.
Description check	✅ Passed	The description clearly explains the defect, implementation, failure handling, and verification, despite not using every template heading.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}