feat: harden knowledgebase retrieval and role skills

[OpenCodeEngineer]

Summary

• consolidate OpenHands runtime to one agent_service.openhands.agent.Agent and preload roles from agents/agents.yaml
• add shared + role-specific knowledgebase-search skills and inject required KB skill instructions into agent context
• harden docs_tools retrieval:
  • explicit indexing includes agents/shared/knowledgebase
  • root markdown indexing is non-recursive
  • deterministic keyword fallback when BM25 returns no positive scores
• add SoftwareEngineer KB ingestion skill using docs_tools verification flow (instead of raw rg)
• add SupportEngineer KB-ingestion response compaction to keep KB eval responses concise/focused
• add OpenClaw docs/KB context injection controls and tests
• add dedicated Slack eval scenario for cross-agent KB ingestion/retrieval

Validation

• uv run python -m pytest tests/test_agents_md_loader.py tests/test_docs_tools.py tests/test_openclaw_docs_context.py tests/test_support_engineer_kb_response.py -v
• uv run python scripts/eval_slack_e2e.py --scenario knowledgebase_cross_agent_support_to_product --channel C0AATPSADB8 --timeout 600 (re-run after merge/deploy)

Fixes #328

[OpenCodeEngineer]

Added new Slack eval scenario in this branch:

• software_engineer_tooling_vibe_dev_health

It was executed against Slack (C0AATPSADB8, thread 1772666970.207689).

Result summary:

• Overall: ❌ Failed
• ToolingBootstrap: 0.20
• KubernetesAccess: 1.00
• HealthAndLogs: 0.90
• ResponseEfficiency: 0.90

Report path:

• results/eval_reports/eval_software_engineer_tooling_vibe_dev_health_20260304_233616.md

Failure reason is specifically tooling bootstrap evidence: agent reported jq missing and apt install permission error, so jq readiness was not proven.

[OpenCodeEngineer]

Rerun status after runtime sudo fix:

• Redeployed openhands-svc and vibeteam-gateway.
• Verified in live pod: jq-1.7, sudo -n true works, sudo apt-get update -qq works.
• Re-ran software_engineer_tooling_vibe_dev_health.

Run details:

• Thread: 1772678937.628759
• Report: results/eval_reports/eval_software_engineer_tooling_vibe_dev_health_20260305_025614.md
• Outcome: ❌ failed (same metric)
  • ToolingBootstrap: 0.20
  • KubernetesAccess: 1.00
  • HealthAndLogs: 0.80
  • ResponseEfficiency: 0.80

Conclusion:

• The previous permission blocker is fixed.
• Remaining failure is evaluator/rubric alignment: final Slack response provides summarized versions but not explicit command-output style evidence the judge expects, and includes off-scope Sentry/rollback content.

[OpenCodeEngineer]

Addressed prompt source duplication identified in review.

Root cause:

• OpenClaw gateway read prompt copies from k8s/base/openclaw-prompts/*, while canonical prompts live in agents/*/AGENTS.md.
• MarketingManager copy had drift from canonical prompt.

Changes:

• k8s/base/openclaw-gateway.yaml: mount prompt files from shared agents-config PVC (ProductManager/AGENTS.md, MarketingManager/AGENTS.md), add agents-sync init container.
• k8s/base/kustomization.yaml: remove openclaw-agent-prompts ConfigMap generator.
• Remove duplicate files under k8s/base/openclaw-prompts/.
• Update docs (docs/openclaw-introduction.md, docs/design.md).

Validation:

• kubectl kustomize k8s/base succeeds.

[OpenCodeEngineer]

Update pushed on branch chore/openhands-single-agent-config (commit 39f843d).

This addresses the prompt-source inconsistency you flagged:

• Legacy k8s/base/openclaw-prompts/* path is not used anymore.
• Live openclaw-gateway in AKS now mounts prompts from canonical agents/*/AGENTS.md via agents-config PVC subpaths.

Also fixed deploy convergence on AKS:

• k8s/overlays/dev/agents-config-patch.yaml now matches existing immutable PVC attributes (ReadWriteMany + azurefile-csi) so kubectl apply -k k8s/overlays/dev succeeds.

Operational notes from rollout:

• Pod crashlooped once due to missing LITELLM_BASE_URL / LITELLM_API_KEY in vibeteam-secrets (required by openclaw.json env interpolation).
• Secret patched; deployment recovered and rollout completed.

Verification done in cluster ~/.kube/aks-1, namespace vibeteam:

• openclaw-gateway volumeMount subPaths: ProductManager/AGENTS.md, MarketingManager/AGENTS.md
• stale openclaw-agent-prompts configmap removed.

[OpenCodeEngineer]

Post-change validation run:

• Executed Slack eval:
  • uv run python scripts/eval_slack_e2e.py --scenario support_400_errors --channel C0AATPSADB8 --timeout 600
  • Thread: 1772681024.141429
  • Report: results/eval_reports/eval_support_400_errors_20260305_032517.md

Result:

• Overall: FAILED
• InvestigationQuality: 0.60 (threshold 0.7)
• EvidenceBasedDecision: 1.00
• HandoffCompletion: 0.90
• ResponseEfficiency: 0.70

Note: this eval exercises SupportEngineer quality heuristics and is not directly tied to the OpenClaw prompt mount change; OpenClaw rollout and health checks are green in-cluster.

[OpenCodeEngineer]

Pushed follow-up commit 77b3981 to chore/openhands-single-agent-config.

Scope:

• Ensure both OpenHands and OpenClaw runtimes can self-modify shared agent config on request.

Changes:

1. k8s/base/openclaw-gateway.yaml

• Added AGENTS_DIR=/app/agents
• Added AGENTS_CONFIG_PATH=/app/agents/agents.yaml
• Added agents-config PVC mount at /app/agents
• Hardened agents-sync init script (lock cleanup + timeout)

2. k8s/base/openclaw-svc.yaml

• Added agents-sync init container
• Added AGENTS_DIR / AGENTS_CONFIG_PATH
• Added agents-config PVC volume + mount at /app/agents
• Hardened agents-sync init script (lock cleanup + timeout)

Runtime validation in AKS (~/.kube/aks-1):

• Rollout OK: openclaw-gateway, openclaw-svc, openhands-svc
• Write access checks OK for /app/agents in all three pods
• OpenClaw /run smoke test OK (CONFIG_WRITE_READY)

Post-change Slack eval run:

• support_400_errors thread 1772684106.444049
• report results/eval_reports/eval_support_400_errors_20260305_041641.md
• failed only on InvestigationQuality (0.60), unrelated to this infra/config writeability change.

[OpenCodeEngineer]

Docs update pushed in 8ffb8e0:

• Updated docs/design.md to explicitly document knowledgebase architecture:
  • layered model (filesystem instructions/config, docs retrieval, live operational evidence tools, session memory)
  • freshness/cache semantics (PVC visibility vs in-process role-routing cache behavior)
  • current OpenHands/OpenClaw config read/write model

This aligns the design doc with the current runtime behavior after the OpenClaw/OpenHands self-modification changes.

[OpenCodeEngineer]

Added KB management documentation update in 3c9650f.

docs/design.md now explicitly covers:

• current state: no end-user KB upload/manage API exposed by gateway
• target design: tenant-scoped KB upload/index/retrieve pipeline
• minimal API surface (/api/kb/documents, /api/kb/reindex, /api/kb/search)
• operational requirements (tenant isolation, versioning, async indexing, observability, safety checks)