[Feature]: Implement persistent organizational knowledge base for contextual AI responses

[MatthewGrimshaw]

What problem are you trying to solve?

The Azure FinOps agent treats every conversation as if it knows nothing about the user's organization. Users in large enterprises have domain-specific context that would dramatically improve the quality of the agent's analysis — subscription-to-application mappings, cost center ownership hierarchies, SLA/SLO/RTO/RPO targets, preferred analysis patterns, tagging conventions, and custom instructions. Today, users must repeat this context in every conversation, which is tedious, error-prone, and doesn't scale across teams.

What users want to tell the agent once and have it remember:

• "Subscription abc-123 runs our Patient Portal app (Tier 1, RTO 4h, RPO 1h). Subscription def-456 is the Discovery research platform (Tier 3, no SLA)."
• "Our cost centers are mapped by the CostCenter tag. The finance team owns CC-1000 through CC-1099. Engineering owns CC-2000+."
• "When analyzing costs, always group by the ApplicationPortfolio tag first, then by Environment (prod/staging/dev). Ignore sandbox subscriptions matching rg-sandbox-*."
• "Our budget cycle runs April–March (fiscal year). Q1 is Apr–Jun."
• "Flag any resource in westeurope that costs more than $500/month — we're migrating out of that region."

Root cause analysis based on the current codebase:

1. The system prompt is static and generic — CopilotSessionFactory.cs defines a ~3,000-word system prompt that covers response formatting, tool usage, and FinOps frameworks. It contains zero organization-specific context. The prompt is compiled once at startup and shared across all users.

2. Per-request context injection is limited to connection state and uploads — ChatEndpoints.cs prepends two dynamic blocks to every user prompt: a [CONTEXT: ...] block with Azure connection status, and an [UPLOADED FILES ...] block listing session file uploads. There is no mechanism to inject persistent organizational knowledge.

3. Uploaded files are ephemeral — UploadedFileTools.cs stores user-uploaded files in /tmp with a 30-minute TTL. Files are cleared on chat reset and lost on container restart. Users cannot persist reference data (subscription mappings, cost center tables, analysis templates) across sessions.

4. No per-user or per-tenant knowledge store — there is no equivalent of GitHub Copilot's .github/copilot-instructions.md or custom instructions. The agent has no way to learn and retain organizational context.

5. No tool for querying knowledge — even if knowledge were stored, the LLM has no tool to look up organizational context during a conversation. When a user asks "what's the cost of the Patient Portal app?", the LLM cannot resolve "Patient Portal" to a set of subscriptions without the mapping being provided in-session.

Dependency: This issue builds on the persistent user sessions issue. Deterministic Entra ID-based user identity is required so that knowledge articles are scoped to the correct user/tenant and persist across sessions.

Proposed solution

A three-layer approach: a persistent knowledge store with CRUD API, context injection into every LLM prompt, and an LLM tool for querying knowledge during conversations.

Layer 1: Knowledge store (KnowledgeStore)

New file: src/Dashboard/Services/KnowledgeStore.cs

A file-based, per-user store for organizational knowledge articles. Each article is a named document containing structured or unstructured text that the LLM can reference.

Data model:

public class KnowledgeArticle
{
    public string Id { get; set; }             // Auto-generated 8-char hex
    public string Title { get; set; }          // Short name: "Subscription Mappings", "Cost Analysis Instructions"
    public string Category { get; set; }       // "subscriptions", "cost_centers", "instructions", "architecture", "sla", "custom"
    public string Content { get; set; }        // The actual knowledge text (markdown, CSV, JSON, or free text)
    public long UserId { get; set; }           // Entra OID-derived deterministic ID
    public DateTime CreatedUtc { get; set; }
    public DateTime UpdatedUtc { get; set; }
    public bool Active { get; set; } = true;   // Soft-delete: inactive articles are excluded from context
}

Storage:

• Per-user directory: ~/finops-agent-knowledge/{userId}/
• Each article stored as {id}.json — individual files enable atomic updates without locking the entire store.
• The ~/ path resolves to /home/ on Azure App Service (durable across restarts) or $TEMP for local dev.

Size constraints:

• Maximum 20 knowledge articles per user — prevents unbounded context growth.
• Maximum 10,000 characters per article — keeps individual articles within a reasonable LLM context window.
• Maximum 50,000 characters total across all articles per user — ensures the combined context injection doesn't exceed ~12,500 tokens (~25% of a 50K-token context window), leaving room for the system prompt, conversation history, and tool results.

Implementation:

public static class KnowledgeStore
{
    private static string UserDir(long userId) =>
        Path.Combine(Environment.GetEnvironmentVariable("HOME") ?? Path.GetTempPath(),
            "finops-agent-knowledge", userId.ToString());

    public static List<KnowledgeArticle> ListForUser(long userId) { ... }
    public static KnowledgeArticle? Get(long userId, string id) { ... }
    public static KnowledgeArticle Save(long userId, KnowledgeArticle article) { ... }
    public static bool Delete(long userId, string id) { ... }

    /// <summary>
    /// Returns the concatenated context block for all active articles,
    /// formatted for LLM injection. Called on every chat request.
    /// </summary>
    public static string BuildContextBlock(long userId)
    {
        var articles = ListForUser(userId).Where(a => a.Active).ToList();
        if (articles.Count == 0) return "";

        var sb = new StringBuilder();
        sb.AppendLine("[ORGANIZATIONAL KNOWLEDGE — The user has provided the following reference information about their environment. Use this to enrich your analysis, resolve entity names, apply their preferred conventions, and follow their instructions. This knowledge persists across sessions.]");
        foreach (var article in articles.OrderBy(a => a.Category))
        {
            sb.AppendLine($"\n### {article.Title} ({article.Category})");
            sb.AppendLine(article.Content);
        }
        return sb.ToString();
    }
}

Layer 2: Inject knowledge context into every LLM prompt

File: src/Dashboard/AI/ChatEndpoints.cs

Insert the knowledge context block between the connection context and the user's prompt, following the same pattern used for uploaded files.

Implementation — add after the existing uploadsContext construction (around line 115):

// Surface persistent organizational knowledge for this user
var knowledgeContext = KnowledgeStore.BuildContextBlock(userId);

prompt = string.IsNullOrEmpty(uploadsContext)
    ? string.IsNullOrEmpty(knowledgeContext)
        ? $"{connectionContext}\n{prompt}"
        : $"{connectionContext}\n{knowledgeContext}\n{prompt}"
    : string.IsNullOrEmpty(knowledgeContext)
        ? $"{connectionContext}\n{uploadsContext}\n{prompt}"
        : $"{connectionContext}\n{knowledgeContext}\n{uploadsContext}\n{prompt}";

The resulting prompt structure for each chat request becomes:

[CONTEXT: User IS connected to Azure. Available APIs: ...]
[ORGANIZATIONAL KNOWLEDGE — ...]
### Subscription Mappings (subscriptions)
abc-123 = Patient Portal (Tier 1, RTO 4h)
def-456 = Discovery Platform (Tier 3)
...
### Cost Analysis Instructions (instructions)
Always group by ApplicationPortfolio tag first...
...
[UPLOADED FILES IN THIS SESSION — ...]
{user's actual question}

This injection happens per-request (not baked into the system prompt), so knowledge updates take effect immediately without session recycling.

Layer 3: Knowledge CRUD API

New file: src/Dashboard/Endpoints/KnowledgeEndpoints.cs

REST API for managing knowledge articles:

GET    /api/knowledge              — list all articles for current user (title, category, id, active, updatedUtc)
GET    /api/knowledge/{id}         — get full article content
POST   /api/knowledge              — create a new article (body: { title, category, content })
PUT    /api/knowledge/{id}         — update an existing article (body: { title?, category?, content?, active? })
DELETE /api/knowledge/{id}         — permanently delete an article
POST   /api/knowledge/import       — import knowledge from an uploaded file (CSV/JSON/TXT)

All endpoints require an authenticated session. Articles are scoped by the deterministic Entra OID-derived userId — users can only access their own knowledge.

The POST /api/knowledge/import endpoint enables bulk import from structured files:

• CSV/TSV with subscription_id, application, tier, owner columns → auto-creates a "Subscription Mappings" article formatted as a markdown table.
• JSON with key-value or array structure → auto-creates an article with the JSON formatted for LLM readability.
• TXT/MD file → creates an article with the raw text as content.

This reuses the existing file upload infrastructure (UploadedFileTools.RegisterAsync for parsing) but persists the extracted content as a knowledge article instead of a temporary file.

Layer 4: LLM tool for querying knowledge

New file: src/Dashboard/AI/Tools/KnowledgeTools.cs

An AI-callable tool that lets the LLM explicitly look up knowledge articles during a conversation.

public class KnowledgeTools
{
    private readonly long _userId;

    public KnowledgeTools(long userId) => _userId = userId;

    public IEnumerable<AIFunction> Create()
    {
        yield return AIFunctionFactory.Create(QueryKnowledge, "QueryKnowledge",
            @"Look up the user's organizational knowledge base. Use this when you need to resolve
entity names (app names → subscriptions), check SLA/SLO targets, find cost center owners,
or apply user-defined analysis instructions. The knowledge base contains persistent reference
data the user has provided about their organization.

Modes:
  list     — list all knowledge article titles and categories
  search   — search article content for a keyword/phrase (param: query)
  get      — get full content of a specific article (param: id)

Use 'search' when the user mentions an application name, team, cost center, or convention
that isn't in the current conversation — it may be defined in their knowledge base.");
    }

    private string QueryKnowledge(
        [Description("Mode: 'list', 'search', or 'get'")] string mode,
        [Description("Search query string (for 'search' mode) or article ID (for 'get' mode)")] string? param = null)
    {
        var articles = KnowledgeStore.ListForUser(_userId).Where(a => a.Active).ToList();

        return mode.ToLowerInvariant() switch
        {
            "list" => JsonSerializer.Serialize(articles.Select(a => new
            {
                a.Id, a.Title, a.Category,
                contentPreview = a.Content.Length > 100 ? a.Content[..100] + "..." : a.Content
            })),

            "search" when !string.IsNullOrWhiteSpace(param) =>
                JsonSerializer.Serialize(articles
                    .Where(a => a.Content.Contains(param, StringComparison.OrdinalIgnoreCase)
                             || a.Title.Contains(param, StringComparison.OrdinalIgnoreCase))
                    .Select(a => new { a.Id, a.Title, a.Category, a.Content })),

            "get" when !string.IsNullOrWhiteSpace(param) =>
                KnowledgeStore.Get(_userId, param) is { } article
                    ? JsonSerializer.Serialize(new { article.Id, article.Title, article.Category, article.Content })
                    : JsonSerializer.Serialize(new { error = "Article not found" }),

            _ => JsonSerializer.Serialize(new { error = "Invalid mode. Use 'list', 'search', or 'get'." })
        };
    }
}

Register the tool in CopilotSessionFactory.cs alongside other per-user tools, passing the userId.

Layer 5: Frontend knowledge management UI

File: src/Dashboard/frontend/src/components/ChatView.vue

Add a "Knowledge Base" section to the existing sidebar (below the maturity scores section) for managing knowledge articles.

Implementation:

• Sidebar section with a book/brain icon labeled "Knowledge Base". Shows a count badge of active articles.
• List view: clicking the section expands to show article titles with category badges and edit/delete icons.
• Add article: a modal/panel with fields for title (text input), category (dropdown: subscriptions, cost_centers, instructions, architecture, sla, custom), and content (textarea with markdown support). Submit calls POST /api/knowledge.
• Edit article: same modal pre-filled with existing data. Submit calls PUT /api/knowledge/{id}.
• Import from file: a "Import" button that opens a file picker (CSV/JSON/TXT). Calls POST /api/knowledge/import.
• Toggle active/inactive: a toggle switch on each article that calls PUT /api/knowledge/{id} with { active: false/true }. This lets users temporarily disable an article without deleting it.
• Character count: show {current}/{max} character count on the content textarea. Disable submit when over 10,000 chars.

System prompt update

File: src/Dashboard/AI/CopilotSessionFactory.cs

Add a brief instruction to the system prompt so the LLM knows to use knowledge context:

## Organizational Knowledge
When the user's prompt includes an [ORGANIZATIONAL KNOWLEDGE] block, treat it as ground truth about their environment. Use it to:
- Resolve application names to subscription IDs
- Apply their tagging conventions and cost center mappings
- Follow their analysis instructions and reporting preferences
- Reference their SLA/SLO/RTO/RPO targets when assessing risk
- Respect their fiscal calendar and budget cycle definitions
If the user asks about an entity you don't recognize, call QueryKnowledge('search', entityName) to check their knowledge base before asking them.

Files to Modify

File	Change
src/Dashboard/AI/ChatEndpoints.cs	Insert KnowledgeStore.BuildContextBlock(userId) into the prompt construction pipeline between connection context and uploads context
src/Dashboard/AI/CopilotSessionFactory.cs	Add "Organizational Knowledge" section to system prompt; register KnowledgeTools as per-user tool
src/Dashboard/Program.cs	Register KnowledgeEndpoints via app.MapKnowledgeEndpoints()

New Files to Create

File	Purpose
src/Dashboard/Services/KnowledgeStore.cs	Per-user file-based knowledge article persistence with BuildContextBlock() for prompt injection
src/Dashboard/Endpoints/KnowledgeEndpoints.cs	REST API for knowledge CRUD: list, get, create, update, delete, import
src/Dashboard/AI/Tools/KnowledgeTools.cs	LLM-callable tool for searching/retrieving knowledge articles during conversation

Frontend Changes

File	Change
src/Dashboard/frontend/src/components/ChatView.vue	Add "Knowledge Base" sidebar section with list/add/edit/delete/import/toggle UI

Acceptance Criteria

• A KnowledgeStore persists articles to ~/finops-agent-knowledge/{userId}/ as individual JSON files, surviving container restarts
• Maximum 20 articles per user, 10,000 characters per article, 50,000 characters total — enforced on create/update with clear error messages
• KnowledgeStore.BuildContextBlock(userId) returns a formatted [ORGANIZATIONAL KNOWLEDGE ...] block injected into every chat prompt
• Knowledge context appears in the prompt between [CONTEXT: ...] and [UPLOADED FILES ...] blocks
• QueryKnowledge LLM tool supports list, search (keyword match), and get (by ID) modes
• GET /api/knowledge returns all articles for the authenticated user (title, category, id, active, updatedUtc)
• POST /api/knowledge creates a new article; PUT /api/knowledge/{id} updates; DELETE /api/knowledge/{id} deletes
• POST /api/knowledge/import accepts CSV/JSON/TXT file upload and creates a knowledge article from the content
• Articles can be toggled active/inactive via PUT without deletion; inactive articles are excluded from context injection
• Frontend sidebar shows "Knowledge Base" section with article count, list, add/edit/delete/import/toggle UI
• The system prompt includes instructions for the LLM to use organizational knowledge and the QueryKnowledge tool
• Knowledge is scoped per user — no user can read or modify another user's articles
• Knowledge context injection adds zero latency when a user has no articles (empty string, no file I/O)

References

• GitHub Copilot custom instructions (inspiration for the knowledge-as-instructions pattern)
• Existing context injection in ChatEndpoints.cs (connection context and uploaded files blocks)
• Existing UploadedFileTools.cs (file registration and schema summarization pattern)

Area

New AI tool (Azure / Graph / Log Analytics)

Alternatives considered

1. Embed knowledge in the system prompt (baked at session creation)

Inject knowledge into the system prompt in CopilotSessionFactory.cs when creating the Copilot session, rather than per-request. Rejected because the Copilot session is long-lived (recycled only on token expiry, ~1 hour). Knowledge updates would not take effect until the session is recreated. Per-request injection in ChatEndpoints.cs ensures knowledge changes are immediately available.

2. RAG with vector embeddings (Azure AI Search / FAISS)

Implement Retrieval-Augmented Generation: embed knowledge articles as vectors, store in Azure AI Search or a local FAISS index, and retrieve relevant chunks per query. Rejected because (1) it adds a significant dependency (Azure AI Search at ~$70/mo or FAISS library + embedding model), (2) the knowledge base is small (≤50KB per user, ≤20 articles) — full injection into the context window is feasible and more reliable than retrieval, (3) vector search adds latency per request, and (4) the LLM may miss relevant context if the retrieval step doesn't surface the right chunks. Full injection ensures the LLM always has complete organizational context. RAG should be reconsidered if knowledge bases grow beyond 100KB per user.

3. Fine-tune a custom model per tenant

Fine-tune an Azure OpenAI model with organization-specific data. Rejected because fine-tuning is expensive ($25+/hour for training), requires a separate model deployment per tenant, takes hours to days to process, cannot be updated in real-time, and is designed for learning patterns (writing style, domain terminology), not for storing factual reference data (subscription mappings, cost center tables) that change frequently.

4. Store knowledge in Azure Cosmos DB

Use Cosmos DB as the persistence layer for knowledge articles. Rejected because it adds ~$25/mo infrastructure cost for a simple key-value use case, and the App Service /home directory already provides durable file storage at zero additional cost. The knowledge store has low throughput (a few reads/writes per session, not thousands per second) and small data size (≤50KB per user), making file-based storage the right fit.

5. Store knowledge in the browser (localStorage)

Persist knowledge articles in the browser's localStorage, similar to the conversation history approach in the sessions issue. Rejected because knowledge must be accessible to the backend (for prompt injection into the LLM context), not just the frontend. localStorage is client-side only — the server cannot read it. Server-side storage is required so the ChatEndpoints.cs prompt construction can inject knowledge into every request.

6. Use the existing file upload mechanism for persistent knowledge

Let users upload CSV/JSON files via the existing upload endpoint and treat them as persistent knowledge. Rejected because uploaded files have a 30-minute TTL, are stored in /tmp (volatile), are cleared on chat reset, and are designed for ad-hoc analysis, not persistent reference data. The knowledge store needs different lifecycle semantics: long-lived, survives restarts, persists across sessions, and is injected into every conversation. The import endpoint (POST /api/knowledge/import) bridges the two — users can upload a file that gets converted into a persistent knowledge article.

7. Per-tenant shared knowledge (admin-managed)

Allow a tenant admin to define knowledge articles shared across all users in the tenant. Deferred — this is a valuable future feature (e.g. a central team maintains the subscription-to-app mapping for everyone), but it requires tenant-level RBAC, admin role detection, and careful access control design. The per-user knowledge base is the right starting point. Shared tenant knowledge can be added as a second tier, with per-user articles taking precedence over tenant-level defaults.

8. GitHub Copilot-style instruction files in a git repo

Let users point the agent at a git repository containing .md instruction files. Rejected because it introduces a dependency on git/GitHub access from the container, requires additional OAuth scopes (repo access), adds latency to fetch files on every request, and assumes users store their organizational context in git (most don't — it's in SharePoint, Confluence, or people's heads). The in-app knowledge editor is more accessible.