title	Memory Module
anchor	memory
type	module
status	standard
version	2.0
normative	true

Memory Module: Design Standard

AI-Native Data Product Architecture

Document Control

Attribute	Value
Status	STANDARD
Type	Module Design Standard (platform-agnostic)
Scope	Memory module: agent state and learning (runtime), and design memory (documentation)
Extends	Master Design
Notation	Design Language
Implementations	`implementation/teradata/modules/memory/`

Memory is the module that provides DocumentationCapture: the capability every other module soft-requires to record its design decisions. It is also the store of agent runtime state.

1. Purpose

Memory enables agent learning, continuity, and collaboration across sessions, users, and agent instances, and it holds the product's design memory: the decisions, glossary, and change history that make the product self-describing.

AI-native characteristic	Purpose
Session continuity	Agents remember context across interactions.
Cross-agent learning	Agents share strategies that worked.
Preference learning	Agents adapt to user and business preferences.
Meta-learning	Agents learn what works and improve over time.
Privacy-aware	Every runtime record is scoped (user, team, organisation, agent).
Design memory	Captures decisions, rationale, glossary, and change history via the documentation facet.

2. Facets

Memory is one module with two facets (see the composition mechanism), enabled independently:

Facet	Holds	Provides
`documentation` (design memory)	Module registry, design decisions, glossary, query cookbook, implementation notes, change log.	`DocumentationCapture`: consumed by every module.
`runtime` (agent state)	Sessions, interactions, learned strategies, preferences, discovered patterns.	Agent continuity and learning, consumed by agents.

A Data Asset takes the documentation facet only (Domain + Memory[documentation] + Access Layer). An AI-Native Data Product takes both. Neither facet has a hard dependency on another module, so Memory can be deployed alongside Domain alone.

3. Scope and Boundaries

Two principles govern what Memory stores:

Entity = table, not instance. Memory references entities (tables), never the individual instance keys or rows from a query's results (INV-MEMORY-001).

Big questions, small answers. Agents process millions of records; Memory stores the metadata about those processes (the query run, the tables involved, the outcome, the counts) and never the result data (INV-MEMORY-002). Memory holds thousands to tens of thousands of rows, not millions.

In scope: agent interaction metadata (what was asked, what query ran, which tables, the outcome), agent learning metadata (strategies, patterns, success rates), preferences, session state, and, via the documentation facet, design decisions, glossary, cookbook, registry, and change history.

Out of scope: business domain data (→ Domain), query results (→ Domain or temporary tables), individual record keys/ids, and detailed personal profiles (→ Domain, referenced by key).

3.1 Process context against result data

INV-MEMORY-001 is the invariant designs violate most often, and they violate it in good faith. Session continuity naturally suggests storing what the agent last found, and the line between what the agent is doing and what a query returned is genuinely not obvious from the prohibition alone. It is worth stating from the compliant side.

Process context (store it). A record of what the agent is doing. "The agent is investigating agreement AGR-00042" is process context: store current_entity_key. So is "the last similarity search asked for overdue renewals and matched 37 rows": store the query text and the count.

Result data (do not store it). A record of what a query returned. "The last search matched AGR-00001, AGR-00042, AGR-00099" is result data, whatever it is called. Storing that key list makes Memory a result cache, and a stale one, since nothing invalidates it when the underlying rows change. If the agent needs those keys again it re-executes the search, which is cheap and always correct.

The test is not what the attribute is named but what it holds: an attribute holding one key the agent chose to work on is context; an attribute holding the set a query produced is a cached result. retrieved_*_keys, *_key_list, and result_* are the shapes this failure takes.

The same distinction governs counts: query_result_count is an aggregate about the process and belongs here; the ids it counted do not.

4. Entity Model: Runtime Facet

Runtime entities are append-oriented operational records. Every one carries a privacy scope (see Privacy and Scoping). None stores business content: table references are table-level, and content is obtained by join-back to Domain.

Entity: AgentSession              [kind: Record]
  session_id: Identifier  // surrogate key
  session_key: NaturalKey [required]  // business session identifier
  agent_key: ShortText [required]  // which agent instance
  user_key: ShortText [optional]  // which user
  session_start_dts: Timestamp [required]
  session_end_dts: Timestamp [optional]  // null while active
  session_status: Enum{ACTIVE|COMPLETED|ABANDONED}
  session_goal: Text [optional]
  session_context: Json [optional]  // flexible context, processed by the consumer
  scope_level: Enum{USER|TEAM|ORGANIZATION|AGENT} [required]
  scope_identifier: ShortText [required]  // user/team/org/agent key matching scope_level

Entity: AgentInteraction          [kind: Record]
  interaction_id: Identifier
  session_id: Reference [required] [-> AgentSession]
  interaction_seq: Integer [required]  // order within the session
  interaction_type: Enum{QUERY|ACTION|DECISION|EXPLANATION}
  interaction_dts: Timestamp [required]
  user_input: Text [optional]
  agent_response: Text [optional]
  action_taken: Text [optional]
  referenced_tables: Text [optional]  // qualified table names, comma-separated; TABLE-LEVEL only (INV-MEMORY-001)
  query_executed: Text [optional]  // the query text, not its results
  query_result_count: Integer [optional]  // aggregate count only, never the ids
  execution_time_ms: Integer [optional]
  outcome_status: Enum{SUCCESS|PARTIAL|FAILED}
  user_feedback: Enum{POSITIVE|NEUTRAL|NEGATIVE} [optional]
  scope_level: Enum{USER|TEAM|ORGANIZATION|AGENT} [required]
  scope_identifier: ShortText [required]

Entity: LearnedStrategy           [kind: Record]
  strategy_id: Identifier
  strategy_name: ShortText [required]
  strategy_category: Enum{QUERY_OPTIMIZATION|FEATURE_SELECTION|ERROR_HANDLING}
  strategy_pattern: Text [optional]  // the pattern/approach, described
  strategy_metadata: Json [optional]
  success_rate: Decimal(5,4) [optional]  // 0.0-1.0
  times_used: Integer [optional]
  is_active: Flag
  is_validated: Flag
  scope_level: Enum{USER|TEAM|ORGANIZATION|AGENT} [required]
  scope_identifier: ShortText [required]

Entity: UserPreference            [kind: Record]
  preference_id: Identifier
  user_key: ShortText [required]
  preference_category: Enum{REPORT_FORMAT|DATA_FILTER|AGGREGATION_LEVEL|VISUALIZATION_TYPE}
  preference_name: ShortText [required]
  preference_value: Text [optional]
  preference_json: Json [optional]
  confidence: Decimal(5,4) [optional]
  is_active: Flag
  scope_level: Enum{USER|TEAM|ORGANIZATION|AGENT} [required]
  scope_identifier: ShortText [required]

Entity: DiscoveredPattern         [kind: Record]
  pattern_id: Identifier
  pattern_name: ShortText [required]
  pattern_type: Enum{CORRELATION|TEMPORAL|TABLE_RELATIONSHIP|ANOMALY}
  pattern_definition: Json [optional]
  sample_size: Integer [optional]  // how many records analysed (summary, not the records)
  confidence_score: Decimal(5,4) [optional]
  involved_tables: Text [optional]  // TABLE-LEVEL references only (INV-MEMORY-001)
  is_validated: Flag
  scope_level: Enum{USER|TEAM|ORGANIZATION|AGENT} [required]
  scope_identifier: ShortText [required]

All runtime entities Apply patterns: object-placement, access-layer and Require: RichMetadata.

5. Entity Model: Documentation Facet

The documentation facet is design memory: it records why a product is the way it is, how to use it, and what changed. Runtime memory records what agents did; this records what the designers decided.

Boundary with Semantic. Semantic stores what exists and how it connects (tables, columns, join paths); documentation stores why it exists, how to use it, and what changed. Documentation never duplicates Semantic metadata (INV-MEMORY-004).

5.1 Documentation entities

Documentation entities are temporally versioned (they apply temporal-lifecycle-metadata); corrections supersede prior versions rather than overwriting them (INV-MEMORY-005).

Entity: ModuleRegistry            [kind: History]
  module_registry_id: Identifier
  module_name: Enum{DOMAIN|SEARCH|PREDICTION|OBSERVABILITY|SEMANTIC|MEMORY} [required]
  container_name: ShortText [required]  // where the module is deployed
  deployment_status: Enum{DEPLOYED|PLANNED|DEPRECATED} [required]
  module_version: ShortText [required]
  module_purpose: LongText [required]
  key_entities: Text [optional]
  dependencies: Text [optional]

Entity: DesignDecision            [kind: History]
  decision_id: NaturalKey [required]  // DD-{MODULE}-{NNN}
  decision_version: Integer [required]
  decision_title: ShortText [required]
  context: LongText [optional]
  alternatives: LongText [optional]
  rationale: LongText [optional]
  consequences: LongText [optional]
  decision_status: Enum{PROPOSED|ACCEPTED|SUPERSEDED|DEPRECATED} [required]
  decision_category: Enum{ARCHITECTURE|SCHEMA|NAMING|PERFORMANCE|SECURITY|INTEGRATION|OPERATIONAL} [required]
  source_module: ShortText [required]
  superseded_by: NaturalKey [optional]

Entity: BusinessGlossary          [kind: History]
  term: ShortText [required]
  term_category: Enum{ENTITY|ATTRIBUTE|METRIC|BUSINESS_RULE|CLASSIFICATION|REFERENCE_CODE} [required]
  definition: LongText [required]
  source_module: ShortText [required]

Entity: QueryCookbook             [kind: History]
  recipe_id: NaturalKey [required]  // QC-{MODULE}-{NNN}
  recipe_title: ShortText [required]
  use_case: ShortText [required]
  target_module: Enum{DOMAIN|SEARCH|PREDICTION|OBSERVABILITY|SEMANTIC|MEMORY|CROSS} [required]
  query_template: LongText [required]  // parameterised query, consumed by agents
  complexity: Enum{SIMPLE|MODERATE|COMPLEX|ADVANCED} [required]
  is_batch: Flag  // 1 = batch only; 0 = safe for interactive agent use
  source_module: ShortText [required]

Entity: ImplementationNote        [kind: History]
  note_id: NaturalKey [required]  // IN-{MODULE}-{NNN}
  note_title: ShortText [required]
  note_content: LongText [required]
  note_category: Enum{DEPLOYMENT|WORKAROUND|KNOWN_ISSUE|PERFORMANCE_TIP|OPERATIONAL|SECURITY} [required]
  severity: Enum{LOW|MEDIUM|HIGH|CRITICAL} [optional]
  source_module: ShortText [required]

Entity: ChangeLog                 [kind: History]
  change_id: NaturalKey [required]  // CL-{MODULE}-{NNN}
  version_number: ShortText [required]
  change_title: ShortText [required]
  change_type: Enum{INITIAL_RELEASE|SCHEMA_CHANGE|FEATURE_ADDITION|BUG_FIX|PERFORMANCE|DEPRECATION} [required]
  source_module: ShortText [required]
  related_decision_id: NaturalKey [optional] [-> DesignDecision]

Use source_module on every documentation entity except ModuleRegistry (which uses module_name to identify the registered module). Never add module_name to the other entities. source_module holds one registered module name; the capture protocol below states why that matters and what to do with a decision that spans several.

5.2 Capture protocol (the `DocumentationCapture` contract)

When any module is designed for the product, it records its documentation here. Each deployed module must produce, at minimum:

Record	Minimum	Id convention
Module registry entry	1 per module considered (with `deployment_status`)	-
Design decision	3 per deployed module	`DD-{MODULE}-{NNN}`
Change-log entry	1 (initial release)	`CL-{MODULE}-{NNN}`
Business-glossary term	3	-
Query-cookbook recipe	1 per deployed module; 1 cross-module recipe per deployed pair	`QC-{MODULE}-{NNN}`
Implementation note	as needed	`IN-{MODULE}-{NNN}`

{MODULE} is the short module name (DOMAIN, SEARCH, …). Additional required records: a design decision for every deferred or deprecated module; a design decision for every deviation from a design standard (category ARCHITECTURE); and the ERD recipe QC-SEMANTIC-002 when Semantic is present. This protocol is the provider side of INV-MASTER-002: it is what Domain's the Designer Responsibilities section and Search's the Implementation section point at.

What counts toward the three, and what source_module may hold. Two different things are both called decisions, and reading one as the other is how a module ends up appearing under its minimum, or above it, without anyone being wrong:

A catalogued decision (DEC-TEMPORAL-PATTERN, DEC-DELETE-STRATEGY, …) is a question a module obliges a designer to settle, listed under Designer Responsibilities. It is part of this standard.
A design decision (DD-{MODULE}-{NNN}) is a record written here. It is part of the product.

The three-per-module minimum counts records, not questions. Settling a catalogued decision does not by itself satisfy anything: the answer is recorded as a DD- record naming the DEC- id it settles, and it is that record which counts. A module that settles three catalogued decisions therefore meets its minimum, but by writing three records rather than by pointing at three rows of a table it did not write.

Two rules follow, and both are checkable:

source_module names a module registered in ModuleRegistry for this product. A value naming anything else, a pattern, a layer, a concern, resolves to no module, so the record counts toward nothing and the shortfall is reported against a module that looks compliant. Access-layer and other pattern decisions are recorded against the module whose behaviour they govern.
A decision affecting several modules is recorded once per module, each with the rationale that applies to that module. A single blended record under one source_module under-counts every other module it governs, and loses the per-module reasoning that made it worth recording. Where the reasoning is genuinely identical, say so; where it differs, as the temporal choice does between design memory and a feature store, the difference is the substance.

6. Privacy and Scoping

Every runtime record carries a privacy scope, both a scope_level (USER/TEAM/ORGANIZATION/AGENT) and a scope_identifier, with no exceptions (INV-MEMORY-003). Retrieval always filters on scope, so a user sees only their own records, a team its shared records, and so on.

The filter lives in the access surface, not in the consumer. Carrying a scope and enforcing one are different things, and a scope enforced only by whatever queries the table is not enforced. The consumer grant reaches a view that applies the scope predicate; the runtime base tables are not part of the consumer surface. This matters most where both facets share a container, because the grant that makes documentation readable is the grant that reaches session records: see the access-layer pattern.

Data minimisation. Store the user_key (an identifier), never names, emails, or demographics: those are obtained by join-back to Domain when genuinely needed.

7. Applied Patterns

Pattern	Contribution to Memory
`temporal-lifecycle-metadata`	Version-chains the documentation entities; corrections supersede, never overwrite.
`object-placement`	Which container the Memory tables and views are created in, and who may reach them.
`access-layer`	Standard views over sessions, interactions, current decisions, active recipes, etc.
`validation`	The conformance checks run before the module is declared done.

8. Capabilities and Composition

Memory is cross-cutting and soft: nothing hard-depends on it, and it hard-depends on nothing, so it composes with any product and either facet can be deployed alone. See the composition mechanism.

Provides:

Capability	Facet	Made available to
`DocumentationCapture`	`documentation`	Every module, to record its design memory.
`AgentContinuity`	`runtime`	Agents, across sessions and instances: prior sessions, interactions, learned strategies, and preferences.

Requires:

Capability	Strength	Provider	Why
`RichMetadata`	`[hard]`	`self` / `platform`	Agent-readable metadata on every object and attribute.
`DocumentationCapture`	`[soft]`	`self` (`documentation` facet)	Memory records its own design decisions.
`SemanticRegistration`	`[soft]`	`module:Semantic`	Register Memory's entities in the Semantic map when present (`INV-MASTER-002`).
`EntityJoinBack`	`[soft]`	`module:Domain`	Resolve a referenced table to Domain entity context when needed.
`QualityScore`	`[soft]`	`module:Observability`	Learn strategies from observed outcomes and quality evidence when Observability is present.
`NearestNeighbors`	`[soft]`	`module:Search`	Find similar past sessions when Search is present.

9. Integration with Other Modules

Observability → Memory. Memory learns strategies from observed outcomes (which query patterns performed well). Soft: absent Observability simply means no outcome-driven learning.
Memory + Search: find similar historical sessions via Search's NearestNeighbors. Soft.
Memory + Semantic: apply learned rules alongside Semantic's business rules; register Memory's own entities in the Semantic map. Soft.
Memory + Domain: table-level references resolve to Domain entity context by join-back when needed. Memory never copies Domain content.

10. Invariants

INV-MEMORY-001: Memory references entities at the table level (qualified names); it never stores individual instance keys/ids from query results.
INV-MEMORY-002: Memory stores process metadata (query text, patterns, outcomes, counts), never result data or business content; content is obtained by join-back to Domain.
INV-MEMORY-003: every runtime record carries a privacy scope (scope_level and scope_identifier).
INV-MEMORY-004: documentation records why/how/what-changed; they never duplicate Semantic's what-exists/how-connects metadata.
INV-MEMORY-005: documentation records are temporally versioned; corrections supersede prior versions rather than overwriting them.
INV-MEMORY-006: when the documentation facet is present, every deployed module records its documentation here per the capture protocol (the provider side of INV-MASTER-002).

11. Designer Responsibilities

Designers supply:

Element	Example
Agent types	analytics agent, customer-service agent
Session patterns	query session, analysis session
Learning categories	query patterns, feature importance, preferences
Privacy scoping	which scope levels are in use
Retention policies	sessions 90 days, interactions 1 year, validated strategies 2 years
Facets enabled	`documentation` only (Data Asset) or both (AI-native)

Design review checklist:

11.1 Decisions to settle

These are the catalogued decisions a Memory module design must settle. The recommendation is this standard's default; the question is what shifts it. The design skill walks a designer through each one at design time and records the answer in the product's own design.

Decision	Recommended	Settle it by asking
`DEC-TEMPORAL-PATTERN`	`scd2`	Recommended over the advocated `bi-temporal` because design memory is superseded by a later version rather than corrected retrospectively: there is no transaction-time question to answer. Choose `bi-temporal` if the product must reconstruct what it recorded as of a past instant.
`DEC-DELETE-STRATEGY`	`soft-delete`	Is a withdrawn decision or retired glossary term still evidence?
`DEC-TIMESTAMP-ZONE`	`zone-aware`	Do agent sessions span regions?

12. Implementation

The Teradata binding (the runtime and documentation tables, the standard views, the capture-protocol templates, and the invariant checks) lives in implementation/teradata/modules/memory/. Other platforms add sibling directories under implementation/ without changing this document.

End of Memory Module Design Standard

FilesExpand file tree

memory.md

Latest commit

History