Skip to content

Plan sharded EvaluationSession state before parallel batch evaluation #25

@hardbyte

Description

@hardbyte

Context

PR #18 keeps EvaluationSessionInner simple and safe: one Mutex<HashMap<TypeId, ...>> each for sources, caches, and in-flight loads, and no lock is held across FactSource::load_many().await. That is fine for the current sequential policy-outer batch driver.

If Gatehouse later parallelises bulk evaluation across one shared session, these coarse locks become a scalability and contention point. The public &EvaluationSession shape is right, but the internals should be sharded before parallel fanout becomes a documented feature.

Design direction

  • Keep source loading outside locks; no mutex guard should be held across .await.
  • Treat sources as effectively immutable after session construction.
  • Split state by fact key type, e.g. TypeId -> FactState<K>, instead of global cache/in-flight mutexes for all key types.
  • Inside each fact type, consider per-key or sharded in-flight state so unrelated keys can load concurrently.
  • Prefer sharding/per-key state over simply switching to async-aware mutexes; tokio::sync::Mutex is only useful if a guard must be held across .await, which this design should avoid.
  • Preserve the current cancellation invariant: a cancelled loader must wake waiters and leave a deterministic fail-closed result for the request.

Acceptance criteria

  • Add a concurrency test showing unrelated fact types/keys do not contend on one global session lock under parallel batch evaluation.
  • Add a cancellation test for parallel loaders sharing one session.
  • Document whether parallel evaluation is a checker-level feature, a combinator-level feature, or left to policy implementations.
  • Benchmark the current coarse-lock session against the sharded design with a hot in-RAM source.

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions