Context
PR #18 keeps EvaluationSessionInner simple and safe: one Mutex<HashMap<TypeId, ...>> each for sources, caches, and in-flight loads, and no lock is held across FactSource::load_many().await. That is fine for the current sequential policy-outer batch driver.
If Gatehouse later parallelises bulk evaluation across one shared session, these coarse locks become a scalability and contention point. The public &EvaluationSession shape is right, but the internals should be sharded before parallel fanout becomes a documented feature.
Design direction
- Keep source loading outside locks; no mutex guard should be held across
.await.
- Treat sources as effectively immutable after session construction.
- Split state by fact key type, e.g.
TypeId -> FactState<K>, instead of global cache/in-flight mutexes for all key types.
- Inside each fact type, consider per-key or sharded in-flight state so unrelated keys can load concurrently.
- Prefer sharding/per-key state over simply switching to async-aware mutexes;
tokio::sync::Mutex is only useful if a guard must be held across .await, which this design should avoid.
- Preserve the current cancellation invariant: a cancelled loader must wake waiters and leave a deterministic fail-closed result for the request.
Acceptance criteria
- Add a concurrency test showing unrelated fact types/keys do not contend on one global session lock under parallel batch evaluation.
- Add a cancellation test for parallel loaders sharing one session.
- Document whether parallel evaluation is a checker-level feature, a combinator-level feature, or left to policy implementations.
- Benchmark the current coarse-lock session against the sharded design with a hot in-RAM source.
Related
Context
PR #18 keeps
EvaluationSessionInnersimple and safe: oneMutex<HashMap<TypeId, ...>>each for sources, caches, and in-flight loads, and no lock is held acrossFactSource::load_many().await. That is fine for the current sequential policy-outer batch driver.If Gatehouse later parallelises bulk evaluation across one shared session, these coarse locks become a scalability and contention point. The public
&EvaluationSessionshape is right, but the internals should be sharded before parallel fanout becomes a documented feature.Design direction
.await.TypeId -> FactState<K>, instead of global cache/in-flight mutexes for all key types.tokio::sync::Mutexis only useful if a guard must be held across.await, which this design should avoid.Acceptance criteria
Related