-
Notifications
You must be signed in to change notification settings - Fork 9
feat: scaffold permissioned surplus pools #248
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
kayibal
wants to merge
4
commits into
main
Choose a base branch
from
feat/permissioned-surplus-scaffold
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
d6d745d
feat: scaffold permissioned surplus pools
kayibal cf3a161
refactor: address scaffold review feedback
kayibal 5de1b18
refactor: improve surplus-scaffold doc readability and naming
kayibal 05671cc
refactor: rename eg_amount to surplus_amount
kayibal File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,237 @@ | ||
| //! Per-worker permission scoping for permissioned components. | ||
| //! | ||
| //! "Permissioned" means accessible only to Fynd: such components must never appear in a normal | ||
| //! public quote, yet a dedicated worker is allowed to route through them to capture the surplus | ||
| //! they offer above the best public-market rate. Isolation is achieved by filtering each worker's | ||
| //! local graph topology/events through a `PermissionPolicy` according to its `ComponentScope` — the | ||
| //! shared `MarketState` is never duplicated. | ||
|
|
||
| use std::{ | ||
| collections::{HashMap, HashSet}, | ||
| sync::Arc, | ||
| }; | ||
|
|
||
| use tycho_simulation::tycho_common::models::{protocol::ProtocolComponent, Address}; | ||
|
|
||
| use crate::{ | ||
| feed::{events::MarketEvent, market_data::MarketState}, | ||
| types::ComponentId, | ||
| }; | ||
|
|
||
| /// Classifies a [`ProtocolComponent`] as permissioned or public. | ||
| /// | ||
| /// The predicate is supplied by the caller rather than hard-coded against an id or protocol name, | ||
| /// so the notion of "permissioned" can evolve (e.g. a hook address allowlist) without touching the | ||
| /// routing core. | ||
| #[derive(Clone)] | ||
| pub struct PermissionPolicy { | ||
| /// Returns `true` when the component is permissioned and therefore excluded from public | ||
| /// quotes. | ||
| is_permissioned: Arc<dyn Fn(&ProtocolComponent) -> bool + Send + Sync>, | ||
| } | ||
|
|
||
| impl PermissionPolicy { | ||
| /// Creates a policy from a predicate identifying permissioned components. | ||
| pub fn new<F>(predicate: F) -> Self | ||
| where | ||
| F: Fn(&ProtocolComponent) -> bool + Send + Sync + 'static, | ||
| { | ||
| Self { is_permissioned: Arc::new(predicate) } | ||
| } | ||
|
|
||
| /// Returns `true` if the component is permissioned. | ||
| pub fn is_permissioned(&self, component: &ProtocolComponent) -> bool { | ||
| (self.is_permissioned)(component) | ||
| } | ||
| } | ||
|
|
||
| impl std::fmt::Debug for PermissionPolicy { | ||
| fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { | ||
| f.debug_struct("PermissionPolicy") | ||
| .finish_non_exhaustive() | ||
| } | ||
| } | ||
|
|
||
| /// The set of components a worker is allowed to see in its local graph. | ||
| /// | ||
| /// Determines whether permissioned components are filtered out before the worker builds or updates | ||
| /// its graph. Each worker gets exactly one scope for its lifetime, derived from its pool's role. | ||
| #[derive(Debug, Clone, Copy, PartialEq, Eq)] | ||
| pub enum ComponentScope { | ||
| /// Public workers: permissioned components are dropped so they can never appear in a quote. | ||
| ExcludePermissioned, | ||
| /// Surplus workers: every component is visible, including permissioned ones. | ||
| IncludeAll, | ||
| } | ||
|
|
||
| /// Per-worker permission scoping: which components a worker may ingest into its local graph. | ||
| /// | ||
| /// Bundles a [`ComponentScope`] with an optional [`PermissionPolicy`]. All graph filtering for a | ||
| /// worker flows through this type, so the worker never reasons about permissioned-ness itself — it | ||
| /// just hands its topology and incoming events here. | ||
| #[derive(Clone, Debug)] | ||
| pub(crate) struct PermissionContext { | ||
| /// Scope governing whether permissioned components are filtered out. | ||
| scope: ComponentScope, | ||
| /// Predicate classifying components as permissioned. `None` ⇒ no filtering is ever applied. | ||
| policy: Option<PermissionPolicy>, | ||
| } | ||
|
|
||
| impl PermissionContext { | ||
| /// Default context: see every component, apply no permission filtering. | ||
| pub(crate) fn include_all() -> Self { | ||
| Self { scope: ComponentScope::IncludeAll, policy: None } | ||
| } | ||
|
|
||
| /// Context derived from a pool's role: a scope plus the (optional) classifying policy. | ||
| pub(crate) fn new(scope: ComponentScope, policy: Option<PermissionPolicy>) -> Self { | ||
| Self { scope, policy } | ||
| } | ||
|
|
||
| /// Returns the policy to enforce, or `None` when this worker filters nothing (scope is | ||
| /// `IncludeAll`, or no policy was configured). | ||
| fn active_policy(&self) -> Option<&PermissionPolicy> { | ||
| match (self.scope, &self.policy) { | ||
| (ComponentScope::ExcludePermissioned, Some(policy)) => Some(policy), | ||
| _ => None, | ||
| } | ||
| } | ||
|
|
||
| /// Filters a full topology map to the components this worker may see. | ||
| /// | ||
| /// Public workers drop permissioned components; surplus workers (and workers with no policy) | ||
| /// receive the map unchanged. | ||
| pub(crate) fn filter_topology( | ||
| &self, | ||
| market: &MarketState, | ||
| topology: HashMap<ComponentId, Vec<Address>>, | ||
| ) -> HashMap<ComponentId, Vec<Address>> { | ||
| let Some(policy) = self.active_policy() else { | ||
| return topology; | ||
| }; | ||
| // TODO: drop entries whose ComponentId resolves (via market.get_component(id)) to a | ||
| // ProtocolComponent for which policy.is_permissioned is true. | ||
| let _ = (policy, market); | ||
| todo!("filter permissioned components from the worker's initial topology") | ||
| } | ||
|
|
||
| /// Restricts a market event to the components this worker may see. | ||
| /// | ||
| /// Public workers drop permissioned component ids from the added/updated/removed lists so a | ||
| /// permissioned component is never ingested mid-stream; other workers see the event unchanged. | ||
| pub(crate) fn scope_event(&self, market: &MarketState, event: MarketEvent) -> MarketEvent { | ||
| let Some(policy) = self.active_policy() else { | ||
| return event; | ||
| }; | ||
| let MarketEvent::MarketUpdated { added_components, removed_components, updated_components } = | ||
| event; | ||
|
|
||
| let added_ids: Vec<ComponentId> = added_components | ||
| .keys() | ||
| .cloned() | ||
| .collect(); | ||
| let permitted_added: HashSet<ComponentId> = self | ||
| .filter_component_ids(policy, market, &added_ids) | ||
| .into_iter() | ||
| .collect(); | ||
| let added_components = added_components | ||
| .into_iter() | ||
| .filter(|(id, _)| permitted_added.contains(id)) | ||
| .collect(); | ||
| let removed_components = self.filter_component_ids(policy, market, &removed_components); | ||
| let updated_components = self.filter_component_ids(policy, market, &updated_components); | ||
|
|
||
| MarketEvent::MarketUpdated { added_components, removed_components, updated_components } | ||
| } | ||
|
|
||
| /// Keeps only the component ids that are NOT permissioned under `policy`. | ||
| fn filter_component_ids( | ||
| &self, | ||
| policy: &PermissionPolicy, | ||
| market: &MarketState, | ||
| ids: &[ComponentId], | ||
| ) -> Vec<ComponentId> { | ||
| // TODO: keep only ids whose ProtocolComponent (via market.get_component(id)) is NOT | ||
| // permissioned per policy.is_permissioned. | ||
| let _ = (policy, market, ids); | ||
| todo!("filter permissioned component ids from an incremental market event") | ||
| } | ||
| } | ||
|
|
||
| #[cfg(test)] | ||
| mod tests { | ||
| use super::*; | ||
| use crate::{ | ||
| algorithm::test_utils::{component, token}, | ||
| feed::{events::MarketEvent, market_data::MarketState}, | ||
| }; | ||
|
|
||
| /// A predicate that treats the `vm:permissioned` protocol system as permissioned. | ||
| fn permissioned_protocol_policy() -> PermissionPolicy { | ||
| PermissionPolicy::new(|c: &ProtocolComponent| c.protocol_system == "vm:permissioned") | ||
| } | ||
|
|
||
| fn permissioned_component(id: &str) -> ProtocolComponent { | ||
| let mut c = component(id, &[token(0x01, "A"), token(0x02, "B")]); | ||
| c.protocol_system = "vm:permissioned".to_string(); | ||
| c | ||
| } | ||
|
|
||
| fn public_component(id: &str) -> ProtocolComponent { | ||
| component(id, &[token(0x01, "A"), token(0x02, "B")]) | ||
| } | ||
|
|
||
| fn market_with(components: Vec<ProtocolComponent>) -> MarketState { | ||
| let mut market = MarketState::new(); | ||
| market.upsert_components(components); | ||
| market | ||
| } | ||
|
|
||
| #[test] | ||
| #[ignore = "scaffold: filter helpers are todo!()"] | ||
| fn is_permissioned_reflects_predicate() { | ||
| let policy = permissioned_protocol_policy(); | ||
| assert!(policy.is_permissioned(&permissioned_component("perm-1"))); | ||
| assert!(!policy.is_permissioned(&public_component("pub-1"))); | ||
| } | ||
|
|
||
| #[test] | ||
| #[ignore = "scaffold: filter helpers are todo!()"] | ||
| fn filter_topology_excludes_permissioned() { | ||
| let policy = permissioned_protocol_policy(); | ||
| let market = market_with(vec![public_component("pub-1"), permissioned_component("perm-1")]); | ||
| let topology = market.component_topology(); | ||
|
|
||
| let public = PermissionContext::new(ComponentScope::ExcludePermissioned, Some(policy)); | ||
| let public_view = public.filter_topology(&market, topology.clone()); | ||
| assert!(public_view.contains_key("pub-1")); | ||
| assert!(!public_view.contains_key("perm-1")); | ||
|
|
||
| let surplus_view = | ||
| PermissionContext::include_all().filter_topology(&market, topology.clone()); | ||
| assert_eq!(surplus_view.len(), topology.len()); | ||
| } | ||
|
|
||
| #[test] | ||
| #[ignore = "scaffold: filter helpers are todo!()"] | ||
| fn scope_event_excludes_permissioned() { | ||
| let policy = permissioned_protocol_policy(); | ||
| let market = market_with(vec![public_component("pub-1"), permissioned_component("perm-1")]); | ||
| let event = MarketEvent::MarketUpdated { | ||
| added_components: HashMap::from([ | ||
| ("pub-1".to_string(), vec![]), | ||
| ("perm-1".to_string(), vec![]), | ||
| ]), | ||
| removed_components: vec!["pub-1".to_string(), "perm-1".to_string()], | ||
| updated_components: vec!["pub-1".to_string(), "perm-1".to_string()], | ||
| }; | ||
|
|
||
| let public = PermissionContext::new(ComponentScope::ExcludePermissioned, Some(policy)); | ||
| let MarketEvent::MarketUpdated { added_components, removed_components, updated_components } = | ||
| public.scope_event(&market, event); | ||
| assert!(added_components.contains_key("pub-1")); | ||
| assert!(!added_components.contains_key("perm-1")); | ||
| assert_eq!(removed_components, vec!["pub-1".to_string()]); | ||
| assert_eq!(updated_components, vec!["pub-1".to_string()]); | ||
| } | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.