From 894742f6c68e860780e831415a14017615d64462 Mon Sep 17 00:00:00 2001 From: Felix Hennig Date: Thu, 18 Jan 2024 15:36:15 +0100 Subject: [PATCH 01/12] first draft --- ...DRXXX-authorization-abstraction-layer.adoc | 97 +++++++++++++++++++ 1 file changed, 97 insertions(+) create mode 100644 modules/contributor/pages/adr/ADRXXX-authorization-abstraction-layer.adoc diff --git a/modules/contributor/pages/adr/ADRXXX-authorization-abstraction-layer.adoc b/modules/contributor/pages/adr/ADRXXX-authorization-abstraction-layer.adoc new file mode 100644 index 000000000..4fab08ad1 --- /dev/null +++ b/modules/contributor/pages/adr/ADRXXX-authorization-abstraction-layer.adoc @@ -0,0 +1,97 @@ += ADRXXX + +== Context + +As a user I want to be able to very easily define my access control for everything in the platform. Access control rules per product should be defined on groups or projects, and then I assign users as members of these projects - defined as relations maybe? I want to be able to simply say "John is a member of the data science team" and that gives John access to a number of resources in the platform. + +== Problem Statement + +How should we design the user facing part of this authorization spec? Do we create Custom Resources for some of this? Or simply ConfigMaps? How many, how are the rules split across resources? + +== Decision Drivers + +* The design should be flexible to allow to easily represent verious organizational structures +* The design should validate as much of the input as possible, to prevent misspellings from invalidating rules. Nothing should just silently not do anything. + +== Proposed Design + +We design a relation based access control system (ReBAC), inspired by Google Zanzibar. + +We define a CustomResource that allows users to define ReBAC stuff in CRs. we translate that into JSON that we can then give into OPA as data context. + +There are three kinds of rules that users can define easily in a CR: +- relations of objects and users +- relations of objects and user sets +- relations of objects and other objects + +Users are pre defined in either LDAP or keycloak, and referenced by their name/ID. +Resolving users is done in OPA, using the UserInfoFetcher. +For this ADR, we will assume users are specified by their username, i.e. "alice" or "bob". + +Other objects in the system can be entities inside of the products, such as a `trino-table`. +We also allow users to define their own objects and relations, mainly to organize their users and permissions. +Every user-defined object and relation needs to be specified in an RebacType object. + +Example: + +[source,yaml] +---- +kind: RebacType +metadata: + name: project # <1> +spec: + relations: # <2> + - name: member + objects: # <3> + - secret-project + - datascience-spike + - ladida + - otherproj +---- + +<1>: The name of the RebacType. This will be referenced in rule definitions. +<2>: The relations that are defined for this type. In this case there is a "member" definition. Users can be members of a project. +<3>: Object defintions. This is optional; if given, only these objects can be referenced, adding another layer of verification. + +For products, the SDP comes with predefined objects that make sense for the product. +For example in Trino we will define `trino-table` and `trino-catalog`, each with a `reader`, `editor` and `owner` relation. +For Superset we can define a `superset-dashboard` with the same relations. +For Kafka we can define a `kafka-topic` type. + +We can group some resources together into a project, and then assign users to the project. +Users can be assigned individually or based on their group memberships. + +Here is an example of what it could look like in yaml: + +[source,yaml] +---- +kind: RebacRules +metadata: + name: project-ladida + labels: + opa.stackable.tech/bundle: "true" +# Project Ladida manages the mydata table in Trino +# User 13 and all the members of the 'otherproj' project are members +# of Project Ladida +relations: + # Which resources are part of the project? + - subject: "project:ladida" + relation: "owner" + object: "trino-table:mydata" + - subject: "project:ladida" + relation: "reader" + object: "kafka-topic:datasource" + - subject: "project:ladida" + relation: "editor" + object: "superset-dashboard:ladida-viz" + # Who is part of the project? + - user: "alice" + relation: "member" + object: "project:ladida" + - userset: + object: "department:datascience" + relation: "member" + relation: "member" + object: "project:ladidi" +---- + From 3edebe7e2932a02d509c3934f57c22e7c680475a Mon Sep 17 00:00:00 2001 From: Felix Hennig Date: Thu, 18 Jan 2024 15:42:51 +0100 Subject: [PATCH 02/12] ... --- .../adr/ADRXXX-authorization-abstraction-layer.adoc | 10 +++++++--- modules/contributor/partials/adr-nav.adoc | 2 ++ 2 files changed, 9 insertions(+), 3 deletions(-) diff --git a/modules/contributor/pages/adr/ADRXXX-authorization-abstraction-layer.adoc b/modules/contributor/pages/adr/ADRXXX-authorization-abstraction-layer.adoc index 4fab08ad1..0fdc9b5c8 100644 --- a/modules/contributor/pages/adr/ADRXXX-authorization-abstraction-layer.adoc +++ b/modules/contributor/pages/adr/ADRXXX-authorization-abstraction-layer.adoc @@ -65,11 +65,9 @@ Here is an example of what it could look like in yaml: [source,yaml] ---- -kind: RebacRules +kind: RebacRelations metadata: name: project-ladida - labels: - opa.stackable.tech/bundle: "true" # Project Ladida manages the mydata table in Trino # User 13 and all the members of the 'otherproj' project are members # of Project Ladida @@ -95,3 +93,9 @@ relations: object: "project:ladidi" ---- +=== Implementation + +The RebacType and RebacRelations can both be annotated with the `opa.stackable.tech/bundle: "true"` label to include it into the OPA bundle. + +The relation definitions can the be verified - to a degree - and subsequently everything is serialized as JSON and provided as `data` to OPA. +We then need to define a RegoRule framework inside OPA to evaluate these rules correctly. diff --git a/modules/contributor/partials/adr-nav.adoc b/modules/contributor/partials/adr-nav.adoc index 8942b3630..352635ff0 100644 --- a/modules/contributor/partials/adr-nav.adoc +++ b/modules/contributor/partials/adr-nav.adoc @@ -6,3 +6,5 @@ include::partial$current_adrs.adoc[] * *** Drafts **** xref:adr/drafts/ADRx-choose_authorization_engine.adoc[] +**** xref:adr/ADRXXX-authorization-abstraction-layer.adoc[] + From 5fcb3687425964c2952de9edb0cfe46989945ad5 Mon Sep 17 00:00:00 2001 From: Felix Hennig Date: Thu, 18 Jan 2024 15:49:09 +0100 Subject: [PATCH 03/12] ... --- modules/contributor/partials/adr-nav.adoc | 1 - modules/contributor/partials/current_adrs.adoc | 1 + 2 files changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/contributor/partials/adr-nav.adoc b/modules/contributor/partials/adr-nav.adoc index 352635ff0..cf90ab22b 100644 --- a/modules/contributor/partials/adr-nav.adoc +++ b/modules/contributor/partials/adr-nav.adoc @@ -6,5 +6,4 @@ include::partial$current_adrs.adoc[] * *** Drafts **** xref:adr/drafts/ADRx-choose_authorization_engine.adoc[] -**** xref:adr/ADRXXX-authorization-abstraction-layer.adoc[] diff --git a/modules/contributor/partials/current_adrs.adoc b/modules/contributor/partials/current_adrs.adoc index 77966be43..e0846d1e2 100644 --- a/modules/contributor/partials/current_adrs.adoc +++ b/modules/contributor/partials/current_adrs.adoc @@ -29,3 +29,4 @@ **** xref:adr/ADR030-allowed-pod-disruptions.adoc[] **** xref:adr/ADR031-resource-labels.adoc[] **** xref:adr/ADR032-oidc-support.adoc[] +**** xref:adr/ADRXXX-authorization-abstraction-layer.adoc[] From 7b5ae5fa1037b9877bcc1e042d744008d1d226e1 Mon Sep 17 00:00:00 2001 From: Felix Hennig Date: Thu, 18 Jan 2024 16:06:01 +0100 Subject: [PATCH 04/12] ... --- ...DRXXX-authorization-abstraction-layer.adoc | 42 +++++++++++++++---- 1 file changed, 34 insertions(+), 8 deletions(-) diff --git a/modules/contributor/pages/adr/ADRXXX-authorization-abstraction-layer.adoc b/modules/contributor/pages/adr/ADRXXX-authorization-abstraction-layer.adoc index 0fdc9b5c8..f53b9012c 100644 --- a/modules/contributor/pages/adr/ADRXXX-authorization-abstraction-layer.adoc +++ b/modules/contributor/pages/adr/ADRXXX-authorization-abstraction-layer.adoc @@ -1,23 +1,44 @@ -= ADRXXX += ADRXXX: Authorization abstraction layer +Felix Hennig +v0.1, 2024-01-18 +:status: draft -== Context +* Status: {status} +* Deciders: TBD +* Date: 2024-01-18 -As a user I want to be able to very easily define my access control for everything in the platform. Access control rules per product should be defined on groups or projects, and then I assign users as members of these projects - defined as relations maybe? I want to be able to simply say "John is a member of the data science team" and that gives John access to a number of resources in the platform. +Technical Story: https://github.com/stackabletech/issues/issues/439 == Problem Statement -How should we design the user facing part of this authorization spec? Do we create Custom Resources for some of this? Or simply ConfigMaps? How many, how are the rules split across resources? +How should we design the user facing part of an authorization layer for the platform? Do we create Custom Resources for some of this? Or simply ConfigMaps? How many, how are the rules split across resources? + +== Context + +As a user I want to be able to very easily define my access control for everything in the platform. +Access control rules per product should be defined on groups or projects, and then I assign users as members of these groups, departments and projects. +I want to be able to simply say "John is a member of the data science team" and that gives John access to a number of resources in the platform based on his team membership. +I also want to be able to easily specify one-of access to individuals for specific resources, such as given Alice access to a specific Trino table. == Decision Drivers -* The design should be flexible to allow to easily represent verious organizational structures +* The design should be flexible to allow to easily represent various organizational structures. +* It should be possible to group together access to different resources across products. * The design should validate as much of the input as possible, to prevent misspellings from invalidating rules. Nothing should just silently not do anything. +* Rules should be defined as Manifests and put into Kubernetes. + +== Constraints + +* We use OPA as the underlying policy engine, so any design needs to be implementable with OPA. == Proposed Design We design a relation based access control system (ReBAC), inspired by Google Zanzibar. +ReBAC allows for role-based access control as well (RBAC) but goes beyond that to also allow hierarchies of objects that users can be related to. +This allows for more flexibility when defining organizational structures, and rules can be attached at any level. -We define a CustomResource that allows users to define ReBAC stuff in CRs. we translate that into JSON that we can then give into OPA as data context. +We define a CustomResource that allows users to define ReBAC stuff in CRs. +This allows for more validation instead of putting rules in a DSL or in JSON into a ConfigMap. There are three kinds of rules that users can define easily in a CR: - relations of objects and users @@ -87,12 +108,17 @@ relations: relation: "member" object: "project:ladida" - userset: - object: "department:datascience" + object: "ad-group:datascience" relation: "member" relation: "member" - object: "project:ladidi" + object: "project:ladida" ---- +The first three relations define the project as the owner/reader/editor of a number of resources. +Transitively, any member of the project will get these relations. + +The last two relations define the user "alice" as a member of the project, as well as any member of the ActiveDirectoy group "datascience". + === Implementation The RebacType and RebacRelations can both be annotated with the `opa.stackable.tech/bundle: "true"` label to include it into the OPA bundle. From cfbcea2182a6ae3c67bb228314005d46eda23849 Mon Sep 17 00:00:00 2001 From: Felix Hennig Date: Thu, 18 Jan 2024 16:11:05 +0100 Subject: [PATCH 05/12] formatting --- .../adr/ADRXXX-authorization-abstraction-layer.adoc | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/modules/contributor/pages/adr/ADRXXX-authorization-abstraction-layer.adoc b/modules/contributor/pages/adr/ADRXXX-authorization-abstraction-layer.adoc index f53b9012c..e9dd846b1 100644 --- a/modules/contributor/pages/adr/ADRXXX-authorization-abstraction-layer.adoc +++ b/modules/contributor/pages/adr/ADRXXX-authorization-abstraction-layer.adoc @@ -41,9 +41,10 @@ We define a CustomResource that allows users to define ReBAC stuff in CRs. This allows for more validation instead of putting rules in a DSL or in JSON into a ConfigMap. There are three kinds of rules that users can define easily in a CR: -- relations of objects and users -- relations of objects and user sets -- relations of objects and other objects + +* relations of objects and users +* relations of objects and user sets +* relations of objects and other objects Users are pre defined in either LDAP or keycloak, and referenced by their name/ID. Resolving users is done in OPA, using the UserInfoFetcher. @@ -70,9 +71,9 @@ spec: - otherproj ---- -<1>: The name of the RebacType. This will be referenced in rule definitions. -<2>: The relations that are defined for this type. In this case there is a "member" definition. Users can be members of a project. -<3>: Object defintions. This is optional; if given, only these objects can be referenced, adding another layer of verification. +<1> The name of the RebacType. This will be referenced in rule definitions. +<2> The relations that are defined for this type. In this case there is a "member" definition. Users can be members of a project. +<3> Object defintions. This is optional; if given, only these objects can be referenced, adding another layer of verification. For products, the SDP comes with predefined objects that make sense for the product. For example in Trino we will define `trino-table` and `trino-catalog`, each with a `reader`, `editor` and `owner` relation. From ba7d1a326482066fb9bcb254e58fc89d7a7d6d27 Mon Sep 17 00:00:00 2001 From: Felix Hennig Date: Thu, 18 Jan 2024 16:34:52 +0100 Subject: [PATCH 06/12] Add more context --- ...DRXXX-authorization-abstraction-layer.adoc | 24 +++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/modules/contributor/pages/adr/ADRXXX-authorization-abstraction-layer.adoc b/modules/contributor/pages/adr/ADRXXX-authorization-abstraction-layer.adoc index e9dd846b1..5bf082d7f 100644 --- a/modules/contributor/pages/adr/ADRXXX-authorization-abstraction-layer.adoc +++ b/modules/contributor/pages/adr/ADRXXX-authorization-abstraction-layer.adoc @@ -20,6 +20,23 @@ Access control rules per product should be defined on groups or projects, and th I want to be able to simply say "John is a member of the data science team" and that gives John access to a number of resources in the platform based on his team membership. I also want to be able to easily specify one-of access to individuals for specific resources, such as given Alice access to a specific Trino table. +=== Different authorization models: RBAC, ABAC, ReBAC and more + +Out-of-the-box, OPA uses RegoRules to define policies. +This is very powerful, but also more complex that other mechanisms such as RBAC or ACLs. +We want to pick an authorization model to build on top of OPA and abstract away from the RegoRules for 95% of use cases that the typical Stackable user might encounter. + +Relation-based access control (ReBAC) was popularized by Google Zanzibar and goes beyond RBAC. +The relational model allows for more flexibility when defining rules. + +Learn more: + +* https://authzed.com/zanzibar +* https://authzed.com/blog/what-is-google-zanzibar +* https://gruchalski.com/posts/2022-05-07-zanzibar-style-acls-with-opa-rego/ +* https://www.permit.io/blog/oparebac +* https://www.permit.io/blog/policy-engines + == Decision Drivers * The design should be flexible to allow to easily represent various organizational structures. @@ -126,3 +143,10 @@ The RebacType and RebacRelations can both be annotated with the `opa.stackable.t The relation definitions can the be verified - to a degree - and subsequently everything is serialized as JSON and provided as `data` to OPA. We then need to define a RegoRule framework inside OPA to evaluate these rules correctly. + +== TODO: Gaia-X considerations + +TSA? OCM? + +"TSA ist OPA, unser Authr. ist OPA, aber bauen wir oder TSA dinge drumherum, die die Integration schwer machen?" + From 1074943b9d952c66639f656f2b5ed4de7d6f416c Mon Sep 17 00:00:00 2001 From: Felix Hennig Date: Mon, 22 Jan 2024 17:29:12 +0100 Subject: [PATCH 07/12] WIP --- ...DRXXX-authorization-abstraction-layer.adoc | 81 +++++++++++++++++-- 1 file changed, 75 insertions(+), 6 deletions(-) diff --git a/modules/contributor/pages/adr/ADRXXX-authorization-abstraction-layer.adoc b/modules/contributor/pages/adr/ADRXXX-authorization-abstraction-layer.adoc index 5bf082d7f..af933b477 100644 --- a/modules/contributor/pages/adr/ADRXXX-authorization-abstraction-layer.adoc +++ b/modules/contributor/pages/adr/ADRXXX-authorization-abstraction-layer.adoc @@ -11,14 +11,34 @@ Technical Story: https://github.com/stackabletech/issues/issues/439 == Problem Statement -How should we design the user facing part of an authorization layer for the platform? Do we create Custom Resources for some of this? Or simply ConfigMaps? How many, how are the rules split across resources? +How should we design the user facing part of an authorization layer for the platform? +Do we create Custom Resources for some of this, or are ConfigMaps sufficient? +How many, how are the rules split across resources? == Context -As a user I want to be able to very easily define my access control for everything in the platform. -Access control rules per product should be defined on groups or projects, and then I assign users as members of these groups, departments and projects. -I want to be able to simply say "John is a member of the data science team" and that gives John access to a number of resources in the platform based on his team membership. -I also want to be able to easily specify one-of access to individuals for specific resources, such as given Alice access to a specific Trino table. +What is the current state of authorization in the SDP, what do users want to define and which authorization models are widespread already? + +=== Current state of authorization and policy in the SDP + +Currently the Stackable Data Platform supports authorization policies through OPA. +OPA is a general _policy_ agent and does not out of the box have any particular framework in place for _authorization_ as a special case of policy. +OPA uses policy-as-code to define policy, and as such supports a wide variety of approaches to policy definitions. + +=== Authorization settings that users might want to model + +Some use case examples: + +* rules for individuals: Alice needs one-of read access to a Trino Table +* group based access control: Bob joins the company in the data science team and should get access to all the resources he needs to stark working +* resource grouping and ad-hoc groups: A new data analysis task force is formed that needs access to specific resources. Resources should be grouped and then all task force members need access. +* Class based permissions: Andy needs to be able to read _all_ Trino tables, and not just a pre-defined selection of tables. + +A common complaint seems to be that in RBAC systems, roles end up getting copy pasted. +A role might have many permissions attached to it, so if you want to modify a particular permission for just one user, you might end up copy-pasting the role. + +Also, users should be able to treat resources in general the same way across all supported products. +I.e. there should be an abstraction over resources such as Trino tables, Superset dashboards and Kafka topics. === Different authorization models: RBAC, ABAC, ReBAC and more @@ -26,9 +46,15 @@ Out-of-the-box, OPA uses RegoRules to define policies. This is very powerful, but also more complex that other mechanisms such as RBAC or ACLs. We want to pick an authorization model to build on top of OPA and abstract away from the RegoRules for 95% of use cases that the typical Stackable user might encounter. +Role-based access control (RBAC) is a common authorization model where users are assigned roles, and roles come with certain sets of permissions. + Relation-based access control (ReBAC) was popularized by Google Zanzibar and goes beyond RBAC. The relational model allows for more flexibility when defining rules. +https://www.permit.io/blog/rbac-vs-rebac[RBAC vs. ReBAC]. + +https://www.keycloak.org/docs/latest/authorization_services/index.html#_overview[The Keycloak model] + Learn more: * https://authzed.com/zanzibar @@ -37,18 +63,61 @@ Learn more: * https://www.permit.io/blog/oparebac * https://www.permit.io/blog/policy-engines +=== What models do the applications use? + +Druid: Uses an RBAC model + +Kafka: RBAC, group based with LDAP, ACLs + == Decision Drivers * The design should be flexible to allow to easily represent various organizational structures. * It should be possible to group together access to different resources across products. * The design should validate as much of the input as possible, to prevent misspellings from invalidating rules. Nothing should just silently not do anything. * Rules should be defined as Manifests and put into Kubernetes. +* Solution needs to be safely implemented. This means that it might be good to keep complexity low. This is a security component! +* Solution needs to work well with existing authorization models in the applications we support. +* Expressive enough so users do not have to copy-paste roles or lists of permissions. == Constraints * We use OPA as the underlying policy engine, so any design needs to be implementable with OPA. -== Proposed Design +== Expected outcome + +We should decide on a general authorization model, what we want it to look like to the user and also have a rough idea of how it will be implemented. + +== RBAC vs ReBAC + +_RBAC_ is the default choice for authorization models, as it is widely known an understood already, and simple to implement and understand. +Kubernetes itself uses RBAC with Role, ClusterRole, RoleBinding etc.. + +In RBAC it is difficult to give one-of permissions to individual users, since permissions are always assinged to a role, and then the role is assigned to a user. + +How are policies actually derived from the relationships in ReBAC? + +I think it makes sense to stay close to the applications here, i.e. for druid there should still be read/write permissions on resources. + +we still need to define things like "an editor on thsi resource group can write to this topic" + +It would be cool to have "resource groups" and then just relations like "owner", "reader", "edtior" relating to a whole group of resources. + +A mixed model might make sense, maybe a hierarchical tree of user groups on the one side, and resource groups on the other side? or composing roles and also composing groups, and then mapping groups and roles? I got this idea from Keycloak. + +I think maybe the idea of having the permission for an action on a resource is still very central to all the products we support, and it makes sense to keep that. +Maybe just in a "permission bundle" and those can also be grouped? + +I think in i.e. GDrive we could give someone read permission on a folder of files, but this relation needs to be already embedded in the application. +We cannot give someone read access on a group of Trino tables, because this concept does not exist in Trino. +The group will still need to be maintained outside of Trino, and so we do not have a lot to gain here. + +== Design A + +RBAC-based design. + +Let users define + +== Design B We design a relation based access control system (ReBAC), inspired by Google Zanzibar. ReBAC allows for role-based access control as well (RBAC) but goes beyond that to also allow hierarchies of objects that users can be related to. From e586aeb489863d0031ce091fd8bd8e961913eae8 Mon Sep 17 00:00:00 2001 From: Felix Hennig Date: Mon, 22 Jan 2024 17:51:40 +0100 Subject: [PATCH 08/12] Added terminology --- ...DRXXX-authorization-abstraction-layer.adoc | 29 +++++++++++++++---- 1 file changed, 23 insertions(+), 6 deletions(-) diff --git a/modules/contributor/pages/adr/ADRXXX-authorization-abstraction-layer.adoc b/modules/contributor/pages/adr/ADRXXX-authorization-abstraction-layer.adoc index af933b477..597f3fb36 100644 --- a/modules/contributor/pages/adr/ADRXXX-authorization-abstraction-layer.adoc +++ b/modules/contributor/pages/adr/ADRXXX-authorization-abstraction-layer.adoc @@ -19,12 +19,34 @@ How many, how are the rules split across resources? What is the current state of authorization in the SDP, what do users want to define and which authorization models are widespread already? +=== Terminology + +Resource:: A resource in the authorization context is commonly something that can be accessed, read, edited etc., like a DAG in Airflow, a Table in Trino or a file in a file system. Resources can also be grouped, like a folder in a file system containing multiple files. A resource is specific, so it does not refer to Trino tables in general, but to a specific Foo table (for example). +Action:: An action is defined in context of a resource. Examples are "Viewing", "Editing", "Deleting", "Creating". +Permission:: A permission is the combination of an action and a resource. Like "view table Foo". A permission can also be more general, like "view all tables" (i.e. no specific resource is specified, just a class/type of resource). +Policy:: A policy is a generic term that does not only exist in authorization. It is a rule, like "The cluster should always have 10% free memory left" or "Only the HR team can access the employee database". +RBAC:: Role-based access control. +Role:: A role in RBAC generally means a collection of permissions. In RBAC, permissions are assigned to roles. For example, an _admin_ role might have the permission to view and edit all data. A _marketting-employee_ role grants viewing access to a specific set of tables. +ReBAC:: Relation-based access control. +Relation:: A relation is pretty generic, and refers to relations between object and and other objects (or resources), between resources and users or between users and other users or user groups. Examples: "Alice is a _reader_ of a table." "Bob is a _member_ of the data science team." "The `pictures` folder is the _parent_ of the `cat.jpg` file." +Group:: A group is typically a collection of users. Groups can also be organized hierarchically. Groups can sometimes be used to attach roles to, so users can simply be grouped together and their permissions be managed as a whole. + === Current state of authorization and policy in the SDP Currently the Stackable Data Platform supports authorization policies through OPA. OPA is a general _policy_ agent and does not out of the box have any particular framework in place for _authorization_ as a special case of policy. OPA uses policy-as-code to define policy, and as such supports a wide variety of approaches to policy definitions. +Some products do not support OPA yet, but we want to support OPA in them in the future. +Some products like Airflow and Superset do not support OPA and we do not plan to add support at this moment. + +The products themselves also have access control models: + +* Druid: Uses an RBAC model +* Kafka: RBAC, group based with LDAP, ACLs +* Airflow: Uses roles to group permissions, and then assign roles to users. Roles can also be assigned to LDAP groups. + + === Authorization settings that users might want to model Some use case examples: @@ -32,6 +54,7 @@ Some use case examples: * rules for individuals: Alice needs one-of read access to a Trino Table * group based access control: Bob joins the company in the data science team and should get access to all the resources he needs to stark working * resource grouping and ad-hoc groups: A new data analysis task force is formed that needs access to specific resources. Resources should be grouped and then all task force members need access. +* group hierarchies: there might be multiple data science teams that share access to some common resources, but also have specific resources that are only relevant to each team. * Class based permissions: Andy needs to be able to read _all_ Trino tables, and not just a pre-defined selection of tables. A common complaint seems to be that in RBAC systems, roles end up getting copy pasted. @@ -63,12 +86,6 @@ Learn more: * https://www.permit.io/blog/oparebac * https://www.permit.io/blog/policy-engines -=== What models do the applications use? - -Druid: Uses an RBAC model - -Kafka: RBAC, group based with LDAP, ACLs - == Decision Drivers * The design should be flexible to allow to easily represent various organizational structures. From 5da01803a0baab72ac4f78e84ed7b9bf2e1f6e6c Mon Sep 17 00:00:00 2001 From: Felix Hennig Date: Thu, 22 Feb 2024 14:08:07 +0100 Subject: [PATCH 09/12] some changes --- ...DRXXX-authorization-abstraction-layer.adoc | 73 +++++++++++-------- 1 file changed, 44 insertions(+), 29 deletions(-) diff --git a/modules/contributor/pages/adr/ADRXXX-authorization-abstraction-layer.adoc b/modules/contributor/pages/adr/ADRXXX-authorization-abstraction-layer.adoc index 597f3fb36..90e423ecd 100644 --- a/modules/contributor/pages/adr/ADRXXX-authorization-abstraction-layer.adoc +++ b/modules/contributor/pages/adr/ADRXXX-authorization-abstraction-layer.adoc @@ -14,23 +14,13 @@ Technical Story: https://github.com/stackabletech/issues/issues/439 How should we design the user facing part of an authorization layer for the platform? Do we create Custom Resources for some of this, or are ConfigMaps sufficient? How many, how are the rules split across resources? +Where do users want to "hook in"? +How are policies deployed? == Context What is the current state of authorization in the SDP, what do users want to define and which authorization models are widespread already? -=== Terminology - -Resource:: A resource in the authorization context is commonly something that can be accessed, read, edited etc., like a DAG in Airflow, a Table in Trino or a file in a file system. Resources can also be grouped, like a folder in a file system containing multiple files. A resource is specific, so it does not refer to Trino tables in general, but to a specific Foo table (for example). -Action:: An action is defined in context of a resource. Examples are "Viewing", "Editing", "Deleting", "Creating". -Permission:: A permission is the combination of an action and a resource. Like "view table Foo". A permission can also be more general, like "view all tables" (i.e. no specific resource is specified, just a class/type of resource). -Policy:: A policy is a generic term that does not only exist in authorization. It is a rule, like "The cluster should always have 10% free memory left" or "Only the HR team can access the employee database". -RBAC:: Role-based access control. -Role:: A role in RBAC generally means a collection of permissions. In RBAC, permissions are assigned to roles. For example, an _admin_ role might have the permission to view and edit all data. A _marketting-employee_ role grants viewing access to a specific set of tables. -ReBAC:: Relation-based access control. -Relation:: A relation is pretty generic, and refers to relations between object and and other objects (or resources), between resources and users or between users and other users or user groups. Examples: "Alice is a _reader_ of a table." "Bob is a _member_ of the data science team." "The `pictures` folder is the _parent_ of the `cat.jpg` file." -Group:: A group is typically a collection of users. Groups can also be organized hierarchically. Groups can sometimes be used to attach roles to, so users can simply be grouped together and their permissions be managed as a whole. - === Current state of authorization and policy in the SDP Currently the Stackable Data Platform supports authorization policies through OPA. @@ -46,23 +36,6 @@ The products themselves also have access control models: * Kafka: RBAC, group based with LDAP, ACLs * Airflow: Uses roles to group permissions, and then assign roles to users. Roles can also be assigned to LDAP groups. - -=== Authorization settings that users might want to model - -Some use case examples: - -* rules for individuals: Alice needs one-of read access to a Trino Table -* group based access control: Bob joins the company in the data science team and should get access to all the resources he needs to stark working -* resource grouping and ad-hoc groups: A new data analysis task force is formed that needs access to specific resources. Resources should be grouped and then all task force members need access. -* group hierarchies: there might be multiple data science teams that share access to some common resources, but also have specific resources that are only relevant to each team. -* Class based permissions: Andy needs to be able to read _all_ Trino tables, and not just a pre-defined selection of tables. - -A common complaint seems to be that in RBAC systems, roles end up getting copy pasted. -A role might have many permissions attached to it, so if you want to modify a particular permission for just one user, you might end up copy-pasting the role. - -Also, users should be able to treat resources in general the same way across all supported products. -I.e. there should be an abstraction over resources such as Trino tables, Superset dashboards and Kafka topics. - === Different authorization models: RBAC, ABAC, ReBAC and more Out-of-the-box, OPA uses RegoRules to define policies. @@ -86,6 +59,34 @@ Learn more: * https://www.permit.io/blog/oparebac * https://www.permit.io/blog/policy-engines +== Requirements + +The overall design should make it easy for the majority of users to define rules, without needing to write RegoRules. +This should be done with CRDs that can deployed, and it works out of the Box. + +For the remaining users it should be possible to hook into various places of the system to write their own more specific rules. + +* 80% of users can use the CRDs that allow coarse access control in a unified way across the platform, possibly hiding some product specific things. +* 10% of users can drop down one layer into specifying custom JSON data for the Stackable provided Rego rules, + allowing a little bit more detailed access to product specific access control rules such as column masking in Trino. +* 10% of users will want to write completely custom Rego rules, which is currently already possible and will still be supported. + +=== Authorization settings that users might want to model + +Some use case examples: + +* rules for individuals: Alice needs one-of read access to a Trino Table +* group based access control: Bob joins the company in the data science team and should get access to all the resources he needs to stark working +* resource grouping and ad-hoc groups: A new data analysis task force is formed that needs access to specific resources. Resources should be grouped and then all task force members need access. +* group hierarchies: there might be multiple data science teams that share access to some common resources, but also have specific resources that are only relevant to each team. +* Class based permissions: Andy needs to be able to read _all_ Trino tables, and not just a pre-defined selection of tables. + +A common complaint seems to be that in RBAC systems, roles end up getting copy pasted. +A role might have many permissions attached to it, so if you want to modify a particular permission for just one user, you might end up copy-pasting the role. + +Also, users should be able to treat resources in general the same way across all supported products. +I.e. there should be an abstraction over resources such as Trino tables, Superset dashboards and Kafka topics. + == Decision Drivers * The design should be flexible to allow to easily represent various organizational structures. @@ -236,3 +237,17 @@ TSA? OCM? "TSA ist OPA, unser Authr. ist OPA, aber bauen wir oder TSA dinge drumherum, die die Integration schwer machen?" +== Appendix + +=== Terminology + +Resource:: A resource in the authorization context is commonly something that can be accessed, read, edited etc., like a DAG in Airflow, a Table in Trino or a file in a file system. Resources can also be grouped, like a folder in a file system containing multiple files. A resource is specific, so it does not refer to Trino tables in general, but to a specific Foo table (for example). +Action:: An action is defined in context of a resource. Examples are "Viewing", "Editing", "Deleting", "Creating". +Permission:: A permission is the combination of an action and a resource. Like "view table Foo". A permission can also be more general, like "view all tables" (i.e. no specific resource is specified, just a class/type of resource). +Policy:: A policy is a generic term that does not only exist in authorization. It is a rule, like "The cluster should always have 10% free memory left" or "Only the HR team can access the employee database". +RBAC:: Role-based access control. +Role:: A role in RBAC generally means a collection of permissions. In RBAC, permissions are assigned to roles. For example, an _admin_ role might have the permission to view and edit all data. A _marketting-employee_ role grants viewing access to a specific set of tables. +ReBAC:: Relation-based access control. +ABAC:: Attribute-based access control. +Relation:: A relation is pretty generic, and refers to relations between object and and other objects (or resources), between resources and users or between users and other users or user groups. Examples: "Alice is a _reader_ of a table." "Bob is a _member_ of the data science team." "The `pictures` folder is the _parent_ of the `cat.jpg` file." +Group:: A group is typically a collection of users. Groups can also be organized hierarchically. Groups can sometimes be used to attach roles to, so users can simply be grouped together and their permissions be managed as a whole. From b0c23b7dd20d48202e07e0025bb07b13d7ad29c0 Mon Sep 17 00:00:00 2001 From: Felix Hennig Date: Thu, 22 Feb 2024 14:19:39 +0100 Subject: [PATCH 10/12] delete old design --- ...DRXXX-authorization-abstraction-layer.adoc | 137 ++---------------- 1 file changed, 15 insertions(+), 122 deletions(-) diff --git a/modules/contributor/pages/adr/ADRXXX-authorization-abstraction-layer.adoc b/modules/contributor/pages/adr/ADRXXX-authorization-abstraction-layer.adoc index 90e423ecd..c5de339f7 100644 --- a/modules/contributor/pages/adr/ADRXXX-authorization-abstraction-layer.adoc +++ b/modules/contributor/pages/adr/ADRXXX-authorization-abstraction-layer.adoc @@ -105,137 +105,30 @@ I.e. there should be an abstraction over resources such as Trino tables, Superse We should decide on a general authorization model, what we want it to look like to the user and also have a rough idea of how it will be implemented. -== RBAC vs ReBAC +== Proposed design -_RBAC_ is the default choice for authorization models, as it is widely known an understood already, and simple to implement and understand. -Kubernetes itself uses RBAC with Role, ClusterRole, RoleBinding etc.. +=== Stackable Rego rule library -In RBAC it is difficult to give one-of permissions to individual users, since permissions are always assinged to a role, and then the role is assigned to a user. +For every product (and every supported version of a product) we ship a ruleset that users can use (and might be used as a default). +Since the rules are dependent on the product version, the product operator needs to ship these rules. +What about the OPA version? Rules need to also be compatible with the OPA version? -How are policies actually derived from the relationships in ReBAC? - -I think it makes sense to stay close to the applications here, i.e. for druid there should still be read/write permissions on resources. - -we still need to define things like "an editor on thsi resource group can write to this topic" - -It would be cool to have "resource groups" and then just relations like "owner", "reader", "edtior" relating to a whole group of resources. - -A mixed model might make sense, maybe a hierarchical tree of user groups on the one side, and resource groups on the other side? or composing roles and also composing groups, and then mapping groups and roles? I got this idea from Keycloak. - -I think maybe the idea of having the permission for an action on a resource is still very central to all the products we support, and it makes sense to keep that. -Maybe just in a "permission bundle" and those can also be grouped? - -I think in i.e. GDrive we could give someone read permission on a folder of files, but this relation needs to be already embedded in the application. -We cannot give someone read access on a group of Trino tables, because this concept does not exist in Trino. -The group will still need to be maintained outside of Trino, and so we do not have a lot to gain here. +=== product specific JSON data policies -== Design A +The rules work with product specific JSON policies. +These policies should expose every feature that the authorizer supports. -RBAC-based design. +=== Unified policy CRs -Let users define +The unified policy CRD is modeled as ABAC. +Resources and users have attributes which get matched in a policy. +If a decision request matches to a policy, the permissions from the policy apply. -== Design B +Resource attributes are resource specific, i.e. for a Trino table, there is a "catalog" attribute, but that only exists on Trino tables. -We design a relation based access control system (ReBAC), inspired by Google Zanzibar. -ReBAC allows for role-based access control as well (RBAC) but goes beyond that to also allow hierarchies of objects that users can be related to. -This allows for more flexibility when defining organizational structures, and rules can be attached at any level. +More advanced stuff like masking properties is maybe not supported. maybe the access levels are also only "read", "write" and "full". -We define a CustomResource that allows users to define ReBAC stuff in CRs. -This allows for more validation instead of putting rules in a DSL or in JSON into a ConfigMap. - -There are three kinds of rules that users can define easily in a CR: - -* relations of objects and users -* relations of objects and user sets -* relations of objects and other objects - -Users are pre defined in either LDAP or keycloak, and referenced by their name/ID. -Resolving users is done in OPA, using the UserInfoFetcher. -For this ADR, we will assume users are specified by their username, i.e. "alice" or "bob". - -Other objects in the system can be entities inside of the products, such as a `trino-table`. -We also allow users to define their own objects and relations, mainly to organize their users and permissions. -Every user-defined object and relation needs to be specified in an RebacType object. - -Example: - -[source,yaml] ----- -kind: RebacType -metadata: - name: project # <1> -spec: - relations: # <2> - - name: member - objects: # <3> - - secret-project - - datascience-spike - - ladida - - otherproj ----- - -<1> The name of the RebacType. This will be referenced in rule definitions. -<2> The relations that are defined for this type. In this case there is a "member" definition. Users can be members of a project. -<3> Object defintions. This is optional; if given, only these objects can be referenced, adding another layer of verification. - -For products, the SDP comes with predefined objects that make sense for the product. -For example in Trino we will define `trino-table` and `trino-catalog`, each with a `reader`, `editor` and `owner` relation. -For Superset we can define a `superset-dashboard` with the same relations. -For Kafka we can define a `kafka-topic` type. - -We can group some resources together into a project, and then assign users to the project. -Users can be assigned individually or based on their group memberships. - -Here is an example of what it could look like in yaml: - -[source,yaml] ----- -kind: RebacRelations -metadata: - name: project-ladida -# Project Ladida manages the mydata table in Trino -# User 13 and all the members of the 'otherproj' project are members -# of Project Ladida -relations: - # Which resources are part of the project? - - subject: "project:ladida" - relation: "owner" - object: "trino-table:mydata" - - subject: "project:ladida" - relation: "reader" - object: "kafka-topic:datasource" - - subject: "project:ladida" - relation: "editor" - object: "superset-dashboard:ladida-viz" - # Who is part of the project? - - user: "alice" - relation: "member" - object: "project:ladida" - - userset: - object: "ad-group:datascience" - relation: "member" - relation: "member" - object: "project:ladida" ----- - -The first three relations define the project as the owner/reader/editor of a number of resources. -Transitively, any member of the project will get these relations. - -The last two relations define the user "alice" as a member of the project, as well as any member of the ActiveDirectoy group "datascience". - -=== Implementation - -The RebacType and RebacRelations can both be annotated with the `opa.stackable.tech/bundle: "true"` label to include it into the OPA bundle. - -The relation definitions can the be verified - to a degree - and subsequently everything is serialized as JSON and provided as `data` to OPA. -We then need to define a RegoRule framework inside OPA to evaluate these rules correctly. - -== TODO: Gaia-X considerations - -TSA? OCM? - -"TSA ist OPA, unser Authr. ist OPA, aber bauen wir oder TSA dinge drumherum, die die Integration schwer machen?" +The OPA operator should read these CRs and convert them into JSON data policies. == Appendix From e55b48041210ba72ae418787eeaffe989b925f0d Mon Sep 17 00:00:00 2001 From: Felix Hennig Date: Thu, 22 Feb 2024 16:35:32 +0100 Subject: [PATCH 11/12] ~ --- ...DRXXX-authorization-abstraction-layer.adoc | 2 +- .../ADRXXX-authorization-decision-layer.adoc | 112 ++++++++++++++++++ 2 files changed, 113 insertions(+), 1 deletion(-) create mode 100644 modules/contributor/pages/adr/ADRXXX-authorization-decision-layer.adoc diff --git a/modules/contributor/pages/adr/ADRXXX-authorization-abstraction-layer.adoc b/modules/contributor/pages/adr/ADRXXX-authorization-abstraction-layer.adoc index c5de339f7..adaba4ab6 100644 --- a/modules/contributor/pages/adr/ADRXXX-authorization-abstraction-layer.adoc +++ b/modules/contributor/pages/adr/ADRXXX-authorization-abstraction-layer.adoc @@ -1,4 +1,4 @@ -= ADRXXX: Authorization abstraction layer += ADRXXX: Authorization decision layer Felix Hennig v0.1, 2024-01-18 :status: draft diff --git a/modules/contributor/pages/adr/ADRXXX-authorization-decision-layer.adoc b/modules/contributor/pages/adr/ADRXXX-authorization-decision-layer.adoc new file mode 100644 index 000000000..aaddd7d7a --- /dev/null +++ b/modules/contributor/pages/adr/ADRXXX-authorization-decision-layer.adoc @@ -0,0 +1,112 @@ += ADRXXX: Authorization decision layer +Felix Hennig +v0.1, 2024-01-18 +:status: draft + +* Status: {status} +* Deciders: TBD +* Date: 2024-01-18 + +Technical Story: https://github.com/stackabletech/issues/issues/439 + +== Problem Statement + +The Stackable Data Platform provides the OpenPolicyAgent as a policy engine, but we currently do not supply examples or rule frameworks to the users to easily get started with writing authorization policies. + +We want to supply a rego rule library that platform users can use as a default or as a starting point to write their own Rego rules. +These rules (and accompanying data structures) should expose all the product specifics that each product offers. +A simplified and abstracted authorization layer will be built later, on top of this one. + +== Decision Drivers + +Users can already write their own rego rules, but we want to make it easier for them and allow them to only write JSON policies. +At the same time we still want them to have as much control over the product as possible, without having to write their own Rego rules. + +== Proposed design + +We have specific rego rules per product. +These need to be highly specific, because every authorizer has a different request structure. + +While there are some commonalities across products (they all have a 'resource' concept), details are product specific and difficult to generalize +without losing out on fine grained control. +We want to keep as much control as possible. + +The RegoRules are deployed by the product operator as ConfigMaps. +The package name contains the version of the ruleset, the product and the product version: `stackable.v1.trino.v439` + +NOTE: Should we simply version the stackable rules with the platform version? + +=== Cluster/Stacklet information in the requests + +Resources are already organized hierarchically, for example in Trino: Catalog, Schema, Table. +The Stacklet sits on top of this, and can be seen as another layer. +Because of this, it makes sense to add the Stacklet name, namespace and labels to the authorization request. + +The information could be added by the specific authorizer plugin, but at least for Kafka and Trino, this would require patching the upstream authorizer. + +Alternatively we could add a little intermediate package: + +[source] +---- +package enrichRequest.simpleTrino # auto generated package name + +import rego.v1 +import myRules # package name taken from the clusterConfig + +allow if { + myRules.allow with input as { # package name taken from the clusterConfig + "product": "trino", + "cluster": { # the name and labels are taken from the kubernetes metadata + "name": "simple-trino", + "namespace": "foo", + "labels": { + "dev": true + } + }, + "request": input + } +} +---- + +This could be generated by the product operators and would be "invisible" to the user. + +=== Using the Stackable Rego Framework + +Currently, the user specifies a `package` when using the OPA authorizer. + +[source,yaml] +---- +kind: TrinoCluster +metadata: + name: simple-trino + labels: + dev: true +spec: + image: + productVersion: "428" + clusterConfig: + authorization: + opa: + configMapName: my-opa + package: myRules +---- + +To make it easy to use the framework, the framework should either be the default (and is maybe versioned with the platform version) +or you select the version of the rule framework like this: + +[source,yaml] +---- +kind: TrinoCluster +metadata: + name: simple-trino + labels: + dev: true +spec: + image: + productVersion: "428" + clusterConfig: + authorization: + opa: + configMapName: my-opa + stackableRules: v1 +---- \ No newline at end of file From 49c5ce9d40b90fe14f598dad1e750cac13285296 Mon Sep 17 00:00:00 2001 From: Felix Hennig Date: Thu, 22 Feb 2024 17:09:46 +0100 Subject: [PATCH 12/12] ~ --- .../pages/adr/ADRXXX-authorization-decision-layer.adoc | 2 ++ 1 file changed, 2 insertions(+) diff --git a/modules/contributor/pages/adr/ADRXXX-authorization-decision-layer.adoc b/modules/contributor/pages/adr/ADRXXX-authorization-decision-layer.adoc index aaddd7d7a..3cc3fc74a 100644 --- a/modules/contributor/pages/adr/ADRXXX-authorization-decision-layer.adoc +++ b/modules/contributor/pages/adr/ADRXXX-authorization-decision-layer.adoc @@ -31,6 +31,8 @@ While there are some commonalities across products (they all have a 'resource' c without losing out on fine grained control. We want to keep as much control as possible. +We can also keep it close to what the software is already doing - in the case of Trino - which makes it easier for users that are migrating. + The RegoRules are deployed by the product operator as ConfigMaps. The package name contains the version of the ruleset, the product and the product version: `stackable.v1.trino.v439`