Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

AWS Organizations L2s #2877

Open
jdiazcano opened this issue Jun 14, 2019 · 54 comments
Open

AWS Organizations L2s #2877

jdiazcano opened this issue Jun 14, 2019 · 54 comments
Labels
effort/large Large work item – several weeks of effort feature-request A feature should be added or improved. p2

Comments

@jdiazcano
Copy link

I'd like to have AWS Organizations in CDK in order to automate the creation of the accounts, permissions etc!

https://aws.amazon.com/organizations/
@jdiazcano jdiazcano added the feature-request A feature should be added or improved. label Jun 14, 2019
@hoegertn
Copy link
Contributor

What exactly would the use cases be? Can you elaborate? Not many things in organizations are in CFN.

@jdiazcano
Copy link
Author

I was thinking on not having to create accounts manually and since I'm already using CDK I would like to go with CDK instead of going for the API.

It's nothing super urgent or whatever, I guess that if there is no CFN support from Organizations then there's little that can be done

@hoegertn
Copy link
Contributor

For creating accounts I think one thing could be a step function (deployed using CDK) that can be used to create accounts. That would be as far as you can get I think. You could then use this as a custom resource. But beware that you cannot simply delete accounts.

@NGL321 NGL321 added the needs-triage This issue or PR still needs to be triaged. label Jun 17, 2019
@NGL321 NGL321 added no-cfn-supp and removed needs-triage This issue or PR still needs to be triaged. labels Jul 29, 2019
@NGL321 NGL321 mentioned this issue Jul 29, 2019
Closed
@pgollucci
Copy link

I'd like to have AWS Organizations in CDK in order to automate the creation of the accounts, permissions etc!

https://aws.amazon.com/organizations/

I know many federal government agencies that would use this!

@NGL321 NGL321 added needs-cfn This issue is waiting on changes to CloudFormation before it can be addressed. and removed no-cfn-supp labels Aug 23, 2019
@pgollucci
Copy link

step functions are what I was thinking too. (Understand and ignoring deleting org accounts).
However step function are not FedRAMP yet....

@nonken nonken mentioned this issue Oct 13, 2019
Open
@rix0rrr rix0rrr added @aws-cdk/aws-iam Related to AWS Identity and Access Management effort/large Large work item – several weeks of effort labels Jan 23, 2020
@jakubgaj
Copy link

jakubgaj commented Apr 15, 2020 •
edited
Loading

For automated accounts creation/bootstrapping the ADF can be used:
https://github.com/awslabs/aws-deployment-framework

I'd like to see Organizations support in CDK at least for defining OU structure & SCPs (Service Control Policies). CloudFormation doesn't support it either, although on their roadmap.

@rrrix
Copy link

rrrix commented Jun 15, 2020

I'm also interested in having support for Service Control Policies and OUs. Currently we have to manage these either via the AWS Console or build one-off custom scripts.

SCPs are resources which are applied on the Organization Root account and can be automated via relatively scripts (using the AWS CLI for example).

I think this is something that could possibly be done via the AwsCustomResource (Custom Resources for AWS APIs) Constructs.

Service Control Policies (SCPs)

For SCPs, I suppose there would be two "types" of SCP constructs - one to CrUD the SCP's themselves, and another to handle Attachment (e.g. CreateAttachment and DeleteAttachment, Update could be a no-op).

Organizational Units (OUs)

For OUs, this maps 1:1 to the AwsCustomResource construct CrUD interface.

I haven't used the AwsCustomResource construct before... I'll play around to see if I can get something basic working.

@jakubgaj
Copy link

@rrrix OU structure can be up to 5 levels down and SCPs can be applied to any OU or account. For more complex scenarios you need to build some logic. I'd expect CDK to simply support it similar way as Terraform does:

https://www.terraform.io/docs/providers/aws/r/organizations_organizational_unit.html
https://www.terraform.io/docs/providers/aws/r/organizations_policy.html
https://www.terraform.io/docs/providers/aws/r/organizations_policy_attachment.html

@delprofundo
Copy link

+1

We are building an ops portal and have some very nice web application templates that rely on CDK for infrastructure and Serverless to manage the application. Pretty cool stuff that lays down event sourcing microservice stacks within an account for example.

So in the master billing account we would like to basically have a pane of glass to select a domain to sub from, some meta params and say "ok, give me 5 developer accounts, QA, staging and prod accounts pointing to this repo'. Laying down the accounts is the only manual step in our processes, for now we can use ADF but it would be a good native for CDK.

@ragona
Copy link

ragona commented Jul 21, 2020

I'm considering using just plain boto3 (the python aws sdk) here since AWS accounts are weird. They're not really resources in the same way other things are, as Thorsten points out above. I'm wondering if it makes more sense to make these two tools interact via a shared config file. Also curious to hear how AWS is thinking about this. (I guess maybe you'd have to support deleting accounts first!)

@flochaz
Copy link
Contributor

flochaz commented Aug 21, 2020

My team is planning on developing this (waiting for CFN to support it :) ) and we are considering 2 approaches:

  1. wrap aws-orgs tool in a custom resource and therefore leverage the yaml schema to control the construct
  2. leverage AWSSDKCall + https://docs.aws.amazon.com/organizations/latest/APIReference/API_Operations.html to create a kind of L1 (API based not CFN) construct

Option 2 is cleaner from my point of view but implies many custom resources created which might slowdown deployment (spinup a lambda for each API call etc.) and make it harder than option 1 to create the necessary resources (yaml structure of aws-orgs is quite handy) ...

any opinion ? is cdk team interested in having a PR for one of those options ?

@hoegertn
Copy link
Contributor

What do you think about option 3 and creating a CloudFormation resource provider that does this and then using CDK to generate these new CFN resources?

@fbouteruche
Copy link

I may be wrong but it seems that a CloudFormation resource provider is local to an AWS account. You have to deploy it before you can use it in CDK. Am I right?

@hoegertn
Copy link
Contributor

That is correct and would be a downside of this approach

@tsykora-verimatrix
Copy link

tsykora-verimatrix commented Oct 2, 2020 •
edited
Loading

Terraform supports this:
https://registry.terraform.io/modules/barundel/organizations/aws/1.1.1?tab=inputs

Boto3 supports it as well, but relying on boto3 to create resources and mix it with CDK is not a good practice.
It would be much cleaner for CDK (CF) to catch up with Terraform and support creation of OUs, nested OUs, AWS accounts, SCPs, Tagging polices natively.

@0xjjoyy
Copy link

0xjjoyy commented Nov 15, 2020

@flochaz did you end up creating a custom CDK approach ?

@flochaz
Copy link
Contributor

flochaz commented Nov 16, 2020

@0xjjoyy we ended up creating pure CDK custom resources to create OUs and Accounts. We leveraged a lot CDK AWSCustomResource and CustomResource Provider:

There is still a lot to develop to have all the features of tools like aws-orgs or org-formation ...

@0xjjoyy
Copy link

0xjjoyy commented Nov 16, 2020

@flochaz Got it, thanks. Does your CDK solution handle service control policies? I was trying to look in your repo though couldn't find. Want to avoid duplicating efforts for a custom CDK resource :)

@flochaz
Copy link
Contributor

flochaz commented Nov 16, 2020 via email

@pflorek
Copy link
Contributor

pflorek commented Jan 15, 2022

@coreycoto thank you!

The project wraps AWS Organization API just by CDK Custom Resources.

The Github Actions are the defaults from projen, which is just CDK for projects from the AWS devs.

Currently, the cdk-organizations lacks some better testing. I still need to figure out how to write proper tests for all the Custom Resources...

Also we will need some import logic in a cdk-ish style like ECR Repository.fromRepositoryAttributes. The only question to do so is, which AWS Organizations APIs are idempotent. Then it can be implemented fast and lazy.

One point I didn't got after reading all the docs. Why there is a listRoots endpoint, when one account can only belong to one organization at a time. (API_CreateOrganization_Errors)

@jmjava
Copy link

jmjava commented Jan 27, 2022 via email

@jeromevdl
Copy link

Actually, this is not true. The organization API is in us-east-1 only but you can provision organizations and accounts in other regions.

@jmjava
Copy link

jmjava commented Jan 27, 2022 via email

@peterwoodworth
Copy link
Contributor

Just want to reiterate that the best way for CDK to directly support Organizations is if CloudFormation supports it first. See the coverage roadmap and share your feedback there 🙂

@sholtomaud
Copy link

Just want to reiterate that the best way for CDK to directly support Organizations is if CloudFormation supports it first. See the coverage roadmap and share your feedback there 🙂

Not really. CDK Constructs/CustomResources help us get around Cfn limitations yeah?

@dls314
Copy link

dls314 commented Sep 1, 2022 •
edited
Loading

The ability to generate SCPs with a CDK-like syntax would be very welcome, even without the ability to deploy them using CDK-like tooling or through CFN.

I know that things like Cloud Development Kit for Terraform (CDKTF) exist -- Does anyone who might respond know of a generation language for SCPs that exists now?

https://github.com/pepperize/cdk-organizations#policy <-- maybe this is what I'm looking for, but I would just like to get the SCP as a JSON and carry it over to an existing CfCT for deployment

I'd rather have CDK-similar generation, but would use anything better than mustache templates ;-)

@kiernan
Copy link

kiernan commented Sep 17, 2022

The ability to generate SCPs with a CDK-like syntax would be very welcome, even without the ability to deploy them using CDK-like tooling or through CFN.

I know that things like Cloud Development Kit for Terraform (CDKTF) exist -- Does anyone who might respond know of a generation language for SCPs that exists now?

https://github.com/pepperize/cdk-organizations#policy <-- maybe this is what I'm looking for, but I would just like to get the SCP as a JSON and carry it over to an existing CfCT for deployment

I'd rather have CDK-similar generation, but would use anything better than mustache templates ;-)

Given the limited number of allowed SCPs per OU/Account they do need to be able to be easily combined together / automatically generated to stay within the limit.

@pflorek
Copy link
Contributor

pflorek commented Nov 19, 2022

Native CFN support for Account, Policy and OrganizationalUnit is there

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-organizations-account.html

🚀

@jeromevdl
Copy link

OrganizationalUnit is not in CDK v2.51.1, only Account and Policy.

It should come soon (

aws-cdk/packages/@aws-cdk/cfnspec/spec-source/specification/000_cfn/000_official/000_AWS_Organizations.json

Line 61 in e5a77e8

"AWS::Organizations::OrganizationalUnit": {
)

@pflorek
Copy link
Contributor

pflorek commented Nov 19, 2022

@jeromevdl I bet it get's generated into the CDK with the upcoming CFN spec update?

Did a small proof of concept changing a higher level construct using the organizations.CfnPolicy. I like the idea of the targetIds property. How many items may that list support?

The AWS::Organizations::Account looks almost like our async custom providers. One difference it lacks auto opt-in from ClickOps. So we need something like Account.fromAttributes(). Why does it support a list of parentIds, either the parent is an OU or the (implizit) organizations root?

Currently we still need something like AwsCustomResource for the organizations root and maybe for the organization itself.

Is it possible to an existing org using https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/resource-import-existing-stack.html with the CDK?

pflorek added a commit to pflorek/aws-cdk that referenced this issue Nov 20, 2022
@pflorek
feat: organization base
2b57818 
> basic higher level constructs

**features:**
- adds higher level constructs `Account`, `OrganizationalUnit`, `Policy` building up the org tree
- adds utility construct `OrganizationRoot` to retrieve the root for the first organizational units (singleton `AwsCustomResource`)

**todo:**
- [] decide how to sequentially chain the organization tree
- [] add doc blocks, usage example and howtos
- [] improve tests (unit coverage and integ tests)

> sequentially chain resources is an important feature. The AWS Organizations API can create accounts only sequentially. Also adding policies, delegating administration and enabling trusted services needs to sequentially chained. Here is a solution that uses the construct tree walking `Aspect`: [https://github.com/pepperize/cdk-organizations/blob/main/src/dependency-chain.ts](https://github.com/pepperize/cdk-organizations/blob/main/src/dependency-chain.ts). Another option could be to chain the dependencies in the `Account` and `OrganizationalUnit`

**inversion of parentship:**
It could be useful to inverse the parent child relation, for example

```typescript
organizationalUnit.addAccount(account);
```

instead of
```
new Account(scope, id, {
  parent: ou,
});
```

also it could be useful to inverse the policy attachment

```typescript
export class Account {
  public function attachPolicy(policy: IPolicy): void {
    policy.addAccount(this);
  }
}

```

_Delegation of the attachment could also be useful if explicit dependency chaining is used._

**next (later on):**

- add `ScpPolicy`, `BackupPolicy`, `TagPolicy`, `AiPolicy` as flavors of `PolicyBase`
- add `Organization` construct to enable AWS Organizations
- add  enabling `PolicyType`, `DelegatedAdministrator`, `TrustedService`

Fixes: aws#2877
pflorek added a commit to pflorek/aws-cdk that referenced this issue Nov 20, 2022
@pflorek
feat: organization base
fc3fe18 
> basic higher level constructs

**features:**
- adds higher level constructs `Account`, `OrganizationalUnit`, `Policy` building up the org tree
- adds utility construct `OrganizationRoot` to retrieve the root for the first organizational units (singleton `AwsCustomResource`)

**todo:**
- [] decide how to sequentially chain the organization tree
- [] add doc blocks, usage example and howtos
- [] improve tests (unit coverage and integ tests)

> sequentially chain resources is an important feature. The AWS Organizations API can create accounts only sequentially. Also adding policies, delegating administration and enabling trusted services needs to sequentially chained. Here is a solution that uses the construct tree walking `Aspect`: [https://github.com/pepperize/cdk-organizations/blob/main/src/dependency-chain.ts](https://github.com/pepperize/cdk-organizations/blob/main/src/dependency-chain.ts). Another option could be to chain the dependencies in the `Account` and `OrganizationalUnit`

**inversion of parentship:**
It could be useful to inverse the parent child relation, for example

```typescript
organizationalUnit.addAccount(account);
```

instead of
```
new Account(scope, id, {
  parent: ou,
});
```

also it could be useful to inverse the policy attachment

```typescript
export class Account {
  public function attachPolicy(policy: IPolicy): void {
    policy.addAccount(this);
  }
}

```

_Delegation of the attachment could also be useful if explicit dependency chaining is used._

**next (later on):**

- add `ScpPolicy`, `BackupPolicy`, `TagPolicy`, `AiPolicy` as flavors of `PolicyBase`
- add `Organization` construct to enable AWS Organizations
- add  enabling `PolicyType`, `DelegatedAdministrator`, `TrustedService`

Fixes: aws#2877
This was referenced Nov 20, 2022
Closed
Closed
@raajheshkannaa
Copy link

I'm still unable to import 'aws_organizations' using the latest version of cdk v2.51.1.

Do I have to do something else specific ?

@pflorek
Copy link
Contributor

pflorek commented Nov 22, 2022

@raajheshkannaa Here pepperize/cdk-organizations#753 and here #23001 are some drafts with the new organizations in the cdk 😉

@peterwoodworth
Copy link
Contributor

@raajheshkannaa The l1 resources should be there (there are no L2s yet), this works for me on the latest version 2.51.1

import * as orgs from 'aws-cdk-lib/aws-organizations';

export class MyStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props: MyStackProps) {
    super(scope, id, props);

    new orgs.CfnAccount(this, 'CfnAccount', {})
  }
}

@TheRealAmazonKendra TheRealAmazonKendra added feature/coverage-gap Gaps in CloudFormation coverage by L2 constructs and removed needs-cfn This issue is waiting on changes to CloudFormation before it can be addressed. labels Jan 25, 2023
@peterwoodworth peterwoodworth changed the title AWS Organizations AWS Organizations L2s May 2, 2023
@peterwoodworth peterwoodworth added p2 and removed feature/coverage-gap Gaps in CloudFormation coverage by L2 constructs @aws-cdk/aws-iam Related to AWS Identity and Access Management labels May 2, 2023
@github-actions
Copy link

github-actions bot commented May 7, 2023

This issue has received a significant amount of attention so we are automatically upgrading its priority. A member of the community will see the re-prioritization and provide an update on the issue.

@github-actions github-actions bot added p1 and removed p2 labels May 7, 2023
@sholtomaud
Copy link

Looks like after 4 years this might be a priority now :) Great!

Does the CDK team have sufficient information to generate requirements for this feature?

Some of the things we wanted were:

  • ability to create SCPs
  • ability to create and delete accounts
  • ability to configure Control Tower etc.
  • ability to configure everything required for the vending, management, updating, deletion of Organisation and it's accounts
  • support enterprise scale account vending with, "network stitching" for an account where the Network design uses Hub-Spoke model with Gateway Loadbalancer, Transit Gateway, Palo Altos etc
  • abitlity to configure account permissions
  • ability to configure roles that have access to an account in an enterprise that has Active Directory for Federated (SAML) IAM.
  • etc.

@pahud pahud added p2 and removed p1 labels Jun 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
effort/large Large work item – several weeks of effort feature-request A feature should be added or improved. p2
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat(organizations): add basic organizations higher level constructs pflorek/aws-cdk