Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support AWS_MSK_IAM authentication #2519

Merged
merged 7 commits into from
Mar 12, 2025
Merged

Support AWS_MSK_IAM authentication #2519

merged 7 commits into from
Mar 12, 2025

Conversation

dpkp
Copy link
Owner

@dpkp dpkp commented Mar 11, 2025

Update @mattoberle PR #2255 to new sasl module.

@dpkp dpkp mentioned this pull request Mar 11, 2025
4 tasks
mattoberle and others added 5 commits March 11, 2025 14:44
Adds an AWS_MSK_IAM authentication mechanism which is described here:
* https://github.com/aws/aws-msk-iam-auth#uriencode

To use the mechanism pass the following keyword arguments when
initializing a class:

```
security_protocol='SASL_SSL',
sasl_mechanism='AWS_MSK_IAM',
bootstrap_servers=[
    'b-1.cluster.x.y.kafka.region.amazonaws.com:9088',
    ...
],
```

The credentials and region will be pulled using `botocore.session.Session`.
Using the mechanism requires the `botocore` library which can be
installed with:

```sh
pip install botocore
```

**TODO:**

- [ ] Documentation
- [ ] Tests
- [ ] Refresh mechanism for temporary credentials?
The two tests in `test/test_msk.py` should ensure that the changes to
`kafka/msk.py` do not break the authentication payload.

The authentication payload was validated using a real AWS Kafka cluster
before adding tests with the hard-coded signatures.
@dpkp dpkp force-pushed the mattoberle/aws-msk-iam-sasl branch from 08f6273 to 875b420 Compare March 11, 2025 21:46
@dpkp dpkp merged commit 02dd98f into master Mar 12, 2025
16 checks passed
@dpkp dpkp deleted the mattoberle/aws-msk-iam-sasl branch March 12, 2025 00:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants