Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Filter: load JSON definitions and policies from file #7

Merged
merged 2 commits into from
Jul 3, 2023

Conversation

robobario
Copy link

@robobario robobario commented Jun 22, 2023

Why:
We want users to have access to the full Smorgasbord of configuration including different encryption policies per-topic and support for multiple KMS implementations.

With this change the Kroxylicious Filter will require configuration to point to the two JSON files. This gives us the closest integration with the encryption module because JsonPolicyLoader.loadTopicPolicies works in terms of File objects and does some useful work like validating and associating policies and KMS implementations.

example Kroxylicious configuration

filters:
  - type: TopicEncryption::EncryptProduce
    config:
       inMemoryPolicyRepository:
         kmsDefinitionsFile: "/path/to/kms-def.json"
         topicPoliciesFile: "/path/to/topic-policies.json"
  - type: TopicEncryption::DecryptFetch
    config:
       inMemoryPolicyRepository:
         kmsDefinitionsFile: "/path/to/kms-def.json"
         topicPoliciesFile: "/path/to/topic-policies.json"

In future, if we want to be able to embed the KMS definitions and topic policies in the Filter configuration it might be nice if JsonPolicyLoader had another method like:
List<TopicPolicy> loadTopicPolicies(List<TopicPolicy> topicPolicies, Map<String, KmsDefinition> kmsDefs) that did the same work without the dependency on loading JSON from File.

Implements #3

@robobario
Copy link
Author

I've now refactored the tests, splitting them into tests covering Produce Encryption and Fetch Decryption, testing each side in isolation. This combined with adding tests for per-topic encryption policies would catch the issues fixed by commit #de60abe. The tests were working because of the hardcoded wildcard policy combined with the metadata request failure caching an empty string. We found a name for the topic (empty string), and that got the wildcard policy applied to it, test passes ✔️

@robobario robobario requested a review from SamBarker June 30, 2023 05:09
@robobario
Copy link
Author

To catch a comment from the demos, @SamBarker pointed out that it would be good if we could get credentials into the key manager definition file (like the vault token) in a kubernetes friendly way. So instead of directly in our JSON file, refer to a secret file or environment variable. Captured in #8

Copy link
Member

@SamBarker SamBarker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few cosmetic thoughts but looks good to me.

Why:
At apiVersion 10, the MetadataRequest is only allowed to take null topic ids.
So it responds with an error when we set the id. We need to be at apiVersion
12. We were also missing a guard to prevent caching in this error condition
which allowed an empty topic name to be put in the cache on error. Resulting
in no decryption being applied.

Signed-off-by: Robert Young <[email protected]>
Why:
We want users to have access to the full Smorgasbord of configuration
including different encryption policies per-topic and support for
multiple KMS implementations.

With this change the Kroxylicious Filter will require configuration
to point to the two JSON files. This gives us the closest integration
with the encryption module because `JsonPolicyLoader.loadTopicPolicies`
works in terms of File objects and does some useful work like validating
and associating policies and KMS implementations.

In future, if we want to be able to embed the KMS definitions and
topic policies in the Filter configuration it might be nice if
`JsonPolicyLoader` had another method like:
`List<TopicPolicy> loadTopicPolicies(List<TopicPolicy> topicPolicies, Map<String, KmsDefinition> kmsDefs)`
that did the same work without the dependency on loading JSON from File.

Signed-off-by: Robert Young <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants