Possible to cache graphql.parse() results? #5377

edulop91 · 2022-11-10T01:56:21Z

Short description

I am wondering if there is an easy way to cache the graphql schema AST that is generated by graphql.parse function call. I have been working on integrating OPA for our graphql layer and, unfortunately, am running into performance issues due how long it takes to generate the AST from a raw query.

Our graphQL schema does not change between each authorization decisions and therefore it makes sense to generate thes chema AST once and re-use it for each authorization query.

I am running OPA embedded into Go mode. This is running locally from my laptop.

Steps To Reproduce

Nothing really interesting here. Rego looks like:

package graphql

import future.keywords

query_ast := graphql.parse(input.query, input.schema)[0]

The only difference is having farily large input.query and input.schema.

Expected behavior

I would like to find a way to cache my graphql schema AST between authorization decisions.

Additional context

The text was updated successfully, but these errors were encountered:

srenatus · 2022-11-10T08:28:16Z

I think it should be possible to do this transparently. I.e. any call to graphql.parse(..) would check if the schema was already parsed, and retrieve that from the inter-query cache.

🤔 I wonder if this should be a feature to opt-in to, or if we need to worry about the cache growing too large... @philipaconrad What do you think?

philipaconrad · 2022-11-11T21:17:39Z

@srenatus I agree that we could probably borrow the inter-query cache system for this, but it easily could cause a memory usage spike for folks with many unique GraphQL schemas that get cached.

srenatus · 2022-11-16T12:12:29Z

Mentioned OOB -- maybe some sort of LRU cache could help here. If we cache 10 schemas, we'd probably solve the problem for most users, while not blowing up the memory usage for those special cases where many unique schemas are parsed. For those special cases, the situation would get neither better nor worse from having an LRU.

stale · 2022-12-16T16:47:40Z

This issue has been automatically marked as inactive because it has not had any activity in the last 30 days.

stale · 2023-04-07T01:09:24Z

This issue has been automatically marked as inactive because it has not had any activity in the last 30 days.

adepretis · 2024-04-23T09:33:07Z

I am hitting the same issue. A schema with about 2000 lines takes about 2-3 seconds to parse - for every request. Without some kind of cache this is not feasible :-(

stale · 2024-05-24T17:51:27Z

This issue has been automatically marked as inactive because it has not had any activity in the last 30 days. Although currently inactive, the issue could still be considered and actively worked on in the future. More details about the use-case this issue attempts to address, the value provided by completing it or possible solutions to resolve it would help to prioritize the issue.

robmyersrobmyers · 2025-03-13T15:23:06Z

In case the workaround isn't obvious, you can pass the schema AST into OPA so it doesn't have to be parsed each time. Rego Playground Example.

anderseknert · 2025-03-13T16:39:23Z

That'll still parse the schema every time though, won't it? Which won't be a problem in your case, but apparently so when the schema is thousands of lines.

If anyone would want to have a go at fixing this, I did something very similar some time ago for another built-in function. Could likely be mostly copy-pasted #7081

robmyersrobmyers · 2025-03-13T19:03:37Z

Yep, but it helps a ton with the running into performance issues due how long it takes to generate the AST from a raw query aspect of the original issue. It takes about 30s for OPA to generate the AST for the schema I'm testing with. Thanks for the link to your previous work; I'll take a look.

robmyersrobmyers · 2025-03-17T19:03:10Z

I've got some initial results and wanted to see if I am on the right track before reworking the other functions in a similar fashion.

Are there docs or best practices for putting things on the InterQueryValueCache?
- For now I'm passing string inputs to builtinCryptoSha256() to compute a unique key
  - This results in the object representation and the string representation having different keys, but both can be cached
- I could potentially use the text itself as a key, but it didn't seem like a great idea to put extra MBs in the cache
- I am aware of the pkg.go.dev docs

goarch: amd64
pkg: github.com/open-policy-agent/opa/v1/topdown
cpu: Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
BenchmarkGraphQLSchemaIsValid/Trivial_Schema_-_string-16                     	   16966	     68579 ns/op
BenchmarkGraphQLSchemaIsValid/Trivial_Schema_with_cache_-_string-16         	 1000000	      1062 ns/op
BenchmarkGraphQLSchemaIsValid/Trivial_Schema_-_object-16                    	   12900	     92484 ns/op
BenchmarkGraphQLSchemaIsValid/Trivial_Schema_with_cache_-_object-16         	   92516	     12064 ns/op
BenchmarkGraphQLSchemaIsValid/Complex_Schema_-_string-16                    	      60	  19294987 ns/op
BenchmarkGraphQLSchemaIsValid/Complex_Schema_with_cache_-_string-16         	     342	   3443940 ns/op
PASS
ok  	github.com/open-policy-agent/opa/v1/topdown	12.613s

For the complex benches I'm using the GitHub GraphQL Schema, which is about 1.2 M

…gent#5377) This commit stores parsed GraphQL schemas to the cache, which improves the performance of GraphQL operations that parse the schema more than once. Rough Draft Complete: - graphql.schema_is_valid TODO: - graphql.is_valid - graphql.parse - graphql.parse_and_verify - graphql.parse_query - graphql.parse_schema goarch: amd64 pkg: github.com/open-policy-agent/opa/v1/topdown cpu: Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz BenchmarkGraphQLSchemaIsValid/Trivial_Schema_-_string-16 16966 68579 ns/op BenchmarkGraphQLSchemaIsValid/Trivial_Schema_with_cache_-_string-16 1000000 1062 ns/op BenchmarkGraphQLSchemaIsValid/Trivial_Schema_-_object-16 12900 92484 ns/op BenchmarkGraphQLSchemaIsValid/Trivial_Schema_with_cache_-_object-16 92516 12064 ns/op BenchmarkGraphQLSchemaIsValid/Complex_Schema_-_string-16 60 19294987 ns/op BenchmarkGraphQLSchemaIsValid/Complex_Schema_with_cache_-_string-16 342 3443940 ns/op PASS ok github.com/open-policy-agent/opa/v1/topdown 12.613s Fixes: open-policy-agent#5377

anderseknert · 2025-03-17T20:04:55Z

Very nice @robmyersrobmyers 👍 No docs or best practices I'm aware of, but hashing the input seems like a good idea. The cache limit can be configured, but I'm not sure if it accounts for the key size (I'd have to check).. so keeping that small at the cost of hashing seems like a reasonable trade-off to me.

…gent#5377) This commit stores parsed GraphQL schemas to the cache, which improves the performance of GraphQL operations that parse the schema more than once. Rough Draft Complete: - graphql.schema_is_valid TODO: - graphql.is_valid - graphql.parse - graphql.parse_and_verify - graphql.parse_query - graphql.parse_schema goarch: amd64 pkg: github.com/open-policy-agent/opa/v1/topdown cpu: Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz BenchmarkGraphQLSchemaIsValid/Trivial_Schema_-_string-16 16966 68579 ns/op BenchmarkGraphQLSchemaIsValid/Trivial_Schema_with_cache_-_string-16 1000000 1062 ns/op BenchmarkGraphQLSchemaIsValid/Trivial_Schema_-_object-16 12900 92484 ns/op BenchmarkGraphQLSchemaIsValid/Trivial_Schema_with_cache_-_object-16 92516 12064 ns/op BenchmarkGraphQLSchemaIsValid/Complex_Schema_-_string-16 60 19294987 ns/op BenchmarkGraphQLSchemaIsValid/Complex_Schema_with_cache_-_string-16 342 3443940 ns/op PASS ok github.com/open-policy-agent/opa/v1/topdown 12.613s Fixes: open-policy-agent#5377

robmyersrobmyers · 2025-03-20T02:33:31Z

Status

I am still working on this. I've got tests for all the graphql builtinss and benches for most of them, but I've still only added caching on one of the builtins.

Questions

While testing with a complex schema I've run into an issue where it appears that ast.InterfaceToValue() is called from builtinGraphQLParseSchema() and allocates a ton of memory and never returns. I've made a reproducer here. Perhaps this representation of the schema causes a cycle in the tree somewhere? Perhaps I'm doing something incorrectly?
For now, I'm downloading and embedding large schemas for testing without adding them to the repo. I figure we don't need to carry around a lot of extra MBs or incur any licensing or copyright issues with 3rd party GQL schemas. Is there a preferred way to make it easy to benchmark with large schemas while not polluting the git repo?

Edit: Updated branch of reproducer to keep it out of the MR.

anderseknert · 2025-03-20T10:20:19Z

Great work!

Hmm, ast.InterfaceToValue never returning back doesn't sound good. There are no means in that to function to check for self-reference cycles, and I'm not sure how that would even be accomplished. But other than the map/slice handling, does it even deal with pointers? 🤔 I'd be very interested in knowing exactly where it gets stuck, and if we can reduce that to a minimal example. That never happens in the course of "normal" OPA handling of data.

EDIT: looks like it might be the default condition of that switch. I've gotta look into what data structure it's trying to convert there.

As for the tests, I think the ideal soultion would be to generate the schema to use for benchmarking in the benchmark. I don't know much about GraphQL schemas though ... perhaps that's not feasible?

robmyersrobmyers · 2025-03-20T15:50:01Z

I agree it would be best to autogenerate the gql schemas as part of the benchmark, but that's a bit more work than I have time for at the moment. :) Instead, I've removed the complex schema stuff to a different branch and I'll add the results to this comment for posterity. All tests and benches are working, so I will push up an MR in a bit.

pkg: github.com/open-policy-agent/opa/v1/topdown
cpu: Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
BenchmarkGraphQLSchemaIsValid/Trivial_Schema_-_string-16                           15519            100178 ns/op
BenchmarkGraphQLSchemaIsValid/Trivial_Schema_with_cache_-_string-16               371311              3383 ns/op
BenchmarkGraphQLSchemaIsValid/Trivial_Schema_-_object-16                            2133            542355 ns/op
BenchmarkGraphQLSchemaIsValid/Trivial_Schema_with_cache_-_object-16                 3471            528579 ns/op
BenchmarkGraphQLSchemaIsValid/Trivial_Schema_-_AST_object-16                        7105            193325 ns/op
BenchmarkGraphQLSchemaIsValid/Trivial_Schema_with_cache_-_AST_object-16            66594             18093 ns/op
BenchmarkGraphQLSchemaIsValid/Complex_Schema_-_string-16                              63          19117859 ns/op
BenchmarkGraphQLSchemaIsValid/Complex_Schema_with_cache_-_string-16                  328           3115964 ns/op
BenchmarkGraphQLSchemaIsValid/Complex_Schema_-_object-16                              16          68812732 ns/op
BenchmarkGraphQLSchemaIsValid/Complex_Schema_with_cache_-_object-16                   21          56968709 ns/op
BenchmarkGraphQLParseSchema/Trivial_Schema_-_string-16                              6429            173773 ns/op
BenchmarkGraphQLParseSchema/Trivial_Schema_with_cache_-_string-16                   6523            170819 ns/op
BenchmarkGraphQLParseQuery/Trivial_Query_-_string-16                               16352             72777 ns/op
BenchmarkGraphQLParseQuery/Trivial_Query_with_cache_-_string-16                    16083             73548 ns/op
BenchmarkGraphQLIsValid/Trivial_Schema_-_string-16                                 14320             83589 ns/op
BenchmarkGraphQLIsValid/Trivial_Schema_with_cache_-_string-16                      71486             15463 ns/op
BenchmarkGraphQLIsValid/Complex_Schema_-_string-16                                    60          19134990 ns/op
BenchmarkGraphQLIsValid/Complex_Schema_with_cache_-_string-16                        348           3162399 ns/op
BenchmarkGraphQLParse/Trivial_Schema_-_string-16                                    3380            321490 ns/op
BenchmarkGraphQLParse/Trivial_Schema_with_cache_-_string-16                        13909             87633 ns/op
BenchmarkGraphQLParseAndVerify/Trivial_Schema_-_string-16                           3435            327646 ns/op
BenchmarkGraphQLParseAndVerify/Trivial_Schema_with_cache_-_string-16               13844             85213 ns/op
PASS
ok      github.com/open-policy-agent/opa/v1/topdown     112.465s

…gent#5377) This commit stores parsed GraphQL schemas to the cache, which improves the performance of GraphQL operations that parse the schema more than once. Queries are not cached. pkg: github.com/open-policy-agent/opa/v1/topdown cpu: Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz BenchmarkGraphQLSchemaIsValid/Trivial_Schema_-_string-16 15519 100178 ns/op BenchmarkGraphQLSchemaIsValid/Trivial_Schema_with_cache_-_string-16 371311 3383 ns/op BenchmarkGraphQLSchemaIsValid/Trivial_Schema_-_object-16 2133 542355 ns/op BenchmarkGraphQLSchemaIsValid/Trivial_Schema_with_cache_-_object-16 3471 528579 ns/op BenchmarkGraphQLSchemaIsValid/Trivial_Schema_-_AST_object-16 7105 193325 ns/op BenchmarkGraphQLSchemaIsValid/Trivial_Schema_with_cache_-_AST_object-16 66594 18093 ns/op BenchmarkGraphQLParseSchema/Trivial_Schema_-_string-16 6429 173773 ns/op BenchmarkGraphQLParseSchema/Trivial_Schema_with_cache_-_string-16 6523 170819 ns/op BenchmarkGraphQLParseQuery/Trivial_Query_-_string-16 16352 72777 ns/op BenchmarkGraphQLParseQuery/Trivial_Query_with_cache_-_string-16 16083 73548 ns/op BenchmarkGraphQLIsValid/Trivial_Schema_-_string-16 14320 83589 ns/op BenchmarkGraphQLIsValid/Trivial_Schema_with_cache_-_string-16 71486 15463 ns/op BenchmarkGraphQLParse/Trivial_Schema_-_string-16 3380 321490 ns/op BenchmarkGraphQLParse/Trivial_Schema_with_cache_-_string-16 13909 87633 ns/op BenchmarkGraphQLParseAndVerify/Trivial_Schema_-_string-16 3435 327646 ns/op BenchmarkGraphQLParseAndVerify/Trivial_Schema_with_cache_-_string-16 13844 85213 ns/op PASS ok github.com/open-policy-agent/opa/v1/topdown 112.465s Resolves: open-policy-agent#5377

…gent#5377) This commit stores parsed GraphQL schemas to the cache, which improves the performance of GraphQL operations that parse the schema more than once. Queries are not cached. pkg: github.com/open-policy-agent/opa/v1/topdown cpu: Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz BenchmarkGraphQLSchemaIsValid/Trivial_Schema_-_string-16 15519 100178 ns/op BenchmarkGraphQLSchemaIsValid/Trivial_Schema_with_cache_-_string-16 371311 3383 ns/op BenchmarkGraphQLSchemaIsValid/Trivial_Schema_-_object-16 2133 542355 ns/op BenchmarkGraphQLSchemaIsValid/Trivial_Schema_with_cache_-_object-16 3471 528579 ns/op BenchmarkGraphQLSchemaIsValid/Trivial_Schema_-_AST_object-16 7105 193325 ns/op BenchmarkGraphQLSchemaIsValid/Trivial_Schema_with_cache_-_AST_object-16 66594 18093 ns/op BenchmarkGraphQLParseSchema/Trivial_Schema_-_string-16 6429 173773 ns/op BenchmarkGraphQLParseSchema/Trivial_Schema_with_cache_-_string-16 6523 170819 ns/op BenchmarkGraphQLParseQuery/Trivial_Query_-_string-16 16352 72777 ns/op BenchmarkGraphQLParseQuery/Trivial_Query_with_cache_-_string-16 16083 73548 ns/op BenchmarkGraphQLIsValid/Trivial_Schema_-_string-16 14320 83589 ns/op BenchmarkGraphQLIsValid/Trivial_Schema_with_cache_-_string-16 71486 15463 ns/op BenchmarkGraphQLParse/Trivial_Schema_-_string-16 3380 321490 ns/op BenchmarkGraphQLParse/Trivial_Schema_with_cache_-_string-16 13909 87633 ns/op BenchmarkGraphQLParseAndVerify/Trivial_Schema_-_string-16 3435 327646 ns/op BenchmarkGraphQLParseAndVerify/Trivial_Schema_with_cache_-_string-16 13844 85213 ns/op PASS ok github.com/open-policy-agent/opa/v1/topdown 112.465s Resolves: open-policy-agent#5377 Signed-off-by: Rob Myers <[email protected]>

anderseknert · 2025-03-20T15:55:31Z

Excellent! Yeah if I wasn't clear before, I did in no way mean to imply that you should fix that self-referential infinite loop issue as part of your work :) That's a bug we'll need to look into separately. Looking forward to your PR — the numbers look incredible!

…gent#5377) This commit stores parsed GraphQL schemas to the cache, which improves the performance of GraphQL operations that parse the schema more than once. Queries are not cached. pkg: github.com/open-policy-agent/opa/v1/topdown cpu: Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz BenchmarkGraphQLSchemaIsValid/Trivial_Schema_-_string-16 15519 100178 ns/op BenchmarkGraphQLSchemaIsValid/Trivial_Schema_with_cache_-_string-16 371311 3383 ns/op BenchmarkGraphQLSchemaIsValid/Trivial_Schema_-_object-16 2133 542355 ns/op BenchmarkGraphQLSchemaIsValid/Trivial_Schema_with_cache_-_object-16 3471 528579 ns/op BenchmarkGraphQLSchemaIsValid/Trivial_Schema_-_AST_object-16 7105 193325 ns/op BenchmarkGraphQLSchemaIsValid/Trivial_Schema_with_cache_-_AST_object-16 66594 18093 ns/op BenchmarkGraphQLParseSchema/Trivial_Schema_-_string-16 6429 173773 ns/op BenchmarkGraphQLParseSchema/Trivial_Schema_with_cache_-_string-16 6523 170819 ns/op BenchmarkGraphQLParseQuery/Trivial_Query_-_string-16 16352 72777 ns/op BenchmarkGraphQLParseQuery/Trivial_Query_with_cache_-_string-16 16083 73548 ns/op BenchmarkGraphQLIsValid/Trivial_Schema_-_string-16 14320 83589 ns/op BenchmarkGraphQLIsValid/Trivial_Schema_with_cache_-_string-16 71486 15463 ns/op BenchmarkGraphQLParse/Trivial_Schema_-_string-16 3380 321490 ns/op BenchmarkGraphQLParse/Trivial_Schema_with_cache_-_string-16 13909 87633 ns/op BenchmarkGraphQLParseAndVerify/Trivial_Schema_-_string-16 3435 327646 ns/op BenchmarkGraphQLParseAndVerify/Trivial_Schema_with_cache_-_string-16 13844 85213 ns/op PASS ok github.com/open-policy-agent/opa/v1/topdown 112.465s Resolves: open-policy-agent#5377 Signed-off-by: Rob Myers <[email protected]>

…gent#5377) This commit stores parsed GraphQL schemas to the cache, which improves the performance of GraphQL operations that parse the schema more than once. Queries are not cached. pkg: github.com/open-policy-agent/opa/v1/topdown cpu: Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz BenchmarkGraphQLSchemaIsValid/Trivial_Schema_-_string-16 15519 100178 ns/op BenchmarkGraphQLSchemaIsValid/Trivial_Schema_with_cache_-_string-16 371311 3383 ns/op BenchmarkGraphQLSchemaIsValid/Trivial_Schema_with_cache_-_AST_object-16 66594 18093 ns/op BenchmarkGraphQLParseSchema/Trivial_Schema_-_string-16 6429 173773 ns/op BenchmarkGraphQLParseSchema/Trivial_Schema_with_cache_-_string-16 6523 170819 ns/op BenchmarkGraphQLParseQuery/Trivial_Query_-_string-16 16352 72777 ns/op BenchmarkGraphQLParseQuery/Trivial_Query_with_cache_-_string-16 16083 73548 ns/op BenchmarkGraphQLIsValid/Trivial_Schema_-_string-16 14320 83589 ns/op BenchmarkGraphQLIsValid/Trivial_Schema_with_cache_-_string-16 71486 15463 ns/op BenchmarkGraphQLParse/Trivial_Schema_-_string-16 3380 321490 ns/op BenchmarkGraphQLParse/Trivial_Schema_with_cache_-_string-16 13909 87633 ns/op BenchmarkGraphQLParseAndVerify/Trivial_Schema_-_string-16 3435 327646 ns/op BenchmarkGraphQLParseAndVerify/Trivial_Schema_with_cache_-_string-16 13844 85213 ns/op PASS ok github.com/open-policy-agent/opa/v1/topdown 112.465s Resolves: open-policy-agent#5377 Signed-off-by: Rob Myers <[email protected]>

edulop91 added the bug label Nov 10, 2022

srenatus mentioned this issue Nov 28, 2022

Add built-in funtion to support JSON Schema Validation #1449

Closed

stale bot added the inactive label Dec 16, 2022

ashutosh-narkar added performance builtins labels Mar 8, 2023

stale bot removed the inactive label Mar 8, 2023

stale bot added the inactive label Apr 7, 2023

stale bot removed the inactive label Apr 23, 2024

stale bot added the inactive label May 24, 2024

stale bot removed the inactive label Mar 13, 2025

robmyersrobmyers linked a pull request Mar 20, 2025 that will close this issue

v1/topdown/graphql: Cache GraphQL schema parse results (#5377) #7457

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Possible to cache graphql.parse() results? #5377

Possible to cache graphql.parse() results? #5377

edulop91 commented Nov 10, 2022

srenatus commented Nov 10, 2022

philipaconrad commented Nov 11, 2022

srenatus commented Nov 16, 2022

stale bot commented Dec 16, 2022

stale bot commented Apr 7, 2023

adepretis commented Apr 23, 2024

stale bot commented May 24, 2024

robmyersrobmyers commented Mar 13, 2025 •

edited

Loading

anderseknert commented Mar 13, 2025

robmyersrobmyers commented Mar 13, 2025

robmyersrobmyers commented Mar 17, 2025

anderseknert commented Mar 17, 2025

robmyersrobmyers commented Mar 20, 2025 •

edited

Loading

anderseknert commented Mar 20, 2025 •

edited

Loading

robmyersrobmyers commented Mar 20, 2025

anderseknert commented Mar 20, 2025

Possible to cache graphql.parse() results? #5377

Possible to cache graphql.parse() results? #5377

Comments

edulop91 commented Nov 10, 2022

Short description

Steps To Reproduce

Expected behavior

Additional context

srenatus commented Nov 10, 2022

philipaconrad commented Nov 11, 2022

srenatus commented Nov 16, 2022

stale bot commented Dec 16, 2022

stale bot commented Apr 7, 2023

adepretis commented Apr 23, 2024

stale bot commented May 24, 2024

robmyersrobmyers commented Mar 13, 2025 • edited Loading

anderseknert commented Mar 13, 2025

robmyersrobmyers commented Mar 13, 2025

robmyersrobmyers commented Mar 17, 2025

anderseknert commented Mar 17, 2025

robmyersrobmyers commented Mar 20, 2025 • edited Loading

Status

Questions

anderseknert commented Mar 20, 2025 • edited Loading

robmyersrobmyers commented Mar 20, 2025

anderseknert commented Mar 20, 2025

robmyersrobmyers commented Mar 13, 2025 •

edited

Loading

robmyersrobmyers commented Mar 20, 2025 •

edited

Loading

anderseknert commented Mar 20, 2025 •

edited

Loading