Add Level 2 Caching #2619

jodydonetti · 2025-03-14T22:40:38Z

Why make this change?

Work for #2543 (Add Level 2 Caching).

What is this change?

First code pushed on this branch.

Summary of the changes:

updated package reference for FusionCache from 1.0.0 to 2.1.0
added package references to ZiggyCreatures.FusionCache.Serialization.SystemTextJson, Microsoft.Extensions.Caching.StackExchangeRedis and ZiggyCreatures.FusionCache.Backplane.StackExchangeRedis to support L2+backplane on Redis
added support for L2+backplane in the initial config (Program.cs)
added RuntimeCacheOptions on top of EntityCacheOptions: EntityCacheOptions was used to, I suppose, avoid having 2 similar classes since they had the same shape, but now that is not true anymore
added initial version of RuntimeCacheLevel2Options to contain the related config values
aligned some tests that broke because they were directly instantiating and passing EntityCacheOptions instead of the new RuntimeCacheOptions
added new support classes (eg: RuntimeCacheOptionsConverterFactory)
added CACHE_ENTRY_TOO_LARGE message instead of the previously used CACHE_KEY_TOO_LARGE one, which was used when the entry was too large (not the key)
changed the generic T in DabCacheService.GetOrSetAsync, from JsonElement (same name but a totally different thing than JsonElement class) to TResult (which was already used in the other GetOrSetAsync overload, to avoid confusion
fixed some typos (eg: "ommitted" instead of "omitted" etc)
added some temporary // TODO comments to get back to them later

Still missing:

although I already worked on the config generation side of things (eg: not just reading the config, but also generating it via the CLI), I think I haven't finished it yet
there are also some points where I'm not sure the convention I extrapolated is fully correct (eg: a config sub-object should be generated only if it's not null or should we do a comparison with the inner default values to see if we should skip it as a whole?)
since DAB is using FusionCache, it's also using Auto-Recovery: becaus of this, I'd like to tweak some of the default settings (eg: allow background distributed operations to make things faster, etc). But we'd have to first check if we want to expose some options to drive this or we go full-auto-mode

Not (necessarily) part of this effort, but already mentioned and to keep in mind:

avoid calculating each cache entry size, since SizeLimit is not currently used, nor it is possible to use it via some config (so, for now, it's useless)
start thinking about enabling some resiliency features like Fail-Safe, Soft Timeouts, etc
in the future it may be nice to introduce Tagging to automatically evict all the cached entries containing data about an entity after an update to said entity. It would be great

How was this tested?

Tests run locally:

Unit Tests

Sample Request(s)

N/A

jodydonetti · 2025-03-14T22:43:54Z

@microsoft-github-policy-service agree

jodydonetti · 2025-03-14T22:50:46Z

I'll add that both L2 and the backplane are working, but I'd like to see the backplane in action on a real multi-instance DAB test: any advice on the best way to do this?

Also, I have to add the specific config part for the backplane, but I'm still thinking if it's really necessary or not: the "it just works" way can be to auto-use the backplane if the underlying L2 provider supports it (in our case: "redis"). But I'm pondering between ease of use of the default experience and total control...

Aniruddh25 · 2025-03-17T00:09:35Z

/azp run

azure-pipelines · 2025-03-17T00:09:57Z

Azure Pipelines successfully started running 6 pipeline(s).

jodydonetti · 2025-03-18T21:36:14Z

/azp run

azure-pipelines · 2025-03-18T21:36:20Z

Commenter does not have sufficient privileges for PR 2619 in repo Azure/data-api-builder

This reverts commit db2b271.

jodydonetti · 2025-03-18T22:16:41Z

After a couple of minor commits I think this is ready for a review.

Tests

If someone with the needed privileges (maybe @Aniruddh25 ?) can do an /azp run it should now pass all the tests.

Backplane Experience: It Just Works

About the backplane: my opinion is that in the future we may add the ability to specifcy a different provider for the backplane than for L2 (maybe with different options, like connection string, etc), but the default experience should be as much as possible an "it just works" way, based on the specific provider.

The only supported provider currently, Redis, has both abilities (L2 and backplane): because of this, I think we can be good like this for the first release, without any additional explicit option (and btw, I'm already sharing the same IConnectionMultiplexer instance between L2 and backplane, to use less connections).

Some Notes

A couple of additional notes:

although I already worked on the config generation side of things (eg: not just reading the config, but also generating it via the CLI), I'm not 100% sure it is already completely correct. Are there best practices for this?
there are also some points where I'm not sure the convention I extrapolated is fully correct (eg: a config sub-object should be generated only if it's not null or should we do a comparison with the inner default values to see if we should skip it as a whole?). Are there hints on this?

Pending

Still pending (but not related to this PR):

we should discuss about not calculating each cache entry size, since SizeLimit is not currently used, nor it is possible to use it via some config (so, for now, it's useless). Any opinion?

Please let me know, thanks!

Aniruddh25 · 2025-03-19T21:22:12Z

/azp run

azure-pipelines · 2025-03-19T21:22:37Z

Azure Pipelines successfully started running 6 pipeline(s).

jodydonetti · 2025-03-19T21:33:38Z

PS: although with this PR there would be full support for L2+backplane, as mentioned elsewhere it would be good to have some sort of "app name" or "app id", to use as a prefix or similar to support multiple DAB apps running over the same Redis instance. If it's complicated to come with such a new global config/concept, an alternative would be to have a new config in the top-level cache config node, something like "prefix" here:

{
	"$schema": "https://github.com/Azure/data-api-builder/releases/download/v0.10.23/dab.draft.schema.json",
	"data-source": {
		// ...
	},
	"runtime": {
		"rest": {
			// ...
		},
		"graphql": {
			// ...
		},
		"cache": {
			"enabled": true,
			"ttl-seconds": 10,
			"prefix": "blahblah", // HERE
			"level-2": {
				// ...
			}
		},
		// ...
	}
}

This would be cache-specific, and quite easy to add.

After the merge of this first PR (if all goes well) I can add this pretty fast.

aaronburtle · 2025-03-20T15:54:32Z

src/Config/ObjectModel/EntityCacheOptions.cs

@@ -33,6 +33,7 @@ public record EntityCacheOptions
    [JsonConstructor]
    public EntityCacheOptions(bool? Enabled = null, int? TtlSeconds = null)


Are we still using EntityCacheOptions as a part of the request flow, seems its been usurped by RuntimeCacheOptions, do we still need this class?

To me yes because the shapes of the 2 configs are different.
For example in the runtime one there's the provider, which is not in the entity one.
My 2 cents.

Ya, makes sense, these 2 classes share a lot of code though, so maybe we can split the common functionality out to a base record that they both inherit?

aaronburtle · 2025-03-20T15:56:53Z

src/Config/Converters/RuntimeCacheOptionsConverterFactory.cs

+    private class RuntimeCacheOptionsConverter : JsonConverter<RuntimeCacheOptions>
+    {
+        /// <summary>
+        /// Defines how DAB reads an entity's cache options and defines which values are


reads an entity's cache options or reads the runtime cache options? maybe a copy paste error.

Yeah sorry, copy/paste error in the comments.

aaronburtle · 2025-03-20T15:57:37Z

src/Config/Converters/RuntimeCacheOptionsConverterFactory.cs

+        }
+
+        /// <summary>
+        /// When writing the EntityCacheOptions back to a JSON file, only write the ttl-seconds


Similar as above, this should be RuntimeCacheOptions right?

aaronburtle · 2025-03-20T17:43:01Z

src/Core/Services/Cache/DabCacheService.cs

@@ -27,8 +27,9 @@ public class DabCacheService

    // Log Messages
    private const string CACHE_KEY_EMPTY = "The cache key should not be empty.";
-    private const string CACHE_KEY_TOO_LARGE = "The cache key is too large.";
+    //private const string CACHE_KEY_TOO_LARGE = "The cache key is too large.";


nit: remove commented code

Agree, I left there for now because I was unsure if there was also a need for a "key too large" check (currently missing).
Can remove it if that is not the case.

aaronburtle · 2025-03-24T18:33:28Z

/azp run

azure-pipelines · 2025-03-24T18:33:49Z

Azure Pipelines successfully started running 6 pipeline(s).

Add L2+backplane support + tests + fix typos

9ecbd3e

jodydonetti changed the title ~~Add L2+backplane support + tests + fix typos~~ Add Level 2 Caching Mar 14, 2025

jodydonetti mentioned this pull request Mar 14, 2025

⭐ [Enhancement]: Add Level 2 Caching #2543

Open

12 tasks

Merge branch 'main' into dev/jodydonetti/add-l2-cache

dad001b

jodydonetti added 2 commits March 18, 2025 22:34

Fix tests

e6d481d

Merge branch 'main' into dev/jodydonetti/add-l2-cache

c3d1936

jodydonetti added 3 commits March 18, 2025 22:37

Fix after merge

da6aab4

Newlines normalization (per VS suggestion)

db2b271

Revert "Newlines normalization (per VS suggestion)"

fed0617

This reverts commit db2b271.

jodydonetti marked this pull request as ready for review March 18, 2025 22:16

jodydonetti requested review from Aniruddh25, aaronburtle, RubenCerna2079, ravishetye, rohkhann, neeraj-sharma2592, sourabh1007 and sezal98 as code owners March 18, 2025 22:16

aaronburtle reviewed Mar 20, 2025

View reviewed changes

simonsabin mentioned this pull request Mar 20, 2025

Make parameter length based on underlying column length except for complex ops #2627

Open

1 task

Merge branch 'main' into dev/jodydonetti/add-l2-cache

eb31a8d

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Level 2 Caching #2619

Add Level 2 Caching #2619

jodydonetti commented Mar 14, 2025 •

edited

Loading

jodydonetti commented Mar 14, 2025

jodydonetti commented Mar 14, 2025

Aniruddh25 commented Mar 17, 2025

azure-pipelines bot commented Mar 17, 2025

jodydonetti commented Mar 18, 2025

azure-pipelines bot commented Mar 18, 2025

jodydonetti commented Mar 18, 2025 •

edited

Loading

Aniruddh25 commented Mar 19, 2025

azure-pipelines bot commented Mar 19, 2025

jodydonetti commented Mar 19, 2025

aaronburtle Mar 20, 2025

jodydonetti Mar 20, 2025

aaronburtle Mar 24, 2025

aaronburtle Mar 20, 2025

jodydonetti Mar 20, 2025

aaronburtle Mar 20, 2025

jodydonetti Mar 20, 2025

aaronburtle Mar 20, 2025

jodydonetti Mar 20, 2025

aaronburtle commented Mar 24, 2025

azure-pipelines bot commented Mar 24, 2025

		@@ -33,6 +33,7 @@ public record EntityCacheOptions
		[JsonConstructor]
		public EntityCacheOptions(bool? Enabled = null, int? TtlSeconds = null)

Add Level 2 Caching #2619

Are you sure you want to change the base?

Add Level 2 Caching #2619

Conversation

jodydonetti commented Mar 14, 2025 • edited Loading

Why make this change?

What is this change?

How was this tested?

Sample Request(s)

jodydonetti commented Mar 14, 2025

jodydonetti commented Mar 14, 2025

Aniruddh25 commented Mar 17, 2025

azure-pipelines bot commented Mar 17, 2025

jodydonetti commented Mar 18, 2025

azure-pipelines bot commented Mar 18, 2025

jodydonetti commented Mar 18, 2025 • edited Loading

Tests

Backplane Experience: It Just Works

Some Notes

Pending

Aniruddh25 commented Mar 19, 2025

azure-pipelines bot commented Mar 19, 2025

jodydonetti commented Mar 19, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aaronburtle commented Mar 24, 2025

azure-pipelines bot commented Mar 24, 2025

jodydonetti commented Mar 14, 2025 •

edited

Loading

jodydonetti commented Mar 18, 2025 •

edited

Loading