Skip to content

⭐ [Enhancement]: Add Level 2 Caching #2543

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
12 tasks
JerryNixon opened this issue Jan 29, 2025 · 5 comments
Open
12 tasks

⭐ [Enhancement]: Add Level 2 Caching #2543

JerryNixon opened this issue Jan 29, 2025 · 5 comments
Assignees
Labels
enhancement New feature or request
Milestone

Comments

@JerryNixon
Copy link
Contributor

JerryNixon commented Jan 29, 2025

Configuration Change

Original

Runtime

{
  "runtime": {
    "cache": {
      "enabled": <true> | <false> (default),
      "ttl-seconds": <integer; default: 5>
    }
  }
}

Entity

{
  "entities": {
    "<entity-name>": {
      "cache": {
        "enabled": <true> (default) | <false>,
        "ttl-seconds": <integer; default: 5>
      }
    }
  }
}

Proposed

Runtime

{
  "runtime": {
    "cache": {
      "enabled": <true> | <false> (default),
      "ttl-seconds": <integer; default: 5>,
      "level-2": { // new section
        "enabled": <true> (default) | <false>, 
        "provider": "redis" (default), // only supporting redis for now
        "connection-string": <string>,
        "ttl-seconds": <integer; default: 60>,
        "backplane": {  
          "enabled": <true> (default) | <false>,
          "channel": <string; default: "dab-cache-channel">
        }
      }
    }
  }
}

Summary

Property Type Default Required Description
enabled boolean true No Enables level 2 caching.
provider string "redis" No Cache provider, only Redis is supported.
connection-string string N/A Yes Connection string for the Redis cache.
ttl-seconds integer 60 No Time-to-live (TTL) for level 2 cache entries.

Entity

{
  "entities": {
    "<entity-name>": {
      "cache": {
        "enabled": <true> (default) | <false>,
        "ttl-seconds": <integer; default: 5>,
        "level": <one (default) | two> // new enum
      }
    }
  }
}

Summary

Property Type Default Required Description
level enum (one, two) one No Chooses cache level.

Ancillary requirements

  • Config/Runtime updated
  • Config/Entity updated
  • CLI support: dab configure --runtime.cache.level2.enabled
  • CLI support: dab configure --runtime.cache.level2.provider
  • CLI support: dab configure --runtime.cache.level2.connection-string
  • CLI support: dab configure --runtime.cache.level2.ttl-seconds
  • CLI support: dab validate (rules)
  • Docs updated CLI
  • Docs updated Config/Runtime
  • Docs updated Config/Entities
  • Hot Reload updated
  • .NET Aspire Sample
@jodydonetti
Copy link
Collaborator

jodydonetti commented Jan 29, 2025

Hi team, FusionCache creator here: awesome feature!
The feature description is quite detailed 👍
I'm definitely interested as you can imagine, so I'll try to put together something and will update you on it.

I have a couple of observations and questions right now, since I'm not yet familiar with the codebase yet and don't know 100% of the product roadmap or supported scenarios.

Runtime config level-2/ttl-seconds

It says integer, but also not required. Does this means that if not specified it will use the base ttl-seconds?

(btw sounds good)

Entity config level

Can have a value of one or two, I suppose they mean one = "use only L1" and two means "use both" right? I'm asking because there are some FusionCache users that want to skip L1 and use only L2, but that... is frequently not the best idea.

Backplane

I see the config is explicit about a "level 2", but I'm not sure if that automatically contemplates the use of a Backplane or not.

On one hand, without it there will be out-of-sync issues when in a multi-node environment (apropo, is it possible/supported? Asking as a DAB non-expert) and people may complain.

On the other hand, if people will use it only as an out-of-process level to ease cold starts, a Backplane would not be required (since there are no other nodes to notifiy).

Finally, if single-node is the supported scenario here, there's an alternative to consider, meaning this.

Careful: L2 is "kinda" like a database

What I mean here is that up until today, the cache was only L1: this means that at every change to the code/schemas the app would be restarted, and that will basically clean the cache and start from scratch.

Introducing an L2 means that as soon as something is saved in L2 it will stay there even after a restart: this means that if the schema or something similar is changed between restarts (or between multiple instances on different nodes) there may be problems when getting data from L2 (in particular during the deserialization phase).
There are ways around this, but it's something that must be know upfront and must be designed for.

If you let me know what are the supported scenarios I can give you some ideas for this.

Thanks for the great product!

@JerryNixon
Copy link
Contributor Author

JerryNixon commented Jan 29, 2025

  1. "level-2":"ttl-seconds": <integer; default: 60> When omitted, then default is 60. What I did not indicate in the design was the min and max values we would support. I will let you suggest those.
  2. "entity": { "cache": "level": Let's change the enum to L1Only & L1ThenL2. Let L1ThenL2 be the default? I think this captures your concern and certainly clarifies the behavior to the developer setting it up. Please advise if L2Only makes any sense?
  3. We assume a backplane, yes. Is this overkill?
{
  "runtime": {
    "cache": {
      ...
      "level-2": {
        ...
        "backplane": {  // New Section
          "enabled": <true> (default) | <false>,
          "channel": <string; default: "dab-cache-channel">
        }
      }
    }
  }
}
  1. I am open to ideas here, for sure. It is worth remembering that the engine has the schema available to it and could validate? Though that constant interrogation could interfere with perf. So, open to ideas.

Stole this from a recent tweet. Re: level

Image

@aaronburtle aaronburtle self-assigned this Feb 12, 2025
@aaronburtle aaronburtle added the enhancement New feature or request label Feb 12, 2025
@aaronburtle aaronburtle added this to the 1.5 milestone Feb 12, 2025
@jodydonetti
Copy link
Collaborator

Hi all, I'm making progress on this, and will soon make a draft PR we can discuss about.

One thing I was wondering (thanks to a recent issue on the FusionCache repo) is if we need to support connecting to an Azure Redis instance via a Service Principal (see here).

On one hand the mantra is "let's start with the basics", on the other hand it's Azure Redis, so I'm asking. Maybe it's not even something so common, I don't have usage stats.

Will update soon on the rest.

@jodydonetti
Copy link
Collaborator

Draft PR created here.

@jodydonetti
Copy link
Collaborator

jodydonetti commented Mar 30, 2025

🔔 UPDATE 2025-03-30

(Updated 2025-04-03 after our chat)

Hi all, after our recent talk and while getting to the bottom of this I had some time to reflect on the overall experience for the user, so I added a couple of open points here.

Meanwhile I converted the PR back to draft for now.

L2 Enabled By Default

I noticed that in the proposal level-2 > enabled has a default of true: this is probably wrong, because by being enabled by default and by (obviously) not having a default connection string, any user that enables the normal caching will probably receive an error (still to decide what to do in that case, see my comment in the PR code: log? exception?).

Therefore, the default should be false imho.

The user can enable caching via runtime > cache > enabled and be good with it. Want also L2/backplane? Simply set runtime > cache > level-2 with something minimal like:

"level-2": {
  "enabled": true,
  "connection-string": "..."
}

Note

STATUS: confirmed

  • L2 is disabled by default
  • throw an exception when L2 is enabled without a connection string
  • mark distinguisher (now "partition", see below) as optional, default to null

Backplane

Since the best user experience would be to have everything "just works", we also established (correct me if I'm wrong) that the backplane will not have any specific configuration for now: since "redis" is the only supported provider, by enabling L2 caching we'll also automatically have the backplane functionality too, so everything "just works" (good devex).

Regarding the channel name, I've found a solution that I think it's simpler and works well: see "distinguisher" below.

Note

STATUS: confirmed

  • enabled by default, if L2 is enabled (for the only current provider, "redis")

Distinguisher Partition

Another thing is the "prefix" I touched upon earlier: since without it both the L2 cache and the backplane will have collisions when multiple apps run against the same Redis instance (which is very frequent), I added something for it called "distinguisher".

I haven't used the term "prefix" to keep it more conceptual and to avoid being too specific to the position in the cache key/backplane channel, but we can change it easily if you prefer. It is fully optional and, if valued, will be added to both the cache keys and the backplane channel name.

Note

STATUS: confirmed

  • change the name from "distinguisher" to "partition"

Careful: L2 is "kinda" like a database

The partition can also be used by the user, if they want, to solve the schema evolution problem I touched upon above.

As an example they can simply specify "partition": "myapp" and be good with it, then after a breaking change to the schema they can simply change it to "partition": "myapp-v2" and so on. Problem solved.

NOTE: to reiterate, this is not something specific to FusionCache, but to any distributed cache in general.

Note

STATUS: confirmed, maybe more down the road

  • ok to allow users to use the "partition" to avoid schema evolution problems (but not strictly mandatory, see below)
  • set right now ReThrowDistributedCacheExceptions to false to ease schema evolution
  • in the future, explore the possibility to automatically generate something (eg: a hash) based on a reductio ad minimum of the schema of the entities, so that any change in the entities' schema will translate to an automatic no-conflict situation

The "level" option

In the following proposal I updated this based on the inputs received, but what is in your mind the rationale to skip L2 for some entities? I'm asking just to be sure, since I have a couple of cases in mind but would like to cover the ones you are thinking about.

Note

STATUS: confirmed

  • as a note for blogpost/docs, this is useful for 2 key scenarios: to avoid large entities getting serialized/deserialized/into L2 (network consumption etc) + avoid storing sensitive data in L2 but only in L1 (memory, wiped at every restart)

On Different TTLs

Finally, about multiple TTLs for both:

  • the 2 levels
  • different entities

My first question is: considering we have the backplane, and so any update is instantly propagated to the other nodes, why a different TTL for L1 (5s) and L2 (60s)?

Also, if we keep L1/L2 specific TTLs at a global level but only one TTL per-entity, what should happen when changing the entity one? Should it change only the one for L1? This may create strange situations when setting an entity to, say, 10m with the underlying L2 TTL still at 60s. And if we change both at the entity level, why having 2 different ones for L1 and L2 at a global level?

In my experience, setting a different L2 TTL is mostly used when not having a backplane (but we have it), and when a user wants to change the TTL for an certain entity, it usually wants the same for both L1 and L2.

Note

STATUS: confirmed

  • for now, remove runtime > cache > level-2 > ttl-seconds since with the automatic backplane updates on multi-nodes will be instantaneous, and what are left are a very small set of nice scenarios we can always add in a future version

Updated Proposal

With all of this, the updated proposals are:

Runtime

{
  "runtime": {
    "cache": {
      "enabled": <true> | <false> (default),
      "ttl-seconds": <integer; default: 5>,
      "level-2": { // new section
        "enabled": <true> | <false> (default),
        "provider": "redis" (default), // only supporting redis for now
        "connection-string": <string>,
        "partition": <string> | null (default)
      }
    }
  }
}

Entity:

{
  "entities": {
    "<entity-name>": {
      "cache": {
        "enabled": <true> (default) | <false>,
        "ttl-seconds": <integer; default: 5>,
        "level": <L1 | L1L2 (default)> // new enum
      }
    }
  }
}

PR already updated with everything we discussed.

jodydonetti added a commit to jodydonetti/data-api-builder that referenced this issue Apr 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants