feat: SpilloverCache — memory-first cache with disk overflow

[oliverhaas]

Summary

Add a django_cachex.cache.SpilloverCache backend that uses in-memory storage as the primary tier and spills evicted entries to disk instead of discarding them.

Motivation

TieredCache (LocMem + Redis/File) writes to L2 on every set() because L2 is the source of truth. This means every write hits disk/network, defeating the purpose of an in-memory cache for local-only use cases.

SpilloverCache inverts this: memory is authoritative, disk is overflow storage. Disk I/O only happens on eviction (demotion) and L1 cache misses (promotion). This gives near-LocMem performance for hot keys while retaining cold data on disk instead of losing it.

Design

Operation	Behavior
set()	Write to L1 (memory) only
get() hit	Serve from L1
get() miss	Check L2 (disk), promote to L1 if found
L1 eviction (cull)	Demote to L2 instead of discarding
delete()	Delete from both L1 and L2
clear()	Clear both

Source of truth: L1 for hot data, L2 for cold data. No single authoritative tier — the union of both is the full dataset.

Key difference from TieredCache

	TieredCache	SpilloverCache
set()	write L2, then L1	write L1 only
L1 eviction	data still in L2 (by design)	must demote to L2
source of truth	L2	L1 ∪ L2
primary use case	LocMem + Redis (shared state)	LocMem + disk (single-process)

Implementation ideas

• Override LocMemCache's _cull() to demote evicted entries to L2 before removing from memory
• L2 could be any BaseCache backend but FileBasedCache or DatabaseCache (SQLite) are the natural choices
• Configuration via OPTIONS["OVERFLOW"] pointing to another CACHES alias (same pattern as TieredCache's TIERS)
• L1 size controlled by existing MAX_ENTRIES / CULL_FREQUENCY settings
• TTL handling: preserve original TTL when demoting to L2

Example configuration

CACHES = {
    "spillover_disk": {
        "BACKEND": "django_cachex.cache.FileBasedCache",  # or DatabaseCache
        "LOCATION": "/tmp/cache_overflow",
    },
    "default": {
        "BACKEND": "django_cachex.cache.SpilloverCache",
        "LOCATION": "spillover-locmem",
        "OPTIONS": {
            "MAX_ENTRIES": 1000,
            "OVERFLOW": "spillover_disk",
        },
    },
}

Open questions

• Should promotion from L2→L1 delete from L2 (save disk space) or leave it (avoid re-demotion if L1 is thrashing)?
• Should set() write-through to L2 as well for crash durability, maybe as an opt-in?
• Is there value in an LRU eviction order for L1 instead of Django's random cull?

Related

• feat: FileBasedCache drop-in replacement backend #50 — FileBasedCache drop-in replacement
• feat: explicit cachex backends for LocMem, Database, File caches #49 — explicit cachex backends for LocMem and Database caches