-
Notifications
You must be signed in to change notification settings - Fork 16
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
INTPYTHON-451 Add support for database caching #253
INTPYTHON-451 Add support for database caching #253
Conversation
c7cbf04
to
1ee2a04
Compare
5f356db
to
5be1867
Compare
django_mongodb_backend/cache.py
Outdated
|
||
class MongoDBCache(BaseDatabaseCache): | ||
def create_indexes(self): | ||
self.collection.create_index("expires_at", expireAfterSeconds=0) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is the expiry set to 0 here? I know we talked about doing the filtering afterwards, but this is basically saying "expire immediately" is it not?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If it's getting deleted almost instantly, but we also want to manually do the culling, wouldn't this just be better as a regular sorted index?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, it expires when the time touch the expires_at value, I got the idea from here:
https://www.mongodb.com/docs/manual/tutorial/expire-data/#expire-documents-at-a-specific-clock-time
We also do culling. look at the method _cull
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The idea is to handle different expiration times, so the set the expiration after 0 so we can chose whether we expires data
2fb2337
to
b2c664b
Compare
django_mongodb_backend/management/commands/createcachecollection.py
Outdated
Show resolved
Hide resolved
It's not just the cache tests in Django's test suite that may need the
table. If a Django project is using the Django test runner, what creates
the table?
… Message ID: <mongodb/django-mongodb-backend/pull/253/review/2646403770@
github.com>
|
Ah ok, understood. I thought it was only used by unit test. Given that, yes, I should add it after the super |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You might compare the implementation with https://github.com/django-nonrel/mongodb-cache/blob/develop/django_mongodb_cache/backend.py. One thing I noticed is that your code doesn't consult router.db_for_write()
.
Yeah, my bad. I created the method get_collection and forgot to handle the collection to write. I think I have to add the route test to the unit test. |
91d0e58
to
eae823d
Compare
210c542
to
361df71
Compare
361df71
to
39a468e
Compare
f859a29
to
9ea8dc6
Compare
9ea8dc6
to
7453c14
Compare
Co-authored-by: Tim Graham <[email protected]>
7453c14
to
64b1c10
Compare
Unlike Django's built-in database cache backend, this backend supports | ||
automatic culling of expired entries at the database level. | ||
|
||
In addition, the cache is culled based on ``CULL_FREQUENCY`` when ``add()`` | ||
or ``set()`` is called, if ``MAX_ENTRIES`` is exceeded. See | ||
:ref:`django:cache_arguments` for an explanation of these two options. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These two paragraphs replaced this one which was copied from the Django docs:
Unlike other cache backends, the database cache does not support automatic
culling of expired entries at the database level. Instead, expired cache
entries are culled each timeadd()
,set()
, ortouch()
is called.
The MongoDB cache uses two different indexes:
Unique for the key and TTL index for the expiration date.
The Mongo's demon that deletes data from the collection is called every 60 seconds (if I am not mistaken) we have to deal with virtually deleted records (just filter if the record is expired)
So an extra exclude or include expired query are necessary.
Some scenarios to take in account:
fixes #229