Quick start¶
Requirements¶
- Django 2.0-2.2, 3.0-3.1
- Python 3.5-3.9
- a cache configured as
'default'
with one of these backends:- django-redis
- memcached (using either python-memcached or pylibmc)
- filebased
- locmem (but it’s not shared between processes, see locmem limits)
- one of these databases:
- PostgreSQL
- SQLite
- MySQL (but on older versions like MySQL 5.5, django-cachalot has no effect, see MySQL limits)
Usage¶
pip install django-cachalot
- Add
'cachalot',
to yourINSTALLED_APPS
- If you use multiple servers with a common cache server, double check their clock synchronisation
- If you modify data outside Django – typically after restoring a SQL database –, use the manage.py command
- Be aware of the few other limits
- If you use
django-debug-toolbar,
you can add
'cachalot.panels.CachalotPanel',
to yourDEBUG_TOOLBAR_PANELS
- Enjoy!
Settings¶
CACHALOT_ENABLED
¶
Default: | True |
---|---|
Description: | If set to False , disables SQL caching but keeps invalidating
to avoid stale cache. |
CACHALOT_CACHE
¶
Default: |
|
---|---|
Description: | Alias of the cache from Warning After modifying this setting, you should invalidate the cache using the manage.py command or the API. Indeed, only the cache configured using this setting is automatically invalidated by django-cachalot – for optimisation reasons. So when you change this setting, you end up on a cache that may contain stale data. |
CACHALOT_DATABASES
¶
Default: | 'supported_only' |
---|---|
Description: | List, tuple, set or frozenset of database aliases from DATABASES against
which django-cachalot will do caching. By default, the special value
'supported_only' enables django-cachalot only on supported database
engines. |
CACHALOT_TIMEOUT
¶
Default: |
|
---|---|
Description: | Number of seconds during which the cache should consider data as valid.
Warning Cache timeouts don’t work in a strict way on most cache backends. A cache might not keep data during the requested timeout: it can keep it in memory during a shorter time than the specified timeout. It can even keep it longer, even if data is not returned when you request it. So don’t rely on timeouts to limit the size of your database, you might face some unexpected behaviour. Always set the maximum cache size instead. |
CACHALOT_CACHE_RANDOM
¶
Default: | False |
---|---|
Description: | If set to True , caches random queries
(those with order_by('?') ). |
CACHALOT_INVALIDATE_RAW
¶
Default: | True |
---|---|
Description: | If set to False , disables automatic invalidation on raw
SQL queries – read raw queries limits for more info. |
CACHALOT_ONLY_CACHABLE_TABLES
¶
Default: | frozenset() |
---|---|
Description: | Sequence of SQL table names that will be the only ones django-cachalot
will cache. Only queries with a subset of these tables will be cached.
The sequence being empty (as it is by default) doesn’t mean that no table
can be cached: it disables this setting, so any table can be cached.
CACHALOT_UNCACHABLE_TABLES has more weight than this:
if you add a table to both settings, it will never be cached.
Run ./manage.py invalidate_cachalot after changing this setting. |
CACHALOT_UNCACHABLE_TABLES
¶
Default: | frozenset(('django_migrations',)) |
---|---|
Description: | Sequence of SQL table names that will be ignored by django-cachalot.
Queries using a table mentioned in this setting will not be cached.
Always keep 'django_migrations' in it, otherwise you may face
some issues, especially during tests.
Run ./manage.py invalidate_cachalot after changing this setting. |
CACHALOT_ADDITIONAL_TABLES
¶
Default: | list() |
---|---|
Description: | Sequence of SQL table names that are not included in your Django apps such as unmanaged models. Cachalot caches models that Django does not manage, so if you want to ignore/not-cache those models, then add them here. |
CACHALOT_QUERY_KEYGEN
¶
Default: | 'cachalot.utils.get_query_cache_key' |
---|---|
Description: | Python module path to the function that will be used to generate
the cache key of a SQL query.
Run ./manage.py invalidate_cachalot
after changing this setting. |
CACHALOT_TABLE_KEYGEN
¶
Default: | 'cachalot.utils.get_table_cache_key' |
---|---|
Description: | Python module path to the function that will be used to generate
the cache key of a SQL table.
Clear your cache after changing this setting (it’s not enough
to use ./manage.py invalidate_cachalot ). |
manage.py
command¶
manage.py invalidate_cachalot
is available to invalidate all the cache keys
set by django-cachalot. If you run it without any argument, it invalidates all
models on all caches and all databases. But you can specify what applications
or models are invalidated, and on which cache or database.
Examples:
./manage.py invalidate_cachalot auth
- Invalidates all models from the ‘auth’ application.
./manage.py invalidate_cachalot your_app auth.User
- Invalidates all models from the ‘your_app’ application, but also
the
User
model from the ‘auth’ application. ./manage.py invalidate_cachalot -c redis -p postgresql
- Invalidates all models, but only for the database configured with the ‘postgresql’ alias, and only for the cache configured with the ‘redis’ alias.
Template utils¶
Caching template fragments can be extremely powerful to speedup a Django application. However, it often means you have to adapt your models to get a relevant cache key, typically by adding a timestamp that refers to the last modification of the object.
But modifying your models and caching template fragments leads to stale contents most of the time. There’s a simple reason to that: we rarely only display the data from one model, we often want to display related data, such as the number of books written by someone, display a quote from a book of this author, display similar authors, etc. In such situations, it’s impossible to cache template fragments and avoid stale rendered data.
Fortunately, django-cachalot provides an easy way to fix this issue,
by simply checking when was the last time data changed in the given models
or tables. The API function
get_last_invalidation
does that,
and we provided a get_last_invalidation
template tag to directly
use it in templates. It works exactly the same as the API function.
Django template tag¶
Example of a quite heavy nested loop with a lot of SQL queries (considering no prefetch has been done):
{% load cachalot cache %}
{% get_last_invalidation 'auth.User' 'library.Book' 'library.Author' as last_invalidation %}
{% cache 3600 short_user_profile last_invalidation %}
{{ user }} has borrowed these books:
{% for book in user.borrowed_books.all %}
<div class="book">
{{ book }} ({{ book.pages.count }} pages)
<span class="authors">
{% for author in book.authors.all %}
{{ author }}{% if not forloop.last %},{% endif %}
{% endfor %}
</span>
</div>
{% endfor %}
{% endcache %}
cache_alias
and db_alias
keywords arguments of this template tag
are also available (see
cachalot.api.get_last_invalidation()
).
Jinja2 statement and function¶
A Jinja2 extension for django-cachalot can be used, simply add
'cachalot.jinja2ext.cachalot',
to the 'extensions'
list of the OPTIONS
dict in the Django TEMPLATES
settings.
It provides:
- The API function
get_last_invalidation
directly available as a function anywhere in Jinja2. - An Jinja2 statement equivalent to the
cache
template tag of Django.
The cache
does the same thing as its Django template equivalent,
except that cache_key
and timeout
are optional keyword arguments, and
you need to add commas between arguments. When unspecified, cache_key
is
generated from the template filename plus the statement line number, and
timeout
defaults to infinite. To specify which cache should store the
saved content, use the cache_alias
keyword argument.
Same example than above, but for Jinja2:
{% cache get_last_invalidation('auth.User', 'library.Book', 'library.Author'),
cache_key='short_user_profile', timeout=3600 %}
{{ user }} has borrowed these books:
{% for book in user.borrowed_books.all() %}
<div class="book">
{{ book }} ({{ book.pages.count() }} pages)
<span class="authors">
{% for author in book.authors.all() %}
{{ author }}{% if not loop.last %},{% endif %}
{% endfor %}
</span>
</div>
{% endfor %}
{% endcache %}
Signal¶
cachalot.signals.post_invalidation
is available if you need to do something
just after a cache invalidation (when you modify something in a SQL table).
sender
is the name of the SQL table invalidated, and a keyword argument
db_alias
explains which database is affected by the invalidation.
Be careful when you specify sender
, as it is sensible to string type.
To be sure, use Model._meta.db_table
.
This signal is not directly triggered during transactions,
it waits until the current transaction ends. This signal is also triggered
when invalidating using the API or the manage.py
command. Be careful
when using multiple databases, if you invalidate all databases by simply
calling invalidate()
, this signal will be triggered one time
for each database and for each model. If you have 3 databases and 20 models,
invalidate()
will trigger the signal 60 times.
Example:
from cachalot.signals import post_invalidation
from django.dispatch import receiver
from django.core.mail import mail_admins
from django.contrib.auth import *
# This prints a message to the console after each table invalidation
def invalidation_debug(sender, **kwargs):
db_alias = kwargs['db_alias']
print('%s was invalidated in the DB configured as %s'
% (sender, db_alias))
post_invalidation.connect(invalidation_debug)
# Using the `receiver` decorator is just a nicer way
# to write the same thing as `signal.connect`.
# Here we specify `sender` so that the function is executed only if
# the table invalidated is the one specified.
# We also connect it several times to be executed for several senders.
@receiver(post_invalidation, sender=User.groups.through._meta.db_table)
@receiver(post_invalidation, sender=User.user_permissions.through._meta.db_table)
@receiver(post_invalidation, sender=Group.permissions.through._meta.db_table)
def warn_admin(sender, **kwargs):
mail_admins('User permissions changed',
'Someone probably gained or lost Django permissions.')