Cusbi - An example of speeding up the rendering of your Django templates

For one of our companies, Monit, we have the “Meldman” application running for some 4 years now. It is used by some 20 municipalities, mainly for registering and keeping track of solving equipment failures and for daily reporting.

All was well for a long time, but we noticed that, with the amount of data stored in the database ever increasing, the performance of the front-end was deteriorating. So, it was time to dive into the inner workings of the application and fix this.

The first screen we tackled shows all the equipment a client has with the number of failures and number of times maintenance has been executed in the last year. It had the following bits of code:

Model: PieceOfEquipment.

To count the number of failures / number of times maintenance was executed for a piece in the last year of equipment the model for a piece of equipment had this property added :

def nr_failures(self):
    last_year = make_aware(datetime.today() - timedelta(days=365), 
                           get_current_timezone())
    return FailureMaintenance.objects.filter(apparaat=self,
                                             type__failure_maintenance='F',
                                             start_dt__gte=last_year,
                                             ).count()

def nr_maintenance(self):
    last_year = make_aware(datetime.today() - timedelta(days=365), 
                           get_current_timezone())
    return FailureMaintenance.objects.filter(apparaat=self,
                                             type__failure_maintenance='M',
                                             start_dt__gte=last_year,
                                             ).count()

Views.py

eq = PieceOfEquipment.objects.filter(client=client)
return render(request, 'meldman/overview_equipm.html', {'eq': eq})

HTML template

<td style="text-align:left”>
  <a href="{% url 'some_url' %}?device={{ eq.id }}">{{ eq.nr_failures }}

<td style="text-align:left”>
  <a href="{% url 'some_url' %}?device={{ eq.id }}">{{ eq.nr_maintenance }}

Beautiful, relative simple code with the logic stored in the model (as most blogs tell you to do). Well done, pat on the shoulder.

And this worked fine, however, somehow clients seemed to register more and more equipment which they tracked in the “Meldman” (which I guess can be seen is good news, seemingly they like the application…). The result was that the peformance began te degrade to a level which was not acceptable anymore. So, what was happening? Using the Django-debug-toolbar it was quickly evident that one (expensive) query was used over-and-over again. If a client has 300 pieces of equipment a similar query would run 300 time for calculating the “nr_failures” and 300 times for calculating the “nr_maintenance”.

So, what did we change:

1) we stopped using the properties on the model

2) we changed the view to:

from django.db.models import Count, Q
last_year = make_aware(datetime.today() - timedelta(days=365), get_current_timezone())
eq = Equipment.objects.filter(Q(klant=klant) & 
                              Q(FailureMaintenance__start_dt__gte=last_year))
eq = eq.annotate(nr_failures=Count('failuremaintenance', 
                 filter=Q(failuremaintenance__type__failure_maintenance='F')))
eq = eq.annotate(nr_maintenance=Count('failuremaintenance', 
                 filter=Q(failuremaintenance__type__failure_maintenance='M')))
return render(request, 'meldman/overview_equipm.html', {'eq': eq})

Now, no matter how many pieces of equipment a client has only one query (yes, “1”, as in “the first positive integer”) is executed. Obviously this solved the performance problem quite handsomely.

All in all a fairly painless exercise with a huge benefit.