Speeding up built in Django Templates

My Django template redering is unbearably slow. I found what I thing is a surprising way to speed up my Django template.

I have a site that has lots of point objects with latitude and longitude. When I render a template with 1500 of these points, and the template contains a for loop to output them, the redering time was about 12 seconds. This of course is an unacceptably long time for loading a fairly simple web page that just happens to have a lot of numbers on it.

{% for point in points %}
polylinePoints.push( new YGeoPoint( {{ point.lat }}, {{ point.lng }} ) );
{% endfor %}

I started to play around with ways to speed things up. Using python to format the points into a long string before passing it to the template would cause the template to render in less than 1 second. The problem with this particular optimization is that it transfers part of the responsibility of rendering the data from the template engine to the python code. It would be better if the python code doesn’t need to know this much about how the data will be rendered.

I decided to try using a python code profiler to see if it would yield some clue as to what is causing the template to render so slowly. Using the information about Django profiling found at http://code.djangoproject.com/wiki/ProfilingDjango I got the following profiling data.

618620 function calls (618335 primitive calls) in 15.518 CPU seconds
Ordered by: internal time, call count
List reduced from 425 to 20 due to restriction <20>
ncalls  tottime  percall  cumtime  percall filename:lineno(function)
64288    2.317    0.000    2.317    0.000 /usr/lib/python2.5/posixpath.py:168(exists)
36758    1.909    0.000    9.553    0.000 /usr/lib/python2.5/site-packages/django/utils/translation/__init__.py:23(delayed_loader)
18375    1.436    0.000    1.711    0.000 /usr/lib/python2.5/site-packages/django/utils/importlib.py:18(import_module)
82766    1.429    0.000    1.429    0.000 /usr/lib/python2.5/posixpath.py:56(join)
9184    1.228    0.000    5.588    0.001 /usr/lib/python2.5/gettext.py:421(find)
9184    0.800    0.000   12.763    0.001 /usr/lib/python2.5/site-packages/django/utils/formats.py:10(get_format_modules)
18370    0.613    0.000    0.802    0.000 /usr/lib/python2.5/site-packages/django/utils/translation/trans_real.py:212(get_language)
91937    0.588    0.000    0.588    0.000 /usr/lib/python2.5/site-packages/django/utils/functional.py:274(__getattr__)
9184    0.495    0.000    0.727    0.000 /usr/lib/python2.5/gettext.py:130(_expand_lang)
36762    0.472    0.000   10.026    0.000 /usr/lib/python2.5/site-packages/django/utils/functional.py:54(_curried)
9184    0.448    0.000    6.488    0.001 /usr/lib/python2.5/site-packages/django/utils/translation/trans_real.py:311(check_for_language)
3054    0.346    0.000    0.420    0.000 /usr/lib/python2.5/site-packages/django/utils/numberformat.py:3(format)
9184    0.304    0.000   13.218    0.001 /usr/lib/python2.5/site-packages/django/utils/formats.py:37(get_format)
18368    0.274    0.000    0.274    0.000 /usr/lib/python2.5/site-packages/django/utils/translation/trans_real.py:34(to_locale)
18368    0.260    0.000    0.260    0.000 /usr/lib/python2.5/site-packages/django/utils/importlib.py:4(_resolve_name)
9184    0.232    0.000    0.232    0.000 /usr/lib/python2.5/locale.py:281(normalize)
9188    0.168    0.000    0.168    0.000 /usr/lib/python2.5/posixpath.py:74(split)
1517    0.145    0.000    0.162    0.000 /usr/lib/python2.5/site-packages/django/db/models/base.py:251(__init__)
2    0.144    0.072   14.697    7.348 /usr/lib/python2.5/site-packages/django/template/defaulttags.py:124(render)
3089    0.130    0.000    0.182    0.000 /usr/lib/python2.5/site-packages/django/template/__init__.py:710(_resolve_lookup)

It looks like Django is spending a lot of time trying to translate, format and/or localize my numbers. They are just simple floating point latitude and longitude, and are just being written out to be part of some JavaScript code. They don’t need fancy formatting.  This prompted me to change my loop to the following :

{% for point in points %}
polylinePoints.push( new YGeoPoint( {{ point.lat|stringformat:"s" }}, {{ point.lng|stringformat:"s" }} ) );
{% endfor %}

And re-ran my profiling, with a time of less than 2 seconds for rendering the template.

102767 function calls (102482 primitive calls) in 1.620 CPU seconds
Ordered by: internal time, call count
List reduced from 392 to 20 due to restriction <20>
ncalls  tottime  percall  cumtime  percall filename:lineno(function)
3089    0.121    0.000    0.157    0.000 /usr/lib/python2.5/site-packages/django/template/__init__.py:710(_resolve_lookup)
3090    0.108    0.000    0.457    0.000 /usr/lib/python2.5/site-packages/django/template/__init__.py:550(resolve)
3034    0.105    0.000    0.105    0.000 /usr/lib/python2.5/site-packages/django/template/defaultfilters.py:230(stringformat)
1517    0.090    0.000    0.107    0.000 /usr/lib/python2.5/site-packages/django/db/models/base.py:251(__init__)
2    0.089    0.045    1.101    0.550 /usr/lib/python2.5/site-packages/django/template/defaulttags.py:124(render)
1517    0.081    0.000    0.111    0.000 /usr/lib/python2.5/site-packages/django/db/backends/mysql/compiler.py:4(resolve_columns)
3076    0.079    0.000    0.186    0.000 /usr/lib/python2.5/site-packages/django/utils/formats.py:77(localize)
18/1    0.074    0.004    1.144    1.144 /usr/lib/python2.5/site-packages/django/template/__init__.py:792(render)
16994    0.071    0.000    0.079    0.000 /usr/lib/python2.5/site-packages/django/utils/encoding.py:54(force_unicode)
3076    0.068    0.000    0.888    0.000 /usr/lib/python2.5/site-packages/django/template/debug.py:87(render)
3076    0.064    0.000    0.110    0.000 /usr/lib/python2.5/site-packages/django/utils/html.py:30(escape)
3099    0.056    0.000    0.166    0.000 /usr/lib/python2.5/site-packages/django/utils/functional.py:254(wrapper)
6134    0.055    0.000    0.055    0.000 /usr/lib/python2.5/site-packages/django/utils/safestring.py:89(mark_safe)
1517    0.045    0.000    0.045    0.000 /var/lib/python-support/python2.5/MySQLdb/times.py:43(DateTime_or_None)
1518    0.039    0.000    0.429    0.000 /usr/lib/python2.5/site-packages/django/db/models/query.py:213(iterator)
3090    0.036    0.000    0.036    0.000 /usr/lib/python2.5/site-packages/django/template/context.py:41(__getitem__)
3094    0.033    0.000    0.190    0.000 /usr/lib/python2.5/site-packages/django/template/__init__.py:692(resolve)
1    0.029    0.029    0.075    0.075 /var/lib/python-support/python2.5/MySQLdb/cursors.py:282(_fetch_row)
3951    0.025    0.000    0.025    0.000 /usr/lib/python2.5/site-packages/django/utils/functional.py:274(__getattr__)
1518    0.022    0.000    0.022    0.000 /usr/lib/python2.5/site-packages/django/db/utils.py:122(_route_db)

{{ point.lat }} takes much more time than {{ point.lat|stringformat:”s” }}

Let me repeat that.

{{ point.lat }} takes much more time than {{ point.lat|stringformat:”s” }}

So the result that I found to be rather surprising is that by using the stringformat function on my floating point numbers I was able to drastically speed up the rendering of my latitude and longitudes in by Django template. It seems that by explicitly telling it to use stringformat, I cause it to skip all sorts of number formatting code that is not necessary when you are simply producing JavaScript output.