Django optimizing Memcache and Nginx

I have a django application which complete and ready to deploy. One of the requirement are :

• It will run 24x7 with minimum interupption
• Down Time will be only 3 hours every monday
• User will access concurrently around 17 users

So with this requirement, i want to have my django app high performance without glitch and fast. What already implemented in code are using some method which are :

• Using Postgresql Database in a separate machine.
• Using memcached to do caching
• Using django-compressor to compress static files
• Using Nginx web server to server static files
• using Gunicorn to server the Django application

So that all basic optimization for Django application or Web based application. The application is a simple single table query and an insert to database.

So the application will have insert to table when external user hit the web apps. Then the client will have pool the server for the input every 2 seconds. so the access will be all the time.

So I put some testing of the performance. First using single machine hosting a memcache and postgresql. The setup are :

• Cache using memcache localy
• Session using Memcache
• Nginx server static files

 and here are the results :

Test 1 with single machine setup

Then i try other setup, which difference are put session in postgresql and here the results :

Test 2 Django session using Postgresql

And then last one, i setup a load balanced setup with 2 django app server, here are the setup :

• Memcached setup as shared memcache between 2 Django app
• Nginx as a single entry load balance the application request to 2 Django app server
• Postgresql is still separated machine.
• Django Compressor is still used.

And here are the result of the test :

Django app load balanced with Nginx in front of application

As you see, the result is better if we horizontal scale the python serving django applications. The load balance in Nginx is a standard setup with 2 upstream point to django app server.

Even the Nginx server still hosting 1 django app, and other machine hosting only django app,with memcache in each server as shared memcache pool, and the postgresql is a single machine.

The improvement is huge. From 40 Req/sec to 117 Req/sec , 2x improvements. This still not optimize the database cache, as i using Django 1.6 and johny cache still not available at this time to be use with Django 1.6

On problem to be note, the django-compressor got problem when setup as multiple python app server, because the key to generate key cache prefix is using the host name, so in one another of the server, there will be missing static compressed file. So we just generate the compressed static in 1 machine, and copied to the other machine and this solved.

Posted in: Django,FreeBSD,Python