Motivation
It is quite common for modern web sites to include real-time notifications and alerts. Implementing this feature on sites based on Django is difficult due to the limitations on the architecture and underlying technology.
There are several alternatives, like our very own Telegraphy that uses a Twisted server for message delivery to and from the clients. Although this more than enough for our purposes, we wanted to show you how to integrate popular and mature solutions into an existing Django-based site, so we chose Node.js, Socket.io and Redis for message delivery and Django with django-notifications for the notification generation within the site itself.
The solution is based on the one proposed here, but it is extended for the use case of notifications which can be directed to a specific user that might be logged from several locations.
Another extension to Max Burnstein’s solution, is that we’re going to use Nginx as reverse proxy for our site. The access to the site itself is straight forward, but the WebSockets require special consideration.
The application
We’re going to build a Django application on an Ubuntu Server 12.04 LTS, using the following packages, python and node.js modules:
curl, python-software-properties, g++ and make
python
python-virtualenv
redis-server
nginx (from the official Nginx ppa)
nodejs (from Chris Leas’s ppa)
npm, socket.io and cookie
django
django-notifications-hq
django-user-sessions
The environment
Let us start by installing the necessary Ubuntu packages:
Next, we are going to install the node.js modules. We chose to set them up as global modules to make things easier, but we could have just installed them locally and then made sure that they are available to our application:
Now we have everything in place but the python-specific dependencies. In order to avoid messing with the system’s python modules, we’re going to create a new virtualenv, and install the dependencies on it:
At this point we have everything we need to start building our Django application. We’ll follow the first steps of the Django tutorial so you can have a complete picture of the inner workings of the application.
Initial steps
Unless otherwise indicated, every command listed from now on is supposed to be run from within the virtual environment we created on the previous section. Use:
to activate the virtual environment. We’ll use the django-admin.py script to perform the initial set-up of the application:
This will create a new directory called realtime_notifications and create an empty Django project called realtime_notifications. Change into the realtime_notifications directory and create an empty application called rn with the following command:
The contents of the realtime_notifications directory should look like this:
Read the Django documentation for an explanation of the purpose of each file.
We’re going to use the default SQLite database, but the instructions that follow are valid for any of the supported back-ends. Run the following commands to set-up the database:
Next, we are going to set-up the django-notifications-hq and django-user-sessions modules. Open the settings.py file and change the INSTALLED_APPS ad MIDDLEWARE_CLASSES definitions so they look like the following:
and add the following to the file:
What we did is add notifications to the site’s apps, replaced the default session implementation with the one provided by django-user-sessions and set the database back-end for the session engine. Add the django-notifications-hq and django-user-sessions to the urls.py file:
Run the syncdb command one more time to set-up the necessary tables for the django-notifications-hq and django-user-sessions modules:
Non real-time notifications
At this point we can start working on the application. Our site needs only very basic features to demonstrate the traditional (i.e. non real-time) notifications, as described in the following user stories:
As a User I want to to log into and out of the site
As a User I want to see the unread notifications sent to me
As a User I want to send notifications to other users
As a User I want to mark all received notifications as read
We’re going to make use of the existing Django login and logout views. So we only need to cover the rest of the use cases.
This is how our rn/views.py module looks like:
We’ve defined a view for the home page that displays the unread notifications for the user, a view to send notifications to one or all the users of the site and a view to mark all unread notifications as read.
These are the contents of the urls.py module:
All we need now is a way to present the views. We created a template in rn/templates/index.html with the following content:
Notice that we reference files from Bootstrap and jQuery. You can download them from Bootstrap’s and jQuery’s home pages and put them in the rn/static/ directory. We could have avoided these dependencies, but we wanted to make the example re-usable for real websites.
We now have covered all of the use cases, but there’s a problem: If a notification is generated, the user won’t see it until it reloads the page. We need a way to push those notifications to the user.
Real-time notifications
There have been a lot of solutions to the problem of pushing content to the client, but none of them had wide-spread adoption due to the limitations of the HTTP protocol. WebSockets provide a solution to the problem as it allows bi-directional full-duplex communication between the server and the client. Sadly, the WebSockets specification hasn’t been standarized yet and support for it has only recently been included on all the popular web browsers.
In order to have real-time communication with the client on every possible web browser, we need a more general solution. socket.io provides such solution. socket.io is a JavaScript library with components for both the client and the server. It has support for WebSockets, but if they aren’t available it can fallback to other means of real-time communication. The server part of the library runs on top of node.js, which provides a high-performance event-driven framework to manage the message exchange with the clients.
All we need now is a way to connect the socket.io server running on node.js with our Django site. This can be easily done using Redis. Redis is a basically a key-value store, but it also provides a way to subscribe and publish to keys, so it basically becomes a message bus. With this architecture, the socket.io server will subscribe a user specific key, onto which Django is going to write the notifications. Once the message is received, the server will send it to the connected client.
node.js server
There’s not much to our node.js/socket.io server:
Once the client connects, the handler function is invoked, which connects to the redis server, subscribes to key unique to that session and configures the event handler for when a message is written to the key. When a message is received, it is sent (in its JSON form) down the client’s channel. This file can be stored anywhere as long as node.js has access to the socket.io and cookie dependencies (that’s why we used the global installation). We put the file in nodejs/notifications.js under the root of the site directory. You can run the server typing:
Real-time client
Before we make the change that’s responsible for writing the Django notifications onto the Redis key, we’ll change the client code to connect to the socket.io server and avoid reloading the page whenever we create a notification o mark them as read.
First we need to create views to allow AJAX calls to send_notification and mark_as_read and a new view that points to the real-time version of the home template. These are the changes to rn/views.py:
We’ve created a new home_realtime view that points to the new template for realtime notifications, and two views similar to the send_notification and mark_as_read but inteded to be used with AJAX calls. The ajax_mark_as_read view also writes to the user’s redis notification key to signal that all the notifications were marked as read, and that the UI should be updated accordingly.
Now that we have the views, we need a template (which will be stored in rn/templates/index_realtime.html) to exposed them:
The template is very similar to the non real-time notifications version. The most important change is in the fourth script element. Within this script we use the typical $(document).ready(function() {} jQuery statement to have our event handler run once the document is ready. Let us go statement by statement analyzing what each does:
Whenever the element with id mark-as-read-button we should make an AJAX call to the ajax_mark_as_read URL. We do nothing if the call succeeds. We haven’t done any error handling since it’s not the purpose of this article to teach the reader how to properly do AJAX calls.:
We hook our Bootrap popover to the click event of the element with id send-notification-button:
Whenever a button element with id send-notification-form-submit under the body element (this is defined in the popover content), we make an AJAX call to the ajax_send_notification url, using the existing form as data. When the call succeeds, we hide the popover.:
This is the most important part of the template. We create a socket.io channel to localhost on port 8002, which was the one we used for our node.js service and hook an event handler to the message event for the channel. On this event handler we do three things:
We parse the payload of the message as JSON.
If mark_as_read is present in the message, we clear the notifications table
If mark_as_read is not present in the message, we create a new row at the top notifications table using the message information for each column.
So, whenever something is written to the session’s events key on Redis, that is picked up by the node.js service, which takes the message and sends it to the client through the channel, and when that is received, the UI is updated accordingly without reloading the page.
The last thing we need to do, is add the urls for the views into the urls.py module:
Run the notifications.js node.js module, and our Django site, and open the URL http://localhost:8000/realtime/. If everything went right, you’ll see that everything is updated without reloading the page. In order to properly test the functionality, we suggest you open a different browser and log in with the same user. Whenever a notification is sent to that user, you’ll see the page update itself. If you click on the “Mark as read” button, you’ll see that both tables are cleared.
Nginx support
It’s not uncommon for Django sites to be behind an Nginx reverse proxy. Starting from version 1.4, Nginx has support for WebSockets, but it needs to be configured to do so. This is a possible configuration for our site:
We also need to modify the rn/templates/index_realtime.html template:
You should now be able to access the site at http://localhost/realtime.
Conclusion
We hope we were able show you how easy it is to have real-time communications with the client, if you leverage the existing functionality of node.js, socket.io, Redis and Django with the django-notifications-hq module. Something we haven’t mentioned yet is that this solution works on pretty much every web browser, as socket.io abstracts the problem of constructing the actual communication channel between the client and the node.js service.
Feel free to send us comments, suggestions of improvements to the solution.