2013-01-29

I'm taking up the gauntlet. A gauntlet that has been thrown down with
considerable force and eloquence by my colleague Gijs on his new weblog in a
post called Django is just an API. The title
comes from a similar article about Rails being just an API.

Transitioning to modern times

We have a big stack of Django apps (called Lizard, 50+ apps) and some 40 sites. I have a quite good
description of how it all fits together in a previous post where I told about
a pyramid experiment.

Summary: we use all of Django (models, templates, views, staticfiles,
etc). Reusable apps. Two big core apps that everything inherits from for basic
user interface and basic map handling.

To quote Gijs: ... we are currently transitioning to API-based de-coupling of
back-end and front-end. In other words: a client-side user interface which
talks to an API for data. This web app should just be one of the api-consuming
clients, not privileged in any way compared to an iPhone or Android native
client.

So: Django is relegated to the dustbin of history and is mercifully
allowed to serve up an API (probably until it is replaced in due course with
some node.js goodness). Yes, I'm being over-the-top melodramatic here :-)

Not too quick, though: the "we are currently transitioning" quote should be
qualified as being limited, at the moment, to one single full-blown all-out
experiment on a big project. All our other projects still use ye olde Lizard
with the whole Django stack.

"We are transitioning" is right in that there seems to be management buy-in.

My internal doubt. I'm an obsolete heathen.

That dustbin-of-history stack of Django apps called Lizard? The basis was made
by me and another colleague. Apps as nicely releaseable python packages? With
repeatable buildouts? Internal python package server? Current user interface?
Documentation? Test setup? There's a really big piece of me and my
thinkwork and my preferences in there.

This would mostly be obsoleted by a html+javascript client-side user
interface? An interface that takes over most of the work and that simply talks
to an API? Wow. That turns the whole structure upside down. The flow of
control is reversed. No more UI controlled by (and structured by) Django apps.

Gijs calls this a paradigm shift. When someone has worked on traditional
websites using Django, Rails, PHP, Plone or you name it, the paradigm shift to
API based application design is incredibly hard to wrap your head around.

Is it a problem for me? Am I resisting change? I got some flak at a previous
job for sticking too long to a particular tool ("archgenxml" for those who
know it) when I should really have moved along. Since then I'm a bit anxious
to not be too conservative.

According to Gijs, programmers can develop the technological conservatism
syndrome as they get older. Age-wise I'm number 4 in a company of 50 at the
ripe age of 40 :-) Is it potentially a career-ending syndrome? In any case I
"should be ready to learn new things" or there's a nice place for me right
next to the dustbin where Django already is in.

I'm a heathen. I don't unquestingly believe the new gospel. The new
hmtl+javascript-only party line. I dared to answer the rethorical question
"who doesn't totally believe in html+javascript+api?" with a "no". I do not
believe in that party line in that manner. At least I do not believe in it
with the down-with-django mentality attached to it.

My theoretical misgivings (positive for Django)

With html+javascript+API, you can totally make great webpages/sites/apps that totally
rock. A twitter app, a kanban board on top of github. Google's apps.

But there are sites/apps/pages that are best done in a different way. And some
sites just need more server-side power and control. Here are some of the
theoretical and practical notes I'd like to give to you for consideration:

When I look at https://github.com and look at the http traffic, I see a
different picture from html+javascript+api. Github has a great API and I had great
fun with a little script that queries
that API. But... those pages are Plain Old HTML. Look what comes over the
wire! Responsive as hell, but even the line numbers have their own span or
div! Totally generated server-side. With some great javascript-based partial
page loads and history rewriting (probably pjax?) to make it even quicker.

If javascript-only webpages where the Absolute Truth, wouldn't github have
gotten the message? As they apparently have some measure of success (wow)
and are obviously ridiculously fast, perhaps there's something good to plain
old HTML rendering with some javascript sprinkled on top?

Google doesn't index empty html pages with javascript. To get into google's
index, you need html content.

(Ok, most of our lizard sites couldn't care less if they're indexed or
not. At least we're not actively aiming and designing for that. The nature
of the data doesn't require it. But it would be fun to try and make google
results meaningful for our sites.)

According to Gijs, ... this web app should just be one of the api-consuming
clients, not privileged in any way compared to an iPhone or Android native
client.

Wait one minute. You give all the control and quite a lot of the
functionality to the client side javascript code (per the new paradigm) so
that native clients have a level playing field. This means you have to
duplicate the whole effort another two times to get an iphone and android
native client? For a twitter client, ok, but there's a shocking amount of
functionality buried in all those lizard Django apps. I do not want to have
to code that three times!

The only economically sane way forward for us as a company is to have a good
responsive multi-device layout. But then there's no theoretical need to
shift lots of code to the client side (at least not because of the
equal-footing-for-iphone reasons).

Django doesn't look too bad

To be honest, I cannot see a good theoretical reason why an html+javascript+api
interface is inherently better in each and every case.

I do want to use API's and javascript a lot, I just don't want the user
interface to be king. That kind of control is better left in the hands of good
old reliable Python code. At least for the kinds of elaborate sites we're
making.

I want Python code to manage the URLs (unless it is more practical in the
browser for specific pages and workflows). I want the templates to be very
dumb and the javascript code to be reasonably dumb and the Python code pretty
smart (though I'd personally love some smart backbone.js+api to clean up several
dirty custom javascript plus dirty python code corners).

Some extra points I'd like to raise:

i18n is easier to manage in Python/Django than in javascript. And
internationalization is getting more and more important for us as a company.

We haven't tested a single line of javascript in the three years that we're
now using Lizard. Our Python/Django test coverage is fair (bad in some
places, very good in a few). So... what does that tell me about the expected
test coverage of our code when we've moved a lot of the functionality to
javascript?

The goal of Lizard is to make lots of different kinds of data sources
available and combinable in one system. One of the two cores of the current
Lizard is an "adapter mechanism" that ties the generic
map/search/click/graph handling to arbitrary data sources. The other core is
a generic user interface that you can customize and tweak, for instance by
subclassing. All those various data sources all need different tweaks and
customizations. Lizard allows that.

The core of the new html+javascript+api approach sounds to me like "I want to
build a modern web interface with html+javascript+api". There isn't much "data
source" or "customizations" or "quick modification in a subclass" in the
core of that approach. It sounds like it is aimed at building one single
individual great website. See the next point.

Repeatability, rebuildability, customizability.

I do not doubt the beauty of the end product of a html+javascript+api
interface. I sincerely hope that the current experiment succeeds and that
the big project gets a great interface. I do doubt the applicability to
all our projects and sites. Despite all thoughts about one single website
for all clients, I pretty much know for sure that, next to the central
generic website, there'll always be a bunch of separate websites for
specific clients. Can we manage that with a html+javascript+api codebase? (I
don't know the current state-of-the-art of javascript packaging, perhaps it
is doable).

I started reading the new Django book two scoops of Django last week. Wow, that provided me with some
welcome support: Django is pretty good and useful. Really.

Proposal/wishes

I'd propose continuing with Django. And structuring that mess of javascript we
have now with backbone.js or angular.js or something like that. Which means
bringing in APIs.

Oh, and I'd love to make our Python-internal adapter mechanism a bit more
explicit and turn it partially into an API. Lizard is a bigger project than
just our company. There are quite a lot of partners! (See http://lizard.net/
if you speak Dutch). An API would make it easier to connect their custom data
sources with the rest of Lizard (compared to needing a bunch of Python
code). Up to a certain level this could work real well. And if more
functionality is needed, we can do the real Python integration anyway.

Another wishlist item: a more responsive layout. The previous version worked
OK on the desktop. The current version also works on an iPad. The next version
should also work comfortably on an iPhone.

All those wishes? There's nothing in Django holding them back. And there's a
lot of Django you can use to make it possible.

Closing comment

I've now made myself completely impossible. And I've made a fool out of
myself. Publicly. I'll just slink away quietly now to the house of demented
elderly programmers.

That is, unless somebody shoots me first for being too verbose and for loving
flowery language too much and for making my sentences (and this whole article)
too long :-)

Show more