Hopefully our last post regarding Parse convinced at least some of the existing Parse users to ditch it completely. The purpose of this post is to summarize a simple alternate stack which we used for our customers who wanted to move away from Parse. It is important to note that there are still multiple BaaS options available to host a mobile app backend. For instance here is a neat list of feature wise options available on github. But the question is whether developers can trust yet another platform after being booted by multiple provider over the years. We have successfully used the below stack in most of the migration projects and it has been working out well.
PaaS vs. VM
While for some customers, a pure PaaS system like GAE /AWS BeanStalk/Heroku would have been much less hassle, not all were OK with paying the higher price. While some customers were fine with setting up own infrastructure since they had in-house expertise for managing it. For most clients we just went with bare VMs instead of PaaS.
Web framework
Although we’re mainly a Django shop, this had to be something lightweight. Given that we have a strong bias towards Python, we decided that Flask was a better choice. Flask is a very light-weight framework and very well suited for building API servers. It comes with no ORM bias or any other stack component bias for that matter. (which could be a good or a bad thing depending upon how you’re used to developing webapps).
Database
All the apps didn’t have any unstructured or document-type data requirements per say. There wasn’t much convincing required for the clients, that the storage backend needed to be moved from Mongo to PostgreSQL. I heard one of the clients say something about they planning to use big-data and hence needing mongo... I didn’t hear the rest of it. I am not sure if I managed to convince him that 50,000 rows of data is not really that big a data but we did end up using PostgresSQL for them.
ORM
In the past, for some of the django projects we’ve skipped using a custom ORM where we thought Django’s ORM served the purpose. But since Flask doesn’t come with an inbuilt ORM, we needed to put something which would be robust and beneficial in the long term. I guess some clients would have been OK with not having any ORM at all, but we decided to push for our favourite library for the purpose - SQLAlchemy, an incredibly powerful ORM.
Rest API
With Django, tastypie has been our defacto REST framework for quite some time. With Flask, since we already had some experience with Flask-RESTful, we decided to go with it. On the lines of Flask, Flask-RESTful too feels minimal and bare bones especially compared to something like tastypie. But it seemed to work fine for our purpose and fit nicely into the stack.
Background jobs
Celery has been our choice for a lot of past projects for scheduling background jobs. It is also known to work well with Django. But we wanted something very simple for this part too so that the clients can later manage and maintain without much of the configuration and setup fuss. Fortunately, there is another incredibly simple and light-weight library which we’ve been using exactly for this purpose called RQ, stands for Redis Queue. It has a dead simple API. Although it doesn’t come with in-built monitoring support. One can use rq-dashboard library for monitoring.
Hosting
This one, we let the clients choose as per their individual preference and their access to any possible free credits from the hosting providers. Most of the choices expectedly revolved around AWS, GCE and DO.
Cache
We already had Redis present in the stack since it was used by RQ, but we decided to use memcached for caching, since that’s what we usually prefer and has worked very well in the past for other projects. We used Flask-Caching library as a wrapper api.
Web Server and Proxy deployment
Both Gunicorn and uwsgi play well for deploying flask in production. We’ve been using Gunicorn for a lot of other projects and it is excellent for deploying django apps too. We chose it as a deployment web server, fronted by nginx proxy.
Push notifications, analytics, crash reporting, etc.
Most clients were already using a range of third party solutions for these and other app specific needs. I guess these are some of the well solved problems of mobile and fortunately there were no client requests for building custom stack components for these.
Data migration and throwing the final switch
This was quite tricky since all the existing app installs in the field had to work smoothly while we migrated from mongo to PostgreSQL. This had to be done in multiple steps. A few python scripts were good enough to import existing data into PostgreSQL. New app releases were made utilising the new endpints but during the average transition period of 3 months it was necessary that we support all the existing endpoints and not break existings apps so we ended up writing the old set of parse endpoints and make them compatible to our DB using a complex and somewhat ugly translation wrapper layer. The average downtime for apps was a couple of hours.
The migrations so far seems to have gone smoothly. We suspect that the endpoint translation layer would be very beneficial to a lot other users ( except the fact that it is tied to this particular stack) and we might decide to open source it if there is enough interest for it. Meanwhile feel free to share your stories of migration.