The rains in Mumbai last week have once again showed why BMC and all its citizens have miles to go before we can truly call this a well managed city. This blog post is not an attempt to finger point but a sincere appeal targeting all like-minded people to join together in creating a series of applications that truly utilize digital services in delivering contextual and timely information about our Monsoon.
Source : Getty Images
This is a long blog post, that goes into various areas. At the end of it all, I hope we developers all come together to do something about it. Let the story begin …
Monsoon App Downpour – Challenge Post
First a bit of history about why I am so interested in this. Last year, the online challenge site, ChallengePost organized a series of contests that invited developers to build out solutions to address key urban problems. This challenge was in collaboration with Ford Motors, where you were encouraged to integrate OpenXC standards into your applications.
Let us keep the technicalities aside for a moment. The key is to take a look at the clear requirement that was put in the challenge. I encourage you to read the first 4-5 paragraphs of this page. Sounds familiar to us Mumbaikars, doesn’t it ? The contest was on from mid-July to mid-October.
There were several submissions and a few winners. There were notable judges too as the home page indicates. I was one of the winners of the contest and did go to some extent to provide high tide, traffic, heavy rainfall information to users of my Web and Android application that I submitted in that contest. I am pretty certain not many folks of heard of my application and the other applications that were placed higher than me in the winners category and which I do believe were better and address key information that Mumbaikars would need.
Note : If you hit my application url, you will get a 404 and this is because the Web Services were brought down and I did not have a choice but to disable the apps.
The key thing that I want to point out is that such contests / hackathons should clearly have been organized with the blessings of our local government bodies and that organizations/individuals should come and participate more in such contests. You may not be able to code but you do have great ideas that could help us. More on that later.
What made my application possible? The MCGM Monsoon Web site and its Web Services. One may complain about them but to be fair to them they had RESTful Web Services that I was able to poke into and integrate into my application. They gave me information on Tides, Rainfall across 30-40 weather stations that they have setup across the city, traffic alerts and more.
What is coming next is a clear reminder why my hopes were crushed via unwittingly by the development team that builds out the MCGM Web Services.
Breakdown of MCGM Monsoon Web Services page
I submitted my application in October 2014 and the results gave me the confidence to build out more features and publicize the application to a wide variety of users.
And guess what happens next? The Web Services are no longer available. Game over ! In one sweeping shot, my applications and intent to take this application to the next level were gone! It is easy to put the blame on BMC but years of working with APIs have given me the common sense that there were no agreements in place. BMC did not make any promise to continue hosting this service.
I thought this was a temporary thing and that maybe they would bring up the Web Services in and around monsoon time. I was even harboring thoughts that there would be richer information in the APIs that I could make use of.
But that did not happen and I took down my website to avoid my own embarrassment. June 19th brought all these memories back again and made me start digging into what is going on the MCGM Monsoon Website. Are they not interested in providing information, data, services that developers like us could use to bring our applications that could help all of us.
It led me down an interesting path, each of which I will share. Let’s move ahead.
Current BMC Disaster Manager Mobile Apps + Websites + Social Media Accounts
A few years back with much fanfare, BMC released a site called www.mumbaimonsoon.in. This site promised to do the following:
“Provide information on rainfall that collected information from 30-40 weather stations that were set across the city. This information was supposed to be updated every 15 minutes. In addition to that, BMC in coordination with traffic police, railways and airport authorities was to provide information about traffic diversions, train and flight disruptions”.
And if you have already clicked the www.mumbaimonsoon.in in a hurry (I don’t blame you!) — you were presented with a domain page that shows that the site is down. And what is shocking in this whole process:
BMC did not even bother to renew the domain. They just let it expire.
A Whois search on the domain shows the owner information as given below:
The owner has put up the domain name for sale at $2000.
There cannot be an excuse for this. The site was announced in 2012 and in 2014-2015 – it was gone with the domain not being renewed too.
Let’s move on to the next level of apathy that is visible in its digital services team. Twitter was by far, a very useful service during these days to get information on what is going on. You would expect that they would be active on that and I reproduce below the following. Be your own judge!
How many of you know about the Disaster Management Twitter account. Look at the following:
1) The profile is not updated and it still references the now unavailable mumbaimonsoon.in
2) The last Tweet was in October 2012.
3) I am not going to even debate why High Tide alert information is essential in the month of October, but that is another discussion.
I found the following Twitter account too, but I really want to believe that this is not the official account. Please tell me that.
So is everything lost in that case? Shall we pack up and go home? The answer is a clear No !
There is an updated website (more on this crazy renaming later and how it broke multiple applications including mine in the next section). But for now, there is a shiny website that provides this information. How long this lasts is anyone’s guess but let’s be confident that it does.
GDMA (Greater Mumbai Disaster Management Authority) Web site
This site was originally available at monsoon.mcgm.gov.in but that site is no longer available.
http://dm.mcgm.gov.in:9080/gmdma/
A screenshot is shown below:
Don’t even ask me about port 9080 and all that. Let’s leave that aside for now.
Key things to note:
On the right navigation menu, you will find 2 important links: Weather and Tide Info and Traffic Diversions.
Click on Traffic Diversions to get notified about any Rail / Road / Air issues. At the time of writing, I was greeted with this page:
There was some diversions for Road and the details are given above. Some progress definitely.
The Weather and Tide information is more exciting and which was there at the older website that they simply changed the name and broke us all. It gives the following page (http://dm.mcgm.gov.in:8080/RESTFulWS/newestindex.html)
The website is quite useful and you can see the rainfall across various time intervals in the area of your choice. This is the result I believe of various weather stations that collect rainfall information. You can see the high tide information too.
A few questions/points (that developers can relate to!):
Check the page name : newestindex.html — what the hell is that?
By the way, index.html page is also available. Try it, it does not work!
Is there really a team developing this ?
Any QA check ?
Anyone who approves sane development standards/processes?
Who developed this website ….anyone knows?
Is the site responsive aka works on various form factors (desktop , mobile browser)? You tell me!
Anyways, answers to these questions do not matter anymore.
What about mobile apps? Yes – there are mobile apps too!
Android App
Here is the link to the Play Store.
Interesting to see the App Id as : com.abel.mmv2 …hmmm? A news report here shows that the site is being developed by Abel Software and Consultants Pvt. Ltd, based out of Mulund, Mumbai. It is the same company that it says developed mumbaimonsoon.in. Sure – that makes complete sense now.
I have only one suggestion to them. At least come up with a proper application id for your Android application.
I have many other questions vis-a-vis who owns the code for this application? Where is it stored? Under whose account? Backups? …. never mind. Let’s move on. Well I am not done yet!
Take a look at the website again :
At the bottom you will find the following (I have put the image here):
A small but important thing. Take a look at the statement “Theme by Abelsoftcons”. This clearly shows or claims that the theme was developed by Abelsoftcons. Click and see where the link goes. I do not have anything else to say here. Atleast attribute it to the right owner guys! That is the least you can do!
iOS App
The link on the main web site : https://itunes.apple.com/us/app/mumbai-monsoon/id885404768?ls=1&mt=8 leads to nowhere. The App is not available. I tried searching it in both Indian and the US App Store and it came up short. More informed users could tell me if they are able to find the application. If not, this is a serious breach which both BMC and the Application Development Company responsible for this app needs to answer.
Other platforms
Tough luck. Use the website ! And good luck using the website in your mobile browser.
MCGM Monsoon Web Services – Updated Documentation
This is a section for developers – so please bear with me.
The first thing that I did on discovering the new site was to check if the old web services that provided information on traffic diversions, rainfall data and high tide information existed in some form or the other.
This is how I went about figuring this one out (It does not take an expert to get to this). The steps were simple:
Launch Chrome browser (other browsers have enough ammo too but I love Chrome Dev Tools!)
Go to http://dm.mcgm.gov.in:8080/RESTFulWS/newestindex.html
Go to Chrome Dev Tools (F12 … all hail the keyboard shortcuts!)
A few JavaScript errors will greet you …. let’s pretend we are blind to those for our sanity!
Click “Network” tab in Dev Tools and refresh the page again.
See the Web Services calls that are getting made as given below:
We can debate forever the shocking number of calls needed. The developers need to consider aggregated calls, cache local data and more. But that is a discussion for another day. We need to have a starting point somewhere right …?
I had my previous private documentation on how these worked and I am reproducing some of that for you today, so that you can get started with the REST APIs that they have made available:
Get Tide Information
On any particular day (date) – you can get the high tide information via the following:
HTTP Method : GET
HTTP Endpoint:
http://dm.mcgm.gov.in:8080/RESTFulWS/rest/MumbaiMonsoonService/getsptidesdata
Go ahead and click it. It gives a nice JSON response as given below:
Few things to note here:
The Unit of Measure is metres (m).
hth stands for high tide height
lth stands for low tide height
htt stands for high tide time
ltt stands for low tide time
At what point this information is updated, I have no clue. I suggest you run a regular CRON Job and then update your application data accordingly.
Get Diversion Information
You can get the regular traffic diversions (Rail, Road and Air) via the following:
HTTP Method : GET
HTTP Endpoint: http://dm.mcgm.gov.in:8080/RESTFulWS/rest/MumbaiMonsoonService/getsptrafficdiversiondata
Go ahead and click it. It gives a nice JSON response as given below:
The response JSON is intuitive enough. I guess having webhooks that can update you in case of changes is a long shot for the development team here. You will need to poll this regularly for any changes. I suggest you providing this service to other apps – should be very useful.
Some other data points missing here is a time stamp for this or indicating the date/time when the problem has started.
Get Rainfall information
This one is a bit tricky but let me explain this.
First you need to know about various locations for which the data is available (there are several across the city for which the rainfall data/weather data is collected). These are locations and then there are sub-locations within that.
To get locations, you can execute the following call:
HTTP Method : GET
HTTP URL : http://dm.mcgm.gov.in:8080/RESTFulWS/rest/MumbaiMonsoonService/getsplocationsdata
You will get the following information (abbreviated screenshot is below):
The important thing to note is the locationidData field. This id field will be useful to retrieve the Weather information for that location id and also the Rainfall information for that location id.
You can also get an information on all sub-locations as given below:
HTTP Method : GET
HTTP URL: http://dm.mcgm.gov.in:8080/RESTFulWS/rest/MumbaiMonsoonService/getspsearchlocationdata
You will get the following information (abbreviated screenshot is below):
The sub-locations are definitely a lot more in number and seems incomplete in terms of the attribute values. But this is what you need to do, if you want to work with sub-locations and not the main locations.
1) Given a sub-location id i.e. idData field, you need to call the following method
HTTP Method : POST
HTTP URL : http://dm.mcgm.gov.in:8080/RESTFulWS/rest/MumbaiMonsoonService/getspsublocationlocationiddata
The POST FORM data is shown below:
{“locationidsubselected”:”67″}
where 67 is the sub location id (idData attribute value).
This will give you a response as follows:
There you go. You have the locationidData value i.e. 33. You can now call the next 2 methods i.e. Get Location Rainfall and Get Weather Data using the location id value that you now have.
Get Location Rainfall
You need the location id for this. Once you have that, execute the following call:
HTTP Method : POST
HTTP URL:
The POST FORM data is shown below:
{“locationidselected”:”25″}
For e.g. 25 is the locationid for the location that you want information on.
The data returned is as follows:
The data returned is in mm. And there are 6 time intervals in which it is returned. For e.g. 15min means rainfall in the last 15 minutes, 1 hour means rainfall in the last 1 hour. Neat !
This data itself could be used for interesting alerts where you could warn users that a particular location is witnessing very heavy rainfall.
Get Weather Data
You need the location id for this. Once you have that, execute the following call:
HTTP Method : POST
HTTP URL : http://dm.mcgm.gov.in:8080/RESTFulWS/rest/MumbaiMonsoonService/getspweatherdata
The POST FORM data is shown below:
{“locationidselected”:”25″}
For e.g. 25 is the locationid for the location that you want information on.
The data returned is as follows:
The temperature is in Celcius. The humidityData is in %. The wind Data is in kmph. The barData stands for pressure and is in Hpa.
There seem to be a lot more web service calls get through that could be interesting to your applications but I will leave that for another day. These basic calls should suffice here for you to make a lot of interesting applications.
MCGM Monsoon Web Services – Road Ahead and some recommendations
The current state of MCGM RESTful services that provide information is something that is far from optimal. In fact, I am bold enough to say that it is abysmal in its design/documentation and road ahead.
Here are my suggestions for whatever it is worth:
Base URL, standard Port Numbers and more
What sort of base URL is this? And port 9080. The API needs to standardize on that and not use these obscure naming conventions.
Introduce Versioning
Granted that your API will change as time passed on. New requirements/changes will come in which will requires that new data formats/capabilities be supported. What is likely to happen and which I fear is that conveniently the API team will change the API (Read that as break the API) and change their website to use the new API. The result is that all applications that are integrated with the API will break instantly, resulting in horrible user experiences and an immediate appearing of fly-by-night experts that will question what went wrong!
We must introduce versioning immediately in the API. And you cannot stop a version without adequate notice/information to its users to upgrade to a newer version.
Client Libraries
If you want to unleash the API on developers and allow them to iterate rapidly through their ideas for Monsoon Apps, you need to provide them client libraries in a variety of programming languages. Not REST APIs in their raw form. That is too much for most developers to even think about spending their time on.
Write client libraries that provide wrappers in popular programming languages like Java, Python, C# and more.
Documentation
Provide API documentation, sample code and a quick “Hello World” that can help demonstrate the information that the API provides in minutes. Any more than 10-15 minutes of the developer time in evaluating your API is as good as a dead API strategy.
API Quotas
You are providing a public API and there seems to be no information or protection vis-à-vis quotas. Perfect recipe for a disaster. Introduce Quotas that limit developers from playing around with your API. Document your API fair usage policy. Think about different API plans. You have to function and execute like an API provider if you want to be successful. There are companies providing API management and gateway solutions. Investigate into those to get a jumpstart.
There are a lot more recommendations for API Best Practices – but you get the gist of what I am trying to say here.
Call for Action
The above information should have been planned from day 1 in case the teams working on this were seriously interested. The technical project execution team should have the above deliverables ideally and a concerted effort should have been in place to make this documentation public and available to all.
Having said that, our life is like a train. Just forget the stations that have passed by and let’s look forward. You may have all the reasons to be sore with BMC, in fact I do. But screw all that and do your own bit. I have given you enough information now, use your offices and individual skills to take this to the next level.
Use the API information. Build out applications. Share it with everyone. Talk to me. Maybe we can do something together. Meetup groups could organize a hackathon. Maybe you could hold a hackathon in your own organization. The possibilities are endless and you are an equal partner.
I would like to hear more from you. Write to me at romin.irani@mindstormsoftware.com