In an effort to increase transparency, the City of Minneapolis has started publishing data that it routinely collects during the course of day-to-day activities. This concept of “open data” has picked up steam as the utility of this data is discovered. By freely publishing this data, institutions create a tool for citizens to piece together disparate sources of information and participate in government in a more informed way.
On December 8, 2014, the City of Minneapolis launched their Open Data Portal, providing online access to city data in order to comply with the Open Data Policy which was passed over the summer. Council Member Andrew Johnson was behind this strong push for an Open Data Policy, bringing official city policy into the 21st century and providing a new resource for transparency activists.
But the open data portal has had problems since its December launch. City Pages called the portal “basically worthless” after documenting numerous problems with the site, and the city was forced to take down private information which was mistakenly made available on the portal. While the most glaring problems have since been resolved, other issues remain.
Platform
An open data portal is a site that provides users with tools to analyze and manipulate data sets. Analysis can include interactive maps of data; imagine a map of Minneapolis comparing neighborhood property values to police-response times. An open data portal should also include the ability to download data sets for offline analysis and have an application programming interface (API). An API allows software developers to create apps which link directly to data and update in realtime.
Socrata is the leader in developing open data portals for city government. Their clients include other large cities such as Austin, Baltimore, Chicago, and New York City. Socrata makes a slick product with plenty of features: the ability to create maps, combine datasets, and share information. They also provide extensive Getting Started guides which allow new users to easily make use of the data. This is important because while some users might be highly tech-savvy (application developers, journalists), it’s important that average citizens can easily use and share this newly available government data. The main competition to Socrata is the open-source CKAN. The software engineers who develop CKAN are available to help set up and deploy their software, but third parties are also available to maintain and support the code; this prevents a client from being forever tied to a single, proprietary vendor. CKAN is currently deployed at organizations including the city of Denver, Minnesota Geospatial Commons, and as the U.S. government’s official data portal.
Another company that offers open data services is ESRI. ESRI has a strong background working specifically with GIS (geographic) data, rather than open data generally. ESRI currently has data portals deployed in Boston, Houston (GIS data only), and Tampa.
Screenshot of current Minneapolis Open Data Portal
Process
There was no bidding process for the creation of Minneapolis’ current data portal. Because of their existing contract with the City, ESRI was granted the right to build the portal. This existing contract means ESRI has not billed the City for the creation of the portal.
While getting a *free* open data portal is an important component of the City’s decision, it does not excuse their unwillingness to invite proposals from competing products. Familiarity with a particular vendor can lead to some biases, and ignoring the offerings of other leading vendors is a failure to objectively shop around. Neither Socrata nor any CKAN vendor was asked to provide a bid, despite both vendors possessing arguably superior products.
Price is clearly a factor in this decision. In 2011, Austin, Texas signed a contract with Socrata which cost the City $3,000 a month after an initial $30,000 startup cost (see Exhibit B). A more complete list of contracts with Socrata can be found here. When asked for documents relating to ESRI and the creation of this portal, I was sent a contract with the City of Minneapolis (this contract has since been extended).
Outreach and Community
There is an existing community in the Twin Cities with an interest in open data. As a strong advocate for open data, Open Twin Cities has been raising awareness for the ways open data can be used to address civic problems. I reached out to several people who are involved locally with the open data movement, and could not find anyone who had been consulted by the City regarding ESRI’s selection.
Considering the strong relationships that Open Twin Cities has forged with both the community and local governments, the lack of outreach is puzzling. While sometimes the cost of these projects is an overriding concern, it would make sense to engage members of the open data community in order to build a portal that is as responsive and useful as possible.
When it comes to seeking community input, government institutions often choose not to bother. Outreach takes effort, and the resulting feedback can make the process more complex, messy, and confusing. This lack of two-way communication is unfortunate, given that many civic hackers volunteer to work on projects that benefit local communities and governments. It only makes sense for the City to solicit feedback from this highly-motivated community and ask “how can Minneapolis better empower open data advocates?”
Screenshot of OMG Transit, a real-time bus-tracking application. OMG Transit started at a hackathon and is possible thanks to Metro Transit’s open APIs.
Groups like Code For America have demonstrated how outreach can lead to better results with open data. CFA has helped to build dozens of apps for cities: an app that allows parents to track the location of their child’s school bus; an app that updates residents on the current status of vacant lots in a neighborhood; and an app that helps business owners navigate the permit process. None of those could have been created without access to useful data sets, and access to better data sets is unlikely without an ongoing dialogue between the City, local interest groups, and the developer community.
Data Sharing
Prior to implementing the Open Data Policy, Minneapolis took steps to utilize data analysis throughout various departments. In 2013, the City signed a $2.8 million dollar deal with IBM to create a data analysis tool for employees. Privacy concerns were raised when some aspects of this IBM “Intelligent Operations Platform” were not shared with the public (though a partial list of capabilities and techniques can be found here).
In a blog post for IBM’s Smarter Cities blog, Minneapolis Chief Information Officer Otto Doll claims the IBM platform is tracking over 1,250 metrics across the city. If the data that fuels this IBM project were shared with the local developer community, the potential applications would be numerous. Privacy issues could also be accurately assessed if the scope of the City’s data sets were subject to public scrutiny. Instead, these valuable metrics exist only within IBM’s custom platform which is inaccessible to citizens.
Screenshot of IBM’s Intelligent Operations Center. For an image displaying more capabilities, see the Star Tribune story.
Minneapolis has been previously criticized for making crime data difficult to work with. In 2014, the City partnered with Bair Analytics to create an online portal where crime data could be mapped. The City created an API which allows Bair Analytics access to daily updates of crime data, but this data is not shared with any other third parties. (Bair Analytics also would not allow any of “their” data be accessed, distributed or used in a commercial publication.)
When it comes to data analysis, City employees have access to useful data sets and powerful tools made possible by IBM, but average citizens only have access to a limited subset of inferior and potentially stale data. Even this data is difficult to obtain, and requires a willingness to spend money acquiring documents within the public domain. These fees can usually only be justified by an entity looking to make a profit.
Continued Problems
Minneapolis’ portal is missing some very basic user interface features. Getting dumped into a screen with a bunch of data sets is confusing and intimidating, especially for users inexperienced with modern data analysis. The website offers no instructions or helpful ideas on how to get started. Compare that to the data portal in Austin, Texas, which includes a helpful Getting Started wiki and a short FAQ.
Screenshot of Socrata-built data portal for Austin, TX
There is also nowhere in the Minneapolis data portal to provide feedback, ask for datasets, or to report bugs. These basic feedback mechanisms are necessary for any online platform which must be responsive to user needs. Even the most well-intentioned developers can’t predict how users will interact with a site without user feedback.
The portal currently hosts 53 individual data sets, though making separate datasets for things like “Fires Confirmed 2013″ and “Fires Confirmed 2014″ effectively inflates that number somewhat. (Incidentally, those two data sets contain the exact same data, but as previously mentioned, it’s not obvious where to report this error.) It’s not clear when or if additional data sets will be online.
None of the existing data sets appear to be frequently updated, eliminating the possibility of developers building helpful real-time apps. Data which could illuminate citywide issues, such as graphics visualizing Minneapolis towing data or the location of bus shelters, are still inaccessible without making a data request (the bus shelter data belongs to Metro Transit, but would obviously be helpful if it were listed in the Minneapolis data portal).
A Way Forward
I spoke at length with Minneapolis Council Member Andrew Johnson about the data portal, and he expressed several reasons for optimism. He feels that the data portal will help foster a new culture of openness and transparency in city government. Currently, citizens who make data requests are sometimes viewed as muckraking adversaries, and this portal could help change that perception.
Johnson also mentioned that it can take years for the culture to shift, and that this is a step in the right direction. He noted that there will be an annual compliance report regarding data sets for city departments to ensure that goals are being met. The city is also currently undergoing several IT-related upgrades (in police records reporting, land management, and legislative management) due to be completed over the next couple of years, which may contain data that can be easily integrated into the portal.
The City of Minneapolis is not permanently committed to ESRI, and could easily move to another vendor if necessary. With the data portal still in its infancy, this drastic step is not immediately necessary. But there is clearly room for improvement, and a good first step is to foster relationships between city departments, stakeholders, and developers.
Streets.mn is a non-profit and is volunteer run. We rely on your support to keep the servers running. If you value what you read, please consider becoming a member.