Stepping into the world of mobile development can be a daunting task. From learning new platforms to understanding the challenges of designing for a multi-device world, there are a lot of concepts involved in mobile development that will probably be new to you. Some things will be familiar, for example, in this article we’ll be talking about data – having access, making sure that access is fast and responsive for users, and displaying it on a variety of devices. But we’re not inside the corporate network anymore and local access speed is between the user and their carrier (I’m looking at you and your ’4G’, AT&T!) so having a fast and responsive backend like the one provided in the Telerik Platform essentially solves our problem, right?
Not entirely. When talking about ‘data’ – the pure bits and bytes we shuffle back and forth to make the magic happen in our applications – we also have to discuss bandwidth. You never really worry about it on your LAN and your home internet likely has a high enough bandwidth cap that you don’t know it exists. More often than not cellular plans are being nicer about how much they allot you every month, but, especially in places where data plans aren’t wild and free, people are very cognizant of the data limits on their plans.
As it turns out, data management in terms of bandwidth allocation is still a huge deal. It’s why we like JSON so much better than XML and the same reason we can’t stand the burden ViewState can bring to an ASP.Net application. Moving data back and forth is necessary but expensive.
In this article, we’re going to explore the intricacies of intelligent data management via a new Android application that uses Telerik Backend Services for our data management. But before we can start comparing and contrasting methods for handling data, we need to build ourselves an application, and before that, we setup our data.
Before We Code, We Data
To explore the concept of data management we need some data to work with. Stepping into my Telerik Backend Services account I went ahead and started a new Workspace for this project as well as a new Backend Services instance:
Within my new instance of Backend Services I have access to all sorts of goodies like reporting on database usage, push notifications, file space allocation, and, importantly for our purposes, bandwidth. Right now it looks a little boring since all the numbers are at zero, but this portal will serve as a great resource after moving forward with publishing our application.
Now that we have a Backend Services instance we can generate some new Types to use within our application. While we want things simple, we do want to make sure our data is beefy enough to make some decent comparisons later in the article, so we’ll start off with a DataTag:
Followed by a DataItem:
For reference, we did the Tag first (even though it is a more auxiliary type) so that we could create the multiple relation reference ‘Tags’ that you see above. Otherwise it would require that we make the DataItem, make the DataTag and then go back into DataItem to create the relation field. I’m also going to make a bunch of fake records and tags, and since the Tags property is really a list of the UUID for DataTag items, I can make some associations to ensure each DataItem has a few tags associated. The end result for DataItems looks something like this:
If this is your first time here, hop over to the Downloads section (link is in the platform menu bar) and grab the Android SDK. We’ll need that. Also, since we’re still here, navigate over to the API Keys link on the sidebar and copy your API Key. We’ll need that too, once we get into our application so we can identify with Backend Services.
Now that we have a live, scalable, testable, analytics-capable, and transferrable-to-other-platforms backend for our application, we need to create something to show off our data.
Let’s Build an Android Application
Maybe it is my J2ME roots, but I like Eclipse so we’re going to use that to start our application. I went ahead and started a fresh workspace and created a new Android Application Project like so:
Everything else is default, so at the end of the wizard I’m left with a new project that has a single, basically blank MainActivity to work with. Now we’ll want to get our application ready to use Backend Services.
We’ve got a documentation page that covers this, but I’ll bullet out the quick details in case you’re all excited to get started and don’t have time for a webpage to load:
Download and extract the SDK, the file we need will be AndroidSDK.jar;
Right-click on your project in Package Explorer and select Properties;
On the left-hand tree view, select Java Build Path;
Under Libraries, click Add External Jar and select the aforementioned AndroidSDK.jar;
Under Order and Export, ensure AndroidSDK.jar is checked.
You should be good to go. As a good exercise, add a new class to your solution named DataTag. Once your basic class is up in Eclipse, extend your class from DataItem. After your [auto-import of choice] method is complete you should now have the following import statement atop your class file:
If not, ensure your project has built correctly up until this point or that there isn’t something else wrong.
Now you may just be thinking: “Tell me more about this DataItem.” If so, you’re in luck because DataItem is basically the base object for any Backend Services type. Along with DataItem you get a bunch of those properties that auto-populated in our DataTag and DataItem like Id, CreatedAt, etc. Also, by default a DataItem knows how to interact with the Backend Services SDK, whereas a base class object wouldn’t know the SDK from any other jar.
Since we have a DataTag (extending DataItem) class just sitting around, let’s put it to use. I like to be ultra-explicit when coding, so while using convention naming our fields to correspond to our server classes should auto-map to Backend Services properties, why take the chance? According to the object-model documentation, we should set both a ServerType annotation on our class as well as ServerProperty annotations on each field we intend to persist, so when all is said and done our DataTag.java class ends up looking like this:
Not too much code for all the functionality we are getting for free from DataItem (and Backend Services).
We want to do something similar for making our DataItem, however since we named our object DataItem and the base class is DataItem, we get to conveniently take advantage of the annotations provided by Backend Services to re-map our class to a different named local class. I’ll provide the short version here with only the useful bits (since you’ve already seen the ServerProperty tag in action up above):
Now our ServerDataItem and DataItem won’t conflict in naming and we’ve got a place to store those related Tag UUIDs that we saw in our Backend Services screenshot.
Moving Data, AndroidSDK Style
Just to summarize, up until this point we have built both a fully-functional Backend Services instance complete with two custom data types as well as laid the plumbing for interacting with that data from our Android application. To prove this all works we’re now going to move some data and display it in the Android app by running a query to get all DataItem objects from our server.
If you’re an Android developer, you know that our next step may just involve the AndroidManifest. This is because in Android we have to explicitly state what rights an application has, be it something simple like network access or something requiring more trust, like enabling access to external storage. In our case we want to set two permissions for both Internet and Access_Network_State, since technically before polling remote resources you should check for network (but I won’t cover that and instead let you add it for extra credit). Here’s the two lines in case you just want to copy/paste them into your manifest:
With that set we can open up our MainActivity.java class. We technically also have a layout file (res/layout/activity_main.xml), but for right now we’re aren’t concerned about UI.
Within MainActivity, locate the onCreate method. After the content view is set we can now load up our EverliveApp instance and pull some results down the wire. Much like in the documentation we are going to want to instantiate an EverliveApp instance and then retrieve some data. When the call returns, we check for success and then we’ll do a simple loop to log out the title value of each item to prove we’re getting the full set. The resulting onCreate looks something like this:
If we run this in our emulator, our resulting Log content should look something like the screenshot below:
Say what? This is where Android is fun – it enforces some best practices, this time it ends up filtered down to us as an EverliveException. The gist behind this error is that networking calls can take time to return and we don’t ever want UI waiting on a network call to process/render, so instead we’ll want to make this an asynchronous call that won’t block the UI but will throw us a callback when it hears back from the server.
For this we have two options. One of them is the RequestResultCallBack, which we cover in the docs, so I’ll show you an alternative that I actually prefer. Rather than using the .executeAsync(*) method in the above referenced documentation we’re instead going to use the same executeSync method seen above, but this time within an asynchronous method. We can then launch this method from onCreate and have the benefit of using the perfectly valid code we already wrote but in an asynchronous manner.
To accomplish this we’ll use an AsyncTask<Params,Progress,Result>. AsyncTask has a method called doInBackground that must be implemented to handle our task. If we passed in any Params, they would be present for use in this method and this method will be expected to return a type of the original Result parameter. Since we don’t need params and won’t report progress, it’s Void all around, giving us the following AsyncTask implementation (note: I just dropped this right into the MainActivity class file to make for better readability, but you’ll probably want it in an external class since it can send result back via a broadcast for better separation-of-concerns):
This is the same code, except now we can add this one line to our onCreate method:
And we now see the right data in our Log window:
Done! But what was that about data management?
Intelligent Data Management Starts With Data
Remember all that talk about data, XML, and JSON back in the beginning of this article? This is where we get to why that matters and what you can do to be more responsible with your data.
To perform the above network call, we made a request through an EverliveApp contained in the AndroidSDK jar. These are really fancy terms that really mean an Android wrapper for the Backend Services REST platform. Since the backend is well RESTed (har har har!) it is smart and returns a full JSON representation of our data type, meaning that the above call we used to display four titles in our log window actually returned a more complex response.
Here’s a look at the full (formatted) raw JSON response:
I’m no math wizard, but a quick check tells me that the length of just the JSON returned is 2183 characters. Let’s live in a world where everything is UTF-8 and say this is 2183 bytes, or, rounded closely, we’ll put it at 2KB per request for four of our items. If one user does this per day it is no problem, since that will only equal about 60KB/month. Knowing our default Backend Services implementation gives us 5,120MB/month, or 5,242,880KB/month, each user is only using a tiny percentage of the overall bandwidth and at this rate we will top out our monthly bandwidth when we reach around 87K users. Of course, this doesn’t take into account the size of other header data, auxiliary data like images we’ll likely transfer, etc.
Waking up from the world of dreams, if we’re lucky most users won’t only access our data once a day. And if our application is going to take off, it’s going to do a lot more than display data in the log window. But this very narrow view does give us numbers we can work with to explore a little further. Before we do that, I want to call your attention to some of what we have in our JSON data and how that further complicates things.
Tags, for Free?
While the JSON output was pretty verbose with all the DataItem data alongside our custom fields, it also included the related fields we set up before:
This provides us with the UUID/Id field for each tag object, but not the actual DataTag itself. This means we have to do another call for tags – or we can include tags in our original request itself! Let’s see how this works by using the new Expander object. Courtesy of the latest version of AndroidSDK, we now have an ExpandDefinition object that allows us to replace the List<UUID> with something more user-friendly – like the actual tags. This requires a bit of extra code in our ServerDataItem to hold the new objects:
Just to explain, because of how Android handles objects, we need to pass in the strong class name for our object, so full namespace plus object name. With this in place, within our AsyncTask, we’re going to add a quick ExpandDefinition to translate the Tags relation field on the server to actual DataTag objects:
That fits right into our original request:
Now our objects return back, fully populated with the entire DataTag object for each related Tag. Yup, the entire one, including fields that we inherit from DataItem like ModifiedBy, CreatedBy, CreatedAt, etc. However, most of these we won’t use, don’t care about, and aren’t relevant to our application.
This gives me a good chance to introduce the FieldsDefinition object, which will allow us to set an included fields ArrayList<String> and will only return the fields we requested. Now instead of the full DataTag object, the SDK will create a new object of the defined type and populate the fields we originally requested, leaving the rest empty. This looks something like the following:
To make this a little easier to consume, I went ahead and threw together a jsFiddle so you can use jQuery along with the API to check your own data. It’s pretty straightforward, just replace the keyType (if not MasterKey), keyValue, apiKey, and modify the urlEnd. Expand expressions as needed if you used different naming than in the examples above. This way you can pop open the console on your favorite browser and see the data results for yourself.
Now that you have the same tools that I do, let’s look at some numbers. Running the following four queries resulted in these corresponding sizes:
Bytes
~KB
Monthly
Users @ Bandwidth Cap
No Expand
2183
2.1
63
83k
Regular Expand
4813
4.7
141
37k
Defined Fields (2)
2801
2.7
81
64k
Defined Fields (2), Reject Id
2449
2.4
72
72k
Those are some dramatically different user counts, although it assumes our unrealistic scenario whereby X users log in once daily and only pull up a single request for four records with their related tags. Realistically those user numbers would be a lot lower when you consider logging in (if implemented), other header data, having more than four records / two tags per record, or users using your app more than once daily.
Shiny != Better
While it isn’t a normal practice to highlight a new feature then show data that says “don’t use it”, this article is all about making intelligent decisions with how we handle our data. The same should be said for all development we work on, since I’m sure more than a few of you out there have been burned by using something cool and new which ended up having serious flaws or shortcomings down the road.
There are definitely cases when it would make sense to pull full expands, like grabbing financial data and all related account records in cases where someone is looking at an entire account. There are also tons of scenarios in which performing a more limited expand to grab an order and a few data points for the related items (but not the full items, though users can drill down into the full item which would fire off a single request as needed) would make sense. But just because we have this expand doesn’t mean we necessarily should use it, especially in a world where an extra KB per user per request can really add up and end up costing us on our bandwidth bill down the road.
Just as an example, I pulled the full list of Tags from the server and the data size was 1855 bytes. Rather than running expands, I can do a single request for the Tags collection, store it locally, and then, when I return individual data items, I can use the ArrayList<UUID> of Tag Ids to pull necessary data into my views from the local collection. This eliminates the extra network call altogether. I envision this would also include some test to check whether the server tag collection has been updated, but that is a topic for another article.
Before We Go
In making the shift to mobile, we are moving from an enterprise, inside-the-wall world where returning twenty records with metadata wasn’t a heavy burden (even if all we needed was an address) to a world of development where trimming an extra KB or two off our requests can have a significant impact on user experience (where connections aren’t as good) and our bottom line (when bandwidth usage gets expensive).
Understanding how to hook our data up to a UI is just as important as understanding how data is getting from our hosting center to our users and how we can optimize that experience for everybody. With this comes the further understanding that features have to be evaluated on a case-by-case basis and not by a gauge of “People are loving this on Twitter! #ShinyNewTech.”
By using Telerik Backend Services and the rest of the Telerik Platform, we take care of problems like data storage, global availability, metrics tracking and reporting, not to mention goodies like testing, analytics, and local app distribution and management. When using this with an Android mobile application we can quickly access single records, lists of data, and related data, all with a few quick API calls. With this comes the responsibility of managing our data and being cognizant of the bandwidth that we’re using to create a compelling user experience, since at the end of the day a few KB’s really do matter.
Github Link: https://github.com/EvanHutnick/IntelligentDataManagement
Note: Due to the nature of how Telerik Backend Services works, we can’t provide a Backend Services link/implementation without providing access to my account. Not happening. If you want to use the exact code from the data dumps we were doing to test bandwidth usage it’s included in the repository in a “rawdata.txt” file. Dev-magic that up and you can populate and then test objects with Backend Services using the same data we used in the article (ids will change, that’s about it).