Mobile application ecosystems — let’s count Android and iOS here — are unbelievably dynamic, but they also suffer from both software and hardware fragmentation. This is especially true for Android, but fragmentation also exists in the iOS ecosystem1, as experienced with the rollout of iOS 8. As the latest version of iOS was released, many existing apps were made clumsy on updated devices.
Even the new iPhone 6 and iPhone 6 Plus have had not-so-typical issues2 for Apple devices. In addition, a significant proportion of users with older devices have very few options: essentially, buy new hardware (i.e. a new device) to get everything working well.
In the Android world, things are different. As OEMs launch new devices, software updates and customizations for their devices, application and game developers get serious headaches3 trying to keep their latest products up to snuff and fully compatible with all possible device variants. Making a certain app or game work only on high-end devices is out of the question, though. Why would a developer want to miss out on a significant chunk of potential users?
4
Automation enables simultaneous testing on hundreds of real devices. (Image credit: Testdroid6854403733282517135) (View large version6)
Professional automated testing software is a solution to a common problem: how to produce high-quality, robust and reliable software with the ever-growing complexity of technology and under massive competitive pressure. Automated software testing is a cost-effective solution to this problem. Not to mention, it provides three business benefits:
increased testing efficiency,
increased testing effectiveness,
faster time to market.
This article walks through a sample use case for test automation and provides a downloadable example to get you started. Also, we’ll focus on different aspects of mobile test automation and explain how this relatively new yet popular topic can help mobile app and game developers to build better, more robust products for consumers. With the advanced example later in the article, we’ll show how image recognition can be used to test mobile games; specifically, we’ll run Appium’s test automation7 framework against Supercell’s Clash of Clan8 game to illustrate how image recognition can be built into the test-automation process.
Test Automation Is Perfect For Mobile App Development
Developing mobile applications is very different from developing PC software or even embedded software. Mobile development is meant to be agile9, and a lot of great tools and practices have been developed for that agility. However, doing something manually — such as testing an app — is never agile, which is why test automation has shown tremendous growth among app and game developers, speeding up their doings and yielding robust and better results.
To achieve compatibility between users, devices and the market, including test automation as a part of the agile development process10 is typical. Fortunately, a lot of tools are available, and test automation is a perfect fit for this process. For example, let’s say your typical development sprint is two weeks. You have daily standups and a lot of scrum activities, and you own internal policies that gear development to the final product. Test automation offers a significant value-add by enabling testing to be done in parallel — for example, as nightly test sessions. By the next morning, the tests will have been finalized and the results of the latest regression will be ready for review. Fixing an issue earlier will save a lot of time and get developers to finalize the product sooner; most importantly, it cumulates to better quality11, with fewer bugs.
12
The value that test automation brings to the agile process. (Image credit: Testdroid6854403733282517135) (View large version14)
Test automation offers the possibility to test mobile apps instantly and effectively. Once tests have been automated, they can be executed quickly and repeatedly, again and again. In almost all cases, this is the most cost-effective method for regression testing15 of software products that have a long maintenance life. In fact, test automation of any mobile app is the best way to increase the effectiveness, efficiency and coverage of the testing process. The true benefit of automation comes not only from the repeatability of tests, but also from the ability to execute tests that probably could not even be performed manually.
Things To Consider When Automating Mobile App Testing
Let’s look at how test automation (as opposed to manual testing) can improve the development process, add value and speed up development.
Costs and Assets
Regardless of whether you go for manual or automated testing, you’ll need the following assets and resources (all of which cost money): time, people, infrastructure, tools and training. Depending on the size of the project and the application, test automation will quite obviously provide a good return on investment. For example, once test cases have been created, automated tests can be run over and over again at no additional cost, and these can be more rapidly completed than any manual tests. Automated software testing can reduce the time required to run repetitive tests from weeks to hours. This is a significant time-saving that translates directly into cost-saving.
16
Continous integration together with mobile app testing. (Image credit: Testdroid6854403733282517135) (View large version18)
Integrated App Development And Testing Cycle
Increasing efficiency and productivity with automation actually starts with adopting a new mindset. Software tests have to be repeated often during all development cycles to ensure the best possible quality of an application. Every time source code is modified19, software tests should be repeated. For each release, your software should be tested on all supported variants of operating systems and all configurations of hardware. Manually repeating these tests would be costly and time-consuming. For example, running comprehensive tests manually on all variants of Android and on actual devices would take a lot of time.
Tools and Technology: Test Automation Frameworks
To get the most out of your efforts and maximize testing coverage, select the most robust and most cross-platform20 method. That automated methods can be used both to validate requirements and to reduce costs through automated test-case generation is well known. However, the full automatization of large software entities also comes with a cost that many companies haven’t been ready to pay for. Historically, one of the reasons has been a concern over the lack of adequate integration with well-established development life cycles.
Test Coverage and Reusability: Open Standards Mean No Vendor Lock-In
Automated testing can increase the depth and scope of tests and significantly improve software quality. Lengthy and thorough tests — often not doable with manual testing — can be run automatically. Ideally, test cases should have full access to an application and test all aspects of it21 — memory contents, data tables, file contents and internal program states — to determine whether the product behaves as expected. Automated tests can easily execute thousands of complex test cases during every test run, providing coverage that is simply not possible with manual testing. Developers, freed from repetitive manual tests, will have more time to create new automated test cases and build more compelling features (and then more test cases).
22
(View large version23)
Improve Effectiveness and Finalize Sooner
In a nutshell, professional automated testing software is a solution to a common problem: how to produce high-quality, robust and reliable software with the ever-growing complexity of technology and under massive competitive pressure. Automated testing improves business results in three ways: greater testing efficiency, greater testing effectiveness and a shorter time to market.
Different Ways to Automate Mobile Testing
In general, there are three ways to automate the testing of mobile apps.
Handwritten Test Scripts
Typically, this is the best choice when you know what you’re doing and when you have programming-capable people doing the test scripts. Plenty of options are available for test automation frameworks, tools and integration — both commercial and open-source offerings. Having your engineers write all of the test scripts will take up some time and tie up resources, but it will get you exactly what you want: well-structured, thorough scripts that test precisely the aspects of your app that you want to test.
24
Manual versus automated testing for mobile apps. (Image credit: Testdroid6854403733282517135) (View large version26)
Record-Playback Approach
This approach is less error-prone because nothing needs to be written in code, but it is typically more limited in functionality. Tests can be quickly recorded and then played back again and again against different OS versions and device configurations. These tests focus on user interactions and user-driven activities. Some things might fall beyond a test’s scope (such as integration with other technologies and compatibility with other software).
Automatic Test Exercisers
Automatic test exercisers provide a great way to smoke-test applications. No specific tests are needed; rather, the focus is on testing user-interface logic (such as opening menus, clicking buttons, swiping and multi-gesture actions). Automatic test exercisers yield the least exact results but provide quick feedback on each iteration of an app.
Focus Areas In Testing For Mobile Apps And Games
27
Different ways to automate mobile app testing. (Image credit: Testdroid6854403733282517135) (View large version29)
User Interface and Functionality
A user interface and its overall functionality will directly affect how successful your app or game30 will be. These two things, which encompass visual appeal and gameplay, are the most important things to get right — and you must ensure that device fragmentation doesn’t break any of these. Various things in the UI need to be tested:
UI layouts and elements
Games especially are typically targeted at a high number of different screen resolutions and screen types. Regression testing should be done each and every time the UI’s layout changes to ensure that the game works.
Menu structure and functions
Testing menu structures, functionality and behavior can be automated with instrumentation and the help of different test-automation frameworks.
Screen orientation
Surprisingly, so many apps and games out there get this wrong. If a screen’s orientation changes during an interaction, for example, what happens? What is supposed to happen? Does the app or game work well in both landscape and portrait modes?
Screen resolution
A lot of screen resolutions exist, especially on Android, and auto-scaling will usually help developers. However, test your game across these resolutions to ensure that the graphics do not stretch.
Graphics Performance
Performance needs to be consistent across all device variants31 among your users. Because of this, test on as many real devices as possible. To determine how well your app or game responds to various levels of usage, including performance and battery usage, consider creating tests that last for hours. To determine whether your game runs effectively under a heavy load for a long time, run load (or stress) tests. These performance tests will measure, for example, how responsive your game is on real devices.
32
Performance testing of CPU load and memory consumption. (Image credit: Testdroid6854403733282517135) (View large version34)
Usability and User Experience (i.e. Good Entertainment)
Testing usability, navigation flow and user experience simply cannot be done on a desktop with a mouse and keyboard. So, forget emulators and use only real devices35. And to test how usable and entertaining your app is, consider these two important things:
User interaction and responsiveness
Testing performance is critical because this will make make or break the user experience. Performance lag, for example, is easy to expose with real devices.
Background events
Interruptions, battery consumption and the effect of battery chargers on overall performance and usage all have a significant impact on the user experience — and entertainment value.
Multi-User Features
Nowadays, multi-user support is common in both apps and games. Testing multi-player capabilities is important and is naturally more challenging, requiring real users to measure performance. A typical case is a game communicating with the back-end server. In this case, connectivity is essential, to synchronize the back end with devices that need to get information about the gameplay. You should test a ton of different scenarios, many of which could severely affect the game’s experience, resulting in negative feedback and the game being uninstalled by users.
Social Integration
36
Testing social integration in mobile apps. (Image credit: Testdroid6854403733282517135) (View large version38)
Integration with social networks is another important factor. Being able to share something across an ecosystem, with friends or just with oneself, is essential in many apps. Test this thoroughly with real Android and iOS devices, with different OS versions and different device configurations, to assess functionality and ease of use.
Security and Liabilities
Nearly all developers use some open-source components in their apps. This practice is widely accepted and highly recommended because it offloads the development of code for non-core functionality. However, identifying vulnerabilities and licensing restrictions with third-party code is often neglected by developers.
Breakdown: Android Test Automation Frameworks
39
Comparison of test automation frameworks. (Image credit: Testdroid6854403733282517135) (View large version41)
Robotium42 is an Android test automation framework that fully supports native and hybrid applications. Robotium makes it easy to write powerful and robust automatic black-box UI tests for Android applications. With the support of Robotium, test case developers can write function, system and user acceptance test scenarios, spanning multiple Android activities.
UIautomator5143, by Google, provides an efficient way to test UIs. It creates automated functional test cases that can be executed against apps on real Android devices and emulators. It includes a viewer, which is a GUI tool to scan and analyze the UI components of an Android app.
Espresso44, by Google, is a pretty new test automation framework that got open-sourced just last year, making it available for developers and testers to hammer out their UIs. Espresso has an API that is small, predictable, easy to learn and built on top of the Android instrumentation framework45. You can quickly write concise and reliable Android UI tests with it.
Calabash46 is a cross-platform test automation framework for Android and iOS native and hybrid applications. Calabash’s easy-to-understand syntax enables even non-technical people to create and execute automated acceptance tests for apps on both of these mobile platforms.
And then there’s Appium. OK, let’s get into this one.
Appium: Executing Tests On Real Devices On A Cloud Service
In a nutshell, Appium47 is a mobile test automation framework (and tool) for native, hybrid and mobile-web apps for iOS and Android. It uses JSONWireProtocol48 internally to interact with iOS and Android apps using Selenium’s WebDriver49.
In its architecture, Appium is an HTTP server written in Node.js that creates and handles multiple WebDriver sessions. Appium starts tests on the device and listens for commands from the main Appium server. It is almost the same as the Selenium server that gets HTTP requests from Selenium client libraries.
In fact, Appium is a pretty good choice for both apps and games because, in many cases, apps and games tend to be identical (or at least very similar) on both platforms, Android and iOS — and so the same test script can be applied to both. Another significant benefit of Appium is that users can write tests using their favorite development tools, environment and programming language, such as Java, Objective-C, JavaScript, PHP, Ruby, Python or C#, among many others.
Appium enables users to execute tests on mobile devices regardless of OS. This is possible because the Appium framework is basically a wrapper that translates Selenium’s WebDriver commands to UIAutomation50 (iOS), UIautomator5143 (Android, API level 17 or higher) or Selendroid52 (Android, API level 16 or lower) commands, depending on the device’s type.
For Android, this is how Appium compares to other test automation frameworks:
53
Family tree of Android test automation frameworks. (Image credit: Testdroid6854403733282517135) (View large version55)
Android test suites are based on JUnit56. In addition, Android provides an architecture and fully integrated testing capabilities with its standard tools57, which help developers to test at every level, from unit to framework. Android instrumentation58 is a set of control methods in the Android system. These methods control an Android component independently of its normal life cycle.
One of the best things about Appium is that, despite sounding architecturally complex, it actually isn’t — at all. For developers, it provides support for various programming languages, freedom from having to select tools, compatibility across the most important platforms (Android and iOS), freedom from having to install and configure devices to test and more.
If you are familiar with Selenium, then you’ve got Appium covered. An Appium test is pretty much the same as a Selenium test: They use the same WebDriver, and DesiredCapabilities59 is used the same way. Configuring an application to run on Appium has a lot of similarities to Selenium — for example, those DesiredCapabilities. We’ll configure a sample test later in this article.
Also, Appium includes a component called the inspector. This inspector enables a host of functionality — for example, showing all of those UI elements in the application and enabling basic recording and playback.
However, you may not need the inspector because everything can be done in the code. The example later provides all of the scripts and instructions step by step, without using it.
More information about Appium can be found in “Getting Started With Appium60.”
Setting Up The Environment
To get started, first download our Appium example61. The example is available in Python, Java and Ruby. Then, depending on which programming language you will be using, select the appropriate client library62. Appium’s documentation63 provides all instructions, help and additional material.
Mac OS X and Linux
Ensure that Python 2.7.x or later is installed (it’s preinstalled on Mac OS X):
Install Python if it’s not already (Linux only):
Check whether Python’s “pip” module is already installed:
If you’re on Mac OS X, install pip if it’s not already:
If you’re on Linux, install pip if it’s not already:
Install the Selenium module for Python:
Verify that Selenium is installed:
Windows
Ensure that Python 2.7.x or later is installed:
If it’s not installed, download and run the setup from Python’s download center64. To add Python environment variables, go to “System properties” → “Advanced System Settings” → “Environment Variables” → “System Variables” → “Edit Path,” and then insert C:\Python27\;C:\Python27\Scripts\ at the end (assuming you have installed Python in the default location). Make sure to restart the command prompt to bring new environment variables into effect.
Check whether Python’s pip module is already installed:
Install pip if it’s not already:
Install Python’s Selenium module:
Running The First Tests
Download the Appium sample test script and sample app65. The sample package includes the APK for Android, the IPA for iOS and one sample test script. In addition, there are three samples:
appium_example_ios.py
appium_example_android.py (for Android API level 17 and above)
appium_example_selendroid.py (for Android API level 16 and below)
Step 1: Create Account and Upload the APK
Create an account.66 Note that this service provides a freemium option, and you don’t need a plan to complete these examples using real devices on a cloud service. If you want to access hundreds of device models, then different plans are available.
Before proceeding with running the test script, you’ll need to upload the APK or IPA to Testdroid’s cloud service via HTTP POST. Let’s try that using cURL.
For Mac OS X and Linux:
For Windows:
Upon successful upload, you will get a JSON response with status:0 and an unique identifier for the uploaded app:
Step 2: Set Credentials and Other Parameters
Open the test script, appium_sample_ios.py, in any text editor. Set screenshotDir to the path where you want those screenshots to be saved on your machine. Set your credentials on testdroid_username and testdroid_password in DesiredCapabilities. Set the myAppFile identifier against testdroid_app in DesiredCapabilities. It should look like this:
Step 3: Run the Test Script
Execute the following command:
The console’s output should look like this:
67
Console’s output after test script. (Image credit: Testdroid6854403733282517135) (View large version69)
Step 4: Get Results From Cloud
The screenshots will be available locally on your machine in the directory that you specified in step 2. Log into your cloud service and navigate to the project’s name, as defined in the _project attribute of DesiredCapabilities, to access the following log files:
Appium’s server log,
Logcat and instruments log.
Advanced Example: Using Appium For Mobile Game Testing With Image Recognition
In this example, We’re using Supercell’s popular mobile game Clash of Clans. It’s a fantastic game and I’ll bet many of you have played it, so you should be pretty familiar with its look and gameplay. This example does basic clicks through the game’s tutorial.
Let’s look some of the stages in this test script. The test_tutorial contains the following steps:
It first figures out whether the test executes on Android (self.isAndroid()) or iOS. As you can see, it looks for UI content differently. On Android, it tries to find a button by using an element’s ID (element_name) and on iOS by using an accessibility ID with a description (“Okay”). The same check happens for iOS’ Game Center login.
Screenshots are taken at various steps and stored in files entered as a parameter in a function call.
A check is done to see whether “goldmine” appears on screen by comparing two PNG files using a self.find_image call. If these pictures match (i.e. “goldmine” appears on screen), then the script continues executing the game’s tutorial.
The tutorial proceeds with the following steps:
Go to shop.
Buy cannon.
Place the cannon.
The information about all three of these items is stored in those PNG files: shopbutton.png, cannon.png and place_the_cannon.png.
Finally, the tutorial finishes and the battle starts. After the battle, the application is closed.
The video below shows how the test executes at each step. For the video, I’ve used three devices: one iOS (iPhone 4S) and two Android phones (Samsung Galaxy S3 Mini and HTC One X). You can also watch the video on YouTube70.
How Is Image Recognition Used Here?
The example shown uses image recognition (i.e. template matching71) to identify which features — basically, pixels and graphic content — are shown on screen and to compare the two pictures to each other. The algorithm built to recognize images was used on real devices and on two different platforms (Android and iOS), and it used a single test script for both platforms. This sort of image comparison is even very handy for recognizing UI elements and graphics that are resized and/or rotated.
Let’s say the template image has some distinctive features, such as text that can be easily abstracted from the background content. In this case, feature-based recognition can be used. In our example, if the “Button” text has been resized or rotated (or otherwise transformed), we can quickly and easily identify this and take further action.
The following functions explain the approach of comparing images:
Conclusion
Testing is crucial to ensuring success in the highly competitive landscape of mobile apps and games72. But even poorly planned testing can take up 20 to 50% of your total development effort, in which case it would also account for the single biggest cost in your budget. To ensure that testing is extremely efficient, covering the breadth of today’s mobile ecosystems and device models, the best option is an online cloud-based service.
If you only start thinking about testing a few days before the app hits the market, it’s too late. You’ll need to test a wealth of elements, data and functionality from day one. Here are some things to consider in making testing a part of your development process:
Plan carefully: Automate generic processes as much as possible.
When you’re building a mobile app, a well thought out strategy is critical, a great user experience and design are paramount, and solid development and testing are fundamental. Many aspects of testing can be automated, and this automation will increase the depth and scope of your testing and significantly improve the app’s quality. Ideally, test cases should have full access to the application and test all aspects of it — memory contents, data tables, file contents and internal program states — to determine whether the product performs as expected.
Your app will change during development: The same goes for testing.
Many things about your app will change as you’re creating it: the user interface, graphics, functionality, language support, the privacy policy, the use of external resources and much more. Even if 10% of your code changes73 or is added to the app, you’ll still need to test 100% of the features. Manual testing can’t keep up with this, so your best option is to build all of your test cases for new features. Then, when a feature is added, all features will be automatically tested. Building your tests to be maintainable over the various development phases of your app is essential.
Choose a testing technology and provider you can grow with.
If you already have an app on the market and are looking to create a similar one, then select a technology74 and vendor that meet your needs. For example, building your test cases according to a certain method or framework will enable you to reuse those test cases for your new application — at least to some extent. So, choose a technology and vendor that is able to handle your needs as your product scales up and as your testing has to cover new geographical areas and even support new platforms (for example, going from Android to iOS).
Test automation is available 24/7.
Automation will reduce the time it takes to test new features and even the app itself by running 24/7.
Use a cloud-based platform for truly global reach.
With an online cloud-based service, you’ll get instant access to hundreds of real Android devices75. Especially with Android, having access to devices that are used in volume holds significant value. Running automated tests on these devices is easy and fast and provides all of the information you’ll need, preprocessed, summarized and in full detail.
What are your thoughts on test automation for mobile? Please let us know in the comments section!
(da, ml, al)
Footnotes
1 http://www.zdnet.com/ios-8-usage-figures-showing-signs-of-stress-fragmentation-after-initial-uptick-7000034389/
2 http://www.gottabemobile.com/2014/10/22/iphone-6-plus-problems-fixes/
3 http://www.smashingmagazine.com/2014/10/02/what-every-app-developer-should-know-about-android/
4 http://www.smashingmagazine.com/wp-content/uploads/2014/12/01-device-rack-opt.jpg
5 http://www.testdroid.com/
6 http://www.smashingmagazine.com/wp-content/uploads/2014/12/01-device-rack-opt.jpg
7 http://appium.io/
8 http://www.supercell.net/games/view/clash-of-clans
9 http://www.gartner.com/newsroom/id/2823619
10 http://testdroid.com/testdroid/5876/test-early-test-often-testing-as-part-of-your-app-development
11 http://autotestcentral.com/how-test-automation-can-help-to-achieve-5-stars/285
12 http://www.smashingmagazine.com/wp-content/uploads/2014/12/02-value-diagram-opt.jpg
13 http://www.testdroid.com/
14 http://www.smashingmagazine.com/wp-content/uploads/2014/12/02-value-diagram-opt.jpg
15 http://en.wikipedia.org/wiki/Regression_testing
16 http://www.smashingmagazine.com/wp-content/uploads/2014/12/03-testing-in-ci-process-opt.png
17 http://www.testdroid.com/
18 http://www.smashingmagazine.com/wp-content/uploads/2014/12/03-testing-in-ci-process-opt.png
19 http://testdroid.com/testdroid/5851/increase-efficiency-and-productivity-with-test-automation
20 http://en.wikipedia.org/wiki/Cross-platform
21 http://agiledata.org/essays/tdd.html
22 http://www.smashingmagazine.com/wp-content/uploads/2014/12/04-time-to-market-opt.jpg
23 http://www.smashingmagazine.com/wp-content/uploads/2014/12/04-time-to-market-opt.jpg
24 http://www.smashingmagazine.com/wp-content/uploads/2014/12/05-different-ways-to-automate-opt.jpg
25 http://www.testdroid.com/
26 http://www.smashingmagazine.com/wp-content/uploads/2014/12/05-different-ways-to-automate-opt.jpg
27 http://www.smashingmagazine.com/wp-content/uploads/2014/12/06-different-ways-to-test-opt.jpg
28 http://www.testdroid.com/
29 http://www.smashingmagazine.com/wp-content/uploads/2014/12/06-different-ways-to-test-opt.jpg
30 http://www.appannie.com/indexes/all-stores/rank/overall/
31 http://www.androidcentral.com/devices
32 http://www.smashingmagazine.com/wp-content/uploads/2014/12/07-performance-testing-opt.png
33 http://www.testdroid.com/
34 http://www.smashingmagazine.com/wp-content/uploads/2014/12/07-performance-testing-opt.png
35 http://testdroid.com/testdroid/5901/rely-only-on-real-emulators-vs-devices
36 http://www.smashingmagazine.com/wp-content/uploads/2014/12/08-social-integrations-opt.jpg
37 http://www.testdroid.com/
38 http://www.smashingmagazine.com/wp-content/uploads/2014/12/08-social-integrations-opt.jpg
39 http://www.smashingmagazine.com/wp-content/uploads/2014/12/09-frameworks-opt.png
40 http://www.testdroid.com/
41 http://www.smashingmagazine.com/wp-content/uploads/2014/12/09-frameworks-opt.png
42 https://code.google.com/p/robotium/
43 http://developer.android.com/tools/help/uiautomator/index.html
44 https://code.google.com/p/android-test-kit/wiki/Espresso
45 http://developer.android.com/tools/testing/testing_android.html#Instrumentation
46 http://calaba.sh/
47 http://appium.io/
48 https://code.google.com/p/selenium/wiki/JsonWireProtocol
49 http://docs.seleniumhq.org/projects/webdriver/
50 https://developer.apple.com/library/mac/documentation/DeveloperTools/Conceptual/InstrumentsUserGuide/UsingtheAutomationInstrument/UsingtheAutomationInstrument.html
51 http://developer.android.com/tools/help/uiautomator/index.html
52 http://selendroid.io/
53 http://www.smashingmagazine.com/wp-content/uploads/2014/12/10-framework-families-opt.png
54 http://www.testdroid.com/
55 http://www.smashingmagazine.com/wp-content/uploads/2014/12/10-framework-families-opt.png
56 http://developer.android.com/tools/testing/testing_android.html#JUnit
57 http://developer.android.com/tools/testing/testing_android.html
58 http://developer.android.com/tools/testing/testing_android.html#Instrumentation
59 https://code.google.com/p/selenium/wiki/DesiredCapabilities
60 http://appium.io/slate/en/tutorial/ios.html?ruby#getting-started-with-appium
61 https://github.com/bitbar/testdroid-samples/tree/master/appium/sample-scripts
62 http://appium.io/downloads.html
63 http://appium.io/introduction.html
64 https://www.python.org/downloads/
65 https://github.com/bitbar/testdroid-samples/tree/master/appium/sample-scripts/python
66 http://testdroid.com/sm-appium
67 http://www.smashingmagazine.com/wp-content/uploads/2014/12/11-console-output-opt.jpg
68 http://www.testdroid.com/
69 http://www.smashingmagazine.com/wp-content/uploads/2014/12/11-console-output-opt.jpg
70 https://www.youtube.com/watch?v=Q5U3HwpD1Ec
71 http://en.wikipedia.org/wiki/Template_matching
72 http://www.appannie.com/indexes/all-stores/rank/overall/
73 http://testdroid.com/testdroid/6186/strive-for-hermetic-but-never-compromise-integrity-of-app
74 http://www.smashingmagazine.com/2014/01/10/four-ways-to-build-a-mobile-app-part2-native-android/
75 http://insights.wired.com/profiles/blogs/the-best-advice-for-app-developers-skip-emulators
The post The Basics Of Test Automation For Apps, Games And The Mobile Web appeared first on Smashing Magazine.