We’ve made a lot of bold promises here at mobileapptesting.com – free cars, dates with celebrities and user-generated content, to name a few. Here to help us make good on at least one of these promises is guest blogger Jigar Patel of India (pictured right), who has written a two-part post on some of the best practices for mobile application testing.

Jigar has over five years experience as a software test engineer, and has recently taken a keen interest on the mobile side of testing. In this post, he tackles dev metrics for browser, device, region, carrier and other criteria, in addition to the testing process. Check back soon for part II.

***

This post is based on my working experience as a software test engineer – specifically, my thoughts on the growing trend of mobile web app testing. Since this trend is new to a lot of people, I hope to provide a road map of sorts for achieving efficient mobile app testing. Let’s start with metrics.

Browser/Device Metrics
Deciding on browser and device metrics should ideally occur in the early stage of the development process. Of course,   much depends on the nature of the application being developed, but below are some general criteria on deciding which metrics should be considered.

• Browser capability: If the application being developed will be dynamic or interactive, then the browser must support the following:
  • JavaScript support: Mobile browsers must support JavaScript in order the build an interactive application UI.
  • XML HTTP Request Object:  XML HTTP request object support is required to communicate with the back-end server and to update the page with new data without reloading the page. This will give the user a smooth browsing experience for the site.
  • CSS support: CSS defines how page elements are to be displayed and enables you to change appearance and layout of all of the pages by editing a single file.

There should also be performance analysis of the browser. If the browser performance is poor, then it should not be a part of the rich mobile application metrics that is going to be developed.

Browser metrics, of course, are subject to change. This can depend on changes in technology, including the growing (or diminishing) popularity of the browser. In this case – when the browser that is not currently part of the metrics is gaining in popularity and major market share – then it should be included in the metrics at some point to support the application. The sooner, the better.

If browser is initially part of the metrics and later starts to show rendering and performance issues which can’t be fixed at lower cost, then it can be kicked out of the metrics if it is not a priority. Apart from this, there should no major changes in the browser metrics, as this can lead to needless risk. This decision, therefore, should made based on the proper analysis at a higher level.

If the application renders correctly in the higher version of the browser (e.g. browser2.0 which ships with the device) and has rendering issues on the older version of the same browser (e.g. browser1.0 which ships with device Y) then you must consider whether these issues can be fixed at a lower cost. If the answer is no, then it is advisable to go ahead with browser2.0 and drop browser1.0 from the metrics, since it would be less popular and would incur high development costs to fix the bugs.

Device popularity
There should be analysis on the popular devices in the market, which hold the major market share. This can include iPhone, HTC g1, Motorola Droid, Google and Nexus One, which seem to be everywhere these days. Of course, these metrics are also going to change frequently due to rapidly growing mobile industry which frequently causes changes to device popularity.

Metrics for regions, carriers
There should be analysis for device/carrier popularity in terms of web traffic from devices in particular countries and regions. For example, the iPhone is extremely popular in the US and because of this, they obviously have more web traffic. We can get this kind of analysis from: http://metrics.admob.com which published monthly analysis. AdMob serves Graphical Banner and Text Link ads on mobile web pages for more than 15,000 mobile sites and applications. This monthly report offers a snapshot of its data to provide insight into trends in the mobile ecosystem, which can be very helpful when deciding the browser/device metrics for the application.

Testing Approach
Testing approach can vary depending on the nature of the application being developed. Here are some basic observations:

Testing process
In the mobile application development & testing process, the first goal should be to develop the basic stable application that works functionally on desktop browser and some of the key devices from the metrics like iPhone, HTC, etc., instead of worrying about supporting all devices in the early stages. Once the basic application is ready, then tester should begin their efforts to find the device specific bugs.

Before you start to determine how it works on mobile device, you should make sure the functionality of the site is working as expected by testing it on desktop browser. We can find functional bugs that are going to be reproducible irrespective of mobile devices and can get them fixed by developers at an early stage before the testing cycles start on the real mobile devices. Testing on the mobile devices before testing on the desktop browser is not good practice, and can lead to the more testing efforts down the road. The best testing approach for the mobile application should be following.

• First test on the desktop browser to find the functional bugs and make sure that functionality of the site is stable
• Once the site or module functionality is complete, then test on the device emulator to get the browser/platform coverage (in case if you don’t have the device).
• If device is already available, then test on the device itself to find the device specific bugs instead of testing on the emulator

This testing process will help us to find the functional bugs at early stage using a desktop browser. The tester can then focus on the real device testing to find the device-specific bugs like any rendering issues or any functional issues that do not reproduce on the desktop browser but do reproduce on the device being tested. This will help developera to get better idea of whether it is a device-specific bugs or not, so they can get the right direction to fix it.