Mobile App Testing

A few months back, we noted that mobile apps are NOT just for DoodleJump, where we discussed a study showing the incredible expansion (and acceptance) of mobile apps into the corporate world. In fact, between 86 percent and 90 percent of respondents identified increasing worker productivity and responsiveness as two key factors for deploying mobile applications.

Since we’re all apparently in agreement that mobile apps are extremely beneficial, the next question becomes:  iPhone or Android?

This is a particularly important question for small businesses – and it’s the subject of Joe Brockmeier’s latest article on smallbusinesscomputing.com. Comparing each device in terms of hardware, mobile apps and carrier (like a mobile usability tester) Joe explains why Android makes more sense for small businesses:

Read more…

A hurricane is about to level your beach house. A tornado is about to take you and your car airborne. Godzilla and King Kong are about to exchange fisticuffs in your city square. You have to act fast, but what should you do?

I recommend a combination of hiding and running. FEMA, however, suggests that you log on to the mobile web, register as a new user, confirm your identity with a security question and then search their application for tips on dealing with (insert disaster here). Starting to see the problem?

So did Marc Ambinder of The Atlantic, who was able to test the mobile application developed by FEMA. He writes:

FEMA allowed me to test it, which I did from my iPhone. It took about 15 minutes for me to register for my fictional disaster. I worried about two things: bandwidth and security. If tens of thousands of people applied for assistance at the same time, would FEMA servers be able to handle it all? And would the information be secure? Fugate, in an interview, said that the mobile platform was built on top of a highly secure disaster assistance infrastructure that could scale up quite quickly in the event of an emergency.  He noted that the program requires users to answer a security question with the aim of weeding out spammers and cyber criminals.

Just another reminder that when testing mobile applications, it’s important to keep the typical end user in mind.

(This post originally appeared on the uTest Blog)

Everyone wanted to know what Apple was going to say at their big press conference. Would the iPhone 4 bugs prompt them to issue a recall? Would they send users a plastic case that supposedly solves the reception problems (yes)? Would they try to fix the defects with a software patch? Would they say they’re sorry and that this will never happen again? Would they tell NY Senator Chuck Schumer to suck an egg?

Here’s one thing they didn’t say (but should have): “We should have listened to our testers!”

One of the biggest pet peeves among testers and engineers (or anyone in involved in quality assurance of technology) is not being taken seriously when a serious issue is uncovered. For most companies, it’s generally a cross-site scripting vulnerability, an SQL injection or a browser compatibility flaw in the UI. For the iPhone 4, it was an antenna issue. As it turns out, many top executives – including Steve Jobs himself – were repeatedly warned about about the “death grip” well in advance of the product’s release. These warnings from respected internal resources were either ignored or not taken seriously. They should have listened to their testers.

But what should testers do when they find themselves in this situation? According to Bill Ricardi, they should report the bug and move on. A member of the uTest community, Bill gave his advice on this matter as part of our Guest Blogger series, writing:

You won’t always see eye to eye with the client. What you consider a critical bug, they might see as a non-issue (or worse, a ‘feature’). What you call a major security flaw, they might consider such a remote possibility that it doesn’t even deserve a mention.

You might ask how you bridge such a gap between your level of testing and the client’s level of acceptance and understanding of product integrity and the testing process in general. The answer is simple:

You don’t.

Read more…

Take that Oracle! You just let Apple capture the lead in the 2010 Bug Marathon, otherwise known as Secunia’s Half Year Report (PDF). Worth the read, the 20-page report identifies the ten largest vendors with the most vulnerabilities (in all their products) and ranks them for the first half of 2010 – great entertainment for those who like to track bugs and keep score.

I mean, World Cup is over and nobody really cares about baseball until September, so perhaps this could help fill the competitive void in the meantime…

Anyway, here are the current “standings”:

1. Apple
2. Oracle
3. Microsoft
4. HP
5. Adobe Systems
6. IBM
7. VMware
8. Cisco
9. Google
10. Mozilla Organization

As noted earlier, this is really more of a marathon than a sprint, so it would be useful if we went back a little longer than six months to crown a winner. Thankfully, Secunia did just that as part of their key findings:

Read more…

Safe to say that mobile app development has greatly outpaced mobile app testing over the last few years. In other words, while the applications, platforms and operating systems have seen tremendous advances, the same cannot be said of mobile testing methodologies.

Case in point: The majority of mobile app developers are still overwhelmingly reliant on internal beta testing.

Here with proof is VisionMobile, who recently published a fascinating report on the mobile app ecosystem. A must-read for anyone – and I mean anyone – involved with mobile applications (i.e. developing, marketing, sales, etc.) the report should serve to demonstrate that mobile app testing has a long way to go before it’s fully matured. Here’s the key excerpt:

Internal beta testing is the most popular technique used by the vast majority (nearly 70 percent) of respondents, with beta testing with users and peer reviewing the next most popular techniques. Only 20 percent of respondents use focus groups or research of their own. Overall, North American developers are somewhat more sophisticated in their application planning, with 97 percent using beta testing as a standard part of application development and with broader use of a portfolio of planning techniques as well.

Yet, small development firms have limited means today to beta test and peer review their applications with a crosssection of representative users. Given the hundreds of thousands of mobile apps, we believe that efficient
(crowd-sourced) testing of apps in a global market of users is considerably under-utilized. This presents an opportunity for the few solution providers in this segment – Mob4Hire and uTest.com, for example – but also for network operators, who can generate a channel for testing applications with end users, and provide an open feedback support system back to developers.

Other interesting findings included:

Read more…

MobileCrunch picked up the news that Firefox Home – what they call the “almost-a-browser” -  is now awaiting App Store approval. That’s great, you’re thinking, but what the heck is an “almost-a-browser”? Here’s Simon Chester with an explanation:

Well, rather than just being a Safari replacement, Firefox Home acts as a bridge between your desktop version of Firefox and your iPhone. This is crucial, as apps that duplicate functionality of native iOS apps without adding anything new are a no-no in the App Store approval process.

It uses the Firefox Sync extension to sync your (desktop) Firefox bookmarks and history with your iPhone, and – more interestingly – will allow you to slide tabs open in Firefox over to your iPhone, much like Google’s Chrome-to-Phone functionality in Android 2.2.

You can search for and view any of the pages in your desktop bookmarks or history, as well as view the tabs open on your desktop copy of Firefox, right on your iPhone. Once you’ve found the page you’re after, you can view it either in Safari (an Apple-friendly move), or from within Firefox Home (which may upset Apple). You can also send the links via email.

Make sense? If not, and you’re one of those “visual” learners, here’s a video demonstration:

Read more…

From the AppleInsider’s Daniel Eran Dilger:

“Readers report that Apple’s tech support forums originally confirmed that a iOS 4.0.1 software fix addressing the issue would ship early next week (as early as Monday), before the comments were subsequently taken down along with all the other related discussion about the matter.

The fix is expected to address a issue in iOS 4 related to radio frequency calibration of the baseband. Readers who saw the original forum discussions say that the issue is believed to occur when switching frequencies; because the lag is allegedly not calibrated correctly, it results in the device reporting “no service” rather than switching to the frequency with the best signal to noise ratio.

iOS 4 introduced some enhancements to how the baseband selects which frequencies to use, so it makes sense that the error may have crept into those changes. Additionally, this explains why iOS 4 has also caused similar problems for iPhone 3GS users.

Additional readers have shared other related experiences that also corroborate the idea that the issue is related to iOS 4′s software control of the baseband, including the fact that the issue seems easily reproducible when connecting to a WWAN 3G network but does not appear when connecting to a Microcell 3G. If the problem were simply hardware related issues of the antenna design, it should only affect iPhone 4 units with that new design and should occur at all times, regardless of the tower type. That is not being observed.

The core software problem is likely augmented by hand placement, as Jobs noted in pointing out that holding the new phone (or any mobile device) in such a way that attenuates the signal should simply be avoided.”

Read the rest of the article.

This one’s for the Seinfeld fans among us. Remember the episode where George bought a cashmere sweater as a gift for Elaine? The sweater was perfect – even better, it was highly discounted – with the only drawback being a small, red-dotted stain. Even though it was barely noticeable, it was enough to cause sit-comic chaos, before Elaine eventually “re-gifted.”

Like that great episode, the new iPhone 4 – a great gift in its own right – is being tainted by a small yellow discoloration on the screen.

Mashable has a nice summary of the issue, along with several other user complaints:

Several early iPhone 4 customers have discovered critical issues with the phone’s display and reception that could pose major problems for Apple’s newest mobile device.

Multiple Apple- and gadget-focused websites are receiving reports that the iPhone’s much-discussed “Retina Display” is susceptible to a yellow discoloration, either as a thin line of yellow or as a circular tint.

That’s not the only problem: There are now countless videos online that show how holding the new iPhone by its sides can decrease reception quality.

Here’s video showcasing the reception difficulties:

Have you gotten the new iPhone 4 yet? Have any bugs to report?

The mobile industry is being increasing cluttered with jargon, says Anuj Gupta, our latest guest blogger for mobileapptesting.com. In this post, Anuj clears the air with an essential primer for anyone remotely involved in the mobile testing space, including discussions on the most popular operating systems and application platforms for iPhone, Blackberry, Windows mobile and more.

Mobile devices are evolving into increasingly sophisticated general purpose computers and this has led to the development of a variety of platforms and operating systems in the mobile space. Today, the operating system lying inside your mobile device is equally important as the device manufacturer (OEM). The various features, GUI, processing speed, and most importantly, the applications available for your device will depend greatly on the underlying OS.

Unfortunately, industry jargon has cluttered our understanding of this emerging technology. It is my intention to clear the air on a few of these misunderstandings. Here goes….

Manufacturers launch devices based on various user segments, and giving rise to a number of Mobile OS and Application platforms. A mobile OS manages the hardware and software resources of a mobile device, similar to that of a computer OS. Some OS platforms cover the entire range of the software stack, while others may only include the lower levels (typically the kernel and middleware layers) and rely on additional software platforms to provide a user interface framework.

Today’s phones are expected to run a growing range of software such as internet browsers, navigation packages, games and music/video players. Application platforms are supposed to provide a ground for them.  Application platforms are built over lower-level kernel operating systems (OS) such as Rex, Linux or compact real-time OS’s such as Nucleus. Software running on the device accesses platform resources through a set of application programming interfaces or APIs. Application development platforms, such as Brew, Symbian UIQ, Android, LIMO, ALP, Qtopia or WIPI provide programming resources for native and Java applications.

Mobile developers typically develop for multiple platforms to maximize their available market. This can be a difficult and time-consuming task as multiple platforms use different API calls for common OS operations and accessing OS resources such as accessing memory and files.

There is a variety of both the OS and the application platforms in the market. Obviously, some are more dominant than others. Here is a list of major mobile operating systems and Application platforms.

Read more…

Everyone’s excited about the big iOS 4 update that launches later today. Okay, well maybe not everyone. But Mashable certainly is, and staff writer Christina Warren has the scoop. Here she is with a couple of new key features:

On the backend, iOS 4 has more than 1,500 changes or additions to the API, which will enable developers to build even better apps on these devices. We’ll be posting a more complete video overview of what is new in iOS 4 later this afternoon, but check out some of the highlights first:

• Multitasking — A combination of persistent-state applications and backgrounding functions for certain services. The end result means better battery performance when doing things like listening to Pandora while browsing the web.
• Folders — Organizing your iPhone Home Screen is easier now that you can use folders to store groups of apps in one icon. This is a huge time saver and makes finding and keeping more apps on your device much easier.
• iBooks — First appearing on the iPad, iBooks is now available for iPhone users. Just like the Kindle app, your purchases and page marks are synchronized across devices.
• Tap to Focus Video — iPhone 3GS owners can now tap to focus their videos just like you can when taking still shots. The new software also includes a 5x digital zoom for the built-in camera.
• Bluetooth Keyboard Support – You can now pair your iPhone with a Bluetooth keyboard, which will make typing long documents much easier.

Of course, as with any major release, there are bound to be some bugs that sneak past production – most likely bugs associated with these new features. Nothing like a few million downloads to help uncover them.