The following post was written by mobile app tester Dom Wolf as part of uTest’s “Crash Courses” series. You can read similar posts in the uTest Forums(membership required).
Background: The release of the iPhone in 2007 introduced not only a new product, but also a new dichotomy to the world of software testing. The iPhone was the first mainstream phone to feature a touchscreen coupled with a 3G/Wi-Fi enabled web browser; two things that caused web developers to consider not only how their sites would appear, but also how they would be interacted with in such a different way.
Approximately one year after the launch of the iPhone, the iPhone SDK and App Store were introduced. The popularity of the iPhone at this point spurred many developers to submit applications at an unprecedented rate. Some apps were simple utilities, easily to develop and easy to test. Some, on the other hand, made use of many frameworks built within the newly introduced iOS and were incredibly complex. As the store’s popularity grew, Apple found itself on the receiving end of criticism over the content, and reliability of applications being submitted. The latter point is where we come in.
So there are two deliverables that are typically evaluated through uTest – native applications and mobile sites (this includes so-called web-apps, which are basically extensions to a mobile or web site). For the most part, the same test procedures can be applied to both types of deliverable, and this is what will be covered in this Crash Course. There may be some forms of testing which are deemed out of scope for a particular project you’re working on.
It’s paramount that you check the scope before you start testing and raising bugs. Not doing so could lead to having bugs rejected, which may affect your eligibility to take part in future releases. So, assuming that nothing is considered ‘out of scope’, we could include testing of the following:
The Next Web’s Matt Brian seems to think so. After months of hearsay and conjecture, the latest update for Google’s Android operating system looks fully baked and ready to serve. Here’s the story, and here’s the scoop:
Due before the end of the year, the Gingerbread update is said to quash a number of bugs, introduce a new open source video codec that can possibly replace Flash, support Near Field Communication (NFC) add more features to the built-in navigation software, support HTML5 and introduce new elements to the default User Interface.
There was no word on a release date, until recently.
Slowly but surely the pieces of the Gingerbread puzzle have come together and it has become apparent that December 6 will be a big day for Android, it is very likely that Google’s next Android update will be unveiled today.
Later today, Google VP Andy Rubin, the man many see as the driving force behind development of the Android operating system, will take to the stage at the D: Dive into Mobile conference, an event founded by All Things D writers Walt Mossberg and Kara Swisher. Here, Rubin is expected to leverage the opportunity to confirm the release of Android 2.3 (Gingerbread), discuss its features and supply us with a release date.
Late last week, security firm viaForensics announced it had discovered security vulnerabilities in mobile banking smart-phone apps for Google’s Android and Apple’s iPhone. Apps from Bank of America, Chase, TD Ameritrade, USAA, Wells Fargo and Vanguard were all targeted by the firm.
“It’s no secret that the mobile channel is screaming for more authentication and data encryption, but mobile nuances have posed some challenges.”
ViaForensics found that neither Google nor Apple has adequately prevented mobile apps from storing sensitive financial information. The firm noted that some apps do not validate security certification, making them susceptible to so-called man-in-the-middle attacks. Some apps also inadvertently saved passwords, because of the lack of encryption, and some saved data to the phone that had previously been viewed in the app itself.
Question: Why do you continue to tolerate buggy mobile apps? Why do you insist on going back to them, day after day, month after month, despite the inconsistent performance and general unreliability?
Answer: You don’t!
An article on BizReport.com confirmed what most mobile app testers & developers know all too well: That users have virtually NO tolerance for glitchy applications. Writes Helen Leggatt:
While most will tolerate a minor glitch, the findings show that many won’t hang around if they are disappointed with the performance of a mobile app or website. Almost a third (31%) said they would be less likely to use that app or website in the future and 21% said they would seek out another to use.
And the damage doesn’t stop there. Word-of-mouth will ensure any bad experiences are soon broadcast.
In a world where even the popular apps have a shorter shelf life than raw oysters, it’s more important than ever that you thoroughly – I and mean thoroughly – test your mobile application across as many platforms, locations, languages and other criteria as possible prior to launch.
This one’s for the Seinfeld fans among us. Remember the episode where George bought a cashmere sweater as a gift for Elaine? The sweater was perfect – even better, it was highly discounted – with the only drawback being a small, red-dotted stain. Even though it was barely noticeable, it was enough to cause sit-comic chaos, before Elaine eventually “re-gifted.”
Like that great episode, the new iPhone 4 – a great gift in its own right – is being tainted by a small yellow discoloration on the screen.
Mashable has a nice summary of the issue, along with several other user complaints:
Several early iPhone 4 customers have discovered critical issues with the phone’s display and reception that could pose major problems for Apple’s newest mobile device.
Multiple Apple- and gadget-focused websites are receiving reports that the iPhone’s much-discussed “Retina Display” is susceptible to a yellow discoloration, either as a thin line of yellow or as a circular tint.
That’s not the only problem: There are now countless videos online that show how holding the new iPhone by its sides can decrease reception quality.
Here’s video showcasing the reception difficulties:
Have you gotten the new iPhone 4 yet? Have any bugs to report?
The WAP Catalog has just posted a terrific app review of Twitter for Blackberry (in case you planned on downloading it). Mobile testers could learn a lot from this thorough content, especially from a usability perspective, so go check it out when you get a second. In the meantime, here are a few brief excerpts:
The Good: Push notifications for Direct Messages (DMs) and integration with BlackBerry’s native message folder make for an impressive experience.
The Bad: This release is still pretty buggy; it hangs and lags too often for a final build, despite perfect 3G and Wi-Fi coverage.
Recommendation: Yes. Despite the flaws, which can honestly get quite annoying at times, the features and execution are such that I would recommend this application to BlackBerry users.
I have been using Twitter for Blackberry Public Beta (T4BB for short) for nearly a month now since its release to the public in early April. Testing of the app has taken place in the Metro Atlanta area with pretty solid 3G and Wi-Fi coverage throughout, on a T-Mobile branded BlackBerry Bold 9700. T4BB is a feature-rich application that contains all the functionality of the Twitter website, with added flare that will make it instantly familiar to seasoned BlackBerry users (such as the ‘t’ and ‘b’ shortcuts that instantly take you to the top or bottom of your timeline). T4BB will suck the life right out of your battery depending on usage and update settings, but so will any other application that updates itself on regular intervals.
Exciting news from the mobile front yesterday, as developer Vladimir Vukićević announced that a more usable “pre-alpha” build of Firefox for Android (aka Fennec) is now available to a “broader set of people.” As you would expect from a project in such an early stage, there a number of bugs and other issues that still need to be resolved. Vladimir acknowledges as much in the post, offering a number of specific examples. Here are few of them (verbatim):
It will likely not eat your phone, but bugs might cause your phone to stop responding, requiring a reboot.
Memory usage of this build isn’t great — in many ways it’s a debug build, and we haven’t really done a lot of optimization yet. This could cause some problems with large pages, especially on low memory devices like the Droid.
You’ll see the app exit and relaunch on first start, as well as on add-on installs; this is a quirk of our install process, and we’re working to get rid of it.
You can’t open links from other apps using Fennec; we should have this for the next build.
This build requires Android 2.0 or above, and likely an OpenGL ES 2.0 capable device.
For more on this development – including how you can install and test the build yourself – go read the entire post.