Mozilla’s Matt Evans on Mobile App Testing

The uTest Blog has just published Part I of a two-part interview with Matt Evans, QA Director at Mozilla. Prior to his role at Mozilla, Evans was a key player at Palm, where he managed the quality program for the WebOS Applications and Services of the Palm Pre smartphone. You should read the entire interview, but here are two questions he answered specific to mobile app testing. Enjoy!

uTest: You were the QA Director for Palm when they launched the Palm Pre Smartphone, as well as the WebOS apps and services. What’s been the biggest difference (if any) between launching a mobile product and a web product?

ME: The biggest difference between a web product and mobile device is the amount of testing and certification that must precede the launch of a mobile product. A smartphone such as the Pre is an incredibly complex and highly integrated piece of technology–much more so than a typical web application. First off, a smartphone contains a fully-functional OS, usually based on some variant of Linux running on very constrained hardware. It must perform all of its concurrent services utilizing limited memory and limited CPU horsepower. The smartphone must also respond correctly to the multitude of many current events, from those generated from the environment–like switching from wifi to a WAN internet connection–to handling data input from the user, as well as handling events from the onboard applications.

Launching a mobile product requires exhaustive certification of individual hardware components such as the CPU, modems, codecs, and displays. Even then, the finished product is really launched by the carrier and must go through their exhaustive certification tests as well. Testing an onboard mobile application is also a much harder testing task. There are so many conditions and constraints that are involved in testing a mobile application.

A typical mobile application is nearly functionally equivalent to any counterpart desktop client-side web application. Take, for example, a mobile email application. It must behave and interact with the server-side application in nearly the same way as a desktop web client. The established protocols were designed for a stable communications environment, but this is just not the case in a mobile environment. The internet connection may be lost and reconnected very rapidly. The connection may even be lost for long periods of time. The application may, at any moment, be swapped out of memory. The system may be shut down abruptly. Lots of system conditions happen in a smartphone that would rarely or never happen in the context of a desktop web client application. However, a mobile application must perform its main functional operations of retrieving and sending messages flawlessly with no loss of data and full operational integrity. Testing mobile applications under these environmental scenarios is a huge challenge. In short, testing a web application is no easy task, but mobile applications and products represent a much tougher and larger testing challenge.

uTest: There’s been a lot of bold predictions about the future of native apps, the mobile web, tablets, and just about anything that relates to mobile. Having lived (and worked) through the dotcom boom and bust of the last decade, do you see any similarities with regard to mobile?

ME: Not really. In the dotcom boom and bust era, the problem was such rampant speculation about how we as consumers would utilize the web for just about every type of activity and commerce transaction. It seemed at that time that anyone with a slick PowerPoint presentation and an outrageous business plan could get startup funding. The main problem was that the technology of the day just couldn’t deliver the user experience that all the web startup companies promised. Here we are, nearly a decade after the bust, and I believe we are just beginning to fulfill some of the wild web usage predictions and capitalization by users and consumers. That’s quite a gap between the vision that was promised and when it was actually delivered. No wonder there was such startup company roadkill during the bust.

Mobile computing, like any other surge in adoption of technology, will have winners and losers. However, there seems to be no end in sight for all things mobile. There is such innovation happening in this space. Take, for instance, the iPad. It is a game-changer, in my opinion. It has defined a whole new computing platform between what we consider a mobile handset and a laptop, and surely is adding to Apple’s vast bank vault. Smartphones continue to wow us with their capabilities and the ability to combine consumer-based technologies into such a convenient package, and we are quickly seeing how these devices are changing how we interact and communicate with each other. Mozilla encourages people to think big thoughts about the future of mobile computing. Just check out this video about the concept mobile phone called seabird and you will see what I mean. I sure would like the phone for Christmas.

So here’s the big difference I see between the dotcom boom/bust period a decade ago: In this mobile era we are leading with innovation and seeing how the marketplace utilizes the technologies. Where as, in the dotcom/bust era it was a lot of empty promises about technology and investor riches that ultimately were never delivered. The current stage in mobile and web computing in general is set for an explosive boom possibly much bigger and broader than than the dotcom period.  But this time I think it will be sustainable. Mobile computing and platforms are here to stay and will be the magnet for innovation for the foreseeable future. The current biggest hurdles are battery life and communication bandwidth. The latter is being addressed as we speak. 4G networks will have broadband connectivity. Hopefully battery life will also be improved shortly. A bright future ahead is my prediction.

Leave a Reply