How USA Today Tests Their Mobile Apps

USAWith mobile app testing being a relatively new discipline, there’s a lot of variation in the way it’s conducted, particularly among the world’s major brands. Some rely primarily on tools. Some outsource their testing. Some crowdsource the effort. Others, surprisingly, don’t test at all.

In today’s post, we’re going to give you a specific example by highlighting how USA TODAY tests their mobile apps. The following has been taken from a uTest customer case study. For the complete story, click here.

For nearly thirty years, USA TODAY has been at the forefront of innovation in the news & media industry across print, web and now mobile platforms. The first national paper to introduce color and one of the first publications to establish an active web presence in the mid-1990s, USA TODAY was also among the first in its class to launch native applications for the iPhone, iPad, Android and the other mobile operating systems.

The company’s dedication to mobile quality has clearly paid off: As of December 2011, USA TODAY saw aggregated mobile page views exceed web page views for the first time in its history.

But with greater success, comes greater responsibility. As USA TODAY began to support more mobile platforms and devices in an increasingly fragmented environment, key players within the company realized the need for an efficient, scalable way to increase their testing capabilities. Tim Carlson – Director of Mobile Product Development and Operations – began this search with some key criteria in mind.

“We needed a QA and testing solution that could be a part of our in-house team,” said Tim, who started his IT career as a tester. “We have such a close knit team and we wanted to keep that culture, so traditional outsourcing was not even an option. Of course, we also needed mobile experts who could hit the ground running. Mobile is a fast-growing space, but there are surprisingly few people who specialize in the testing side of the equation. Those were the main reasons why we chose uTest.”

Continue Reading >>>

Make Your Mobile Apps Pay Off

Trulia Mobile AppTrulia has a pretty cool real estate website. You can search for homes for sale or rent and read up on what’s nearby, the crime rate in the area and other information people looking to move might find interesting. But Trulia comes in even more handy when you’re just driving around. Wondering if there are any nearby homes on the market as you drive down a particularly pretty street? Curious about how much that home with the for sale sign in the yard is going for? Pull up the Trulia app.

Trulia saw their user base shifting from web to mobile – so they shifted right along with them and created mobile apps. When you need to make sure your app works just about anywhere in the country, under all sorts of circumstances (I’ve used Trulia while driving through parts of Maine) how do you test it? Here’s what Trulia was up against and what they decided to do, according to a report from analyst firm IDC:

To test these applications across an extensive geography and broad spectrum of devices, operating systems, and carriers, Trulia chose to go beyond the confines of its testing lab and adopt a “crowdsourcing” model to achieve in-the-wild testing. This model leverages a community of testers with different backgrounds, skills, and testing specialties to put applications through their paces under real-world conditions.

Can you guess who they went with?

After evaluating various crowdsourcing companies, Trulia chose uTest to test its mobile applications.

According to the IDC Spotlight: Trulia ROI, Trulia saw some very specific, very impressive results when working with uTest:

  • ROI of 280%
  • Seeing payback in 7.2 months
  • Reducing testing costs by 95%
  • Increased customer base by 10%-15%

Read the full IDC Spotlight: Trulia ROI >>>

What’s the ROI of your mobile app testing?

Need Your App Tested? There’s a Company For That

Apps GaloreLong-time readers of this site will know that the content presented here comes courtesy of uTest – the world leader for in-the-wild testing. This post is not for them.

Instead, this post for the first-time visitor, or the casual visitor that does not yet know about uTest’s various mobile app testing services (and who might need them). So for them, here is a brief overview:

With uTest’s global community of testers, companies can easily test  their mobile applications under real-world conditions across multiple handset makers & models, wireless carriers, operating systems, locations, languages  and other important criteria. uTest Apphance takes mobile app testing a step further, giving developers over-the-air distribution capabilities plus the ability to get bug reports and feedback from directly within the app while automatically collecting device data and detailed crash reports.

So whether you’re building apps for  iPhone, Android or Windows Phone, uTest is ready to help you meet  your testing needs with services including:

  • Exploratory Testing: Testers will search your mobile app for bugs and defects that  are only discoverable through unexpected, novel usage patterns from  real users.
  • Test Case Execution: A team of  testers will use your test cases to ensure your mobile app is tested across OS versions, browsers, locations, carriers and more.
  • Test Case Writing: Let one of our community experts draft a highly detailed and efficient  test case (aka test script) to ensure comprehensive test coverage.
  • Test Automation: uTest partners with some of the leading test automation companies to  deliver a more affordable, practical automated QA solution.

Other testing types include:

  • Security Testing: Mobile apps are now being targeted by hackers. With uTest, a team of security experts will probe for common vulnerabilities.
  • Load & Performance Testing: Ensure that your mobile application is prepared for peak usage periods with our load & performance services.
  • Localization Testing: uTest can provide third-party verification of context and accuracy for any recently localized mobile application.
  • Usability Testing: Test the usability of your mobile application with the help of our dedicated UX experts and software-savvy professionals.

Complexity across handset makers, carriers, locations and operating systems has made building bug-free mobile apps incredibly difficult and expensive. With uTest’s global community of testers, companies of all sizes can ensure that their applications works properly – for everyone, everywhere, all the time.

Learn More About Mobile App Testing with uTest >>>

uTest Named a Global Mobile Awards Finalist

logo_gma1Back in 2007, when uTest was just getting off the ground, we began to think that there had to be a better way for companies to develop, test and launch mobile applications. We saw firsthand how difficult it was to ensure that a mobile app worked as intended across multiple devices, operating systems, carriers and other real-world conditions. So we got down to business, and by 2009, we had launched our unique mobile app testing solution.

Fast forward to this week. After helping thousands of companies from around the world launch higher quality mobile apps, uTest was named a Global Mobile Awards finalist at Mobile World Congress. It’s basically the Oscars of mobile, but with slightly less singing and dancing. The recognition as a finalist for the Best Enterprise Mobile Service award – which puts uTest in a class with AirWatch, Antenna Software, MobileIron, Nuance and others – is yet another proof point on the effectiveness of in-the-wild testing, especially in the realm of mobile app development.

Our impact was certainly not lost on the judges here at MWC. Here was their criteria for the award:

The Best Enterprise Mobile Service award will be presented to the organisation judged to have introduced the most innovative mobile tools to help corporations or enterprise users to work smarter and “do business better” on the move. This award seeks to recognise innovators and market driving operators, vendors and app developers that deliver best in class products and solutions to the enterprise, from cloud services for accounting, sales, HR, logistics, communications, to outsourcing or point of sale applications, for example.

A round of applause to the folks at MobileIron who took home the award. We felt honored for the nomination and consider it a testament to our success. Just as the mobile ecosystem continues to grow and mature, so too will our mobile testing services. In other words, we’re just getting started.

Congratulations to all the Global Mobile Award winners and finalists who gathered at the Mobile World Congress. We’ll see you next year!

(This post originally appeared on the uTest Blog)

Forrester says In-the-Wild Testing the Key to Successful Apps

User FeedbackForrester recently spoke to mobile developers and user firms to put together the Mobile App Development Playbook. They found that the key to a successful mobile app is in-the-wild testing. Margo Visitacion, writing about the playbook for SD Times, said “Prioritize testing in the real world. Brands live and die by how their customers view their apps.”

In the article, Margo shares three tips for achieving real world success:

1) Feedback: In an ideal world, companies would know what their customers want, when they want it, and how they want it packaged. So for mobile app developers, make each day a potential election: Customers who vote on which functions to include or features to drop can provide fantastic input. But it’s more than just building the right features: Customers now feel that they own a piece of the app. And in an increasingly fickle world, customer loyalty is keenly important. By allowing customers to test the features in which they are interested, mobile app development teams can make loyalty a natural part of the development process,

2) Location testing: Apps that have location-based components should be tested in different locations. This is especially important if you are building a navigation app that guides users to the nearest store or clinic, as mistakes in these apps can be costly and result in people getting lost.

3) Crowdsourcing: It’s time for developers to take testing to the crowd, allowing groups to test the app and report back on their experience. Crowdsourcing is also a great way to gather data on how your app performs in “real-world” 2G, 3G, even 4G environments.

Read the full article at SD Times >>>

At uTest, we’re all about real world testing! To help spread the word, we have quite a few helpful resources for those looking to learn more about in-the-wild testing, crowdsourced testing and real world results. Check out these whitepapers, eBooks and tools:

Mobile App Testing – An Interview with uTest’s Matt Johnston

Greetings, uTest Nation and its followers, I’m back to serve in my role as the guy who does that thing that we don’t like to do too much of around here at blog central – talk about ourselves.

Well, not really. I’m actually here to share an excellent interview that STP‘s Rich Hand conducted with uTest’s Chief Marketing Officer Matt Johnston. In the interview, Hand and Johnston discuss ways in which the current burgeoning app economy has elevated the game for the way enterprise businesses conduct mobile app testing. Johnston also describes the importance of testing mobile apps in the wild and what it means for QA professionals.

The interview comes just a few weeks ahead of Johnston’s Mobile-themed keynote at the annual STP conference in Miami.

Listen to the interview today and look for more updates on uTest’s busy fall conference schedule in the coming weeks!


Test Drive uTest Apphance

ApphanceLast month we launched uTest Apphance, a new mobile quality tool that makes it easy for mobile app developers to understand how their apps are working across a wide range of mobile devices, carriers and locations. Since then, the response has been incredible. Hundreds of customers have already signed up for Apphance, and each day we’re delivering mobile developers incredible amounts of information about crashes and bugs.

Still, as we’ve talked to our customers about Apphance, some of them have asked us to share with them a working example – an app where they could kick the tires and get to know how Apphance works.

We think that trying out Apphance should be as easy as possible. That’s why we’re pleased to announce new Apphance demo apps for iOS and Android (available in Github). These “Hello World” apps are basic projects that already have Apphance installed and ready to go.

Learn more and watch tutorial videos at the Software Testing Blog >>>

Bob Binder on Mobile App Testing

I just posted a great interview with mobile app testing expert Bob Binder on the uTest Blog. Here’s an excerpt where he discusses the unique problems of  – you guessed it – mobile app testing:

The mobile app space is an unprecedented phenomena in many ways. I just finished a study of mobile apps for a certain aspect of driving (cars) — I found about 250 on iOS and Android. In the two weeks it took to complete the study, four simply vanished from their portals and many others were updated. Churn is very high and release cycles are on the order of weeks to months.

Most of these apps were the work of a single person – let’s call them app artisans. The remainder were authored by a big business or as part of a startup product or service. However, some artisan apps had millions of downloads and four or five star ratings.

The course is an attempt to provide something useful to this community. Having worked on mobile app testing since 2002, when I heard that millionth mobile app had been released in December 2012, I wondered “what kind of testing has been done on these apps?” My guess was, very little.

It seemed to me that artisans could benefit the most from good testing, but would probably have little inclination and time to do it. So, the challenge was to produce a course that assumed no prior knowledge of software engineering or testing and that did not require any tool support.

Instead of superficial hints to “explore” an app or platform-specific coding tricks, I provide very specific step-by-step guidance to develop a complete and reusable mobile app test plan. This approach is all manual and can be easily repeated for apps supported on multiple platforms (Android, iOS, etc.)

Read the entire interview >>> 

uTest Acquires Mobile Testing Tool Apphance

In case you missed the biggest news of the day, uTest (the brilliant, talented and incredibly attractive people behind this blog) acquired Apphance – an awesome mobile testing tool.

Since this is a mobile app testing blog this is definitely something you’re going to want to learn more about. Here’s an overview of some of Apphance’s coolest features.

Over-The-Air App Distribution

This one is huge for anyone developing a mobile app where build updates happen frequently. With over-the-air app distribution, you can upload a new build and your testers will be able to download an update either within the app or via email. That means they’re not wasting time testing an out-of-date version of your app or trying to get updates installed on their device.

Devices in ApphanceWhile app distribution is just for pre-production apps, having Apphance track your builds can be very useful for everyone from pre-production to production users. Apphance will keep track of your different builds, so it can tie crash reports and bug data to individual app versions. That means that you’ll be able to see how a given version of your app is performing in testing and in-the-wild. You can even choose to ignore reports from older versions of your app and just focus on data from the latest builds, which is perfect for production apps where there are always users who forget to upgrade from their app store.

Crash Reporting

Apphance Crash ReportNobody likes it when their app crashes, but figuring out why it crashed can be incredibly challenging. Even if the developer is holding the device in their hands, the exact reason the app crashed may not be apparent without extracting the buried system and crash logs.

Apphance solves this by detecting crashes and sending crash reports to the Apphance servers in the cloud. Developers can not only see which versions of their app have crashes, but they can even see the details about why the app crashed. Apphance also lets the developer manually log their own status messages, making it easy to track activities that are significant to individual apps. A developer can then go back and review the steps leading up to specific crashes.

Along with the crash report, Apphance will also send some additional data about the state of the device and the app. The exact data it sends depends on whether Apphance is running in pre-production mode or production mode. In pre-production mode, Apphance will send as much detail as possible, while in production mode Apphance sends a more limited amount of detail out of respect for the users’ privacy. You can learn more about the exact details Apphance sends from our help topics.

In-App Bug Reporting

Apphance Bug ReportWhen testers discover bugs in mobile apps, it can be difficult to capture the needed screenshots, write notes about the steps to reproduce the bug, and then enter all that into a bug tracking system that’s usually opened on a completely separate device. Apphance changes all that, allowing testers to report bugs from within the app itself. All the tester needs to do is shake the device (or any other mechanism the developer can specify), and Apphance will begin the bug reporting process. Apphance will take screenshots and get the details from the tester, all without ever leaving the app.

When the bug gets submitted, Apphance includes the same information as a crash report. That means developers can see the state of the device and the app, along with any additional logging information they add manually.

User Feedback

Even the best apps can have frustrated users, and inevitably frustrated users write negative reviews. Apphance gives developers a new tool to listen to user complaints from within the app. When a user is frustrated or having trouble, they can send their concerns directly to the developer using Apphance’s user feedback feature.


All this adds up to Apphance being one incredible mobile quality tool. Developers for iOS, Android, Windows Phone, Kindle, and Nook now have a tool that can distribute their builds to testers, track crashes and bugs in the wild, and get feedback from customers. To learn more about Apphance, check out the website or read the online help topics.

Upcoming Webinar: Mobile Usability & What You Need to Know

Update: If you missed the webinar, no worries, watch the recording anytime. Or download our free Mobile Usability eBook.

Are you a mobile app designer or developer who’s confused about usability? Or maybe you just wish you knew more. If you feel this way, you’re not alone. In fact, mobile usability is still a great mystery for the vast majority of developers and testers today. Both the processes and tools are immature, and there are few best practices and guidelines.

uTest wants to help, and we’re joining forces with usability expert Inge De Bleecker for a free webinar that will show you some of the best practices for mobile usability. Join us this Thursday June 28th at 2:00 PM ET, and you’ll learn:

  • The right things to consider when building mobile apps and mobile websites
  • Future trends that could change how you think about mobile development and usability
  • Tips and tools for usability testing your mobile apps and websites

At the conclusion of the webinar, we will also send you a free copy of our newest eBook about mobile usability. Be one of the first to receive this eBook with great tips and advice for your mobile projects.

Interested? Register here >>