Users increasingly rely on mobile applications for computational needs. Google Android is a popular mobile platform, hence the reliability of Android applications is becoming increasingly important. Many Android correctness issues, however, fall outside the scope of traditional verification techniques, as they are due to the novelty of the platform and its GUI-oriented application construction paradigm.
Many tools and techniques exist for automating the test- ing of mature, well-established applications, such as desktop or server programs. However, the physical constraints of mo- bile devices (e.g., low-power CPU, small memory, small dis- play), as well as developers’ unfamiliarity with mobile plat- forms (due to their novelty), make mobile applications prone to new kinds of bugs.
For example, an Android application is structured around activities (GUI windows), broadcast receivers, services and content providers; this is different from standard server applications, or from an event-based system used in a desktop GUI application. The tendency of mobile applications to have bugs is evidenced by their high defect density: one study has found that Android applications can have defect densities orders of magnitude higher than the OS.
In this paper we aim to bring novel, Android-specific classes of bugs to light, and show how to construct an effective test automation approach for addressing such bugs, especially GUI bugs, and ensuring the reliability of Android applications.
We first conduct a bug mining study to understand the nature and frequency of bugs affecting Android applications; our study finds that GUI bugs are quite numerous. Next, we present techniques for detecting GUI bugs by automatic generation of test cases, feeding the application random events, instrumenting the VM, producing log/trace files and analyz- ing them post-run.
We show how these techniques helped to re-discover existing bugs and find new bugs, and how they could be used to prevent certain bug categories. We believe our study and techniques have the potential to help devel- opers increase the quality of Android applications.
We found that, while bugs related to application logic are still present, the remaining bugs are Android-specific, i.e., due to the activity- and event-based nature of Android ap- plications. We categorized all confirmed bugs in the bug database based on our observations. To detect and fix these categories of bugs, we employ an automated test approach.
Our approach uses a combination of techniques. First, we employ test and event generators to construct test cases and sequences of events. We then run these test cases (and feed the events, respectively) to the application. Once a test case is running, we record detailed information about the application in the system log file; after each test case run, we perform a log file analysis to detect potential bugs.
To demonstrate the effectiveness of our approach, we present an evaluation on the open source applications that form the object of our bug study. We generated test cases for all projects used in the bug study and compared bugs we found with bugs reported by users. We detected most bugs reported, and found new bugs which have never been reported.
To read this external content in full, download the complete paper from the author archives on line at the University of California.