Detecting Localization and Internationalization Failures in Android Apps: A Semi-Automated Approach and An Empirical Study of Developer Response in Open-Source Projects

Abstract

The process of globalization (g11n) is crucial for expanding the global reach of software. It involves two key components: localization (L10n) and internationalization (i18n). Localization and Internationalization testing are essential to ensure an ideal user experience, regardless of the software’s language settings, as L10n/i18n errors are easily noticed by the end user and can cause discomfort or lead to misinterpretations of the software’s use. This work proposes a semi automated approach for identifying L10n/i18n failures in mobile applications on the Android platform. The proposed approach uses an open-source tool called Droidbot for exploring the interface of applications, executing them in different locales defined by the user. At the end of the exploration, reports are generated containing screenshots organized by locales. Subsequently, a human tester manually analyzes each screenshot in the report to identify and catalog the failures. The evaluation of the usefulness and efficiency of the proposed approach was conducted through an empirical study with 50 open-source Android applications available on the F-Droid platform, automatically explored in up to seven distinct locales, resulting in 237 rounds of automated exploration. The manual analysis enabled the identification of 41 failures related to localization and internationalization, which were reported to the developers of the evaluated open-source projects on F-Droid. Regarding the reported issues, 85.71% were responded by the developers, of which 80.49% failures were accepted, showing the usefulness of the proposed approach in identifying L10n/i18n failures relevant to open-source project developers. Furthermore, it was identified that some type of failures occur significantly more frequently than other types of L10n/i18n failures for the evaluated projects. In this study the most frequent failure found was Missing Translation. The exploration for all 50 open-source projects was executed automatically, taking 711 hours of execution for all the supported locales. In human workdays (considering 8 hours per day) it would be 89 days.