On the challenges and solutions of training localization and internationalization testers in the context of manual testing

Abstract

Internationalization (i18n) and localization (L10n) tests are necessary to guarantee the quality of software that supports more than one language and is targeted to be launched in the global market. In this work, we aimed to understand the state of the art and practice of i18n/L10n testing. To achieve this goal, we conducted a Systematic Mapping Study (SMS) to answer the following research questions: “What are the main challenges faced by L10n and i18n testers?”, “What are the most commonly used testing strategies for L10n and i18n testing?”, “What is the current tooling support for L10n and i18n testing?” and finally, “Are there differences in L10n and i18n testing techniques depending on the context in which they are applied?”. The primary studies retrieved by our search string reported that the main challenges faced by L10n and i18n testers are the lack of scientific research on the topic, high manual labor, very limited automation support, the cost of fixing internationalization faults, and testing right-to- left (RTL) languages. The most commonly applied testing strategy is system level testing since this type of testing is typically performed in the final stages of development. Regarding the current tooling support for this practice, only four tools were reported in the primary studies as specialized for the context of L10n and i18n testing. Lastly, the context (mobile, web, or system) is directly linked to the testing technique applied due to the nuances and unique challenges of each context. From these results, the lack of materials and tooling support caught our attention, as it directly impacts the training of novice L10n and i18n testers. Motivated by this problem, we developed an approach that involved the creation of L10N-TRAINER, this tool simulates i18n/L10n faults in mobile applications to replicate the issues found during this type of testing. To evaluate the effectiveness of L10N-TRAINER, we conducted an empirical study in a real industrial setting. This study with L10N-TRAINER demonstrated that using our tool improved the performance of the group of participants who were first introduced to L10n and i18n testing with an application seeded by the tool. The group that was initially introduced to the practice using the traditional methodology employed by the company showed stability or a decline in their results, while the group trained first with the training application improved their discovery of true bugs, increasing from 2 to 5 true bugs.