Best Practices for Conducting User Testing in Asia
by Yufen Chen
With one fifth of the world’s population, China’s market is divided into Mainland China, Hong Kong, and Taiwan. Hong Kong and Taiwan are considered distinct markets within Greater China, as each have their own spoken and written language. Therefore, products looking to meet the needs of these two markets will have different creative, design, and language considerations. How these customers interact with your products or services from end-to-end can vary quite significantly between these populations and the majority that resides within Mainland China.
When choosing customers to sample from in Mainland China, four city tiers are often considered alongside other demographic, psychographic, and behavioral variables. Most brands choose to conduct research in Tier 1 cities like Shanghai and Beijing, so they get a sample of the largest and wealthiest cities; plus the cultural, political and, technology center of China. However, if the goal is to understand user experience across China and build customer personas, you might consider sampling from China’s other 10 megacities[i] to validate learnings from Beijing and Shanghai. Research on the competitive landscape and how it impacts the user experience must account for different distribution channels and customer access to products across urban cities.
Brands that want to be successful in China have been doing research in Tier 2 and 3 cities. Combined, these tiers have 6x the number of households in Tier 1 cities. In the last decade, many brands have also gone in Tier 4 and even Tier 5 cities to better grasp and map the customer journey.
Appreciating cultural and linguistic diversity is critical to successful data collection in China. There are five major dialectical groups that are mutually unintelligible and covers 200 individual dialects. Experienced Chinese moderators and translators will likely be fluent in two dialects and have no problems distinguishing between local access and comprehending native expressions (common Chinese idioms). For example, a southern Chinese person from Shanghai may often have problems understanding a northerner in Beijing when one speaks too quickly or vice versa. Participants can also be more sarcastic in certain cities over others. Use local moderators and translators whenever possible, particularly as you move beyond testing in Beijing and Shanghai only.
Allowing more time between interviews for mini-debriefs with moderators and translators is also helpful. Often, there is hidden meaning beyond the literal translation, so don’t be afraid to pause between sessions and ask if there is an alternative explanation. An experienced researcher in China will be able to explain the differences in expression, and identify if there is double meaning. Example, use of sarcasm versus someone politely “giving face” and avoiding direct criticism.
Finally, with analysis and recommendations, researchers need to consider that foreign companies are not allowed to wholly own companies in China, which then has an impact on product development and services. Global and regional stakeholders may have limited visibility and control over local implementation. So, whether you’re conducting a study to inform a product launch or market positioning, spend more time in understanding where the research needs are coming from. Particularly with multinational companies, understanding the makeup of your local, regional, and global stakeholder groups will help inform the types of recommendations that have impact and at which level. Otherwise, your recommendations may be interesting but fail in being “actionable”.
Partnering with a larger research agency can help to assure a sense of quality and a more familiar level of service for “new-to-China” companies. As a young industry in China, however, smaller research agencies are often more agile and able to produce research at a lower cost. Fortunately, a number of competitive options have increased dramatically over the last few years.
To identify potential partners, start by joining UXPA China (http://www.upachina.org/en/), which is formerly known as UPA China. Since 2004, this organization has early roots in user experience and can provide a rich network for global companies seeking local partners. The industry is still young and growing in China, providing greater resources for all companies looking to improve their experiences with Chinese customers.
Key Lime Interactive is a global partner with UX Fellows. For more information on conducting global UX testing with our team, email us at firstname.lastname@example.org. Not looking for a usability partner at this time? Email us to say hi, anyways.
[i] China’s megacities range from population sizes of 5-20+ million each