To explain the WEIRD problem, let’s start with a weird analogy…
Every career involves some type of research, which inevitably helps professionals get better at their craft. Consider a baker with a specialization in pies. Custard, cream, and apple pies are all second nature to this person because they learned as much as possible to become an expert in this particular domain. Yet, the credibility of the baker becomes questionable if they start saying they know everything about every type of food using only the knowledge they have about pies. Pies are such a small percentage of all the different types of food in the world- how could this baker possibly make claims over all types of food?
This is the exact situation occurring within social science, but rather than worrying about misrepresenting pasta or pastries, there is a bias in participants recruited. Henrich, Heine, and Norenzayan (2010) were some of the first researchers to expose the massive amount of disproportionate sampling within behavioral science. The demographics for the majority of research fell into the WEIRD acronym coined by these researchers. WEIRD stands for Western, Educated, Industrialized, Rich, and Democratic. Although some consumers do fall into the WEIRD category, it is very safe to say it’s not all-encompassing. Recruiting only individuals within this bracket would not present an issue if all humans shared the same reactions and behaviors, but human beings have a lot of variability. Similar problems have been noted in biology and neuroscience where a lot of research solely studied males in neurobiology animal research. In either situation, overarching claims easily misrepresent populations. Ingrained beliefs, values, past experiences, and social conditions all impact judgment, perception, and behavior. Therefore, measuring only a small subsect of the population and generalizing it to all humans is not only misleading—it’s wrong.
What types of issues does this create?
Cross-cultural studies have demonstrated the need for diversity in research by suggesting the WEIRD population is actually more of an outlier than a norm (Muthukrishna et al., 2020). Patterns within the WEIRD population are cited for being more individualistic, independent, and analytical, and impersonally prosocial (Schulz et al., 2019). To complicate matters even more, variability even exists among, within, and across the WEIRD population (just ask any New Yorker visiting rural Kentucky). These differences highlight the need to understand cross-cultural context in consumer science.
Henrich, Heine, and Norenzayan (2010)’s paper shares that only 12% of the world’s population was utilized in 96% of psychological samples. This implies there is a much greater likelihood of being studied when attending a Western university as opposed to any person you may pass on the street. Additionally, this disproportionate analysis of human behavior is perpetuated by many of the big labs, journals, editors, researchers, and conferences who also are WEIRD. The study of culture is informed by the researchers’ culture. It is important to have more diverse researchers in the field to help appropriately investigate particular individuals. Without fully knowing what it is like to live in someone else’s culture or society, important questions or specifications may be overlooked. Therefore, a lot of the foundational research may be flawed or inaccurate in its generalizations when replicated with other groups of individuals. The cycle of researching only a small subsect of the population and assuming its applicable universally creates distortion about behavior by neglecting trends and patterns within other groups.
Expanding the scope of research is challenging; however, the current literature regarding judgment and decision making is heavily biased since the focus was driven by the WEIRD population. The past literature has a lot to offer in terms of framework, design, as well as cautionary limitations. Findings from any piece of research must be considered within its cultural confines. As scientists, we must acknowledge and appreciate the gaps in research in order to learn more about the human effect. Reviewing the main effects of research under this critical lens may be messier, but it is especially important in consumer science. Exposure to ads, products, or packaging can vary drastically depending on the context. By addressing the differences within cultures and societies around the world, companies can cater to consumers better to meet their personalized needs. While it may feel overwhelming, there is an opportunity to make major improvements by adjusting the research to better serve the target demographic.
The Mess in Measures
The implications of the WEIRD problem extend into the tools used to measure populations. Having a scale or methodology be validated and reliable with one subgroup does not guarantee its effectiveness with another. Advances in technology have provided many benefits for faster and easier communication, data collection, and management; however, not all tools are made equal or accessible. Customizing the choice in methodologies to the research question is crucial when trying to explore a concept for a certain group. It is important to be aware of the constraints about each tool or technique in order to determine if that limitation will compromise the study.
Certain advancements also may isolate certain populations. For example, consider a study using an app as a way to record daily experiences of participants. Having a smart phone compatible with the app is a major limiting factor in recruitment. Furthermore, within that population, many individuals may be technologically inept and find using the app challenging. An inability to handle the app is just as concerning, since the data will not be recorded if the participant cannot access the right interface. As seen in this example, to meet scientific goals, it is important to account for the culture and conditions of the participants. For research innovation and advancement to take place, studies must have strategies in place to understand and account for the differences that may occur.
Blind Spots in Technology
As market researchers, HCD looks to produce the best quality results for specific research questions. By constantly exploring new and emerging technology, it is important to consider how scalable the technology is and how useful its application will be in any of our client’s studies. One of the most important parts of evaluating new technology is understanding how to use the tool, which includes understanding any barriers. Critically evaluating the tools is one of the first safeguards to ensuring useful data. We are motivated to seek out new methods and technology for specific research questions to ensure there is a consideration for the validity of the findings through its connections to the cross-cultural context.
Awareness of WEIRD is important because research can plan and design in ways that address the issue. The way consumers live will have a major impact on the way they see, hear, and experience any product, ad, or package. As the research is crafted, the population explored, and the tools used to measure any effect, should not be taken lightly. Screening questions must be carefully constructed with consideration of complex components such as gender identity and mixed races. These decisions must have a strong justification as they will have ramifications on other research decisions such as sample size and scalability.
Getting comfortable with familiar methods may be tempting, but the objective is to be effective not compliant. In The Art of War by Sun Tzu, he shares wise advice very applicable to consumer science research, “Do not repeat the tactics which have gained you one victory, but let your methods be regulated by the infinite variety of circumstances.” Understanding context is how this field will continue to grow and evolve, but only if the appropriate measures are used to accurately report an experience.
If you are interested in starting a conversation about using the right tool for the right question, please feel free to contact HCD Research via email at info@hcdi.net or call 908.788.9393.
References:
Henrich, J., Heine, S. J., & Norenzayan, A. (2010). The weirdest people in the world?. Behavioral and brain sciences, 33(2-3), 61-83.
Muthukrishna, M., Bell, A. V., Henrich, J., Curtin, C. M., Gedranovich, A., McInerney, J., & Thue, B. (2020). Beyond Western, Educated, Industrial, Rich, and Democratic (WEIRD) psychology: Measuring and mapping scales of cultural and psychological distance. Psychological science, 31(6), 678-701.
Schulz, J. F., Bahrami-Rad, D., Beauchamp, J. P., & Henrich, J. (2019). The Church, intensive kinship, and global psychological variation. Science, 366(6466).
