Manipulation checks in the managerial psychology experiment in China: Current status, problems, and suggestions

Subjects: Psychology >> Management Psychology Subjects: Psychology >> Psychological Measurement submitted time 2021-12-15

卫旭华 汪光炜 陈义

Abstract： Manipulation checks are important to ensure the construct validity of experiment, however, they have been less discussed in Chinese managerial psychology research. To investigate the status quo of manipulation checks among Chinese studies, we coded 144 experiments drawn from 49 research in the field of managerial psychology which were published in 4 top-tier journals in 2020 (i.e., Acta Psychologica Sinica, Journal of Psychological Science, Management World, and Nankai Business Review). We found that among these experiments, the way to implement manipulation checks appeared monistic with self-report dominating. And very few experiments focused on discriminant manipulation checks or instructional manipulation checks. In addition, the choice of location, the time to implement, and the strategy of analysis remained underexplored. Future research should pay attention to the application of non-interventional forms to reduce the interference of manipulation checks on the experimental process. Besides, error control strategies should be advocated to eliminate invalid data and competitive explanations. At the same time, we suggest that researchers should consider the characteristics of manipulation checks when they decide the appropriate position of manipulation checks included in the experiment material. Researchers are also encouraged to use pre and post manipulation checks to ensure flexibility. In the data analysis phase, we recommend researchers use manipulation strength and mediation analysis to guarantee the internal and external validity of the experiment. By doing so, we expect the normativity of manipulation checks will be improved to a great extent."

 Hits 3066  Downloads 626  Comment