Subjects: Psychology >> Statistics in Psychology submitted time 2020-10-17
Abstract： P-value is the most widely used statistical index for inference in science. Unfortunately, researchers in psychological science may not be able to interpret p-value correctly, resulting in possible mistakes in statistical inference. Our specific goal was to estimate how nonsignificant results were interpreted in the empirical studies published in Chinese Journals. Frist, We randomly selected 500 empirical research papers published in 2017 and 2018 in five Chinese prominent journals (Acta Psychological Sinica, Psychological Science, Chinese Journal of Clinical Psychology, Psychological Development and Education, Psychological and Behavioral Studies). Secondly, we screened the abstracts of the selected articles and judged whether they contained negative statements. Thirdly, we categorized each negative statement into 4 categories (Correct-frequentist, Incorrect-frequentist: whole population, Incorrect-frequentist: current sample, Difficult to judge). Finally, we calculated Bayes factors based on the t values and sample size associated with the nonsignificant results to investigate whether empirical data provide enough evidence in favor of null hypothesis. Our survey revealed that: (1) 36% of these abstracts (n = 180) mentioned nonsignificant results; (2) there were 236 negative statements in the article that referred to nonsignificant results in abstracts, and 41% negative statements misinterpreted nonsignificant results; (3) 5.1% (n = 2) nonsignificant results can provide strong evidence in favor of null hypothesis (BF01 > 10). The results suggest that Chinese researchers need to enhance their understanding of nonsignificant results and use more appropriate statistical methods to extract information from non-significant results.
Peer Review Status:
Awaiting Review
Hits
Hits
Downloads
Comment