Subjects: Psychology >> Developmental Psychology submitted time 2023-03-28 Cooperative journals: 《心理科学进展》
Abstract： In recent years, multilevel models (MLM) have been frequently used for studying multilevel moderation in social sciences. However, there still exist sampling errors and measurement errors even after separating the between-group effects from the within-group effects of multilevel moderation. To solve this problem, a new method has been developed abroad by integrating MLM with structural equation models (SEM) under the framework of multilevel structural equation models (MSEM) to set latent variables and multiple indicators. It has been showed that the method could rectify sampling errors and measurement errors effectively and obtain more accurate estimation of moderating effects. After introducing the new method by modeling with random coefficient prediction and with latent moderated structural equations, we propose a procedure for analyzing multilevel moderation by using MSEM. An example is illustrated with the software Mplus. Totally 29 articles, published in Chinese psychological journals from 2010 to 2017, are reviewed for evaluating the situation of using multilevel moderation analysis methods in psychological researches in China. Directions for future study on multilevel moderation and MSEM were discussed at the end of the paper.
Subjects: Psychology >> Social Psychology submitted time 2023-03-28 Cooperative journals: 《心理科学进展》
Abstract： At present, the analysis of moderating effect is mainly based on cross sectional data. This article discusses how to analyze the moderating effect with longitudinal data. If the independent variable X and the dependent variable Y are longitudinal data, longitudinal moderation models can be divided into three categories according to the type of moderator: time-invariant moderator, time-variant moderator, and moderator generated from X or Y. For example, Xtj is divided into two parts, time-varying intra-individual differences Xtj−X¯∙jXtj−X¯∙jX_{t j}-\bar{X}_{\bullet} j and time-invariant inter-individual differencesX¯∙jX¯∙j\bar{X}_{\boldsymbol{\bullet} j}, and then the moderating effect of X¯∙jX¯∙j\bar{X}_{\boldsymbol{\bullet} j} on the relationship between (Xtj−X¯∙j)(Xtj−X¯∙j)(X_{t j}-\bar{X}_{\bullet} j) and Ytj can be analyzed. In that case, there will be no new moderator Z, which is characteristic of moderation research on longitudinal data in contrast to research on cross-sectional data. Four types of longitudinal moderation analysis approaches are summarized: 1) Multilevel model (MLM); 2) Multilevel structural equation model (MSEM); 3) Cross-lagged model (CLM); 4) Latent growth model (LGM). It is found that the decomposition of the moderating effect and the use of the latent moderating structural equation (LMS) method are the two characteristics of the moderation analysis for longitudinal data. Specifically, MLM, MSEM, and CLM divide the moderating effect of longitudinal data into three parts: the time-varying intra-individual part, time-invariant inter-individual part, and the cross-level part. In addition, the moderating effect of longitudinal data can be decomposed into the moderating effect of initial level and rate of change by LGM. In the present study, we propose a procedure to analyze longitudinal mediation analysis. The first step is to decide whether it is necessary to make a causal inference. If the aim of research is to make a causal inference, CLM should be adopted to analyze longitudinal moderation. Otherwise, proceed with the second step. The second step is to decide whether it is necessary to treat longitudinal data as multilevel data. If longitudinal data is treated as multilevel data, MSEM should be adopted to analyze longitudinal moderation, because MSEM and MLM are more suitable for describing individual differences. Otherwise, LGM should be adopted to analyze longitudinal moderation, because only an LGM can simultaneously examine the effect of some variables on change and how the change affects other variables. The third step is to decide whether MSEM converges. If MSEM converges, the result of MSEM should be reported. Otherwise, MLM should be adopted to analyze longitudinal moderation. Compared with MLM, MSEM takes sampling error into account when the group mean is calculated, but the convergence of the MSEM is more difficult. Therefore, the MSEM with sampling error taken into account is preferred. If convergence fails, MLM will be considered. This paper exemplifies how to conduct the proposed procedure by using Mplus. Directions for future research on moderation analysis of longitudinal data are discussed, such as the moderation analysis for intensive longitudinal data based on the dynamic structural equation model.
Subjects: Psychology >> Social Psychology submitted time 2023-03-28 Cooperative journals: 《心理科学进展》
Abstract： A total of 213 articles on psychological statistical methods have been published in 11 journals of psychology in Mainland China from 2001 to 2020. There are mainly 10 areas attractive to researchers (sorted by the number of papers): structural equation models (SEM), test reliability, mediation effect, effect size and testing power, longitudinal study, moderation effect, exploratory factor analysis, latent class analysis, common method bias and hierarchical linear models. Research on structural equation models (with confirmatory factor analysis model as a special case) explore five major aspects: model fit evaluation, model estimation, item parceling, measurement invariance and the extensions of SEM. The last aspect includes exploratory structural equation modeling, factor mixture modeling, high-order factor modeling as well as bifactor modeling. Articles on exploratory factor analysis focus on factor extraction. Modern reliability analysis is inextricably linked with factor models, including three main topics: distinction between coefficientα and internal consistency or homogeneity, confidence interval estimation of composite reliability and homogeneity coefficient, and reliability of multilevel data and longitudinal data. Common method bias is also based on factor analysis and studied in three aspects: the relationship between common method bias and common method variance, the influence of common method bias, and the comparison of approaches for testing and controlling common method bias. Studies on mediation effects can be summarized in four topics: testing approaches and their comparison, mediation effect size, mediation effect testing for categorical variables, and the extensions of mediation models. The simple mediation model was extended to multilevel or multiple mediation models, moderated mediation models and mediated moderation models, as well as mediation models of longitudinal data. Articles on moderation effects mainly explore three issues: the development of latent interaction models from those with mean structure to those without mean structure, and the change from latent interaction models with product indicators to those without product indicators, as well as standardized estimates of latent moderating effect models. Articles on longitudinal data analysis fall into three main groups. The first is the development of models, which includes hierarchical linear models, latent growth models and its mixture models, piecewise growth models and its mixture models, etc. The second is the development of longitudinal data collecting methods, which include intensive longitudinal and accelerated longitudinal design. The last is missing data handing methods of longitudinal data. Hierarchical linear models were studied in three directions: aggregation adequacy testing used in aggregating the ratings of individual level to team level, hierarchical linear model of categorical variables as outcome variables (including multilevel binomial and multilevel multinomial logit models), hierarchical linear modeling of latent variables (i.e., multilevel structural equation model). Research on latent class models investigates three main topics: the use of latent class analysis, latent profile analysis and Taxometric techniques in probing latent class structure; precision of classification; regression mixture model (i.e., latent class model including covariates). Both effect size and testing power are closely associated with hypothesis testing, and studies in this area introduce types and characteristics of effect size, calculation of testing power, alternative approaches and their supplements for testing null hypothesis significance.
Subjects: Psychology >> Social Psychology submitted time 2023-03-28 Cooperative journals: 《心理科学进展》
Abstract： Structural equation modeling (SEM) is an important statistical method in social science research. In the first two decades of the 21st century, great progress has been made in methodological research on SEM in China’s mainland. The publications cover five aspects: model development, parameter estimation, model evaluation, measurement invariance and the special data processing in SEM. SEM development includes the research on measurement models, structural models, and complete models, as well as the SEM in population heterogeneity studies and longitudinal studies. The research on the measurement models involves bi-factor model, exploratory structural equation model, measurement models for special design (e.g., random intercept factor analysis model, fixed-links model, and the Thurston model), and formative measurement models. The research on the structural models involves the actor-partner interdependence model. The research on the complete models focuses on item parceling. The SEM in the study of population heterogeneity involves latent class/profile model, factor mixture model, and multi-level latent class model. The SEM in longitudinal studies includes models describing development trajectories and differences, such as the latent growth model, the piecewise growth model, the latent class growth model, the growth mixture model, the piecewise growth mixture model, the latent transition model and the cross-lagged model. The publications on parameter estimation methods mainly involve the introduction of methodology (including the partial least square method and the Bayesian method) and the comparison of different parameter estimation methods. Advances in the model evaluation include fit indices and their corresponding critical values, selection of fit indices, model evaluation criteria beyond fit indices, and comparison and selection among alternative models. The development of measurement invariance involves three topics: (1) the introduction of different models with testing process and model evaluation criteria for measurement invariance analysis; (2) measurement invariance analysis in a particular model or data (e.g., second order factor model and ordered categorical data); (3) new methods of measurement invariance analysis (e.g., alignment and projection method). In addition, research into special data processing methods in SEM addresses issues of missing data, non-continuous data, non-normal data, and latent variable scores. Finally, recent advances in SEM methodological research abroad are introduced to help researchers understand some cutting-edge topics in this field, which offers implications for future directions of SEM methodological research.
Subjects: Psychology >> Social Psychology submitted time 2023-03-28 Cooperative journals: 《心理科学进展》
Abstract： Hypothesis testing is an important part of inferential statistics. Most reported statistical test results are based on the null hypothesis significance test (NHST). In the first two decades of the 21st century, the studies on hypothesis testing and related topics in China’s mainland cover such topics as the deficiency of the null hypothesis significance test, use of P-value, repeatability of psychological research, effect size, power of a statistical test, and equivalence test, among others. This systematic review summarizes the main findings and gives suggestions. NHST has a wide range of applications to a variety of fields, from mathematical statistics to psychology. In the past two decades, Chinese researchers have experienced a process from knowing, using, misunderstanding, understanding, and questioning it, to constantly proposing improvement methods. NHST still occupies an important position in scientific research, despite some shortcomings. When providing statistically significant results, it is recommended to offer precise P-values in order to better evaluate the type I error rate. When one wants to verify is equivalence (or zero effect), a better approach is to set an equivalent boundary value and put the equivalence hypothesis in the position of alternative hypothesis. NHST has been developed into a set of procedures as follows: First, to ensure the power of a statistical test and save costs, one should do a priori power analysis before sampling, and calculate the required sample size. The only exception is questionnaire studies with more than 160 participants which usually do not need such priori power analysis in the traditional statistical analysis. Second, to collect and analyze data, and report NHST results and confidence intervals. Third, to calculate and report the effect size if the results are statistically significant (at this time only the Type Ⅰ error is possible), and draw conclusions based on the magnitude of the effect size. Fourth, to calculate the effect size if the results are not statistically significant (at this time only the Type Ⅱ error is possible), and accept the null hypothesis if the effect size is small. However, a posterior power analysis is required when the effect size is medium or large. If the test power is high, the null hypothesis will be accepted; if the test power is less than 80%, more participants could be added for further analysis. The process of increasing the sample size should be reported clearly, with the final P-value presented and the type I error rate evaluated. Furthermore, the reproducibility crisis of psychological research is partly attributable to NHST. But the reproducibility of scientific research must be strictly defined. Although the failure to replicate a study may result from inaccurate operations and improper methods, it may also be caused by moderating effect. We can't judge the scientificity of a study simply by whether it is replicable. There are three major aspects for expanding the research on the related issues of hypothesis testing. Firstly, the equivalence test has been extended to the evaluation of structural equation models. Second, the analysis of test power has been extended to models other than those in traditional statistics, such as mediation effect models and structural equation models. Third, the effect size has also been extended to models other than those in traditional statistics, and a new R2-type effect size was proposed by using variance decomposition.
Subjects: Psychology >> Social Psychology submitted time 2023-03-28 Cooperative journals: 《心理科学进展》
Abstract： With the application of confirmatory factor analysis, research on reliability has entered a new stage. In the first two decades of the 21st century, the studies on test reliability (including point estimation and interval estimation) in China’s mainland show three main lines of development. The first line is the development from research centered on the coefficient αto the reliability research based on confirmatory factor models, including the homogeneity coefficient, composite reliability, maximum reliability, single-indicator reliability and reliability of the whole item set scores. Studies have shown that the coefficient αis still useful. In most cases, the α coefficient is the lower bound of the reliability of the composite score (total or average score). As long as the coefficient αis high enough, the test reliability will be even higher. But the coefficient αcannot be used to measure the homogeneity and the internal consistency of a test. The homogeneity coefficient based on the bi-factor model can be adopted to measure the homogeneity of a multidimensional scale, and the composite reliability can be adopted to measure the internal consistency (if consistency is understood as the consistency within each dimension). Furthermore, the Delta method can be employed to estimate the confidence intervals of various reliability. The second line is the expansion of data types collected by scales (or questionnaires), from single-level data to multi-level and longitudinal data. Whether unidimensional or multidimensional, it is recommended to use a multi-level confirmatory factor model to calculate the reliability of multi-level data. As for the longitudinal data, it is recommended to use the test reliability developed on the basis of the linear mixed model, and the longitudinal data can also be used as a special case of the two-level data for reliability analysis. The third line is the extended use of reliability, involving rater reliability, encoder reliability, attribute-level classification consistency in cognitive diagnostic assessment, and reliability of difference scores. In addition, research of reliability generalization and reliability meta-analysis appeared. For a common test with item-errors that can be reasonably assumed uncorrelated, the following procedure of reliability analysis is recommended. When the coefficient αis high enough, report the coefficient α; otherwise calculate the composite reliability on the basis of the factor model. If the composite reliability is high enough, report the composite reliability; otherwise the test reliability is considered unacceptable. If the composite reliability of every variable in a statistical model is very high (over 0.95), modeling with composite scores does not differ much from modeling with latent variables. Otherwise, it is better to use latent variable modeling.
Subjects: Psychology >> Social Psychology submitted time 2023-03-28 Cooperative journals: 《心理科学进展》
Abstract： Longitudinal research could systematically capture the change of the target variable and thus is more convincing than cross-sectional research. It is popular in the fields of social sciences such as psychology, management, statistics, sociology, etc. The present study reviews the methodology study and model development for analyzing longitudinal data in China’s mainland. We aim to retrospect the methods used, the main research questions, and the popular research domains in longitudinal models. The target publications ranged from 1st Jan. 2001 to 31st Dec. 2020 in CNKI core collections in the relative domains, and finally, 75 articles met our selecting criterion. Results also indicated that the research topic widely includes latent growth model, multilevel modeling, autoregression, cross-lagged model, missing data, etc. Among these research topics, latent growth model ranked as the first. Typically, the latent growth model and experience sampling method were favored in the field of psychology. There are mainly four research questions retrieved from the publications. The first research question is to compare the mean difference, which is less popular. The second research question is to examine the reciprocal relationship between variables. It often uses the cross-lag model and the causal model to reveal the autoregressive and cross-lagged relationships within and between variables. The third research question is to depict growth trajectory with individual differences. It uses the latent growth model (LGM) and multilevel model (MLM) as the main methods to show a growth trajectory from the between-person perspective, as well as the individual difference included. The last one is to explore the dynamic changes. This research question does not focus on the general tendency of change but on the fluctuation between different time points. It usually uses autoregression with its extensions, MLM, time-varying effect model, and some newly developed models such as the dynamic structural equation model. The recent 20 years' publication broadens the domains of longitudinal models, such as the extension of the shape and pattern of growth, the combination of latent class analysis leading to growth mixture model and latent transition analysis. The causal effect, longitudinal mediation and moderation models are also introduced to reveal the relationship between variables. Meanwhile, models depicting growth trajectory with individual differences combines with models examining reciprocal relationships, thus they were extended and integrated to random intercept cross-lagged model, latent variable autoregressive latent trajectory, as well as general cross lagged model. Furthermore, research design becomes more complex; the intensive longitudinal data was introduced and thus the models were according developed, such as MLM, time-varying effect model, dynamic structural equation model, group iterative multiple model estimation, and so forth. Particularly, missing data issue is also hot discussed in the field. To summarize, methodology study for analyzing longitudinal data in China’s mainland has made fruitful development on the above topics and are in an advanced position all over the world. However, when comparing to the international scope, publications in China’s mainland are limited in narrow range. Many topics need to keep up with the international pace, which is a direction that Chinese scholars need to make efforts. Another future direction is to learn from other disciplines to promote the development of interdisciplinary.
Subjects: Psychology >> Social Psychology submitted time 2023-03-28 Cooperative journals: 《心理科学进展》
Abstract： Being able to analyze the influence mechanism of independent variables on dependent variables, the analysis of mediation effect has become an important statistical method in multivariate research. Since the first publication of Chinese paper on the mediation effect and its analytical methods in 2004, the mediation effect has become one focus of methodological research in Chinese Mainland, which is systematically reviewed in this paper. Firstly, the simple mediation model is reviewed with concept identification: how to distinguish between mediation and suppression effects, partial and complete mediation effects, and mediation effect and moderation effect. Then, methodological research on mediation effects in China’s Mainland is divided into five aspects: testing method for mediation effects, mediation effect size measure, mediation effect involving categorical variables or longitudinal data, and extended mediation model. They are summarized as follows. To test ab≠0,the easiest way is to test a≠0 and b≠0. These sequential tests are actually not the same as the joint significance tests because the Type-I error rates are rather different. If the test result is a≠0 and b≠0, then ab≠0 can be inferred with the Type-I error rate less than the significance level 0.05 (the preset significance level), while the Type-I error rate of the joint significance tests is 0.0975. However, if at least one of a≠0 and b≠0 does not hold, the sequential tests should not be used, since its statistical power is less than other alternative test methods discussed in the paper. Anyway, Bootstrap methods are preferred because they provide interval estimation of the mediation effect with a higher power. Furthermore, if appropriate prior information is available, the Bayesian method is also recommended. It is believed that κ2, R2-type and so on are not suitable as mediation effect size measures because of no monotonicity. Although υ=(ab)2υ=(ab)2\upsilon ={{(ab)}^{2}} is monotonic, it is not as simple and clear as the mediation effect (ab) itself. It is recommended that when the signs of ab and c are consistent, the standardized estimation of ab and ab/c should be reported. Mediation analysis with multi-categorical independent variables and with a two-condition within-participant design are discussed when categorical variables are concerned in mediation effect models. There are two types of model development in mediation analysis with longitudinal data. One is continuous time model and multilevel time-varying coefficient model that could be used to test time-varying effect of mediation effect. The other is random-effects cross-lagged panel model and multilevel autoregressive mediation model that could be adopted to examine individuals-varying effect of mediation effect. In addition, latent growth mediation model or multilevel mediation model in mediation effect analysis could be adopted only when the involved causal relationship is instant. Otherwise, cross-lagged panel model, continuous time model, or multilevel autoregressive mediation model should be adopted. The extensions of the mediation model include multiple mediation model, multilevel mediation model, single-level and multilevel moderated mediation model as well as mediated moderation model. These extended models can be used for both the analysis of observed variables and latent variables. Finally, the recent development of foreign methodological research on mediation effects is discussed, including potential outcome mediation analysis, confounder control in mediation analysis, robust mediation analysis, and power analysis of mediation effects. Moreover, integration of new statistical techniques has become a new feature of methodological research of mediation effects, for example, exploratory mediation analysis via regularization, bi-factor mediation analysis, latent class mediation analysis, and network mediation analysis.
Subjects: Psychology >> Social Psychology submitted time 2023-03-28 Cooperative journals: 《心理科学进展》
Abstract： The analysis of moderation effects has become an important statistical method in multivariate studies. Methodological research on moderation effects in China’s mainland covers the following topics: moderation effects of observed variables, latent variables, multi-level data and longitudinal data; the single-level moderation effect analysis based on a two-level regression model; the integration model of moderation and mediation (see Wen et al. 2022). Methodological research on the moderation effect of observed variables includes three aspects: standardized resolution, simple slope test, and the moderation effect of category variables. The research on latent moderation includes three aspects too: standardized resolution, model simplification, and comparison of analytical methods. Under the normal condition, latent moderated structural equations (LMS) are recommended to estimate the moderation effect of latent variables. Otherwise, after centralizing all indicators, the unconstrained product indicator method is recommended to establish a latent moderation model; Bayesian method is an alternative, especially in the case of a small sample. The model development of multilevel moderation effect involves the conflated multilevel model, unconflated multilevel model (UMM), and multilevel structural equation model (MSEM). All independent variables at Level-1 are not centered in the conflated multilevel model, whereas in the UMM all independent variables at level-1 are centered using group-mean, and the group mean is included at Level-2. If the group-mean was treated as a latent variable, MSEM is recommended. Further, two ways are adopted to test multilevel moderation in the multilevel structural equation model: random coefficient prediction (RCP) for cross-level moderations, and LMS for same-level moderations. The moderation effect analysis of longitudinal data is divided into three types. The first type is moderation analysis in two-instance repeated measures designs, in which only the dependent variable is repeated measurement. In the second type, there isn’t any moderator, while both the independent and dependent variables are repeated measurement (e.g., the cross-lagged model, and the contextual moderation model). In the third type, all variables are repeated measurement, such as the latent growth model and multilevel model. Two-level regression model is recommended to analyze the moderation effect of single-level data. It can be employed to analyze the moderation effect of both observed variables and latent variables. Some international frontiers of methodological research on moderation analysis are briefly introduced: the combination of LMS and Bayesian method, moderation analysis of multiple moderators; moderation analysis of longitudinal data.
Subjects: Psychology >> Statistics in Psychology Subjects: Psychology >> Psychological Measurement submitted time 2022-03-08
Abstract：
In the first two decades of the twenty-first century, the hotspots of the methodological research on structural equation models (SEM) in China's mainland generally involve the following five aspects: model development, parameter estimation, model evaluation, measurement invariance and special data processing. Remarkably, there is more progress in model development (i.e., different variations of SEM) amongst the above aspects. After an overview of the background knowledge of these hotspots, we presented the main research topics and methodological achievements under each hotspot. We also discussed the recent progress of the foreign methodological studies on SEM and the future research directions.
Peer Review Status:
Awaiting Review
Subjects: Psychology >> Statistics in Psychology submitted time 2021-12-09
Abstract：使用多元回归法进行调节效应分析在社科领域已常有应用。简述了目前多元回归法的调节效应分析存在的不足，包括人为变换检验模型、自变量和调节变量区分不足、误差方差齐性的假设难以满足、调节效应量指标△R2没有直接测量调节变量对自变量与因变量关系的调节程度。比较好的方法是用两水平回归模型进行调节效应分析并使用相应的效应量指标。在介绍新方法和新效应量后，总结出一套调节效应的分析流程，通过一个例子来演示如何用Mplus软件进行两水平回归模型的调节效应及其效应量分析。最后讨论了两水平回归模型的调节效应分析的发展，包括稳健的调节效应分析、潜变量的调节效应分析、有调节的中介效应分析和有中介的调节效应分析等。
Subjects: Psychology >> Statistics in Psychology submitted time 2021-08-26
Abstract： The mediation effect analysis is able to reveal the process and mechanism of the impact of independent variables on a dependent variable. As an important statistical method, the mediation effect analysis has become a hot topic in methodology research in the last twenty years. The development of domestic research on the methodology of mediation effect was systematically reviewed from the five aspects, including testing method, effect size, the mediation effect test of categorical variables and longitudinal data, and model expansion. Specifically, model expansions include multiple mediation models, multilevel mediation models, moderated mediation model and mediated moderation model. Finally, recent progresses of foreign methodological studies on mediation effect and the future research directions were discussed.
Hits
Hits
Downloads
Comment