基于信号检测论的认知诊断评估：构建与应用

• 作者： 郭磊 ^1,2 秦海江 ^1,3
• 作者单位：
  1.  西南大学心理学部，重庆400715
  2.  中国基础教育质量监测协同创新中心西南大学分中心，重庆400715
  3.  贵阳市第三十七中学，贵阳 550003
• 通讯作者： 郭磊  Email:happygl1229@swu.edu.cn
• 提交时间：2023-11-13

摘要: 作答选择题可被看作从噪音中提取信号的过程，研究提出了一种基于信号检测论的认知诊断模型(SDT-CDM)。新模型的优势在于：(1)无需对选项进行属性层面的编码。(2)能获得传统诊断模型无法提供的题目区分度和难度参数。(3)可以直接表达每个选项之间的合理性差异，对题目性能刻画更加细微全面。两个模拟研究结果表明：(1)EM算法可以实现对新模型的参数估计过程，便捷有效。(2)SDT-CDM具备良好性能，分类准确性和参数估计精度较高以外，还能提供选项层面的估计信息，用于题目质量诊断与修订。(3)属性数量、题目质量与样本量等因素会影响SDT-CDM的表现。(4)与称名诊断模型NRDM相比，SDT-CDM在所有实验条件下对被试的分类准确性更高。实证研究表明：SDT-CDM比NRDM具有更好的模型数据拟合结果，其分类准确性和一致性更高，尤其当属性考察次数较少时具有很强的稳定性，难度和区分度参数与IRT模型估计结果的相关性也更高，值得推广。

Abstract: Cognitive diagnostic assessment (CDA) is aimed at diagnose which skills or attributes examinees have or do not have as the name expressed. This technique provides more useful feedback to examinees than a simple overall score got from classical test theory or item response theory. In CDA, multiple-choice (MC) is one of popular item types, which have the superiority on high test reliability, being easy to review, and scoring quickly and objectively. Traditionally, several cognitive diagnostic models (CDMs) have been developed to analyze the MC data by including the potential diagnostic information contained in the distractors.However, the response to MC items can be viewed as the process of extracting signals (correct options) from noises (distractors). Examinees are supposed to have perceptions of the plausibility of each options, and they make the decision based on the most plausible option. Meanwhile, there are two different states when examinee response to items: knows or does not know each item. Thus, the signal detection theory can be integrated into CDM to deal with MC data in CDA. The cognitive diagnostic model based on signal detection theory (SDT-CDM) is proposed in this paper and has several advantages over traditional CDMs. Firstly, it does not require the coding of q-vector for each option. Secondly, it provides discrimination and difficulty parameters that traditional CDMs cannot provide. Thirdly, it can directly express the relative differences between each options by plausibility parameters, providing a more comprehensive characterization of item quality.The results of two simulation studies showed that (1) the marginal maximum likelihood estimation approach via Expectation Maximization (MMLE/EM) algorithm could effectively estimate the model parameters of the SDT-CDM. (2) the SDT-CDM had high classification accuracy and parameter estimation precision, and could provide option-level information for item quality diagnosis. (3) independent variables such as the number of attributes, item quality, and sample size affected the performance of the SDT-CDM, but the overall results were promising. (4) compared with the nominal response diagnostic model (NRDM), the SDT-CDM was more accurate in classifying examinees under all data conditions.Further, an empirical study on the TIMSS 2011 mathematics assessment were conducted using both the SDT-CDM and the NRDM to inspect the ecological validity for the new model. The results showed that the SDT-CDM had better fitting and a smaller number of model parameters than the NRDM. The difficulty parameters of the SDT-CDM were significantly correlated with those of the two- (three-) parameter logical models. And the same was true of the discrimination parameters for the SDT-CDM. However, the correlation between the discrimination parameters of the NRDM and those of the two- (three-) parameter logical models was low and not significant. Besides, the classification accuracy and classification consistency of the SDT-CDM were higher than those of the NRDM. All the results indicated that the SDT-CDM was worth promoting.

• 来自： 郭磊
• 期刊： 心理学报
• 分类： 心理学 >> 心理测量
• 投稿状态： 已被期刊接收
• 引用： ChinaXiv:202311.00128 (或此版本 ChinaXiv:202311.00128V1)
  DOI:10.12074/202311.00128V1
  CSTR:32003.36.ChinaXiv.202311.00128.V1
• 推荐引用方式： 郭磊,秦海江.(2023).基于信号检测论的认知诊断评估：构建与应用.心理学报.doi:10.12074/202311.00128V1 (点此复制)