| Abstract: | The focus on cognitive diagnosis assessment (CDA) has become particularly intense in test theory research in recent years, which can provide detailed information about the strengths and weaknesses of examinees for specific content domains. Meanwhile, computerized adaptive testing (CAT) can provide equivalent or even higher accuracy in the measurement of an examinee’s latent skills, but with reductions in test length of up to 50%, compare to traditional paper-and-pencil testing. Recently, aiming to maximize the benefits of both CDA and CAT, researchers have attempted to combine the two, and naming cognitive diagnostic computerized adaptive testing (CD-CAT).During CD-CAT, many factors can affect the reliability and validity of the test, one of which is the balance of attribute coverage. It is very important to make sure that each attribute is measured adequately, or the reliability of the test will be reduced. Therefore, researchers have developed some attribute balance indices (ABIs) to satisfy the attribute coverage. While a shortcoming of both the ABI and revised ABI (RABI) is that they tend to select items that measuring single attribute, therefore, items measuring single attribute will be overused, as a consequence, an uneven distribution of item exposures will be raised. The improved ABI (IABI), on the contrary, inclines to select items that measuring all attributes even when the minimum number of items that measuring some specific attribute are satisfied. To overcome the shortcomings of these ABIs in some degree, two new attribute coverage control methods?modified ABI (MABI) and ratio of test length to the number of attributes (RTA)? are proposed in current study.To examine the performance of MABI and RTA, a Monte Carlo simulation was conducted. Five factors are manipulated: Number of attributes (4 and 6), test length (20 and 30 items), attribute coverage control method (without attribute coverage control [Non], ABI, IABI, RABI, MABI, and RTA), and item selection method (KL, PWKL, MI, and MPWKL methods), and model type (DINA, RRUM). There are 2 × 2 × 4 × 6 × 2 = 192 conditions in current study, of these, attribute coverage control method and item selection methods are within-group variable, and the rest are between-group variables. In addition, the covariates include number of items in the item bank (500 items), number of individuals, and distribution of item parameters. Furthermore, the minimum items measuring each attribute (Bk) are fixed as 4. The evaluate criteria are pattern correct classification rate (PCCR), attribute correct classification rate (ACCR), and the usage of k-attribute items. The results show that: (a) Methods with attribute coverage control (ABI, IABI, RABI, MABI, and RTA), in general, perform better than the method that without attribute coverage control (Non). (b) The ABI and RABI method produces higher PCCR and ACCR than MABI and RTA method in most conditions, while produces more uneven item usage than MABI and RTA. (c) The performance among IABI, MABI, and RTA are twisted each other under different conditions. (d) The IABI, MABI and RTA methods can deal with the trade-off between correct classification rate and item usage quite well. |
