THE EFFECT OF ERROR IN ITEM PARAMETER ESTIMATES ON LINKING AND EQUATING WITH THE IRT TEST CHARACTERISTIC CURVE METHOD
Kaskowitz, Gary Scott
De Ayala, Rafael J.
MetadataShow full item record
This research looked at the effect of item parameter estimation error on the estimation o f the equating coefficients for the test characteristic curve (TCC) method in Item Response Theory (IRT) equating. Robustness and accuracy of equated scores based upon these equating coefficient estimates were also examined. The study introduced the Improvement Ratio method for analyzing the accuracy of the obtained equating coefficient estimates based on the TCC method of equating. The Improvement Ratio is obtained by calculating the maximum loss function between the two TCCs that are being equated (i.e., how far apart the curves are initially). It compares the distance between two TCCs after they have been linked against the maximum distance of the TCCs prior to the linking to form a ratio of maximum improvement available. This simulation study modeled a horizontal internal anchor equating situation by creating item parameter “estimates” for initial and target metrics generated from known sampling distributions. The crossing of the standard error factor (low, moderate, high), the number o f common items factor (5, 15 or 25), the IRT model factor (2PL or 3PL) and the relationship of the standard error with the b estimates (non-related error or related error) produced 36 cells for study. The results of the study show that for both the non-related error and related error parameters, the use of a minimum of 15 or 25 common items provided much better results than 5 common items. It was seen that the linkings were generally very good even with large initial differences between the TCCs being equated. This was reflected in the values of IR which showed successive improvement as the error condition increased for a given set of parameters. The results also showed that the error in True Score estimation was relatively constant within a given ICC function and parameter set (i.e., non-related error and related error). This finding would indicate that even when there is a fair amount o f error in the item parameter estimates, the TCC equating method is relatively robust.