Methods: Five OSCEs were used in the study: 3 chiropractic licensure OSCEs and 2 nursing licensure OSCEs. Items were assigned to split-halves based on common stimulus. Split-half correlations were compared with Spearman-Brown estimates of reliability based on Cronbach á with all items contributing. Two- and 3-facet MFRM analyses were performed, first with individual items contributing and second with station totals contributing. Correlations were estimated between the 2 MFRM estimates.
Results: Cronbach á estimates with all items contributing were all very high (>.87). Spearman-Brown estimates were all considerably higher than split-half correlations. Correlations between MFRM by items and by stations were all very high (>.993).
Conclusions: The research project provided evidence that OSCEs violate the local item independence assumption. The project also showed that the MFRM is quite robust to such violations. The authors recommend that the MFRM be applied to OSCEs by station totals for estimates of candidate ability, and by items for item performance measures and quality control programs.
This abstract is reproduced with the permission of the publisher. Full text is available by subscription. DOI Link