The Functionality of a Hierarchical Generalized Linear Model Used for Detecting Differential Item Functioning with a Fisher-Tippett Ability Distribution

Abstract

The purpose of this study is to evaluate the hierarchical generalized linear model (HGLM) used for detecting differential item functioning (DIF) as proposed by Kamata when the distribution of examinees’ abilities is a Fisher-Tippett distribution. In the study, 1,000 examinees are divided equally into two groups. In the study, we investigated four factors. The first factor is whether or not there is difference in mean ability between two groups. The second factor is the DIF proportion, which is arbitrary set to 3 different levels, 5%, 15%, and 30%. The third factor is the DIF magnitude, which is set to 3 levels in a logit scale, 0.3, 0.5, and 0.7. The fourth factor is the type of method used for detecting DIF of the HGLM, a simultaneous method or an item-by-item method. This work is a simulation study in which the data are generated by using R Program version 2.5.1. After the data are generated, they are analyzed by using HLM version 6.0, with 100 runs for each set of conditions. The performance of the HGLM used for detecting DIF is evaluated using hit and false alarm rates. The result from this study is that the Fisher-Tippett distribution as an ability distribution does not affect the detection of DIF in a significant manner for the HGLM-DIF model.