Nonparametric IRT analysis

Leognano Ceraudo

Chapter 1: Item Response Theory

Item Response Theory

Item Response Theory is a system of models that define a correspondence between latent variables (unidimensional continuum construct) and their manifestations (items).

  • Focus of interest: Item-Level

  • Item-Type: Dichotomous and Ordered or Unordered Polytomous Items

  • Latent Trait (theta): Continuous and Normally Distribuited

  • Error term: It varies across items and latent trait

  • Item Response Function Shape: Sigmoid-Shape

Item Characteristic Curves

Item Characteristic Curves

Options Characteristic Curves

IRT Assumptions

What if assumptions are weakened or not satisfied?

OCC edition

Chapter 2: KernSmoothIRT

KernSmoothIRT

KerSmoothIRT implements Kernel Smoothing for the estimation of Option Characteristic Curves to evaluate the whole test, the items and the subjects.

Kernel Smoothing IRT ICC: step by step

Kernel Smoothing OCC: Step by Step

Chapter 3: KernSmoothIRT tutorial

library(KernSmoothIRT)

KernSmoothIRT

# A tibble: 6 × 18
  Sex     Age VR4   VR16  VR17  VR19  LN7   LN33  LN34  LN58  MR45  MR46  MR47 
  <chr> <dbl> <chr> <chr> <chr> <chr> <chr> <chr> <chr> <chr> <chr> <chr> <chr>
1 F        19 D2    D3    D1    D3    D2    D4    D3    D4    C     D2    C    
2 F        19 D2    D2    C     C     C     D2    D1    D4    D3    C     C    
3 F        19 C     C     C     C     D5    D1    C     C     C     C     C    
4 M        19 C     C     C     D2    D4    D2    C     D3    D2    D1    D4   
5 F        22 D1    D2    C     C     C     D5    C     D3    D4    D5    D5   
6 M        20 C     D3    C     C     D5    D2    C     D2    C     C     D5   
# ℹ 5 more variables: MR55 <chr>, R3D3 <chr>, R3D4 <chr>, R3D6 <chr>,
#   R3D8 <chr>

KernSmoothIRT

Icar2a<- Icar2[, c(3:18)]
head(Icar2a)
# A tibble: 6 × 16
  VR4   VR16  VR17  VR19  LN7   LN33  LN34  LN58  MR45  MR46  MR47  MR55  R3D3 
  <chr> <chr> <chr> <chr> <chr> <chr> <chr> <chr> <chr> <chr> <chr> <chr> <chr>
1 D2    D3    D1    D3    D2    D4    D3    D4    C     D2    C     D4    D6   
2 D2    D2    C     C     C     D2    D1    D4    D3    C     C     D4    C    
3 C     C     C     C     D5    D1    C     C     C     C     C     D2    D3   
4 C     C     C     D2    D4    D2    C     D3    D2    D1    D4    D5    D1   
5 D1    D2    C     C     C     D5    C     D3    D4    D5    D5    D2    D7   
6 C     D3    C     C     D5    D2    C     D2    C     C     D5    D1    D4   
# ℹ 3 more variables: R3D4 <chr>, R3D6 <chr>, R3D8 <chr>
Icar2a<- ifelse(Icar2a== "C", 0, 
                ifelse(Icar2a=="D1", 1, 
                ifelse(Icar2a=="D2", 2, 
                ifelse(Icar2a== "D3", 3, 
                ifelse(Icar2a== "D4", 4, 
                ifelse(Icar2a== "D5", 5, 
                ifelse(Icar2a== "D6", 6, 
                ifelse(Icar2a== "D7", 7, NA))))))))
head(Icar2a)
     VR4 VR16 VR17 VR19 LN7 LN33 LN34 LN58 MR45 MR46 MR47 MR55 R3D3 R3D4 R3D6
[1,]   2    3    1    3   2    4    3    4    0    2    0    4    6    0    0
[2,]   2    2    0    0   0    2    1    4    3    0    0    4    0    3    0
[3,]   0    0    0    0   5    1    0    0    0    0    0    2    3    3    4
[4,]   0    0    0    2   4    2    0    3    2    1    4    5    1    6    5
[5,]   1    2    0    0   0    5    0    3    4    5    5    2    7    7    7
[6,]   0    3    0    0   5    2    0    2    0    0    5    1    4    6    1
     R3D8
[1,]    3
[2,]    0
[3,]    4
[4,]    3
[5,]    7
[6,]    2
key<- c(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0)

KernSmoothIRT

Mod1<- ksIRT(Icar2a[, 1:16], format = 1, key= key , kernel = c("gaussian"),weights = NULL, miss = c("option"), NAweight = 0, bandwidth = c("Silverman"), RankFun = "sum", thetadist = list("norm",0,1), groups = FALSE)

KSIRT: pca

#Difficulty-Discrimination Items analysis
plot(Mod1, plottype= "PCA")

KSIRT: occ (theta)

plot(Mod1, plottype = "OCC", axistype = "distribution", items= c(10))#P (theta)

KSIRT: OCC (EIS)

plot(Mod1, plottype = "OCC", axistype = "scores", alpha= T, items = c(10))#P(observed scores)

KSIRT: OCC triangle

plot(Mod1, plottype= "triangle", items = c(2))

KSIRT: OCC DIF

Icar2a$gr1<-as.factor(Icar2a$Sex)
Icar2a$gr1<-ifelse(Icar2a$Sex== "F", "F", "M")

Mod2<- ksIRT(Icar2a[, 1:16], format = 1, key= key , kernel = c("gaussian"),weights = NULL, miss = c("option"), NAweight = 0, bandwidth = c("Silverman"), RankFun = "sum", thetadist = list("norm",0,1), groups = Icar2a$gr1)

KSIRT: OCC DIF

plot(Mod2, plottype = "OCCDIF", axistype = "distribution", items = c(10))

KSIRT: OCC DIF (EIS)

plot(Mod2, plottype = "EISDIF", axistype = "scores", items = c(10))

References:

  • Mazza, A., Punzo, A., & McGuire, B. (2014). KernSmoothIRT: An R package for kernel smoothing in item response theory. Journal of Statistical Software, 58(6), 1–34.

  • Junker, B. W., & Sijtsma, K. (2001). Nonparametric item response theory in action: An overview of the special issue. Applied Psychological Measurement, 25(3), 211–220.

  • Ramsay, J. O. (1991). Kernel smoothing approaches to nonparametric item characteristic curve estimation. Psychometrika, 56(4), 611–630.

Thank You!