Investigation of differential item functioning of the PISA 2015 science literacy items with the latent class approach

Onurcan Ceyhan; Hamide Deniz Gülleroğlu

Article Details

Investigation of differential item functioning of the PISA 2015 science literacy items with the latent class approach

Authors:

Onurcan Ceyhan | Hamide Deniz Gülleroğlu

Publisher:

Kocaeli University

Main Topics:

Alan Eğitimleri | Eğitimde Ölçme ve Değerlendirme (Diğer)

Journal:

Kocaeli Üniversitesi Eğitim Dergisi Vol.8 Issue 1, pp.178-204

Document Type:

Article

Publication Date:

5/2025

Document Language:

English

Document ID (Org No.):

2001349096

pISSN:

2636-8846

Keywords:

Abstract:

 This study examines 11 items determined among the science literacy items of the PISA 2015 in terms of DIF using the latent class method. In the initial investigations, the response pattern met the assumptions of unidimensionality, monotony and local independence, which are the assumptions of the IRT for Türkiye study group consisting of 506 people. As a result of the estimations made with the IRT, it was concluded that items were suitable for the Mixed IRT 2PLM with two latent classes. The analyses were conducted based on the Mixed IRT 2PLM with two latent classes, and it was determined that approximately 91 of the respondents were in the first and 9 were in the second latent class. In the DIF analysis based on the Mixed IRT 2PLM with two latent classes, item 4 showed both non-uniform and uniform DIF. It was concluded that the items numbered 5, 6, 9, and 11 only showed uniform DIF. Item 4, 5, 6, 9, and 11 functioned differently in two latent classes. In the literature review, variables reported as DIF sources were included in the model separately as covariant variables. The percentage of classification accuracy increased among these covariant variables such as gender, ICT resources, enjoyment of science, and sense of belonging to the school. Therefore, these specified covariant variables serve as either a predictor of the latent variable or a source of a priori knowledge. Observed variables were added as covariant variables, DIF was determined as a source, and it was concluded that they were insufficient to explain latent classes. This study showed that if the DIF source based on the observed variable had a weak relationship with the latent variable, it did not significantly affect the validity of the test results.

 This study examines 11 items determined among the science literacy items of the PISA 2015 in terms of DIF using the latent class method. In the initial investigations, the response pattern met the assumptions of unidimensionality, monotony and local independence, which are the assumptions of the IRT for Türkiye study group consisting of 506 people. As a result of the estimations made with the IRT, it was concluded that items were suitable for the Mixed IRT 2PLM with two latent classes. The analyses were conducted based on the Mixed IRT 2PLM with two latent classes, and it was determined that approximately 91 of the respondents were in the first and 9 were in the second latent class. In the DIF analysis based on the Mixed IRT 2PLM with two latent classes, item 4 showed both non-uniform and uniform DIF. It was concluded that the items numbered 5, 6, 9, and 11 only showed uniform DIF. Item 4, 5, 6, 9, and 11 functioned differently in two latent classes. In the literature review, variables reported as DIF sources were included in the model separately as covariant variables. The percentage of classification accuracy increased among these covariant variables such as gender, ICT resources, enjoyment of science, and sense of belonging to the school. Therefore, these specified covariant variables serve as either a predictor of the latent variable or a source of a priori knowledge. Observed variables were added as covariant variables, DIF was determined as a source, and it was concluded that they were insufficient to explain latent classes. This study showed that if the DIF source based on the observed variable had a weak relationship with the latent variable, it did not significantly affect the validity of the test results.

 This study examines 11 items determined among the science literacy items of the PISA 2015 in terms of DIF using the latent class method. In the initial investigations, the response pattern met the assumptions of unidimensionality, monotony and local independence, which are the assumptions of the IRT for Türkiye study group consisting of 506 people. As a result of the estimations made with the IRT, it was concluded that items were suitable for the Mixed IRT 2PLM with two latent classes. The analyses were conducted based on the Mixed IRT 2PLM with two latent classes, and it was determined that approximately 91 of the respondents were in the first and 9 were in the second latent class. In the DIF analysis based on the Mixed IRT 2PLM with two latent classes, item 4 showed both non-uniform and uniform DIF. It was concluded that the items numbered 5, 6, 9, and 11 only showed uniform DIF. Item 4, 5, 6, 9, and 11 functioned differently in two latent classes. In the literature review, variables reported as DIF sources were included in the model separately as covariant variables. The percentage of classification accuracy increased among these covariant variables such as gender, ICT resources, enjoyment of science, and sense of belonging to the school. Therefore, these specified covariant variables serve as either a predictor of the latent variable or a source of a priori knowledge. Observed variables were added as covariant variables, DIF was determined as a source, and it was concluded that they were insufficient to explain latent classes. This study showed that if the DIF source based on the observed variable had a weak relationship with the latent variable, it did not significantly affect the validity of the test results.

 Bu çalışmanın amacı, Uluslararası Öğrenci Değerlendirme Programı PISA 2015 uygulamasının fen okuryazarlığı maddeleri arasından belirlenen 11 maddenin, gizil sınıf yöntemi kullanılarak Değişen Madde Fonksiyonu DMF açısından incelenmesidir. Yapılan ilk incelemelerde, yanıt örüntüsü, 506 kişilik Türkiye çalışma grubu için Madde Tepki Kuramı MTK varsayımları olan tek boyutluluk, monotonluk ve yerel bağımsızlık varsayımlarını karşılamıştır. MTK ile yapılan tahminler sonucunda maddelerin 2 Gizil Sınıflı Karma 2PLM Mixed IRT 2PLM with two latent classes için uygun olduğu sonucuna varılmıştır. Analizler 2 Gizil Sınıflı Karma 2PLM temel alınarak yapılmış ve ankete katılanların yaklaşık 91inin birinci, 9unun ise ikinci gizli sınıfta yer aldığı belirlenmiştir. İki gizil sınıflı Karma 2 PLM temel alınarak yapılan DMF analizinde Madde 4ün hem tek biçimli olmayan TBO hem de tek biçimli olan TB DMF gösterdiği belirlenmiştir. 5, 6, 9 ve 11 numaralı maddelerin sadece tek biçimli olan TB DIF gösterdiği sonucuna varılmıştır. 5, 6, 9 ve 11. maddelerin iki gizil sınıfta farklı fonksiyonlaştıkları görülmüştür. DMF içeren maddelerin gizil sınıfları açıklama gücünü incelemek amacıyla literatür taramasında DMF kaynağı olarak raporlanmış değişkenler kovaryant olarak ayrı ayrı modele dahil edilmiştir. Cinsiyet, bilgi ve teknoloji iletişim kaynakları, fen öğretiminden zevk alma ve okula aidiyet duygusu gibi kovaryant değişkenler modele ayrı ayrı eklendiğinde sınıflandırma doğruluğu yüzdesini arttırmıştır. Dolayısıyla bu çalışmada kovaryant değişkenin gizil değişkenin bir yordayıcısı ya da önsel bilgi kaynağı olarak görev yaptığı sonucuna ulaşılmıştır. Bu durum bireylerin fen okuryazarlığı maddelerine ilişkin tepkilerinde bu değişkenlerden farklı bir boyutun etkisi ile gizil sınıflara ayrıldıklarını göstermektedir. Yine bu çalışma gözlenen değişkene dayalı DMF kaynağının gizil değişken ile zayıf düzeyde ilişkisi olması durumunda, test sonuçlarının geçerliğini önemli ölçüde etkilemediğini göstermiştir.

Article Details

Investigation of differential item functioning of the PISA 2015 science literacy items with the latent class approach

Authors:

Onurcan Ceyhan | Hamide Deniz Gülleroğlu

Publisher:

Kocaeli University

Main Topics:

Alan Eğitimleri | Eğitimde Ölçme ve Değerlendirme (Diğer)

Journal:

Kocaeli Üniversitesi Eğitim Dergisi Vol.8 Issue 1, pp.178-204

Document Type:

Article

Publication Date:

5/2025

Document Language:

English

Document ID (Org No.):

2001349096

pISSN:

2636-8846

differential item functioning

latent class approach

covariant

mixture item response theory

PisaPISA

değişen madde fonksiyonu

gizil sınıf yaklaşımı

kovaryant

karma madde tepki kuramı

Issues

Article Details

Authors: Onurcan Ceyhan | Hamide Deniz Gülleroğlu

Publisher: Kocaeli University

Main Topics: Alan Eğitimleri | Eğitimde Ölçme ve Değerlendirme (Diğer)

Journal: Kocaeli Üniversitesi Eğitim Dergisi Vol.8 Issue 1, pp.178-204

Document Type: Article

Publication Date: 5/2025

Document Language: English

Document ID (Org No.): 2001349096

pISSN: 2636-8846

Issues

Authors:

Onurcan Ceyhan | Hamide Deniz Gülleroğlu

Publisher:

Kocaeli University

Main Topics:

Alan Eğitimleri | Eğitimde Ölçme ve Değerlendirme (Diğer)

Journal:

Kocaeli Üniversitesi Eğitim Dergisi Vol.8 Issue 1, pp.178-204

Document Type:

Article

Publication Date:

5/2025

Document Language:

English

Document ID (Org No.):

2001349096

pISSN:

2636-8846