Detection of aberrant testing behaviour in unproctored CAT via a verification test

Ebru Balta; Arzu Uçar

Article Details

Detection of aberrant testing behaviour in unproctored CAT via a verification test

Authors:

Ebru Balta | Arzu Uçar

Publisher:

İzzet KARA

Main Topics:

Computer Based Exam Applications | Similation Study

Journal:

International Journal of Assessment Tools in Education Vol.12 Issue 3, pp.681-700

Document Type:

Article

Publication Date:

4/2025

Document Language:

English

Document ID (Org No.):

2001347639

pISSN:

2148-7456

Keywords:

Abstract:

 Unproctored Computerized Adaptive Testing CAT is gaining traction due to its convenience, flexibility, and scalability, particularly in high-stakes assessments. However, the lack of proctor can give rise to aberrant testing behavior. These behaviors can impair the validity of test scores. This paper explores the use of a verification test to detect aberrant testing behavior in unproctored CAT environments. This study aims to use multiple measures to detect aberrant response patterns in CAT via a paper-and-pencil PP test as well as to compare the sensitivity and specificity performances of the l_z person-fit statistic PFS using no-stage and two-stage l_z is used after the KullbackLeibler divergence KLD measure methods in different conditions. Three factors were manipulated  the aberrance percentage, the aberrance scenario, and the aberrant examinees ability range. The study found that in all scenarios, the specificity performance of l_z in classifying examinees was higher than its sensitivity performance in no-stage and two-stage analyses. However, the sensitivity performance of l_z was higher in two-stage analysis.

 Unproctored Computerized Adaptive Testing CAT is gaining traction due to its convenience, flexibility, and scalability, particularly in high-stakes assessments. However, the lack of proctor can give rise to aberrant testing behavior. These behaviors can impair the validity of test scores. This paper explores the use of a verification test to detect aberrant testing behavior in unproctored CAT environments. This study aims to use multiple measures to detect aberrant response patterns in CAT via a paper-and-pencil PP test as well as to compare the sensitivity and specificity performances of the l_z person-fit statistic PFS using no-stage and two-stage l_z is used after the KullbackLeibler divergence KLD measure methods in different conditions. Three factors were manipulated  the aberrance percentage, the aberrance scenario, and the aberrant examinees ability range. The study found that in all scenarios, the specificity performance of l_z in classifying examinees was higher than its sensitivity performance in no-stage and two-stage analyses. However, the sensitivity performance of l_z was higher in two-stage analysis.

 Unproctored Computerized Adaptive Testing CAT is gaining traction due to its convenience, flexibility, and scalability, particularly in high-stakes assessments. However, the lack of proctor can give rise to aberrant testing behavior. These behaviors can impair the validity of test scores. This paper explores the use of a verification test to detect aberrant testing behavior in unproctored CAT environments. This study aims to use multiple measures to detect aberrant response patterns in CAT via a paper-and-pencil PP test as well as to compare the sensitivity and specificity performances of the l_z person-fit statistic PFS using no-stage and two-stage l_z is used after the KullbackLeibler divergence KLD measure methods in different conditions. Three factors were manipulated  the aberrance percentage, the aberrance scenario, and the aberrant examinees ability range. The study found that in all scenarios, the specificity performance of l_z in classifying examinees was higher than its sensitivity performance in no-stage and two-stage analyses. However, the sensitivity performance of l_z was higher in two-stage analysis.

 Unproctored Computerized Adaptive Testing CAT is gaining traction due to its convenience, flexibility, and scalability, particularly in high-stakes assessments. However, the lack of proctor can give rise to aberrant testing behavior. These behaviors can impair the validity of test scores. This paper explores the use of a verification test to detect aberrant testing behavior in unproctored CAT environments. This study aims to use multiple measures to detect aberrant response patterns in CAT via a paper-and-pencil PP test as well as to compare the sensitivity and specificity performances of the l_z person-fit statistic PFS using no-stage and two-stage l_z is used after the KullbackLeibler divergence KLD measure methods in different conditions. Three factors were manipulated  the aberrance percentage, the aberrance scenario, and the aberrant examinees ability range. The study found that in all scenarios, the specificity performance of l_z in classifying examinees was higher than its sensitivity performance in no-stage and two-stage analyses. However, the sensitivity performance of l_z was higher in two-stage analysis.

Article Details

Detection of aberrant testing behaviour in unproctored CAT via a verification test

Authors:

Ebru Balta | Arzu Uçar

Publisher:

İzzet KARA

Main Topics:

Computer Based Exam Applications | Similation Study

Journal:

International Journal of Assessment Tools in Education Vol.12 Issue 3, pp.681-700

Document Type:

Article

Publication Date:

4/2025

Document Language:

English

Document ID (Org No.):

2001347639

pISSN:

2148-7456

Aberrant testing behaviour

l_z person-fit statistic

Divergence measure

Unproctored CAT

Verification test.Aberrant testing behaviour

l_z person-fit statistic

Divergence measure

Unproctored CAT

Verification test.

Issues

Article Details

Authors: Ebru Balta | Arzu Uçar

Publisher: İzzet KARA

Main Topics: Computer Based Exam Applications | Similation Study

Journal: International Journal of Assessment Tools in Education Vol.12 Issue 3, pp.681-700

Document Type: Article

Publication Date: 4/2025

Document Language: English

Document ID (Org No.): 2001347639

pISSN: 2148-7456

Issues

Authors:

Ebru Balta | Arzu Uçar

Publisher:

İzzet KARA

Main Topics:

Computer Based Exam Applications | Similation Study

Journal:

International Journal of Assessment Tools in Education Vol.12 Issue 3, pp.681-700

Document Type:

Article

Publication Date:

4/2025

Document Language:

English

Document ID (Org No.):

2001347639

pISSN:

2148-7456