Keynotes
Note: All keynotes will be recorded. Please click read more to view each session information in more detail after the conference.
Wim. van der Linden
University of Twente, Netherlands
Presidential Speech
Seung W. Choi
University of Texas at Austin, USA
Optimal test design approach to through-year computerized adaptive testing
Read More
Mariana Curi
University of São Paulo, Brazil
Estimation of MIRT model parameters with deep learning
Read More
Kyung (Chris) T. Han
Graduate Management Admission Council®, USA
Enhancing Test-Taking Experience: Innovative CAT Designs for Improved Learning and Evaluation
Read More
Chia-Ling Hsu (Early Career Award Winner)
Hong Kong Examinations and Assessment Authority
Variable-length adaptive multistage testing
Read More
Peter W. van Rijn
ETS Global, Netherlands
The Impact of Adaptive Testing Designs on Statistical and AI-Based Methods
Read More
Ji Hoon Ryoo
Yonsei University, South Korea
AI-based AIG within a Learning Engineering Framework
Read More
Workshops
Duanli Yan (ETS) and Alina A. von Davier(Duolingo)
Introduction to AI-based Automated Item Generation and Scoring in Adaptive Testing
This workshop introduces a novel framework, "the item factory", for managing large-scale test development including automation of item generation, quality review, quality assurance, and crowdsourcing techniques in adaptive testing. We will present an overview of the latest natural language processing (NLP) techniques and large language models for automatic item generation, alongside evidence-centered design and psychometric principles and practices for test development. We will discuss the application of engineering principles in designing efficient item production processes (Luecht, 2008; Dede et al, 2018; von Davier, 2017).
Kyung (Chris) T. Han (Graduate Management Admission Council®) and Nathan Thompson (ASC)
Introduction to IRT and CAT
This workshop provides a broad overview of item response theory (IRT) and computerized adaptive testing (CAT) for those who are newer to the field. We assume a knowledge of basic psychometrics such as classical test theory. The workshop begins with a background on (IRT), how it can be used to evaluate item and test performance, and how it provides a number of improvements over the classical approach. We then provide an introduction to CAT, describing the components and algorithms necessary to build an effective CAT program: item bank calibrated with IRT, starting point, item selection rule, scoring method, and termination criterion. Finally, we discuss important aspects regarding how you might evaluate and implement CAT for your organization.
Seung W. Choi (University of Texas, Austin) and Wim J. van der Linden (University of Twente)
The Shadow-Test Approach to Adaptive Testing
This short course has four different sections. In the first section, we explain the ideas underlying the shadow-test approach, discuss a few practical aspects of its implementation, and show some of its generalizations to different test formats. The next two sections present two applications of the approach illustrating its versatility. One application is adaptive testing with a mixture of discrete items and items organized as sets around a common stimulus with continued adaptation within each set. The other is adaptive testing with field-test items inserted for adaptive calibration. The final section of the course demonstrates software available for the implementation of the shadow-test approach to adaptive testing and offers the participants hands-on experience with the R package TestDesign. In addition, a brief introduction will be given to Optimal CAT, a currently freely downloadable microservice available for easy integration with current test delivery systems.