Where Assessment Validation and Responsible AI Meet

Jill Burstein, Geoffrey T. LaFlair

doi:10.32038/ltrq.2025.50.09

European KnowledgeDevelopment Institute

Perspective Article

Where Assessment Validation and Responsible AI Meet

Jill Burstein

Geoffrey T. LaFlair

Language Teaching Research Quarterly, Volume 50, Pages 120-137, https://doi.org/10.32038/ltrq.2025.50.09

Abstract
Full text
Metrics

The core principles of validity, reliability, and fairness have been the foundation of ethical assessment practices, as discussed in classical validation theories (e.g., Chapelle et al., 2008; Kane, 1992, 2013) and the American Educational Research Association (AERA), the American Psychological Association (APA), and the National Council on Measurement in Education Standards (NCME) (AERA & APA & NCME, 2014). The Standards have provided best practices for AI use in high-stakes testing, particularly in the automated scoring of written and spoken responses. Responsible AI (RAI) is essential across industry domains, including educational assessments. With recent advancements in generative AI, new policies and guidance on applying RAI principles in assessment have emerged. Expanding on Chapelle et al.'s (2008) work, this paper introduces a unified assessment framework that integrates traditional validation theory with both assessment-specific and domain-agnostic RAI principles. This framework supports responsible AI use, aligns with ethical principles to uphold human values and oversight, and promotes broader social responsibility in AI-driven assessments.

Loading PDF…

Page 1 of

Download Count : 28

Visit Count : 50

Keywords

Language Assessment; Responsible AI; Validity Theory

Previous article in volume

Underestimated Utility: Recording Design Decisions in Language Test Development

Next article in volume

Developing an Adapted English Proficiency Test for Adult Job Seekers with Intellectual and Developmental Disabilities (IDD): Literature Review

Author(s) Information

Jill Burstein

Jill Burstein, Ph.D. is a Principal Assessment Scientist at Duolingo, leading the validity research team for the Duolingo English Test. Her career focuses on equitable and accessible education through technology, intersecting artificial intelligence, assessment research, and responsible AI. She has developed automated writing evaluation systems for high-stakes assessments and online writing support, holding multiple patents for this work. Dr. Burstein authored the Duolingo English Test responsible AI standards and co-founded SIG EDU, an ACL Special Interest Group. She earned her Ph.D. in Linguistics from the Graduate Center, City University of New York.

Geoffrey T. LaFlair

Geoff LaFlair is a Principal Assessment Scientist at Duolingo where he co-leads Assessment Research and Development for the Duolingo English Test. He holds a PhD in Applied Linguistics from Northern Arizona University. Prior to joining Duolingo, he was an Assistant Professor in the Department of Second Language Studies at the University of Hawai’i at Mānoa and the Director of Assessment in the Center for ESL at the University of Kentucky. His research interests are situated at the intersection of language assessment, psychometrics, and AI, focusing on the application of research from these fields in researching and developing operational language assessments.

Publisher’s Note

The claims, arguments, and counter-arguments made in this article are exclusively those of the contributing authors. Hence, they do not necessarily represent the viewpoints of the authors’ affiliated institutions, or EUROKD as the publisher, the editors and the reviewers of the article.

Acknowledgements

We would like to acknowledge Ravit Dotan for her guidance about the alignment between the NIST AI RMF and the DET RAI Standards. Many thanks to our Duolingo colleagues, Ben Naismith, Yena Park, and Alina von Davier for their reviews.

Funding

The article was funded by Duolingo as both authors are employed by Duolingo.

CRediT Authorship Contribution Statement

Jill Burstein: Conceptualization, Investigation, and Drafting, Reviewing and Editing

Geoff LaFlair: Conceptualization, Investigation, and Drafting, Reviewing and Editing

Generative AI Use Disclosure Statement

Generative AI was used to implement copyediting.

Ethics Declarations

World Medical Association (WMA) Declaration of Helsinki–Ethical Principles for Medical Research Involving Human Participants

No human participants were involved.

Competing Interests

Both authors are employed by Duolingo.