Original Research
Assessing the Accuracy of Automated Writing Evaluation in Predicting English Language Arts Proficiency for Middle-Grade English Language Learners and Non-English Language Learners
- Abstract
- Full text
- Metrics
This study examines the accuracy of fall, winter, and spring benchmark writing assessments, scored by the MI Write automated writing evaluation system, for predicting non-proficiency on the Smarter Balanced ELA assessment. This study considers how grade level, seasonality, and language status influence classification accuracy using Receiver Operating Characteristic (ROC) curve analyses. The results indicate that MI Write demonstrated acceptable overall classification accuracy, with the strongest performance among non-ELLs and Grade 7 students. However, accuracy was more inconsistent for ELLs. Across all grades and subgroups, the d-based cutpoints consistently provided the best balance between sensitivity and specificity. Implications for adopting AI-based assessment systems within the middle grades are discussed.
Page 1 of
Download Count : 24
Visit Count : 44
Keywords
Automated Writing Evaluation (AWE); AI-Based Assessment; English Language Arts (ELA) Proficiency; Middle Grades; ROC Curve Analyses
Author(s) Information
Publisher’s Note
The claims, arguments, and counter-arguments made in this article are exclusively those of the contributing authors. Hence, they do not necessarily represent the viewpoints of the authors’ affiliated institutions, or EUROKD as the publisher, the editors and the reviewers of the article.
Acknowledgements
Not applicable.
Funding
The authors received no specific funding for this work.
CRediT Authorship Contribution Statement
Fan Zhang: Conceptualization, Methodology, Formal Analysis, Investigation, Writing – Original Draft, Writing – Review & EditingJoshua Wilson: Conceptualization, Methodology, Formal Analysis, Investigation, Writing – Original Draft, Writing – Review & Editing, Supervision
Generative AI Use Disclosure Statement
The authors utilized ChatGPT 4 to revise portions of this text for clarity.
Ethics Declarations
World Medical Association (WMA) Declaration of Helsinki–Ethical Principles for Medical Research Involving Human Participants
All procedures performed in this educational study adhered to the general ethical principles for research involving human participants, as outlined in the World Medical Association (WMA) Declaration of Helsinki (2013). The study protocol was specifically reviewed and approved by the University of Delaware Institutional Review Board (IRB) under the Exempt category, and the IRB granted a Waiver of Documentation of Consent.
Competing Interests
The authors declear no competing interests relative to this work.
Data Availability
Data is available upon reasonable request to the corresponding author.