All Journals

Language Teaching Research Quarterly

European Journal of Studies in Management and Business

International Journal of Behavior Studies in Organizations

Studies in Educational Management

New Challenges in Accounting and Finance

Language Testing in Focus: An International Journal

Marketing and Branding Research

Individual Differences in Language Education: An International Journal 

Feedback Research in Second Language 

Management Issues in Healthcare System

Language Teacher Education Research

Artificial Intelligence Knowledge Development

Rater reliability, the consistency of marking across different raters and times, is one important component of reliability regarding the quality of test scores. It is essential regarding performance assessment, such as writing, when the fairness of assessment results can come into question due to the subjectivity when scoring. The present study, part of a larger-scale funded research project, aimed to study this overlooked area in the Omani context, i.e., the reliability in scoring the writing section of the final exams in the University of Technology and Applied Sciences (UTAS). More specifically, the study investigated the estimates of inter-rater and intra-rater reliability among 10 writing markers assessing 286 and 156 students' writing scripts belonging to four levels of proficiency at three different levels of analysis: the whole writing tests, tasks 1 and tasks 2, and the constituent criteria of both tasks. The results indicated a rather high value of inter-rater reliability and a moderate one for intra-rater reliability in general. However, when interpreted regarding raters' personal and background information, some low estimates shed light on the importance of factors influencing scoring consistency across different assessors and times.

ltf.2024.10.03.pdf

Original Research 

ltf.2024.10.03.xml

The inter-rater and intra-rater reliability in scoring writing tests in UTAS

Despite the numerous endeavours made to develop questionnaires to assess learners' strategic behaviour in general and learning/use strategies across different language areas and skills, one can surprisingly find no inventories to address learners' grammar learning /use strategies. This study aims to validate a measure of additional language learners' grammar strategies. The validation of the instrument, grammar learning/use strategy questionnaire, involved three phases. To generate an item pool as the first phase, the researcher drew on Oxford's instructional modes of L2 learning, Oxford's strategic self-regulated model of learning, Larsen-Freeman's three-dimension model of grammar, grammar strategies in the inventories of language skills and Brigg's classroom task-based grammar learning strategies. The second phase of the study involved submitting the instrument to extensive piloting in an adequate sample, followed by the third phase, which comprised evaluating the psychometric properties of the revised instrument through exploratory and confirmatory factor analysis. The results indicate that the developed instrument has satisfactory psychometric characteristics and that the hypothesized theoretical model has a great fit with the data. The article closes with some tentative pedagogical recommendations and implications as well as guidelines on how grammar learning strategies could be utilized and investigated.

ltrq.2023.36.02.pdf

ltrq.2023.36.02.xml

Developing a Grammar Learning/Use Strategy Questionnaire: A Mixed Model