Despite a rapidly increasing volume of research on automated written corrective feedback (AWCF) provided by automated writing evaluation systems in recent years, little is known about how L2 students interact with such feedback in the process of their essay revision. This research presents an exploratory study which pilots the combination of eye-tracking and subsequent stimulated recall to investigate L2 learners’ online processing of AWCF provided by Cambridge English Write & Improve, a new automated writing evaluation system. Twenty-four second-year English majors at a Chinese University performed two writing tasks on Cambridge English Write & Improve website and revised their drafts based on the AWCF they received. Participants’ eye-movements when viewing and responding to the feedback were captured by an eye-tracking device. Stimulated recall interviews were conducted using the recorded eye-movements as stimuli. During the interview, participants reflected on their essay revision behaviour with reference to their eye movements. In total, we collected 48 sets of eye gazeplot recordings together with the corresponding stimulated recall interview responses. Participants’ eye gazeplot recordings were manually annotated through NVIVO 12; their interview responses were analysed via a hybrid approach of inductive and deductive thematic analysis. Results showed that participants spent significantly more time on and expended more cognitive effort to understand indirect than direct AWCF. However, they were less successful in making revisions in response to such feedback. Implications of this research include affordances of eye-tracking as a process-tracing method: it enabled us to obtain a time-sensitive, fine-grained representation of participants’ real-time processing of different types of AWCF in their essay revision process. As such, the findings also point to further directions of the development of automated writing evaluation as an educational technology, with specific reference to what types of AWCF should be provided to L2 learners as optimal learning opportunities and how such feedback can be better presented to meet L2 learner’s needs.
Sha Liu, University of Bristol
Guoxing Yu, University of Bristol