Within the field of language testing, eye-tracking has mostly been used in research on reading assessment, while its use for studying listening assessment has been sparse. A number of eye-tracking studies in listening assessment have been conducted on video-based listening tasks, however in this presentation I will argue that eye-movement data can also provide novel insights into test takers’ response processes in audio-only listening tests. The paper will present two studies. In the first, eye-tracking was used to investigate response order effects in an audio-only multiple-choice listening test. We modelled eye-tracking data captured from 30 test takers while they were completing listening items of the Aptis test. Based on this, we then studied the direct effect of correct response position on item difficulty in a large sample of live Aptis items. The second study will outline how eye-tracking can be useful in listening studies that use verbal recalls, by presenting research on the issue of single play vs. double play in listening assessment. In order to study the effects of double play on test takers’ response processes, 16 candidates completed four listening tasks in single play and double play on an eye-tracker and performed retrospective verbal recalls, which were stimulated by their eye-movements while they had been solving the items. The stimulated recalls were analysed in terms of candidates’ strategic behaviour, anxiety levels, and cognitive processes displayed, and the results were triangulated with a questionnaire analysis of over 300 students. The presentation will discuss the strengths and weaknesses of eye-tracking methodology and offer recommendations on how to use it in research on audio-only listening tests.
Franz Holzknecht, University of Innsbruck