# modeling_eye_movements_when_reading_microblogs__c156ccc9.pdf Modeling Eye Movements when Reading Microblogs Maria Barrett and Anders Søgaard Centre for Language Technology University of Copenhagen Njalsgade 140 DK-2300 Copenhagen S Denmark This Ph D project aims at a quantitative description of reading patterns from eye movements when reading tweets and the development of an eye movement relevance model. Introduction The aim of this Ph D project is to study eye movements during tweet reading to gain insight into the cognitive processing of relevant and irrelevant microblogs. I will explore a data driven method based on experimental eye tracking data with the prospect to train a machine learning classifier to distinguish between the eye movements of an interested and an uninterested reader. The findings may - with some modifications - be valid in other domains and, contrary to other measures of subjective relevance, they are scalable and accessible with little cost, once eye trackers are built into mainstream consumer products (San Agustin et al. 2009). Research question How do eyes move when reading tweets, and how can we based on eye tracking data determine whether or not tweet is relevant to the reader? The goal of this project is threefold and best described by the following three research questions. Each question is approached by means of a substudy which draws on established research from its own field. 1. How do eyes move during tweet reading? An experimental study of eye movements during tweet reading will provide eye tracking data which will be analyzed to identify special characteristics of tweet reading. I expect to use a frame like Part-of-speech (POS) tags from the Universal POS tagset (Petrov, Das, and Mc Donald 2012) or dependency trees for a quantitative comparison of tweet reading patterns and newswire reading patterns. 2. Can relevance be inferred? I will develop a machine learning model for relevance from eye tracking data which distinguishes between eye movements of an interested and an uninterested reader. Copyright c 2014, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. 3. Are my findings valid in other domanins or how can they be adapted to become so? I will both explore other types of text and eye tracking data from a cheap consumer eye tracker. The scientific contribution of the project is a deeper understanding of cognitive processes related to microblog reading, implementation of a subjective relevance classifier and an an exploration of cross-domain reading differences. Eye movements in reading Despite the subjective experience of a stable gaze, the eye moves in rapid, ballistic movements (saccades) with as high a velocity as 500 /s. Between the saccades the eye movements are relatively stable for 60 - 500 ms (average for reading 225 ms), which are fixations (Rayner 1998). The strong eye-mind hypothesis proposed by Just and Carpenter (1980) says that information processing occurs during fixation and that fixation continues until processing is completed. Hence features of eye movements during reading (fixation duration, saccade length, occurrence of regressions and a number of variations of these measures) can be used to infer momentby-moment cognitive processing of a text by a reader (Liversedge and Findlay 2000). It is well established in psycholinguistic research that the eyes move differently as a response to different linguistic features. In normal reading, 85% of content words are fixated and 35% of function words are fixated (Rayner 1998), but in tweets, many function words are omitted to fit the message into the allowed 140 characters. Another specific feature of tweets is the use of # as a way of tagging keywords in the tweet and the use of @ to address or mention another user. Tweets also contain many non-canonical abbreviations and typos. I expect this to call for another reading pattern than for instance the well-studied canonical text. Relevance and eye movements User-generated content - like tweets - contains irrelevant and low quality content between the interesting tweets, and users tolerate a lot of irrelevant tweets in their personal feeds (Andr e, Bernstein, and Luther 2012). Relevance in relation to tweet reading is linked to the goal of entertaining or enlightening (Kwak et al. 2010; ?). Thus it is reasonable to say that relevance in relation to tweets equals interest. Some Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence studies has looked at the link between gaze patterns and relevance: Buscher, Dengel, and van Elst (2008) and Brooks et al. (2006) have described eye movements qualitatively when reading relevant and irrelevant text in a known relevance task. Gwizdka (2014) have explored relevance in eye movements quantitatively in a known relevance task when reading news articles. Saloj arvi et al. (2003) and Ajanki et al. (2009) have found that eye tracking data can reliably model user interest. I expect to use eye movement features of relevance inspired by Saloj arvi et al., and Ajanki et al.. Similar to Saloj arvi et al. I expect to find across-subjects characteristics. Saloj arvi et al. also used pupil dilation as a feature. Pupil dilation isolated from gaze paths is well studied in relation to affection and cognitive states and has been linked to both positive and negative excitement (Partala and Surakka 2003) as well as frustration and workload (Marshall 2007). Subjective relevance inferred from eye movements during tweet reading has not been explored besides Counts and Fisher (2011), but detailed reading pattern was not subject to their study. The desired goal is to identify reliable gaze patterns related to interest and disinterest without using pupil dilation as a metric, because this measurement entails constraints to data collection: the lighting conditions must be very controlled, which can only be realized in a lab. Instead I intend to to explore whether my conclusions are valid with data from consumer eye trackers like The Eye Tribe s $99 eye tracker (San Agustin et al. 2009). Brief timeline All work will be carried out by myself except annotating tweets which has been done by my research group. Sep 2014 - Nov 2014 Literature study and plan experiment Dec 2014 - Jan 2015 Perform experiment and preliminary analysis Feb 2015 - Aug 2015 Analysis and develop relevance model Sep 2015 - Dec 2015 Sub-study: develop and evaluate reader model Jan 2016 - Aug 2016 Implement and evaluate relevance model Sep 2016 - Mar 2017 Explore other domains Apr 2017 - Sep 2017 Write thesis Ajanki, A.; Hardoon, D. R.; Kaski, S.; Puolam aki, K.; and Shawe-Taylor, J. 2009. Can eyes reveal interest? implicit queries from gaze patterns. User Modeling and User Adapted Interaction 19(4):307 339. Andr e, P.; Bernstein, M.; and Luther, K. 2012. Who gives a tweet?: Evaluating microblog content value. In Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work, CSCW 12, 471 474. New York, NY, USA: ACM. Brooks, P.; Phang, K. Y.; Bradley, R.; Oard, D.; White, R.; and Guimbretiere, F. 2006. Measuring the utility of gaze detection for task modeling: A preliminary study. In Pro- ceedings of the International Conference on Intelligent User Interfaces 06. Buscher, G.; Dengel, A.; and van Elst, L. 2008. Attentive documents: Eye tracking as implicit feedback for information retrieval and beyond. In CHI 08: CHI 08 extended abstracts on Human factors in computing systems. Counts, S., and Fisher, K. 2011. Taking it all in? visual attention in microblog consumption. In Proceedings of the Fifth International Conference on Weblogs and Social Media. Gwizdka, J. 2014. News stories relevance effects on eyemovements. In Proceedings of the Symposium on Eye Tracking Research and Applications, 283 286. ACM. Just, M. A., and Carpenter, P. A. 1980. A theory of reading: From eye fixations to comprehension. Psychological Review 87(4):329 354. Kwak, H.; Lee, C.; Park, H.; and Moon, S. 2010. What is twitter, a social network or a news media? In WWW 10: Proceedings of the 19th international conference on World wide web, 591 600. New York, NY, USA: ACM. Liversedge, S. P., and Findlay, J. M. 2000. Saccadic eye movements and cognition. Trends in Cognitive Sciences 4(1):6 14. Marshall, S. P. 2007. Identifying cognitive state from eye metrics. Aviation, Space, and Environmental Medicine 78(5). Partala, T., and Surakka, V. 2003. Pupil size variation as an indication of affective processing. Int. J. Human-Computer Studies 59(1-2):185 198. Petrov, S.; Das, D.; and Mc Donald, R. 2012. A universal part-of-speech tagset. In Calzolari, N.; Choukri, K.; Declerck, T.; Do gan, M. U.; Maegaard, B.; Mariani, J.; Moreno, A.; Odijk, J.; and Piperidis, S., eds., Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC 12). Istanbul, Turkey: European Language Resources Association (ELRA). Rayner, K. 1998. Eye movements in reading and information processing: 20 years of research. Psychological Bulletin 372 422. Saloj arvi, J.; Kojo, I.; Simola, J.; and Kaski, S. 2003. Can relevance be inferred from eye movements in information retrieval? In Proceedings of the 4th Workshop on Self Organizing Maps, 261 266. San Agustin, J.; Skovsgaard, H.; Hansen, J. P.; and Hansen, D. W. 2009. Low-cost gaze interaction: Ready to deliver the promises. In CHI 09 Extended Abstracts on Human Factors in Computing Systems, CHI EA 09, 4453 4458. New York, NY, USA: ACM.