Experimental Humanities II
Eye-Tracking Methodology
First things first ...
• Great job, team Anestis + Jiří Č. + Kamila Č.!
▫ Very nice comments on experimental design!
• 25th April, A21, 18:00 „Data Quality Study“ –
Kenneth Holmqvist
• 3 volunteers for presenting on Tue 26th April?
▫ To all of you, be on time!
• Grading – I‘ll find out if I need to create some exam
time slot for which you need to sign up, otherwise
will happen about 1 week since 26th April
• Informal meeting? Doodle poll
Browsing CH9-14
• Questions?:)
First drafts – do‘s and don‘ts
Do‘s
• Go small and neat with the experiment
• Explain the terminology you are using and always define the terms
(especially field-specific)
• Structure your drafts (title, importance, literature overview, project
description – research questions and hypotheses, proposed eyetracker,
experiment, task, cover story, participants, groups,
description of relevant measures and oculomotor events, expected
outcomes, suggested analysis, references)
• Show how your stimulus + predictions look!
• Reflect your hypothesis and predictions in the analysis (be specific)
• Match stimuli – e.g. Pictures on size, colours, objects, luminosity...
• Turn on automatic spelling check in MS Word
First drafts – do‘s and don‘ts
Don‘ts
• Don‘t try to save the world and/or evaluate the eye-tracking
methodology
• Don‘t complain about the lack of space
• Don‘t include emotional statements (phrases such as: the biggest
weakness of experiment, good overview, I feel, I‘m afraid, in my
opinion)
Final project - deadlines
• 23.4. 23:00 DDL final project
Final drafts
• 8-10 A4 pages (1.5 line spacing, 12pt font)
• All sections well written up, no „to-do“ left
• Use comments from your peers and me, decide
if you are going to accept them or not
Lecture 5: Measures
• When to choose the eye-tracking measures?
• How to choose the eye-tracking measures?
• What to choose from?
• Complexity of measures
• Validity and reliability of measures
• Pilot study
When to choose the eye-tracking
measures?
• Always before you start recording your study, so
already during the experimental design planning
phase
• Beware of fishing trips and post-hoc hypotheses
How to choose the eye-tracking
measures?
• Based on the hypothesis and task, draw the
expected eye movement behaviour onto the
intended stimulus
• Run the pilot
• Recording what you expected?
▫ Yes – run the study
▫ No - redesign
• Within paradigms
▫ If your study is very similar to an already published
study, reuse the measures!
▫ When you deviate in design, you have to think for
yourself
What to choose from?
• Previous studies, paradigms
• Experimental design opreationalization to
search between
▫ Movement measures
▫ Position measures
▫ Count measures
▫ Latency measures
▫ Distance measures
▫ All these described in the Book
Movement measures
• Simple
• Complex
Position measures
Count measures
• Number is simply the number
• Proportion 0-1%
• Rate – number / extension by temporal range
Latency measures
Distance measures
Complexity of measures – simple
measure
Complexity of measures – complex
measure
Validity and reliability of measures
• Measures are not all equal
• Some are more verified than others
• Realiability: e.g. Fixation duration is not a reliable
measure of the level of processing, since there are
many other causes of long fixation durations that
could appear in almost any experiment
• Validity: It measures what it is intended to measure.
E.g., does the duration of a fixation on a text unit
actually reflect the processing difficulty of the
reader?
Pilot your measures!
• Your participants may not behave as you
thought they would when you were drawing your
expectations on the stimulus
• Statistical methods cannot be adapted to just
any measures and experimental designs that
you happen to use
>>
• Minimizing the risk of recording lots of data and
being unable to use it
What have we learned so far?
• Choose your measures before you start recording the
experiment
• Always draw the expected eye movements on the
stimulus
• Keep in mind that whenever you are using complex
measures, a part of the information can be lost through
the transformations
• Reuse measures from paradigms or similar studies
• Prefer such measures which have been tested in many
studies before
• Pilot your measures
• Questions?:)
Collecting questions left unanswered
• Unanswered questions on eye-tracking?
For the next lecture...
• Kenneth Holmqvist as a guest
• Your projects presentation
• Q&A session
• I‘ll collect feedback on the course
▫ What do you think about the course?
▫ Did it give you what you expected from it?
▫ What was good, what was less optimal?
▫ Something that you were missing?
▫ Personal message?
• Prepare
▫ Send the final projects on my email address until 23rdApril,
23:00