Pilot study protocol ESF: DXH_MET2 Metodologie 2 Plan for this semester: do a small pilot study for one of your RQ •4 meetings: • •Take stock, see what‘s missing; measures, materials, instructions • •Finalize procedure/protocol from recruitment to debriefing, decide on participants, check ethics and seek approval • • (in the meantime) …. brush up data-analysis skills •Reflect on pilot experience, refine data-analysis plan and power analysis • Focus of DHX_MET2 •Projects with people as the primary objects or sources of data Purpose for a Pilot study •test feasibility of plan •methods work •participants go through the whole procedure without issues •data look as expected •analysis feasibility •do people (participants) function as I imagine them? •identify weak spots - adjust the plan • • • •from Leong, Austin (2006) Lets try out the procedure •Form groups of 3 people with 3 roles •a researcher •a participant •an observer • •Initially do not discuss the topic of your study with the participant. • •Roles may rotate – we should be able to do one round before lunch and one after Lets try out the procedure •Find a suitable space •Researcher administeres the whole procedure incl. brief informed consent and debriefing •Participant is just himself/herself •Observer tries to be invisible and looks for possible misunderstandings •Reflection •Participant reflects experience – thoughts, emotions, motivation & issues •Observer gives feedback, ask questions for clarification, revise materials… •Together review materials, procedure, ethics • •Suggestions for modification • • Possible sources of problems in asking questions. Respondents…. •May not understand, may misunderstand •May not know the answer or how to get to answer •May not be motivated to invest energy in getting the best (truest) possible answer •May have trouble fitting their answer to the response scale you offer •May not want to tell you the (known) answer (even though they agreed to participate) •May not even want to know the answer •May have their own agenda with respect to your study •May just wanna have fun • Possible sources of problems in observing behavior (e.g. games) •Opportunities for observation error – vague definitions of categories, high cognitive demands on observer •Missing important situational factors, determinants •Fundamental attribution error •Missing unobservable personal variables •Not optimal scale of behavior – too micro, too macro • • • •…. unless the observed behavior does not represent something else • Homework for the 4th meeting (May, 17) •Do a small scale pilot study on volunteers – team up for the task •Prepare a short presentation (10 min) of the pilot and what came out of it. • • • Checklist for the pilot study •RQ/H are clear in my mind •Measures, materials •measures valid, reliable; materials representative •clear and standard instructions •assumed/required abilities of my participants •Design, procedure •clear, standard sequence of acitivities •standard instructions, timing („rapport maintenance“) •monitoring the procedure •Recruitment of participants •standard, clear advertisement/info (good to cross-check against consent form) •Practical issues •Settings – lab, „quiet room“, online, field •Assistants, conferedates, other presonell • •Ready for ethical approval – make as much use of IRB as they let you •functional, „informing“ consent •considering and recognizing non-standard participants •debriefing as an ethical and validity-enhancing procedure, not only if deception is used, may uncover previously •Data processing plan • Final task •As a final outcome of the course write a short report (5-10 pages) •Very brief overview of the RQ an H •Method •Data description focusing on possible issues •Critical evaluation with suggestions for modification. •„Econometrists“ will write an essay on where they see the difficulties and advantages of research with people as the primary source of data. • Dedline? End of May? •