Pilot study ESF: DXH_MET2 Metodologie 2 Recap •So far we have discussed mainly the conceptual side of your projects •Research question as a focus and a „handle“ •Hypotheses as informed expectations, and clarifications of RQ •Variables as (observable) representations of the concepts in RQ/H and their more or less precise measures •Designs allowing us to use the measures to create meaningful data to answer RQ • • • Recap •We have not talked much about •Data analysis • •Practical side of the projects •There are numerous details in designs-measures-analyses which are necessary to make it work, or threaten the success of the whole study •It is easier to learn them hands-on trying to understand them • Overview of the projects so far Paul Mark Ablorh Sustainable transport ? Tomáš Oravec DSGE model econometric Lukáš Marek Improving investment portfolio by considering market interdependencies econometric Valeriia Vysotckaia Effectiveness of import quota econometric Zdeněk Strmiska Impact of anti-VAT-evasion measures econometric LL Soňa Raszková Determinants of successful innovative regions econometric LL Ondřej Špetík How to set-up best tenders for train public transport econometric LL - potential for simulations Alla Kachur Which emotions make people more cooperative experimental Katarína Čellárová Being nasty in economic games experimental Diya Abraham The Neural Correlates of Emotion Regulation experimental Jakub Pejcal Financial health of nonprofit organisations observation/survey Peter Kelemen Use of background knowledge in decisions in unexpected situations qual Tadeáš Pala Financial literacy and high-risk mortgages in Ukraine qual/quan Michaela Floriánová Primary Characteristics of an Average Czech Policyholder survey Helena Kubíčková Support of tourism through small and medium businesses survey Conceptually, projects are (more or less) in progress… •… it‘s time to prepare a pilot study • • • •Plan for this semester: do a small pilot study for one of your RQ •4 meetings: •Take stock, see what‘s missing; measures, materials, instructions •Finalize procedure/protocol from recruitment to debriefing, decide on participants, check ethics and seek approval • (in the meantime) …. brush up data-analysis skills •Reflect on pilot experience, refine data-analysis plan and power analysis • Focus of DHX_MET2 •Projects with people as the primary objects or sources of data Purpose for a Pilot study •test feasibility of plan •methods work •participants go through the whole procedure without issues •data look as expected •analysis feasibility •do people (participants) function as I imagine them? •identify weak spots - tune the plan • •Murphy‘s laws apply • • •from Leong, Austin (2006) Choose a RQ/H you want to pilot •in groups of 2 or 3 Take stock • Measures, materials •Full version of your measures + what it measures •If some stimuli materials are used, what exatly are they going to be Instructions, administration •How exactly are you going to administer the measure to a participant Gather what you have and show it to a colleague •„I‘m going to measure the participants‘ CHARACTERISTIC(S) i the following way:….“ •As a colleague look at it from the perspective of •a participant – how do you react to the measure, the intention to measure.. •a colleague – do you believe it will work as expected? •Be sceptical, be critical … of the materials Possible sources of problems in asking questions. Respondents…. •May not understand, may misunderstand •May not know the answer or how to get to answer •May not be motivated to invest energy in getting the best (truest) possible answer •May have trouble fitting their answer to the response scale you offer •May not want to tell you the (known) answer (even though they agreed to participate) •May not even want to know the answer •May have their own agenda with respect to your study •May just wanna have fun • Possible sources of problems in observing behavior (e.g. games) •Opportunities for observation error – vague definitions of categories, high cognitive demands on observer •Missing important situational factors, determinants •Fundamental attribution error •Missing unobservable personal variables •Not optimal scale of behavior – too micro, too macro • • • •…. unless the observed behavior does not represent something else • Homework •Finalize measures, materials, instructions for individual measures • • • •Next, we will combine it with design plans into a protocol for the pilot