Lecture 9 Report Writing DHX_MET1 Methodology 1 Stanislav Ježek Faculty of Social Studies MU The purpose of research is met by publishing it •in the form of research report to those who commisioned the research •in the form of published journal article • •+ •oral presentations, posters • •+ •research proposals, grant applications PUBLICATION BIAS •A bias in our perception of teh existence or size of an effect due to systematic selective publication of findings •A file-drawer bias • •Preregistration & Registered Reports • http://2.bp.blogspot.com/_lYPdFAJCa4o/TSkdqzL97gI/AAAAAAAAAIc/2HpSC220-MQ/s1600/Funnel+Plot.jpg Funnel plot: Effect size by S.E.(~N) Zdroj: http://chrismasterjohnphd.com/2011/01/09/great-unknown-using-statistics-to/ Efekt některých omega-3 kyselin na depresivitu. Černé jsou hypotetizované nepublikované studie, kde dyperesivita signifikantně narostla. Writing standards •Field dependent •In psychology and many related fields the standard is set by the Publication manual of the APA •currently awaiting its 7th edition •https://apastyle.apa.org/products/publication-manual-7th-edition/?tab=1 •Preview: Appelbaum et al. (2018). Journal article reporting standards for quantitative research in psychology: The APA Publications and Communications Board task force report. American Psychologist, 73(1), 3–25. https://doi.org/10.1037/amp0000191 • •Standardizes mainly •Structure of research reports •Content – reporting results of a range of analyses •Language and tone •Citations •Format • No automatic alt text available. Writing up - IMRaD INTRODUCTION •From the topic to RQ/H • •The topic – what is the problem area and why it is important •The problem – what is the problem and and what are solutions in literature à what is unknown, uncertain, missing, controversial…. •My contribution – what my research should bring – RQs/Hs that logically follow from the previous paragraphs • • METHOD •How do I (plan to ) achieve the aims of my research. •In detail allowing the reader to assess •feasibility •validity – interenal and external – and threats to it •researcher competence • •Design •Sample (participants) •Methods (measures, protocol) •Analyses • •Order of the first three varies. •In lab experiments the placement of detailed description of the experimental procedure varies. In method, notice what is stated and what is NOT stated. •“We report how we determined our sample size, all data exclusions (if any), all manipulations, and all measures in the study.” •Simmons, Nelson, Simonshn: 21-word solution https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2160588 • METHOD - DESIGN •The overall strategy of creating new knowledge •Identification of standard type of design + details, deviations •Observational, Survey, Longitudinal, panel, (Quasi-)Experimental, Case study, Evaluation, Ethnography, GT, Simulation …. • METHOD - METHODS •Measures (to be) used •Experimental procedures. •Short descriptions with arguments for sufficient validity and reliability • •In QUAL studies often merged with Design. •Procedures ensuring validity are explicitely proposed as a part od method •What choices will be made during the study and how wil the be made? • • METHOD - SAMPLE •Description of sampling procedure •Argument for sample size •Description of sample (not in a proposal) •Ethics statement (unless in a special section) METHOD - ANALYSIS •Plan (or description) of the analysis •Data processing & diagnostics •Analytical approach •Tools used (software) • •Relevant level of detail – often short unless the analysis is unusual or complex • RESULTS •Final (effective) sample •Missing data and arguments for missingness mechanism, methods applied to address missing data •Descriptive statistics •Inferential statistics •Models •Assumption checking DISCUSSION •Statements of the support (or absence thereof) found for hypotheses. •Similarities and differences between reported results and work of others. •Interpretation of the results, taking into account •Sources of potential bias and threats to internal and statistical validity •Imprecision of measurement protocols •Overall number of tests or overlap among tests •Adequacy of sample sizes and sampling validity •Generalizability (external validity) of the findings, taking into account •Target population (sampling validity) •Other contextual issues (setting, measurement, time; ecological validity) • Implications for •theory •future research, program, or policy. LIMITATIONS ABSTRACT EXECUTIVE SUMMARY •Often the only text that people read – fully autonomous text • •What was to goal •How we did it •What we found • •Every word should count. Research ethics (Ruane, 2016) •Doing no harm •Obtaining informed consent •Competence •participants need to have the ability to know what is in their best interest •Voluntarism •participants must have freedom of choice about study involvement •Full information •participants should receive all relevant information about study •Comprehension •participants must understand information presented •Protecting privacy •More complicated in our technology saturated world •Many public settings still have privacy expectations •Avoiding conflict of interest •a challenge given prevalence of corporate funding of research •Ethics review boards •(https://vyzkum.rect.muni.cz/cs/zazemi/etika-vyzkumu) •