Assignment 1: Finding Neptune Due date: 20. 11. 12:00 (moved the deadline to give you more time) Where to turn in: IS folder “Assignment 1” as PDF Grading: Welcome to the toughest assignment you’ve ever had to turn in, so far! But no worries, remember that it is intentionally designed this way. You can skip questions or make quite a few mistakes and still get an A from this course (see grading in the syllabus). The goal is not to frustrate you but to give you an opportunity to engage with the subject-matter as much as you want and feel rewarded by your effort. I hope we struck a balance! Remember, the assignments are open book, so you are encouraged to find more sources to help you answer! Mandatory reading: Borsboom, Measuring the Mind, Chapter 3 (Latent Variables) The Bollen & Diamantopoulos (2017) article builds on a general understanding of formative models. You don’t need to understand or read the whole thing, just the introduction is important for the assignment. But it might be useful to at least skim through the rest. Bollen, K. A., & Diamantopoulos, A. (2017). In defense of causal-formative indicators: A minority report. Psychological Methods, 22(3), 581–596. doi:10.1037/met0000056 Reading for Q8. Ignore stuff unrelated to the construct definition / data-generating processes. Mu, G. M., & Hu, Y. (2016). Validation of the Chinese Version of the 12-Item Child and Youth Resilience Measure. Children and Youth Services Review, 70, 332–339. doi:10.1016/j.childyouth.2016.09.037 Optional reading: An intro to formative models you were assigned to read in master’s level psychometrics Edwards, J. R., & Bagozzi, R. P. (2000). On the nature and direction of relationships between constructs and measures. Psychological methods, 5(2), 155–174. https://doi.org/10.1037/1082-989x.5.2.155 Embretson’s brilliant showcase of theory-backed cognitive assessment Embretson, S. (1994). Applications of Cognitive Design Systems to Test Development. In: Reynolds, C.R. (eds) Cognitive Assessment. Perspectives on Individual Differences. Springer, Boston, MA. https://doi.org/10.1007/978-1-4757-9730-5_6 * * * Q1: What ontological stance does a user of FA typically adhere to? Why? (0.5) Q2: “Data-generating process” is a term that gets swung around a lot these days. Please, explain what it means. (0.5) Q3: Borsboom’s chapter on latent variables in Measuring the Mind describes several problems with the within-subject interpretation of latent variables. Choose one and elaborate. (0.5) Q4: Ken Bollen is one of those big names in stats. Among others, he invented several model fit indices (some even named after him) or a new estimation method poetically called MIIV2SLS. This guy is just awesome at stats AND naming things. Anyway, he’s also one of the few defenders of formative models as evidenced by the article you are tasked to read. And formative models are something that’s not really in our usual repertoire. Luckily, Ken gives us a great overview of those. 1. Using his article, argue whether the construct you measured in your bachelor thesis is formative (if this doesn’t apply to your thesis, pick any construct you want). 2. If it is not, what would need to change for it to become one? 3. Is there a way you could tell empirically? (1) Q5: Tomorrow and tomorrow and tomorrow has crept in this petty pace from day to day and, suddenly, you find yourself in year three of your Ph.D. studies. Your classmates are slowly reaching the terminal stages of their doctorates while you spent all that time playing grand strategy games deep into the starless and bible-black nights visible to those who have the courage to consistently shirk their academic responsibilities. But no more, you say, it is time to generate some science! Since open science is all the rage now, you decide to measure the p-hacking index. It’s a formative construct summarizing how people differ in their tendency to engage in questionable research practices. You use 7 items to measure it (e.g., “I love seeing stars in my SPSS output”) on a 5-point Likert scale. After collecting the data, you write up the paper, submit it to a journal and go back to your grand campaign emulating the glorious conquests of Genghis Khan. But after two months, the infamous Reviewer 2 comes back with their suggestions. They ask: Why do you not report Cronbach’s Alpha of your p-hacking index scale? Respond! (1) Q6: Imagine a sample variance-covariance matrix of 3 variables (i.e., covariances off the diagonal, variances on the diagonal). How many parameters does my model need to exactly reproduce the observations? What is the danger of using such model? Can it be useful for something? (1) Q7: It’s Berlin, March 1945. You are a young psychologist (even though you can’t call yourself that) employed by the army to screen new recruits in the vein of Army Alpha and Beta, known from the U.S. But wait! The latest bombing destroyed your data! Quelle dommage! (in your head, you swear in French for some reason) It’s wartime Germany so there’s no other choice but to fabricate the data. How would you proceed? The goal is to generate individual observations for 100 persons that would conform to the data-generating process assumed by intelligence testing. There are two ways you can do this (and you can also combine them). Either, 1. Describe each step (e.g., I would sample scores for each item from an F distribution) in the degree of technicality you are comfortable with. Equations are welcomed! Or: 2. Go all-in and do this in R. I’ve uploaded an example R script to show you the functions you will need, along with some hints. Both approaches are equivalent grade-wise, but I may award bonus points to sophisticated answers. Feel free to e-mail me if you need to troubleshoot technical issues. (2+1) Q8: Read the article by Mu and Hu (you can find it in the Study Materials/Assignment 1) about resilience. What measurement model does the theory imply? No need to explore the theory further, you can rely just on this single article for the sake of simplicity. Argue for your selected model – why this one? Which parts of the theory influenced your choice? And do you agree with the pick of the authors? (2) Q9: Do you agree with the following statement? (Reflective) latent variables are latent only until we devise better methods to observe them. In a few decades, we might develop methods so accurate we could make general intelligence an observed variable. Use arguments from whatever approach you want (e.g., Borsboom’s article on latent variable theory, or Bollen’s overview) but, please, state which framework you used. (1) Q10: Isn’t all this formative/reflective mumbo jumbo useless if the models fit the same? What are the consequences of mis-specifying the data generating process, in your opinion? (0.5) Optional Q11: On a scale from 1-10 (least to most) how hard was this assignment for you? What did you like, what did you dislike? How long did it take you? (-)