4ms-winter-school
Slides
Monday
Dorothy Bishop - How our Brains Make it Difficult to Think Like a Scientist / Understanding how cognitive biases affect research [pdf]
Daniel Lakens - Which research is valuable enough to do well? [pdf]
Daniel Lakens - Concerns about Replicability, Theorizing, Applicability, Generalizability, and Methodology across Two Crises in Psychology [pdf]
Filippo Gambarota - Open Science Starter Pack [pdf]
Tuesday
(Interview with researcher [video])
Daniel Lakens - The benefits of preregistration and registered reports [pdf]
Jessica Flake - Measurement not schmeasurement [pdf]
Jessica Flake - Measurement and the next generation of open science reform [pdf]
Jessica Flake - Materials for students [zip] - Additional article [pdf]
Wednesday
Ettore Ambrosini - From Noise to Insight: Improving neuroimaging measurement for credible and reproducible neuroscience [pdf] - see also the recording
Lisa DeBruine - Big Team Setup [link]
Lisa DeBruine - Coding Club: Creating an R Package [link] - see also demopkg
Thursday
Ulrich Schimmack - Theoretical Factor Analysis [pdf]
Giulia Calignano - Multilab and Multiverse: from preprocessing to data analysis [pdf]
Giulia Calignano - additional materials: OSF repo [link]
Livio Finos - PIMA: Post-selection Inference in Multiverse Analysis [pdf]
Livio Finos - additional materials: PIMA dataset [html] / GitHub repo [link]
Friday
Massimo Grassi - Do musicians have better short-term memory than nonmusicians? A multi-lab study [pdf]
Filippo Gambarota - A multi-lab study: Power analysis and data management [pdf]
follow-up
Ettore Ambrosini - 2nd part / “extra” lesson on From Noise to Insight: Improving neuroimaging measurement for credible and reproducible neuroscience (2025-02-26) [recording]
Group Activities
Measurement
Group 1
Elson, M., Hussey, I., Alsalti, T., & Arslan, R. C. (2023). Psychological measures aren’t toothbrushes. Communications Psychology, 1, 25. https://doi.org/10.1038/s44271-023-00026-9 [pdf] — In addition: Select a psychological construct (e.g., cognitive load, word recognition, intrinsic motivation), and compare how five published studies measured it; consider differences in operationalization, scoring, and standardization, and suggest the impact that this variability may have on replicability.
- Zofia Dobrowolska
- Louie Lei Wang
- Stefano Dalla Bona
- Giulia Moncalieri
- Rocío Vizcaíno-Cuenca
- Michela Forcella
Group 2
Stapleton, L. M., Yang, J. S., & Hancock, G. R. (2016). Construct meaning in multilevel settings. Journal of Educational and Behavioral Statistics: A Quarterly Publication Sponsored by the American Educational Research Association and the American Statistical Association, 41, 481–520. https://doi.org/10.3102/1076998616646200 [pdf]
- Lu Zhang
- Yara Alnajjar
- Lucrezia Guiotto Nai Fovino
- Caroline Böhm
- Cemre Çınar
Meta-Research
Group 1
Ulrich, R., & Miller, J. (2020). Questionable research practices may have little effect on replicability. eLife, 9, e58237. https://doi.org/10.7554/eLife.58237 [pdf]
- Rrita Bajraktari
- Giulia Paludo
- Fabrizio Visconti
- Francisco Miguel Soler Martínez
Group 2
Fried, E. I. (2020). Theories and models: What they are, what they are for, and what they are about. Psychological Inquiry, 31, 336–344. https://doi.org/10.1080/1047840x.2020.1854011 [pdf]
- Raphael Merz
- Ruben Lamers James
- Elena Rinallo
- Andreas Voldstad
- Benjamin Kunc
Multi-Lab
Byers-Heinlein, K., Bergmann, C., Davies, C., Frank, M. C., Hamlin, J. K., Kline, M., Kominsky, J. F., Kosie, J. E., Lew-Williams, C., Liu, L., Mastroberardino, M., Singh, L., Waddell, C. P. G., Zettersten, M., & Soderstrom, M. (2020). Building a collaborative psychological science: Lessons learned from ManyBabies 1. Psychologie Canadienne [Canadian Psychology], 61, 349–363. https://doi.org/10.1037/cap0000216 [pdf]
Group 1
- Stefano Ioannucci
- Yashiv Seetahul
- Sana Rehmat
- Giulia Brizzi
Multiverse
Group 1
Steegen, S., Tuerlinckx, F., Gelman, A., & Vanpaemel, W. (2016). Increasing transparency through a multiverse analysis. Perspectives on Psychological Science: A Journal of the Association for Psychological Science, 11, 702–712. https://doi.org/10.1177/1745691616658637 [pdf]
- Marco Roccato
- Isabella Mariani Wigley
- Miłosz Kalinowski
- Clara Alameda
Group 2
Del Giudice, M., & Gangestad, S. W. (2021). A traveler’s guide to the multiverse: Promises, pitfalls, and a framework for the evaluation of analytic decisions. Advances in Methods and Practices in Psychological Science, 4, 251524592095492. https://doi.org/10.1177/2515245920954925 [pdf]
- Marco Badioli
- Ettore Napoli
- Marta Paź
- Victor Auger
- Alice Tupinier
03/03/2025: Final report (in Italian) for internal review and documentation on the International Winter School: The 4 Ms