4ms-winter-school

Slides

Monday

Dorothy Bishop - How our Brains Make it Difficult to Think Like a Scientist / Understanding how cognitive biases affect research [pdf]

Daniel Lakens - Which research is valuable enough to do well? [pdf]

Daniel Lakens - Concerns about Replicability, Theorizing, Applicability, Generalizability, and Methodology across Two Crises in Psychology [pdf]

Filippo Gambarota - Open Science Starter Pack [pdf]

Tuesday

(Interview with researcher [video])

Daniel Lakens - The benefits of preregistration and registered reports [pdf]

Jessica Flake - Measurement not schmeasurement [pdf]

Jessica Flake - Measurement and the next generation of open science reform [pdf]

Jessica Flake - Materials for students [zip] - Additional article [pdf]

Wednesday

Ettore Ambrosini - From Noise to Insight: Improving neuroimaging measurement for credible and reproducible neuroscience [pdf] - see also the recording

Lisa DeBruine - Big Team Setup [link]

Lisa DeBruine - Coding Club: Creating an R Package [link] - see also demopkg

Thursday

Ulrich Schimmack - Theoretical Factor Analysis [pdf]

Giulia Calignano - Multilab and Multiverse: from preprocessing to data analysis [pdf]

Giulia Calignano - additional materials: OSF repo [link]

Livio Finos - PIMA: Post-selection Inference in Multiverse Analysis [pdf]

Livio Finos - additional materials: PIMA dataset [html] / GitHub repo [link]

Friday

Massimo Grassi - Do musicians have better short-term memory than nonmusicians? A multi-lab study [pdf]

Filippo Gambarota - A multi-lab study: Power analysis and data management [pdf]

follow-up

Ettore Ambrosini - 2nd part / “extra” lesson on From Noise to Insight: Improving neuroimaging measurement for credible and reproducible neuroscience (2025-02-26) [recording]

Group Activities

Measurement

Group 1

Elson, M., Hussey, I., Alsalti, T., & Arslan, R. C. (2023). Psychological measures aren’t toothbrushes. Communications Psychology, 1, 25. https://doi.org/10.1038/s44271-023-00026-9 [pdf] — In addition: Select a psychological construct (e.g., cognitive load, word recognition, intrinsic motivation), and compare how five published studies measured it; consider differences in operationalization, scoring, and standardization, and suggest the impact that this variability may have on replicability.

  • Zofia Dobrowolska
  • Louie Lei Wang
  • Stefano Dalla Bona
  • Giulia Moncalieri
  • Rocío Vizcaíno-Cuenca
  • Michela Forcella

Group 2

Stapleton, L. M., Yang, J. S., & Hancock, G. R. (2016). Construct meaning in multilevel settings. Journal of Educational and Behavioral Statistics: A Quarterly Publication Sponsored by the American Educational Research Association and the American Statistical Association, 41, 481–520. https://doi.org/10.3102/1076998616646200 [pdf]

  • Lu Zhang
  • Yara Alnajjar
  • Lucrezia Guiotto Nai Fovino
  • Caroline Böhm
  • Cemre Çınar

Meta-Research

Group 1

Ulrich, R., & Miller, J. (2020). Questionable research practices may have little effect on replicability. eLife, 9, e58237. https://doi.org/10.7554/eLife.58237 [pdf]

  • Rrita Bajraktari
  • Giulia Paludo
  • Fabrizio Visconti
  • Francisco Miguel Soler Martínez

Group 2

Fried, E. I. (2020). Theories and models: What they are, what they are for, and what they are about. Psychological Inquiry, 31, 336–344. https://doi.org/10.1080/1047840x.2020.1854011 [pdf]

  • Raphael Merz
  • Ruben Lamers James
  • Elena Rinallo
  • Andreas Voldstad
  • Benjamin Kunc

Multi-Lab

Byers-Heinlein, K., Bergmann, C., Davies, C., Frank, M. C., Hamlin, J. K., Kline, M., Kominsky, J. F., Kosie, J. E., Lew-Williams, C., Liu, L., Mastroberardino, M., Singh, L., Waddell, C. P. G., Zettersten, M., & Soderstrom, M. (2020). Building a collaborative psychological science: Lessons learned from ManyBabies 1. Psychologie Canadienne [Canadian Psychology], 61, 349–363. https://doi.org/10.1037/cap0000216 [pdf]

Group 1

  • Stefano Ioannucci
  • Yashiv Seetahul
  • Sana Rehmat
  • Giulia Brizzi

Multiverse

Group 1

Steegen, S., Tuerlinckx, F., Gelman, A., & Vanpaemel, W. (2016). Increasing transparency through a multiverse analysis. Perspectives on Psychological Science: A Journal of the Association for Psychological Science, 11, 702–712. https://doi.org/10.1177/1745691616658637 [pdf]

  • Marco Roccato
  • Isabella Mariani Wigley
  • Miłosz Kalinowski
  • Clara Alameda

Group 2

Del Giudice, M., & Gangestad, S. W. (2021). A traveler’s guide to the multiverse: Promises, pitfalls, and a framework for the evaluation of analytic decisions. Advances in Methods and Practices in Psychological Science, 4, 251524592095492. https://doi.org/10.1177/2515245920954925 [pdf]

  • Marco Badioli
  • Ettore Napoli
  • Marta Paź
  • Victor Auger
  • Alice Tupinier

03/03/2025: Final report (in Italian) for internal review and documentation on the International Winter School: The 4 Ms