multiverseRworkshop

Authors
Affiliation

Giulia Calignano

Department of Developmental Psychology and Socialization, University of Padova

Filippo Gambarota

This website contains the materials for the workshop about the Multiverse Analysis in R.



Materials

References

This is an updated selection of papers about the multiverse analysis.

  • Calignano, G., Girardi, P., & Altoè, G. (2023). First steps into the pupillometry multiverse of developmental science. Behavior Research Methods. https://doi.org/10.3758/s13428-023-02172-8
  • Liu, Y., Kale, A., Althoff, T., & Heer, J. (2021). Boba: Authoring and visualizing multiverse analyses. IEEE Transactions on Visualization and Computer Graphics, 27, 1753–1763. https://doi.org/10.1109/TVCG.2020.3028985
  • Harder, J. A. (2020). The multiverse of methods: Extending the multiverse analysis to address data-collection decisions. Perspectives on Psychological Science: A Journal of the Association for Psychological Science, 15, 1158–1177. https://doi.org/10.1177/1745691620917678
  • Steegen, S., Tuerlinckx, F., Gelman, A., & Vanpaemel, W. (2016). Increasing transparency through a multiverse analysis. Perspectives on Psychological Science: A Journal of the Association for Psychological Science, 11, 702–712. https://doi.org/10.1177/1745691616658637
  • Del Giudice, M., & Gangestad, S. W. (2021). A traveler’s guide to the multiverse: Promises, pitfalls, and a framework for the evaluation of analytic decisions. Advances in Methods and Practices in Psychological Science, 4, 251524592095492. https://doi.org/10.1177/2515245920954925
  • Girardi, P., Vesely, A., Lakens, D., Altoè, G., Pastore, M., Calcagnì, A., & Finos, L. (2024). Post-selection inference in multiverse analysis (PIMA): An inferential framework based on the sign flipping score test. Psychometrika, 89, 542–568. https://doi.org/10.1007/s11336-024-09973-6
  • Olsson-Collentine, A., Aert, R. C. M. van, Bakker, M., & Wicherts, J. (2023). Meta-analyzing the multiverse: A peek under the hood of selective reporting. Psychological Methods. https://doi.org/10.1037/met0000559
  • Götz, M., Sarma, A., & O’Boyle, E. H. (2024). The multiverse of universes: A tutorial to plan, execute and interpret multiverses analyses using the r package multiverse. International Journal of Psychology: Journal International de Psychologie, 59, 1003–1014. https://doi.org/10.1002/ijop.13229
  • Patel, C. J., Burford, B., & Ioannidis, J. P. A. (2015). Assessment of vibration of effects due to model specification can demonstrate the instability of observational associations. Journal of Clinical Epidemiology, 68, 1046–1058. https://doi.org/10.1016/j.jclinepi.2015.05.029
  • Klau, S., Felix, Patel, C. J., Ioannidis, J. P. A., Boulesteix, A.-L., & Hoffmann, S. (2023). Comparing the vibration of effects due to model, data pre-processing and sampling uncertainty on a large data set in personality psychology. Meta-Psychology, 7. https://doi.org/10.15626/mp.2020.2556

Packages

  • livioivil/pima: package for the PIMA inferential framework [under development]
  • livioivil/jointest: the main PIMA dependency. Used to compute the multivariate permutation tests and combining p-values
  • specr: main package for the specification curve analysis
  • multiverse: package to create and execute a multiverse analysis using a complex and flexible tidyverse-style approach.