Understanding the Exploratory/Confirmatory Data Analysis Continuum: Moving Beyond the “Replication Crisis”

Dustin A. Fife, Joseph Lee Rodgers

Research output: Contribution to journalArticlepeer-review

8 Scopus citations

Abstract

In light of the “replication crisis,” some advocate for stricter standards and greater transparency in research methods. These efforts push toward a data analysis approach called “confirmatory data analysis” (CDA; see Wagenmakers et al., 2012). However, some (e.g., Baumeister, 2016; Goldin-Meadow, 2016) suggest that emphasizing CDA may restrict creativity and discovery. These scholars argued (sometimes inadvertently) for greater freedom to pursue “exploratory data analysis” (EDA; see Tukey, 1977). Ironically and unfortunately, many who push against stricter CDA standards do not realize EDA exists, or misunderstand the philosophy and proper tools for exploration. In this article, the meaning, tools, philosophy, and ethics associated with EDA, CDA, and a relatively unknown but important approach called “rough CDA” are clarified. Important distinctions are developed between EDA/rough CDA/CDA and other (some problematic) analysis activities including p-hacking, HARKing, and data mining, which are situated in a (graphical) framework that clarifies relationships and ethical boundaries with each. In short, the proper data analytic approach depends on (a) intentions and (b) transparency. Most psychological research is not at a maturity level to justify CDA; researchers have historically used tools mismatched to their research agenda. In the conclusion, recommendations are presented about how these typologies can be integrated into graduate training programs and how a cumulative research program can help psychology move beyond the replication crisis.

Original languageEnglish (US)
Pages (from-to)453-466
Number of pages14
JournalAmerican Psychologist
Volume77
Issue number3
DOIs
StatePublished - Nov 15 2021

All Science Journal Classification (ASJC) codes

  • Psychology(all)

Fingerprint

Dive into the research topics of 'Understanding the Exploratory/Confirmatory Data Analysis Continuum: Moving Beyond the “Replication Crisis”'. Together they form a unique fingerprint.

Cite this