Statistical Sleuthing: Can you spot a problem?

Can you spot problems in a scientific study just by reading the published paper? We put our statistical sleuthing skills to the test with a famous study on procrastination and deadlines that was retracted more than 20 years after it was published. But were there clues hidden in plain sight in the published results—and would we have noticed anything suspicious at the time? We put on our data-detective hats and find a few statistical clues of our own before turning to Data Colada’s investigation of the original data, where the story gets even stranger. Along the way, we hunt for missing participants, question suspiciously enormous effects, meet some unlikely data twins, and discover why sometimes the best statistical tool is simply asking: Do these data behave like real people? Statistical topics Statistical sleuthing Effect sizes Degrees of freedom Replication Missing data Methodologic Morals “A data detective's toolbox is surprisingly simple. Check the degrees of freedom, look for twins, question giant effects, and ask whether the data actually make human sense.” “You don't have to be a statistician to catch a statistical criminal.” References Data Colada blog posts:https://datacolada.org/138 https://datacolada.org/139 Data and code: https://researchbox.org/7135 Retraction Watch article: https://retractionwatch.com/2026/09/03/procrastination-study-duke-dan-ariely-psychological-science-data-colada-tampering-retraction/  Original 2002 paper: https://journals.sagepub.com/doi/10.1111/1467-9280.00441  Retraction notice: https://journals.sagepub.com/doi/10.1177/09567976261488042  Hyndman K, Bisin A. Replication of "Procrastination, Deadlines, and Performance: Self-Control by Precommitment". Psychol Sci. 2026;37(8):557-571. doi:10.1177/09567976261460772 Kristin and Regina’s online courses:  Demystifying Data: A Modern Approach to Statistical Understanding   Clinical Trials: Design, Strategy, and Analysis  Medical Statistics Certificate Program   Writing in the Sciences  Epidemiology and Clinical Research Graduate Certificate Program  Program that Kristin teaches in:: Epidemiology and Clinical Research Graduate Certificate Program  Kristin’s manuscript writing course: https://instats.org/seminar/from-data-to-discussion-writing-a-scient Find us on: Kristin -  LinkedIn & Twitter/X Regina - LinkedIn & ReginaNuzzo.com (00:00) - Introduction (02:15) - The Procrastination Paper and Study Two (08:51) - Suspiciously Tidy Results (16:18) - Study One: Real Classroom Stakes (20:06) - Missing Data and First Red Flags (28:17) - How Data Colada Got the Data (33:43) - When the Replication Failed (35:36) - Red Flags: Data Twins and Sanity Checks (47:21) - Study One Gets the Spreadsheet Treatment (51:45) - The Grading Scheme Unravels (57:51) - Manuscript Versions Reveal the Truth (01:08:11) - Rating the Claim and Methodological Morals