Confidence Intervals: A Deep Dive

Statistics has a habit of taking plain English words and adopting them to mean very specific things. Confidence is not an easy concept in statistics, yet we use it all the time, so what does it actually mean? Chapters: 0:00 Introduction 2:54 Frequentist confidence intervals 4:26 Common misconceptions 5:24 The fundamental confidence fallacy 6:25 The missing submarine 12:32 Bayesian credible intervals 15:37 The elephant in the room 16:48 Real-life comparisons 20:13 Effect of sample size 21:12 Summary 23:23 Closing remarks Scripts mentioned in the video: - Submarine simulation (R): https://github.com/FransRodenburg/Biostatistics/blob/main/ConfidenceIntervals/CI_simulation.R - Submarine simulation (Python): https://github.com/FransRodenburg/Biostatistics/blob/main/ConfidenceIntervals/CI_simulation.py - Confidence versus credibility (Quarto): https://github.com/FransRodenburg/Biostatistics/blob/main/ConfidenceIntervals/conf_vs_cred.qmd - Confidence versus credibility, small data sets (Quarto): https://github.com/FransRodenburg/Biostatistics/blob/main/ConfidenceIntervals/conf_vs_cred_small_data.qmd Articles mentioned in the video: - Morey, R.D., Hoekstra, R., Rouder, J.N. et al. The fallacy of placing confidence in confidence intervals. Psychon Bull Rev 23, 103–123 (2016). https://doi.org/10.3758/s13423-015-0947-8 - Hoekstra, R., Morey, R.D., Rouder, J.N. et al. Robust misinterpretation of confidence intervals. Psychon Bull Rev 21, 1157–1164 (2014). https://doi.org/10.3758/s13423-013-0572-3 - Greenland, S., Senn, S.J., Rothman, K.J. et al. Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. Eur J Epidemiol 31, 337–350 (2016). https://doi.org/10.1007/s10654-016-0149-3 - Belia, S., Fidler, F., Williams, J., & Cumming, G. (2005). Researchers Misunderstand Confidence Intervals and Standard Error Bars. Psychological Methods, 10(4), 389–396. https://doi.org/10.1037/1082-989X.10.4.389 - Kalinowski, Pavel. (2010). Identifying misconceptions about confidence intervals. https://www.researchgate.net/publication/228462422_Identifying_misconceptions_about_confidence_intervals - Siegfried, Tom. (2014). Scientists’ grasp of confidence intervals doesn’t inspire confidence. https://www.sciencenews.org/blog/context/scientists-grasp-confidence-intervals-doesnt-inspire-confidence