Introduction to statistical simulations in health research

Type: Article

Publication Date: 2020-12-01

Citations: 40

DOI: https://doi.org/10.1136/bmjopen-2020-039921

Abstract

In health research, statistical methods are frequently used to address a wide variety of research questions. For almost every analytical challenge, different methods are available. But how do we choose between different methods and how do we judge whether the chosen method is appropriate for our specific study? Like in any science, in statistics, experiments can be run to find out which methods should be used under which circumstances. The main objective of this paper is to demonstrate that simulation studies, that is, experiments investigating synthetic data with known properties, are an invaluable tool for addressing these questions. We aim to provide a first introduction to simulation studies for data analysts or, more generally, for researchers involved at different levels in the analyses of health data, who (1) may rely on simulation studies published in statistical literature to choose their statistical methods and who, thus, need to understand the criteria of assessing the validity and relevance of simulation results and their interpretation; and/or (2) need to understand the basic principles of designing statistical simulations in order to efficiently collaborate with more experienced colleagues or start learning to conduct their own simulations. We illustrate the implementation of a simulation study and the interpretation of its results through a simple example inspired by recent literature, which is completely reproducible using the R-script available from online supplemental file 1.

Locations

  • PubMed Central - View
  • UCL Discovery (University College London) - View - PDF
  • Leiden Repository (Leiden University) - View - PDF
  • DOAJ (DOAJ: Directory of Open Access Journals) - View
  • PubMed - View
  • Open Acess LMU (Ludwig-Maximilians-Universität München) - View - PDF
  • BMJ Open - View - PDF

Similar Works

Action Title Year Authors
+ PDF Chat Conducting Simulation Studies in the R Programming Environment 2013 Kevin A. Hallgren
+ PDF Chat Using simulation studies to evaluate statistical methods 2019 Tim P. Morris
Ian R. White
Michael J. Crowther
+ INTEREST: INteractive Tool for Exploring REsults from Simulation sTudies 2019 Alessandro Gasparini
Tim P. Morris
Michael J. Crowther
+ PDF Chat INTEREST: INteractive Tool for Exploring REsults from Simulation sTudies 2021 Alessandro Gasparini
Tim P. Morris
Michael J. Crowther
+ Computer experiments for epidemiological studies : theory, methodology and models 2011 Zaiyi Guo
+ Simulation Framework for Realistic Large-scale Individual-level Data Generation with an Application in the Health Domain 2020 Santtu Tikka
Jussi Hakanen
Mirka Saarela
Juha Karvanen
+ Simulation Framework for Realistic Large-scale Individual-level Data Generation with an Application in the Health Domain 2020 Santtu Tikka
Jussi Hakanen
Mirka Saarela
Juha Karvanen
+ Exploring Consequences of Simulation Design for Apparent Performance of Statistical Methods. 1: Results from simulations with constant sample sizes 2020 Elena Kulinskaya
David C. Hoaglin
Ilyas Bakbergenuly
+ Clinical Trial Simulations 2023 Jay Park
Edward J. Mills
J. Kyle Wathen
+ PDF Chat What is Computer Simulation? 2015 Kristin L. Sainani
+ Simulation Framework for Realistic Large-scale Individual-level Health Data Generation 2020 Santtu Tikka
Jussi Hakanen
Mirka Saarela
Juha Karvanen
+ Using simulation studies to evaluate statistical methods in Stata: A tutorial 2016 Tim P. Morris
Ian R. White
Michael J. Crowther
+ Using simulation studies to evaluate statistical methods in Stata: A tutorial 2016 Tim P. Morris
Ian R. White
Michael J. Crowther
+ PDF Chat A Review of Clinical Trial Simulation Methods 2013 Chris Schuetz
Sea Hing Ong
+ A Computer-Aided SAS Macro for the Evaluation of the Simulation Performances in Missingness Settings 2015 Urko Aguirre
Inmaculada Aróstegui
José M. Quintana
+ PDF Chat Handling Missingness, Failures, and Non-Convergence in Simulation Studies: A Review of Current Practices and Recommendations 2024 Samuel Pawel
František Bartoš
Björn S. Siepe
Anna Lohmann
+ PDF Chat Simulation methods to estimate design power: an overview for applied research 2011 Benjamin F. Arnold
Daniel R Hogan
John M. Colford
Alan Hubbard
+ PDF Chat Faculty Opinions recommendation of Using simulation studies to evaluate statistical methods. 2020 Robert W. Platt
+ Simulation-Based Sample Size Calculation 2020 Meinhard Kieser
+ PDF Chat Noise-free comparison of stochastic agent-based simulations using common random numbers 2024 Daniel J. Klein
Romesh Abeysuriya
Robyn M. Stuart
Cliff C. Kerr

Works That Cite This (18)

Action Title Year Authors
+ Good Data Science Practice: Moving Toward a Code of Practice for Drug Development 2022 Mark Baillie
Conor Moloney
Carsten Philipp Mueller
Jonas F. Dorn
Janice Branson
David Ohlssen
+ Inverse Probability Weighting to Estimate Exposure Effects on the Burden of Recurrent Outcomes in the Presence of Competing Events 2023 Charles Gaber
Jessie K. Edwards
Jennifer L. Lund
Anne F. Peery
David B. Richardson
Alan C. Kinlaw
+ PDF Chat Adaptive designs in critical care trials: a simulation study 2023 Wei Li
Victoria Cornelius
Simon Finfer
Balasubramanian Venkatesh
Laurent Billot
+ When is Plasmode simulation superior to parametric simulation when estimating the MSE of the least squares estimator in linear regression? 2023 Marieke Stolte
Nicholas Schreck
Alla Slynko
Maral Saadati
Axel Benner
Jörg Rahnenführer
Andrea Bommert
+ Comparing algorithms for characterizing treatment effect heterogeneity in randomized trials 2022 Sophie Sun
Konstantinos Sechidis
Yao Chen
Jiarui Lu
Chong Ma
Ardalan Mirshani
David Ohlssen
Marc Vandemeulebroecke
Björn Bornkamp
+ Methods for determining clinical utility 2023 Francis Bowling
Tony Badrick
+ PDF Chat Rationale and performances of a data-driven method for computing the duration of pharmacological prescriptions using secondary data sources 2022 Laura Pazzagli
David Liang
Morten Andersen
Marie Linder
Abdul Rauf Khan
Maurizio Sessa
+ PDF Chat Comparison of statistical methods for the analysis of patient-reported outcomes in randomised controlled trials: A simulation study 2024 Yirui Qian
Stephen J. Walters
Richard Jacques
Laura Flight
+ PDF Chat Statistical plasmode simulations–Potentials, challenges and recommendations 2024 Nicholas Schreck
Alla Slynko
Maral Saadati
Axel Benner
+ PDF Chat Comparison of variable selection procedures and investigation of the role of shrinkage in linear regression-protocol of a simulation study in low-dimensional data 2022 Edwin Kipruto
Willi Sauerbrei

Works Cited by This (15)

Action Title Year Authors
+ Evidence of subgroup‐specific treatment effect in the absence of an overall effect: is there really a contradiction? 2013 Michał Abrahamowicz
Marie‐Eve Beauchamp
Pierre Fournier
Alexandre Dumont
+ Assessment and comparison of prognostic classification schemes for survival data 1999 Erika Graf
Claudia Schmoor
Willi Sauerbrei
M. Schumacher
+ The design of simulation studies in medical statistics 2006 Andrea Burton
Douglas G. Altman
Patrick Royston
Roger Holder
+ PDF Chat STRengthening Analytical Thinking for Observational Studies: the STRATOS initiative 2014 Willi Sauerbrei
Michał Abrahamowicz
Douglas G. Altman
Saskia le Cessie
James R. Carpenter
+ Two-sided confidence intervals for the single proportion: comparison of seven methods 1998 Robert G. Newcombe
+ PDF Chat Subsampling Versus Bootstrapping in Resampling-Based Model Selection for Multivariable Regression 2015 Riccardo De Bin
Silke Janitza
Willi Sauerbrei
Anne‐Laure Boulesteix
+ A simulation study to compare robust clustering methods based on mixtures 2010 Pietro Coretto
Christian Hennig
+ PDF Chat To test or not to test: Preliminary assessment of normality when comparing two independent samples 2012 Justine Rochon
Matthias Gondan
Meinhard Kieser
+ PDF Chat Play It Again: Teaching Statistics With Monte Carlo Simulation 2016 Matthew Sigal
R. Philip Chalmers
+ PDF Chat Towards evidence-based computational statistics: lessons from clinical research on the role and design of real-data benchmark studies 2017 Anne‐Laure Boulesteix
Rory Wilson
Alexander Hapfelmeier