BIAS: Transparent reporting of biomedical image analysis challenges

Lena Maier‐Hein, Annika Reinke, Michal Kozubek, Anne L. Martel, Tal Arbel, Matthias Eisenmann, Allan Hanbury, Pierre Jannin, Henning Müller, Sinan Onogur

Type: Article

Publication Date: 2020-08-21

Citations: 87

DOI: https://doi.org/10.1016/j.media.2020.101796

View

Locations

Medical Image Analysis - View
PubMed Central - View
arXiv (Cornell University) - View - PDF
Radboud Repository (Radboud University) - View - PDF
Data Archiving and Networked Services (DANS) - View - PDF
ArODES (HES-SO (https://www.hes-so.ch/)) - View - PDF
HAL (Le Centre pour la Communication Scientifique Directe) - View - PDF
PubMed - View

Similar Works

Action	Title	Year	Authors
+	BIAS: Transparent reporting of biomedical image analysis challenges	2019	Lena Maier‐Hein Annika Reinke Michal Kozubek Anne L. Martel Tal Arbel Matthias Eisenmann Allan Hanbuary Pierre Jannin Henning Müller Sinan Onogur
+	BIAS: Transparent reporting of biomedical image analysis challenges	2019	Klaus H. Maier‐Hein Annika Reinke Michal Kozubek Anne L. Martel Tal Arbel Matthias Eisenmann Allan Hanbuary Pierre Jannin Henning Müller Sinan Onogur
+ PDF Chat	Why rankings of biomedical image analysis competitions should be interpreted with care	2018	Lena Maier‐Hein Matthias Eisenmann Annika Reinke Sinan Onogur Marko Stankovic Patrick Scholz Tal Arbel Hrvoje Bogunović Andrew P. Bradley Aaron Carass
+	Methods and open-source toolkit for analyzing and visualizing challenge results	2019	Manuel Wiesenfarth Annika Reinke Bennett A. Landman M. Jorge Cardoso Klaus H. Maier‐Hein Annette Kopp‐Schneider
+	Methods and open-source toolkit for analyzing and visualizing challenge results	2019	Manuel Wiesenfarth Annika Reinke Bennett A. Landman M. Jorge Cardoso Lena Maier‐Hein Annette Kopp‐Schneider
+ PDF Chat	Methods and open-source toolkit for analyzing and visualizing challenge results	2021	Manuel Wiesenfarth Annika Reinke Bennett A. Landman Matthias Eisenmann Laura Aguilera Saiz M. Jorge Cardoso Lena Maier‐Hein Annette Kopp‐Schneider
+	Understanding metric-related pitfalls in image analysis validation	2023	Annika Reinke Minu D. Tizabi Michael Baumgartner Matthias Eisenmann Doreen Heckmann-Nötzel Ali Emre Kavur Tim Rädsch Carole H. Sudre Laura Ación Michela Antonelli
+	Metrics reloaded: Recommendations for image analysis validation	2022	Lena Maier‐Hein Annika Reinke Patrick Godau Minu D. Tizabi Florian Buettner Evangelia Christodoulou Ben Glocker Fabian Isensee Jens Kleesiek Michal Kozubek
+	Biomedical image analysis competitions: The state of current participation practice	2022	Matthias Eisenmann Annika Reinke Vivienn Weru Minu D. Tizabi Fabian Isensee Tim Adler Patrick Godau Veronika Cheplygina Michal Kozubek Sharib Ali
+ PDF Chat	The Image Biomarker Standardization Initiative: Standardized Quantitative Radiomics for High-Throughput Image-based Phenotyping	2020	Alex Zwanenburg Martin Vallières Mahmoud A. Abdalah Hugo J.W.L. Aerts Vincent Andrearczyk Aditya Apte Saeed Ashrafinia Spyridon Bakas Roelof J. Beukinga Ronald Boellaard
+	A Framework for Challenge Design: Insight and Deployment Challenges to Address Medical Image Analysis Problems	2019	Adriënne M. Mendrik Stephen Aylward
+	A Framework for Challenge Design: Insight and Deployment Challenges to Address Medical Image Analysis Problems	2019	Adriënne M. Mendrik Stephen Aylward
+	Common Limitations of Image Processing Metrics: A Picture Story	2021	Annika Reinke Minu D. Tizabi Carole H. Sudre Matthias Eisenmann Tim Rädsch Michael Baumgartner Laura Ación Michela Antonelli Tal Arbel Spyridon Bakas
+	Best Practices and Scoring System on Reviewing A.I. based Medical Imaging Papers: Part 1 Classification	2022	Timothy L. Kline Felipe Kitamura Ian Pan Amine M. Korchi Neil Tenenholtz Linda Moy Judy Wawira Gichoya Igor Santos Steven L. Blumer Misha Ysabel Hwang
+	Community-developed checklists for publishing images and image analysis	2023	Christopher Schmied Michael S. Nelson Sergiy Avilov Gert‐Jan Bakker Cristina Bertocchi Johanna Bischof Ulrike Boehm Jan Brocher Mariana Lombardi Peres de Carvalho Catalin Chiritescu
+ PDF Chat	Bias Assessment and Data Drift Detection in Medical Image Analysis: A Survey	2024	Andrea Prenner Bernhard Kainz
+	Community-developed checklists for publishing images and image analyses.	2023	Christopher Schmied Michael S. Nelson Sergiy Avilov Gert‐Jan Bakker Cristina Bertocchi Johanna Bischof Ulrike Boehm Jan Brocher Mariana Lombardi Peres de Carvalho Catalin Chiritescu
+ PDF Chat	Precision-medicine-toolbox: An open-source python package for the quantitative medical image analysis	2023	Elizaveta Lavrova Sergey Primakov Zohaib Salahuddin Manon Beuque Damon Verstappen Henry C. Woodruff Philippe Lambin
+	Challenge Results Are Not Reproducible	2023	Annika Reinke Georg Grab Klaus H. Maier‐Hein
+	How I failed machine learning in medical imaging -- shortcomings and recommendations	2021	Gaël Varoquaux Veronika Cheplygina

Cited by (31)

Action	Title	Year	Authors
+	The Liver Tumor Segmentation Benchmark (LiTS)	2022	Patrick Bilic Patrick Ferdinand Christ Hongwei Li Eugene Vorontsov Avi Ben-Cohen Georgios Kaissis Adi Szeskin Colin Jacobs Gabriel Efrain Humpire Mamani Gabriel Chartrand
+ PDF Chat	Overview of the HECKTOR Challenge at MICCAI 2021: Automatic Head and Neck Tumor Segmentation and Outcome Prediction in PET/CT Images	2022	Vincent Andrearczyk Valentin Oreiller Sarah Boughdad Catherine Cheze Le Rest Hesham Elhalawani Mario Jreige John O. Prior Martin Vallières Dimitris Visvikis Mathieu Hatt
+	Cutting-edge 3D Medical Image Segmentation Methods in 2020: Are Happy Families All Alike?	2021	Jun Ma
+	Hunting imaging biomarkers in pulmonary fibrosis: Benchmarks of the AIIB23 challenge	2023	Nan Yang Xiaodan Xing Shiyi Wang Zeyu Tang Federico N Felder Sheng Zhang Roberta Eufrasia Ledda Xiaoliu Ding Ruiqi Yu Weiping Liu
+	BIAS: Transparent reporting of biomedical image analysis challenges	2020	Lena Maier‐Hein Annika Reinke Michal Kozubek Anne L. Martel Tal Arbel Matthias Eisenmann Allan Hanbury Pierre Jannin Henning Müller Sinan Onogur
+	Comparative validation of machine learning algorithms for surgical workflow and skill analysis with the HeiChole benchmark	2023	Martin Wagner Beat P. Müller‐Stich Anna Kisilenko Duc Tran Patrick Heger Lars Mündermann David M Lubotsky Benjamin Müller Tornike Davitashvili Manuela Capek
+	Fetal brain tissue annotation and segmentation challenge results	2023	Kelly Payette Hongwei Li Priscille de Dumast Roxane Licandro Hui Ji Md Mahfuzur Rahman Siddiquee Daguang Xu Andriy Myronenko Hao Liu Yuchen Pei
+ PDF Chat	The Medical Segmentation Decathlon	2022	Michela Antonelli Annika Reinke Spyridon Bakas Keyvan Farahani Annette Kopp‐Schneider Bennett A. Landman Geert Litjens Bjoern Menze Olaf Ronneberger Ronald M. Summers
+	Medical deep learning—A systematic meta-review	2022	Jan Egger Christina Gsaxner Antonio Pepe Kelsey L. Pomykala Frederic Jonske Manuel Kurz Jianning Li Jens Kleesiek
+	Multi-Center Fetal Brain Tissue Annotation (FeTA) Challenge 2022 Results	2024	Kelly Payette Céline Steger Roxane Licandro Priscille De Dumast Hongwei Li Matthew J. Barkovich Liu Li Maik Dannecker Chen Chen Cheng Ouyang
+	CholecTriplet2022: Show me a tool and tell me the triplet -- an endoscopic vision challenge for surgical action triplet detection	2023	Chinedu Innocent Nwoye Tong Yu Saurav Sharma Aditya Murali Deepak Alapatt Armine Vardazaryan Kun Yuan Jonas Hajek Wolfgang Reiter Amine Yamlahi
+	Placental vessel segmentation and registration in fetoscopy: Literature review and MICCAI FetReg2021 challenge findings	2023	Sophia Bano Alessandro Casella Francisco Herbert Lima Vasconcelos Abdul Qayyum Abdesslam Benzinou Moona Mazher Fabrice Mériaudeau Chiara Lena Ilaria Anita Cintorrino Gaia Romana De Paolis
+	Hunting imaging biomarkers in pulmonary fibrosis: Benchmarks of the AIIB23 challenge	2024	Nan Yang Xiaodan Xing Shiyi Wang Zeyu Tang Federico N Felder Sheng Zhang Roberta Eufrasia Ledda Xiaoliu Ding Ruiqi Yu Weiping Liu
+ PDF Chat	SegReg: Segmenting OARs by Registering MR Images and CT Annotations	2024	Zeyu Zhang Xuyin Qi Bowen Zhang Biao Wu Hien Le Bora Jeong Zhibin Liao Yunxiang Liu Johan Verjans Minh‐Son To
+ PDF Chat	SimCol3D - 3D reconstruction during colonoscopy challenge	2024	Anita Rau Sophia Bano Yueming Jin Pablo Azagra Javier Morlana Rawen Kader Edward Sanderson Bogdan J. Matuszewski Jae‐Young Lee Dong-Jae Lee
+ PDF Chat	From Nano to Macro: An overview of the IEEE Bio Image and Signal Processing Technical Committee	2023	Selin Aviyente Alejandro F. Frangi Erik Meijering Arrate Muñoz‐Barrutia Michael Liebling Dimitri Van De Ville Jean‐Christophe Olivo‐Marín Jelena Kovačević Michaël Unser
+ PDF Chat	Learn2Reg: Comprehensive Multi-Task Medical Image Registration Challenge, Dataset and Evaluation in the Era of Deep Learning	2022	Alessa Hering Lasse Hansen Tony C. W. Mok Albert C. S. Chung Hanna Siebert Stephanie Häger Annkristin Lange Sven Kuckertz Stefan Heldmann Wei Shao
+	CrossMoDA 2021 challenge: Benchmark of cross-modality domain adaptation techniques for vestibular schwannoma and cochlea segmentation	2022	Reuben Dorent Aaron Kujawa Marina Ivory Spyridon Bakas Nicola Rieke Samuel Joutard Ben Glocker M. Jorge Cardoso Marc Modat Kayhan Batmanghelich
+ PDF Chat	CholecTriplet2022: Show me a tool and tell me the triplet — An endoscopic vision challenge for surgical action triplet detection	2023	Chinedu Innocent Nwoye Tong Yu Saurav Sharma Aditya Murali Deepak Alapatt Armine Vardazaryan Kun Yuan Jonas Hajek Wolfgang Reiter Amine Yamlahi
+	Where is VALDO? VAscular Lesions Detection and segmentatiOn challenge at MICCAI 2021	2023	Carole H. Sudre Kimberlin Van Wijnen Florian Dubost Hieab H.H. Adams David Atkinson Frederik Barkhof Mahlet Birhanu Esther E. Bron Robin Camarasa Nish Chaturvedi
+ PDF Chat	LoDoPaB-CT, a benchmark dataset for low-dose computed tomography reconstruction	2021	Johannes Leuschner Maximilian Schmidt Daniel Otero Baguer Peter Maaß
+	Surgical data science – from concepts toward clinical translation	2021	Lena Maier‐Hein Matthias Eisenmann Duygu Sarıkaya Keno März Toby Collins Anand Malpani Johannes Fallert Hubertus Feußner Stamatia Giannarou Pietro Mascagni
+	Surgical Visual Domain Adaptation: Results from the MICCAI 2020 SurgVisDom Challenge	2021	Aneeq Zia Kiran Bhattacharyya Xi Liu Ziheng Wang Satoshi Kondo Emanuele Colleoni Beatrice van Amsterdam Razeen Hussain Raabid Hussain Lena Maier‐Hein
+	Report of the Medical Image De-Identification (MIDI) Task Group -- Best Practices and Recommendations	2023	David Clunie Adam E. Flanders Adam Taylor Brad Erickson Brian Bialecki David Brundage David A. Gutman Fred Prior J. Anthony Seibert John Perry
+ PDF Chat	Heidelberg colorectal data set for surgical data science in the sensor operating room	2021	Lena Maier‐Hein Martin Wagner Tobias Roß Annika Reinke Sebastian Bodenstedt Peter M. Full Hellena Hempe Diana Mîndroc-Filimon Patrick Scholz Thuy Nuong Tran
+	Surgical Data Science -- from Concepts toward Clinical Translation	2020	Lena Maier‐Hein Matthias Eisenmann Duygu Sarikaya Keno März Toby Collins Anand Malpani Johannes Fallert Hubertus Feußner Stamatia Giannarou Pietro Mascagni
+ PDF Chat	SegRap2023: A benchmark of organs-at-risk and gross tumor volume Segmentation for Radiotherapy Planning of Nasopharyngeal Carcinoma	2025	Xiangde Luo Jia Fu Yunxin Zhong Shuolin Liu Bing Han Mehdi Astaraki Simone Bendazzoli Iuliana Toma-Daşu Yiwen Ye Ziyang Chen
+	The Federated Tumor Segmentation (FeTS) Challenge	2021	Sarthak Pati Ujjwal Baid Maximilian Zenk Brandon Edwards Micah Sheller G. Anthony Reina Patrick Foley Alexey Gruzdev Jason Martin Shadi Albarqouni
+	Heidelberg Colorectal Data Set for Surgical Data Science in the Sensor Operating Room	2020	Lena Maier‐Hein Martin Wagner Tobias Roß Annika Reinke Sebastian Bodenstedt Peter M. Full Hellena Hempe Diana Mîndroc-Filimon Patrick Scholz Thuy Nuong Tran
+	How can we learn (more) from challenges? A statistical approach to driving future algorithm development.	2021	Tobias Roß Pierangela Bruno Annika Reinke Manuel Wiesenfarth Lisa Koeppel Peter M. Full Bünyamin Pekdemir Patrick Godau Darya Trofimova Fabian Isensee
+	Towards responsible research in digital technology for health care.	2021	Pierre Jannin

Citing (7)

Action	Title	Year	Authors
+ PDF Chat	Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: The LUNA16 challenge	2017	Arnaud Arindra Adiyoso Setio Alberto Traverso Thomas de Bel Moira S.N. Berens Cas van den Bogaard P. Cerello Hao Chen Qi Dou M.E. Fantacci Bram Geurts
+ PDF Chat	A survey on deep learning in medical image analysis	2017	Geert Litjens Thijs Kooi Babak Ehteshami Bejnordi Arnaud Arindra Adiyoso Setio Francesco Ciompi Mohsen Ghafoorian Jeroen van der Laak Bram van Ginneken Clara I. Sá‎nchez
+ PDF Chat	Why rankings of biomedical image analysis competitions should be interpreted with care	2018	Lena Maier‐Hein Matthias Eisenmann Annika Reinke Sinan Onogur Marko Stankovic Patrick Scholz Tal Arbel Hrvoje Bogunović Andrew P. Bradley Aaron Carass
+	A Probabilistic U-Net for Segmentation of Ambiguous Images	2018	Simon Kohl Bernardino Romera‐Paredes Clemens Meyer Jeffrey De Fauw Joseph R. Ledsam Klaus H. Maier‐Hein S. M. Ali Eslami Danilo Jimenez Rezende Olaf Ronneberger
+	BIAS: Transparent reporting of biomedical image analysis challenges	2020	Lena Maier‐Hein Annika Reinke Michal Kozubek Anne L. Martel Tal Arbel Matthias Eisenmann Allan Hanbury Pierre Jannin Henning Müller Sinan Onogur
+ PDF Chat	Methods and open-source toolkit for analyzing and visualizing challenge results	2021	Manuel Wiesenfarth Annika Reinke Bennett A. Landman Matthias Eisenmann Laura Aguilera Saiz M. Jorge Cardoso Lena Maier‐Hein Annette Kopp‐Schneider
+	Methods and open-source toolkit for analyzing and visualizing challenge results	2019	Manuel Wiesenfarth Annika Reinke Bennett A. Landman M. Jorge Cardoso Klaus H. Maier‐Hein Annette Kopp‐Schneider