physionet-logo-kp6The PhysioNet/CinC Challenge

The 18th annual PhysioNet/CinC Challenge – Details:
AF classification from a short single lead ECG recording

Background

While planning CinC 2000, local hosts Roger Mark and George Moody proposed to organize an activity that would make effective use of their newly-established PhysioNet web site to stimulate rapid progress on an unsolved problem of practical clinical significance. A timely contribution of data made it possible to create the first PhysioNet/CinC Challenge, which attracted the attention of more than a dozen teams to the subject of detecting sleep apnea from the ECG. Their efforts were broadly successful, they discussed their findings at CinC 2000, and an annual tradition was born.

PhysioNet offers free access via the web to large and growing collections of recorded physiologic signals and related open-source software. Originally established under the auspices of the NIH’s National Center for Research Resources, PhysioNet has been funded since September 2007 under a cooperative agreement with the NIH’s National Institute of Biomedical Imaging and Bioengineering (NIBIB), and with the NIH’s National Institute of General Medical Sciences (NIGMS).

In complementary ways, PhysioNet and Computing in Cardiology catalyze and support scientific communication and collaboration between basic and clinical scientists. The annual meetings of CinC are gatherings of researchers from many nations and disciplines, bridging the geographic and specialty chasms that separate understanding from practice, while PhysioNet provides on-line data and software resources that support collaborations of basic and clinical researchers throughout the year. The annual PhysioNet/CinC Challenges seek to provide stimulating yet friendly competitions, while at the same time offering both specialists and non-specialists alike opportunities to make progress on significant open problems whose solutions may be of profound clinical value. The use of shared data provided via PhysioNet makes it possible for participants to work independently toward a common objective. At CinC, participants can make meaningful results-based comparisons of their methods; lively and well-informed discussions are the norm at scientific sessions dedicated to these challenges. Discovery of the complementary strengths of diverse approaches to a problem when coupled with deep understanding of that problem frequently sparks new collaborations and opportunities for further study.

A new challenge topic is announced each year on the Challenge page at PhysioNet. The PhysioNet team assembles and posts the raw materials needed to begin work. Usually, these raw materials consist of a collection of data to be analyzed; the analyses are provided for a subset of the data (the “learning set”) in each case, and the challenge is to analyze the remaining data (the “test set”).

What will be the topic of the next challenge? It might be image analysis, or simulation, or forecasting…. An ideal challenge problem is interesting, clinically important, and possible to study using available materials that have not been widely circulated previously. Moreover, there must be an objective way to evaluate the quality of a challenge entry (for an analysis problem, this usually means there must be a known set of correct analyses of the data, i.e., a “gold standard” against which entries can be compared). You are invited to submit your ideas for future challenge topics to the Challenge organizer, George Moody; suggestions accompanying relevant data are particularly welcome.

PhysioNet/CinC Challenge Awards

Challenges are open to all. An important milestone for participants is the deadline for submitting abstracts for CinC, which is 15 April each year. Those wishing to qualify as official entrants, with eligibility for awards, must submit an abstract describing their work as well as an entry for scoring by about one week before the abstract deadline. A limited number of revised entries may be submitted between the abstract deadline and the final challenge deadline in early September. Eligibility for awards also requires participants to present their work in a scientific session of CinC. See the Challenge pages on PhysioNet for deadlines and rules for this year’s competition.

Most Challenges are presented as two events (often a narrowly-defined question and a more general one), or sometimes more.

An award is offered for the best solution obtained by any eligible participant.  To qualify for this prize, the solution must include computer code in open-source format.  Each entry should contain a file that defines the specific open-source license under which the software is available. See the Challenge Web page for more details on the problem and the submission process for each year’s challenge.  The winners receive their awards during the final plenary session on Wednesday afternoon. Follow the links below for details about previous challenges.

PhysioNet/CinC Challenge Award Winners

2016:Classification of Normal/Abnormal Heart Sound Recordings

1. Cristhian Potes*, Saman Parvaneh, Asif Rahman, Bryan Conroy, Daniel Schulman and John Ames
Hybrid Feature Aggregation for Detection of Abnormal Heart Sound
Philips Research, Cambridge, MA, USA

2. Morteza Zabihi*1, Ali Bahrami Rad2, Serkan Kiranyaz3, Moncef Gabbouj1 and Aggelos K. Katsaggelos4  PhysioNet/CinC Challenge: Normal/Abnormal PCG Classification using an Ensemble of Support Vector Machines
1Tampere University of Technology, Tampere, Finland 2University of Stavanger, Stavanger, Norway 3Qatar University, Doha, Qatar 4Northwestern University, Evanston, USA

3. Edmund Kay* and Anurag Agarwal Katsaggelos  Neural Networks and the Continuous Wavelet Transform for Classifying Heart Sounds
University of Cambridge, UK

2015: Reducing False Arrhythmia Alarms in the ICU

Overall Winners
Event 1. Filip Plešinger, Petr Klimeš, Josef Halámek, & Pavel Jurák from the Institute of Scientific Instruments of the CAS, Brno, Czech Republic
Event 2. Sibylle Fallet, Sasan Yazdani, and Jean-Marc Vesin from the Applied Signal Processing Group, Swiss Federal Institute of Technology, Lausanne, Switzerland
Event 1 Results
1. Filip Plešinger, Petr Klimeš, Josef Halámek, & Pavel Jurák
2.Vignesh Kalidas and Lakshman Tamil
3.Paula Couto, Ruben Ramalho and Rui Rodrigues
Event 2 Results
1.Sibylle Fallet, Sasan Yazdani, and Jean-Marc Vesin
2.Filip Plešinger, Petr Klimeš, Josef Halámek, & Pavel Jurák
3.Vignesh Kalidas and Lakshman Tamil

2014: Robust Detection of Heart Beats in Multimodal Data

Phase I Results
1. Marcus Vollmer, University of Greifswald, Greifswald, Germany
2. Urska Pangerc and Franc Jager, Faculty of Computer and Information Science, Ljubljana, Slovenia
3. Lars Johannesen, Jose Vicente, Christopher Scully, Loriano Galeotti and David Strauss, CDRH, US FDA, Silver Spring, MD, USA United States
4. Quan Ding, Yong Bai, Yusuf Erol, Rebeca Salas-Boni, Xiaorong Zhang, Lei Li and Xiao Hu, University of California, San Francisco, CA, USA
Phase II Results
1. Thomas De Cooman, Griet Goovaerts, Carolina Varon, Devy Widjaja and Sabine Van Huffel, KU Leuven, Department of Electrical Engineering-ESAT, STADIUS Center for Dynamical Systems, Signal Processing and Data Analytics, Belgium + iMinds, Medical IT Department, Belgium
2. Marcus Vollmer, University of Greifswald, Greifswald, Germany
3. Urska Pangerc and Franc Jager, Faculty of Computer and Information Science, Ljubljana, Slovenia
4. Filip Plesinger, Juraj Jurco, Josef Halamek and Pavel Jurak, Institute of Scientific Instruments of the ASCR, Brno, Czech Republic
Phase III Results
1. Alistair E W Johnson, Joachim Behar, Fernando Andreotti, Gari D Clifford and Julien Oster, University of Oxford, England, United Kingdom
2. Soo-Kng Teo, Bo Yang, Bart Hoeben, Dong Huang, Monterola Christopher and Yi Su, Institute of High Performance Computing, A*STAR, Singapore
3. Thomas De Cooman, Griet Goovaerts, Carolina Varon, Devy Widjaja and Sabine Van Huffel, Center for Dynamical Systems, Signal Processing and Data Analytics, Belgium + iMinds, Medical IT Department, Belgium
4. Jan Jakub Gierałtowski, Kamil Ciuchciński, Iga Grzegorczyk, Katarzyna Kośna, Mateusz Soliński and Piotr Podziemski, Warsaw University of Technology, Poland

2013: Noninvasive Fetal ECG

Events 1 and 2:
Maurizio Varanini, Gennaro Tartarisco, Lucia Billeci, Alberto Macerata, Giovanni Pioggia, and Rita Balocchi, CNR ICP, Pisa, Italy (first place; also third place in events 4 and 5)
Rui Rodrigues, Universidade Nova de Lisboa (third place, event 1)
Luigi Yuri Di Marco, Alberto Marzo, and Alejandro Frangi, University of Sheffield (third place, event 2)
Event 3:
Piotr Podziemski and Jan Gierałtowski, Warsaw University of Technology, Warsaw, Poland (first place; also second place in events 1 and 2)
Events 4 and 5:
Fernando Andreotti, Maik Riedl, Tilo Himmelsbach, Daniel Wedekind, Sebastian Zaunseder, Niels Wessel, and Hagen Malberg, Dresden University of Technology, Dresden, Germany (first place)
Jukka A Lipponen and Mika P Tarvainen, University of Eastern Finland (second place)

Joachim Behar, Julien Oster, and Gari Clifford, University of Oxford, UK, obtained the top scores in events 1 and 2 while participating unofficially.

2012: Predicting Mortality of ICU Patients

Event 1:
Alistair Johnson, Nic Dunkley, Louis Mayaud, Athanasios Tsanas, Andrew Kramer and Gari Clifford, University of Oxford, UK
Event 2:
Luca Citi and Riccardo Barbieri, MIT, Cambridge, Massachusetts, USA, and Massachusetts General Hospital, Boston, Massachusetts, USA

2011: Improving the Quality of ECGs Collected using Mobile Phones

Event 1:
Henian Xia, Gabriel Garcia, Joseph McBride, Adam Sullivan, Thibaut De Bock, Jujhar Bains, Dale Wortham, and Xiaopeng Zhao, University of Tennessee, Knoxville, Tennessee, USA (first place)
Gari Clifford, Daniel Lopez, Qiao Li, and Iead Rezek, University of Oxford, UK (second place)
Event 2:
Henian Xia, Gabriel Garcia, Joseph McBride, Adam Sullivan, Thibaut De Bock, Jujhar Bains, Dale Wortham, and Xiaopeng Zhao, University of Tennessee, Knoxville, Tennessee, USA (first place)
Benjamin Moody, MIT, Cambridge, Massachusetts (second place)
Event 3:
Dieter Hayn, Bernhard Jammerbund, Günter Schreier, AIT, Graz, Austria (first place)
Václav Chudáček, Lukáš Zach, Jakub Kužílek, Jiří Spilka and Lenka Lhotská, Czech Technical University, Prague, Czech Republic (second place)

2010: Mind the Gap!

Event 1 (general and open-source):
Rui Rodrigues, Universidade Nova de Lisboa, Portugal
Event 2 (general and open-source):
The team of Adam Sullivan, Henian Xia, Joseph McBride, and Xiaopeng Zhao, University of Tennessee, Knoxville, Tennessee, USA

2009: Predicting Acute Hypotensive Episodes

Event 1 (general and open-source):
Xiaoxiao Chen, Michigan State University, Lansing, Michigan, USA
Event 2 (general and open-source):
Jorge Henriques and Teresa Rocha, University of Coimbra, Coimbra, Portugal

2008: Detecting and Quantifying T-Wave Alternans

General:
Giovanni Bortolan, ISIB-CNR, Padova, Italy, and
Ivaylo Christov, Bulgarian Academy of Sciences, Sofia, Bulgaria
Open-source:
Alexander Khaustov, Incart, St. Petersburg, Russia

Jubair Saieed, Bangladesh University of Engineering and Technology, Dhaka, Bangladesh, obtained the top score overall while participating unofficially in the open-source division.

2007: Electrocardiographic Imaging of Myocardial Infarction

Event 1:
Hamid SadAbadi, KN Toosi University, Teheran, Iran
Event 2:
Mohamed Mneimneh, Marquette University, Milwaukee, Wisconsin, USA
Event 3 (tie):
Masood Ghasemi, KN Toosi University, Teheran, Iran
Mohamed Mneimneh, Marquette University, Milwaukee, Wisconsin, USA
Hamid SadAbadi, KN Toosi University, Teheran, Iran
Event 4 (tie):
Mohamed Mneimneh, Marquette University, Milwaukee, Wisconsin, USA
Hamid SadAbadi, KN Toosi University, Teheran, Iran

2006: QT Interval Measurement

Division 1 (manual/semi-automated):
Mariano Llamedo Soria, Universidad Tecnologica Nacional FRBA, Buenos Aires, Argentina
Division 2 (automated, general):
Dieter Hayn, ARC Seibersdorf Research GmbH, Graz, Austria
Division 3 (automated, open-source):
Yuri Chesnokov, Cambridge University, Cambridge, UK

2005: The First Five Challenges Revisited

Richard Povinelli, Marquette University, Milwaukee, Wisconsin, USA
C Raab, University of Potsdam, Potsdam, Germany

2004: Spontaneous Termination of Atrial Fibrillation

Event A (general):
Dieter Hayn, ARC Seibersdorf Research GmbH, Graz, Austria
Event A (open-source):
Maurizio Varanini, CNR-ICP, Pisa, Italy
Event B (general and open-source):
Federico Cantini, CNR-ICP, Pisa, Italy

Simona Petrutiu, a member of the research group at Northwestern University that contributed the data, obtained the top scores in both events while participating unofficially.

2003: Distinguishing Ischemic from Non-Ischemic ST Changes

Philip Langley, University of Newcastle upon Tyne, Newcastle, UK

2002: RR Interval Time Series Modeling

Event 1:
DC Lin, Ryerson University, Toronto, Canada
Event 2:
Albert C-C Yang, National Yang-Ming University, Taipei, Taiwan

2001: Predicting Paroxysmal Atrial Fibrillation

Event 1:
Günther Schreier, ARC Seibersdorf, Graz, Austria
Event 2:
Günther Schreier, ARC Seibersdorf, Graz, Austria

Wei Zong, a member of the PhysioNet team at MIT, obtained the top score in event 2 while participating unofficially.

2000: Detecting Sleep Apnea from the ECG

Event 1:
Murray Jarvis, Caltech, Pasadena, California, USA
Event 2:
James McNames, Portland State University, Portland, Oregon, USA