Session P72.5

Measurement of Electrocardiograms Interpretations Variability

AM Andrianov*, J Reinstaedtler, J Gallardo Banez

VIASYS Healthcare GmbH
Hoechberg, Germany

Background: Interpretations of physicians, software algorithms of automatic ECG analysis often vary for one and the same ECG test. This variability depends on expert’s background, proficiency and even subjective reasons. The purpose of this work is to make a critical esteem of ECG interpretations accuracy of medical experts.
Methods: Statistical analysis, Kappa coefficient, Weighted Kappa, Analysis of Variance of the data that were retrieved from 5 observers that have reviewed 233 ECG tests multiple times. In this work we present results of the study made to measure reliability of ECG interpretations of external observers for pharmaceutical companies. It is well known that deviation between interpretations exists since the ECG recordings were invented. Our aim was to measure this deviation scope and try to control it. In order to evaluate the variability not only of QRS measurements but also interpretations of ECG tests we standardized the interpretation annotations, developing a list of grouped statements. For each interpretation statement a weight was defined (from 0.0 to 1.0), afterwards agreements and disagreements of statements were calculated and evaluated a general “annotations score” for each observer. The experiment was separated into measurement of interpretation variability between observers (inter reader variability) and variability in interpretations of the same observer (intra reader variability). To define a “gold standard” all tests were reviewed by a group of very experienced electro-cardiologists. Afterwards we calculated correlation coefficient between “gold standard” and data from the observers (from r = 0.48 to r = 0.82).
Results: As a statistical analysis of variability calculations of confidence limits (P = 91%), and a standard deviation of an error for each observer were used. For agreement measurement Kappa statistics was chosen. For different observers Kappa coefficient varied from moderate (k=0, 12) to good (k=0, 65). Basing on Kappa statistics and correlation with the “gold standard” it was possible to group all observers by stability, nature (Inter- or Intra-), variability range (e.g. low k<0, 20, low r<0, 80) and reliability. Based on this analysis it was possible to develop an individual training and proficiency process for every observer, covering their specific problem(s) with the ECG analysis process. Consequently this made it possible to react faster on any inconsistency in the ECG analysis process.
Conclusion: constant monitoring of inter- intra reader variability analysis is nowadays a necessary part of the ECG analysis process. It improves reliability and quality of the measurements, reduces the disagreement between cardiologists and controls the proficiency of observers. This analysis was automated and integrated in our analysis software (ECG OverRead) making it possible to control the interpretation variability limits and analyze dynamics.

(Abstract Control Number: 203)