Introduction: Automatic detection and classification of cardiac disor-ders plays a critical role in the analysis of clinical electrocardiogram (ECG). Traditional classification methods – e.g., support vector machine, Hidden Markov model and random forest – generally require carefully designed input features, which can be challenging for classifying ECG recordings with multiple abnormalities. Deep learning is effective for automated feature extraction and has shown promising results in ECG classification. Most of these methods, however, assume that multiple cardiac disorders are mutually exclusive. In this work, we have created and trained a novel deep learning architecture that considers inter-class relationships for addressing the multi-label classification of 12-lead ECGs. Methods: Raw ECGs from the Physionet database (6,877 12-ECG re-cordings, 6 s - 60 s duration, sampled at 500 Hz) were used as inputs. Convolutional neural networks (CNNs) were firstly employed to extract local features. In order to make CNNs tractable for optimization, a Resid-ual Neural Network was adopted to add a shortcut connection that skips two convolutional layers. A bi-directional LSTM layer was subsequently applied to extract temporal features from the time series data. Finally, multi-label learning was conducted considering high-order relationship among labels during the training of the proposed model. A class-weighted F-score (F_2 Score) and a generalization of the Jaccard measure (G_2 Score) were used as measures of performance. Results: The proposed model achieved a F_2 Score of 0.823 and a G_2 Score of 0.613 on training dataset using 10-fold cross validation. With the same model, we achieved a Geometric Mean of 0.702 with a F_2 score of 0.817 and a G_2 score of 0.603 on the test data set of Physio-net/CinC Challenge 2020, scored in ‘Unofficial Phase’. Conclusion: We propose a novel deep learning architecture that can potentially help in automated clinical diagnosis of multiple cardiac dis-orders.