Cardiac arrhythmia, associated with abnormal heart activity, presents as changes in electrocardiogram (ECG) signals. In this research, we propose an automatic cardiac arrhythmia (Normal sinus rhythm and 8 types of arrhythmias) classification system, using 12-lead ECG signals. Deep learning has proven effective is many signal processing problems, however, most successful ECG classification algorithms still use manually engineered features. In this study, we investigate how deep models, and more specifically 1-dimensional convolutional neural networks (1D-CNNs) and long-short term memory (LSTM) networks, used for modelling temporal dependencies of time series, can be utilized to achieve a high-performance model without requiring any pre-processing or feature engineering. Our proposed neural architecture with 49 convolution layers, 16 skip connections, and one bi-directional LSTM layer achieved the F2-score of 75.9% and G2-score of 58.6%, using 5-fold cross-validation on the training dataset of PhysioNet challenge 2020. The F2-score and G2-score of this model using the hidden dataset of the challenge are reported as 79.7% and 61%, respectively. These results confirm the efficiency of convolutional neural networks with identity mapping and LSTM units, for the task of cardiac arrhythmia classification. This opens the possibility that through more advanced network design techniques such as automatic neural architecture search (NAS) methods and defining its search space to include both convolution-based layers and LSTM layers, we may find even a more accurate deep neural model for ECG classification. This will be investigated in the official phase of the challenge and our current deep neural network will be used as the baseline network for comparison to future NAS derived models.