Mainly used for clinical monitoring, electrocardiogram (ECG) signal is a non-invasive representation of the cardiac electrical activity recorded from the body surface. Automatic ECG analysis is very promising for early diagnosis, understanding, and possibly, prediction and prevention of cardiovascular diseases (CVD). However, developing an automatic ECG machine learning analysis solution with medical level performance depends on the availability of labeled data to train high-performance models. In this paper to achieve spatial and temporal information extraction, a combination of deep neural networks (DNN) is developed to improve the model performance for 12-lead ECG recording. This multiclass classification model integrates convolutional neural networks (CNN) and recurrent cells and accepts flexible input data length. All the ECG recordings are first transformed into a 3-D matrix and then fed as input data to the classifier. This optimized model is developed based on data augmentation, noise/outlier removal, normalization, regularization, and cross-validation (CV) concepts. To train and test the model for arrhythmia detection, PhysioNet/Computing in Cardiology Challenge 2020 dataset is used which includes nine classes (one normal sinus rhythm and eight abnormal rhythms) in 12 lead ECG recordings. To precisely validate the classifier, 30 % of the dataset is used as a hidden hold-out set. This model shows the performance of an average class-weighted F2 score of 0.768 % and an average class-weighted G2 score of 0.545 % and, reliably discriminate between the classes based on ECG signal morphology. This high-performance model would facilitate doctors and research groups to accurately detect arrhythmia during monitoring and telehealth.