Introduction: The cardiovascular disease (CVD) is one of the major cause of mortality worldwide. Auscultation of heart sounds or phonocardiograms (PCGs) analysis, which is an efficient and non-invasive way, has been shown to be promising and played an important role in preliminary CVD diagnosis. This study investigates an objective and automatic PCG classification algorithm based on deep learning method. Methods: Our proposed method is mainly comprised three steps: pre-processing, PCG patches classification using a novel 1-D deep convolutional neural network (CNN), and final predicting of PCG recordings based on the patch-level results. In order to enhance the information flow of the CNN, a block-stacked style architecture with clique blocks is employed, and in each clique block a bidirectional connection structure is utilized. Using the stacked blocks, the proposed CNN achieves both spatial and channel attention, which leads a superior classification performance. Moreover, a novel separable convolution with inverted bottleneck is introduced to decouple the spatial and channel-wise dependency of the feature efficiently. Results: Experiments on the dataset of PhysioNet/CinC 2016, using the ten-fold cross-validation, reveal a superior classification performance. Our method achieves promising classification accuracy and specificity (93.21% and 95.12%, respectively). And it obtains superior sensitivity of 85.81% and overall score 90.46% with the extremely low parameter consuming, which outperforms state-of-the-art CNN-based PCG classification methods. Conclusion: We developed a novel 1-D CNN architecture for PCG classification. This architecture is designed to efficiently reuse the feature maps with high parameter efficiency, and without any complex pre-processing or post-processing steps. The promising performance of the proposed method makes us hopeful that with further minimization of the memories and trainable parameters consumption, the CNN may be suitable for embedded or mobile applications.