Due to the increasing life pressure in modern society, more and more people are suffering from sleep disorders. The most serious case of sleep disorders called apnea is characterized by a complete breaking block, leading to awakening and subsequent sleep disturbances. In addition to apnea, sleep arousals can also be initiative, cause by partial airway obstructions, teeth grinding, partial airway obstructions, or even snoring. However, great obstacles still exist in automatic identification of arousals. In this study, a novel method was developed to detect non-apnea sources of arousals during sleep using several physiological signals. In the algorithm, wavelet analysis (Sym8) with five decomposition levels was first applied to eliminate noises using soft fixed threshold. In the dataset provided, the duration of arousal regions are much less than the non-arousal regions. In order to address this issue, a set of segments (20000 samples in length) were extracted for model training in which arousal regions take up a much larger proportion than that in the original training set. After the preprocessing, a sequence-to-sequence deep neural networks (DNNs) which consists of a series of convolutional layers with residual connections, a long short-term memory (LSTM) layer and three fully connected layers, was trained to classify samples in the segments. Results show that the area under receiver operating characteristic curve (AUROC) and area under receiver precision recall curve (AUPRC) are 0.852 and 0.244 respectively in test dataset. In this study, an effective algorithm to detect non-apnea arousals was developed, which has great potentials in the clinical diagnosis and treatment of automatic sleep disturbance in the future.