Identification of Most Suitable Regions in Imaging Photoplethysmography by Reference Signal Information

Matthieu Scherpf, Hannes Ernst, Hagen Malberg, Martin Schmidt
TU Dresden, Institute of Biomedical Engineering, Dresden, Germany


Continuous monitoring of the cardiovascular state can be of utmost importance for specific diseases. Biosignals represent one option to provide valuable information for the characterization of the cardiovascular state. One such signal is the blood volume pulse, which conventionally is recorded by photoplethysmography (PPG) via a finger clip. A novel approach, imaging photoplethysmography (iPPG), captures the blood volume pulse remotely with a camera. Next to the advantages of the contactless biosignal acquisition, iPPG exhibits low signal intensity, i.e. low signal-to-noise ratio (SNR), which makes it susceptible to artifacts from motion and discontinuous illumination. This study proposes a novel approach taking advantage of reference signal information, e.g. from the PPG, to identify most suitable regions for iPPG extraction by applying a supervised machine learning technique.

In addition to datasets acquired during clinical trials within our institution, we included the Univ. Bourgogne Franche-Comté Remote PhotoPlethysmoGraphy (UBFC-RPPG) database and the Binghamton-Pittsburgh-RPI Multimodal Spontaneous Emotion Database (BP4D+) in the analysis. To take advantage of reference signal information we developed a novel approach based on a 3-dimensional convolutional neural network (3DCNN). This network was trained to reconstruct the iPPG from video segments of 2 seconds. The finger PPG was used as reference signal for each segment. We extracted segment-wise activation heatmaps from the 3DCNN using a method similar to the gradient-based class activation map visualization (Grad-CAM). Assuming pixel related with high activation have a relatively high SNR, we calculated a weighting mask for each segment according to the heatmap. Subsequently, the calculated mask contains the most suitable regions to reconstruct the iPPG.

Preliminary evaluations indicate our approach to be beneficial for the iPPG extraction compared to two chosen state-of-the-art algorithms (POS and CHROM). Further investigations will focus on improving heart rate estimation from iPPG by our approach.