Accurate Remote PPG Waveform Recovery from Video Using a Multi-Task Learning Temporal Model

Accurate Remote PPG Waveform Recovery from Video Using a Multi-Task Learning Temporal Model

Presenter(s)

Fangshi Zhou

Comments

1:00-1:20, LTC Studio

Description

Remote photoplethysmography (rPPG) is a contactless method for extracting heart-related signals from video. While promising for cardiac health monitoring, most existing methods only estimate heart rate and fail to reconstruct detailed PPG waveforms needed for biometric analysis. To address this, we developed a multi-loss model designed to restore rPPG waveforms with high accuracy. Our approach uses multi-task learning, incorporating losses for overall waveform reconstruction (MSE), peak detection, trough detection, and signal-to-noise ratio (SNR) to improve signal quality. We also integrate Temporal Shift Modules (TSM) and Long Short-Term Memory (LSTM) networks to capture both short-term and long-term signal dependencies, making the model more robust to noisy or cross-dataset data. Experiments on the PURE and UBFC-rPPG datasets show that our model outperforms DeepPhys and TS-CAN by reducing systolic peak and foot/onset estimation errors by over 30%, improving the detection of diastolic peaks and dicrotic notches, and achieving a DTW distance of 6.54, demonstrating superior waveform reconstruction.

Publication Date

4-23-2025

Project Designation

Graduate Research

Primary Advisor

Zhongmei Yao, Tianming Zhao

Primary Advisor's Department

Computer Science

Keywords

Stander Symposium, College of Arts and Sciences

Institutional Learning Goals

Scholarship; Vocation; Practical Wisdom

Accurate Remote PPG Waveform Recovery from Video Using a Multi-Task Learning Temporal Model

Share

COinS