Accurate Remote PPG Waveform Recovery from Video Using a Multi-Task Learning Temporal Model

Accurate Remote PPG Waveform Recovery from Video Using a Multi-Task Learning Temporal Model

    Presenter(s)

    Fangshi Zhou

    Comments

    1:00-1:20, LTC Studio

    Description

    Remote photoplethysmography (rPPG) is a contactless method for extracting heart-related signals from video. While promising for cardiac health monitoring, most existing methods only estimate heart rate and fail to reconstruct detailed PPG waveforms needed for biometric analysis. To address this, we developed a multi-loss model designed to restore rPPG waveforms with high accuracy. Our approach uses multi-task learning, incorporating losses for overall waveform reconstruction (MSE), peak detection, trough detection, and signal-to-noise ratio (SNR) to improve signal quality. We also integrate Temporal Shift Modules (TSM) and Long Short-Term Memory (LSTM) networks to capture both short-term and long-term signal dependencies, making the model more robust to noisy or cross-dataset data. Experiments on the PURE and UBFC-rPPG datasets show that our model outperforms DeepPhys and TS-CAN by reducing systolic peak and foot/onset estimation errors by over 30%, improving the detection of diastolic peaks and dicrotic notches, and achieving a DTW distance of 6.54, demonstrating superior waveform reconstruction.

    Publication Date

    4-23-2025

    Project Designation

    Graduate Research

    Primary Advisor

    Zhongmei Yao, Tianming Zhao

    Primary Advisor's Department

    Computer Science

    Keywords

    Stander Symposium, College of Arts and Sciences

    Institutional Learning Goals

    Scholarship; Vocation; Practical Wisdom

    Accurate Remote PPG Waveform Recovery from Video Using a Multi-Task Learning Temporal Model

    Share

    COinS