Image credit: Main-StructureAbstract
With the development of generative models, abused Deepfakes have aroused public concerns. As a defense mechanism, face forgery detection methods have been intensively studied. Remote photoplethysmography (rPPG) technology extract heartbeat signal from recorded videos by examining the subtle changes in skin color caused by cardiac activity. Since the face forgery process inevitably disrupts the periodic changes in facial color, rPPG signal proves to be a powerful biological indicator for Deepfake detection. Motivated by the key observation that rPPG signals produce unique rhythmic patterns in terms of different manipulation methods, we regard Deepfake detection also as a source detection task. The Multi-scale Spatial–Temporal PPG map is adopted to further exploit heartbeat signal from multiple facial regions. Moreover, to capture both spatial and temporal inconsistencies, we propose a two-stage network consisting of a Mask-Guided Local Attention module (MLA) to capture unique local patterns of PPG maps, and a Temporal Transformer to interact features of adjacent PPG maps in long distance. Abundant experiments on FaceForensics + + and Celeb-DF datasets prove the superiority of our method over all other rPPG-based approaches. Visualization also demonstrates the effectiveness of the proposed method.
Jiahui Wu 吴嘉辉
Master. The most handsome guy among the master’s students.
A master student of this laboratory, research interests include Artificial Intelligence, Deepfake and System Deployment.
Yu Zhu 朱煜
Professor. Experts in artificial intelligence and computer vision. Lab leader.
Leader of this laboratory, research interests include Artificial Intelligence, Computer Vision, Industrial controls, Digital Image and Video Processing, Machine learning, Deep Learning and Applications.
Xiaoben Jiang 蒋晓奔
PhD. One apple a day keep the doctor away.
A doctor student of this laboratory, research interests include Medical image processing, AIGC, and Image denoising.
Yatong Liu 刘雅童
PhD.
A doctoral student of this laboratory, research interests include Medical Image Processing, Deep Learning Algorithm and Multi-source Medical Image Intelligent Analysis.