IEEE International Conference on Multimedia and Expo
Video self modeling (VSM) is a behavioral intervention technique in which a learner models a target behavior by watching a video of him or herself. Its effectiveness in rehabilitation and education has been repeatedly demonstrated but technical challenges remain in creating video contents that depict previously unseen behaviors. In this paper, we propose a novel system that re-renders new talking-head sequences suitable to be used for VSM treatment of patients with voice disorder. After the raw footage is captured, a new speech track is either synthesized using text-to-speech or selected based on voice similarity from a database of clean speeches. Voice conversion is then applied to match the new speech to the original voice. Time markers extracted from the original and new speech track are used to re-sample the video track for lip synchronization. We use an adaptive re-sampling strategy to minimize motion jitter, and apply bilinear and optical-flow based interpolation to ensure the image quality. Both objective measurements and subjective evaluations demonstrate the effectiveness of the proposed techniques.
Copyright © 2011, IEEE
Place of Publication
National Science Foundation
Shen, Ju; Raghunathan, Anusha; Cheung, Sen-ching S.; and Patel, Ravi R., "Automatic Content Generation for Video Self Modeling" (2011). Computer Science Faculty Publications. 56.
Databases and Information Systems Commons, Graphics and Human Computer Interfaces Commons, Information Security Commons, Numerical Analysis and Scientific Computing Commons, OS and Networks Commons, Other Computer Sciences Commons, Programming Languages and Compilers Commons, Software Engineering Commons, Systems Architecture Commons, Theory and Algorithms Commons