Document Type
Conference Paper
Publication Date
7-2011
Publication Source
IEEE International Conference on Multimedia and Expo
Abstract
Video self modeling (VSM) is a behavioral intervention technique in which a learner models a target behavior by watching a video of him or herself. Its effectiveness in rehabilitation and education has been repeatedly demonstrated but technical challenges remain in creating video contents that depict previously unseen behaviors. In this paper, we propose a novel system that re-renders new talking-head sequences suitable to be used for VSM treatment of patients with voice disorder. After the raw footage is captured, a new speech track is either synthesized using text-to-speech or selected based on voice similarity from a database of clean speeches. Voice conversion is then applied to match the new speech to the original voice. Time markers extracted from the original and new speech track are used to re-sample the video track for lip synchronization. We use an adaptive re-sampling strategy to minimize motion jitter, and apply bilinear and optical-flow based interpolation to ensure the image quality. Both objective measurements and subjective evaluations demonstrate the effectiveness of the proposed techniques.
Inclusive pages
1-6
ISBN/ISSN
1945-7871
Copyright
Copyright © 2011, IEEE
Publisher
IEEE
Place of Publication
Barcelona, Spain
Peer Reviewed
yes
Sponsoring Agency
National Science Foundation
eCommons Citation
Shen, Ju; Raghunathan, Anusha; Cheung, Sen-ching S.; and Patel, Ravi R., "Automatic Content Generation for Video Self Modeling" (2011). Computer Science Faculty Publications. 56.
https://ecommons.udayton.edu/cps_fac_pub/56
Included in
Databases and Information Systems Commons, Graphics and Human Computer Interfaces Commons, Information Security Commons, Numerical Analysis and Scientific Computing Commons, OS and Networks Commons, Other Computer Sciences Commons, Programming Languages and Compilers Commons, Software Engineering Commons, Systems Architecture Commons, Theory and Algorithms Commons
Comments
Funded by National Science Foundation Award No. 1018241. Archived in compliance with federal policy. Permission documentation is on file.