Document Type

Conference Paper

Publication Date


Publication Source

Proceedings of the 2016 ACM on Multimedia Conference


In this work, we present a practical system which uses mobile devices for interactive manuals. In particular, there are two modes provided in the system, namely, expert/trainer and trainee modes. Given the expert/trainer editor, experts design the step-by-step interactive manuals. For each step, the experts capture the images by using phones/tablets and provide visual instructions such as interest regions, text, and action animations. In the trainee mode, the system utilizes the existing object detection and tracking algorithms to identify the step scene and retrieve the respective instruction to be displayed on the mobile device. The trainee then follows the displayed instruction. Once each step is performed, the trainee commands the devices to proceed to the next step.

Inclusive pages




Document Version



The document available for download is the authors' accepted manuscript, provided in compliance with the publisher's policy on self-archiving. Permission documentation is on file.


Association for Computing Machinery

Place of Publication

Amsterdam, The Netherlands