Presenter(s)

Vamsi Charan Adari

Download

Download Project (254 KB)

Description

The purpose of this project is to address the problem of interaction between the user and eye-wear devices. In particular, our framework recognizes audio instructions, hand gestures, and human gazes and translate them into commands. First, the audio input from the user is recognized and converted to text by using speech to text recognition system. Second, hand gestures are recognized based on the movements of finger tips in multiple frames. Third, the human gaze is computed as the average of the two distances captured from both eyes. To demonstrate these commands, we developed two applications to demonstrate the effectiveness of the new way of interaction. The first application projects a 3D model explorer that can be manipulated through the commands that we programmed to expand, rotate, or reset. The second application projects a solar system that demonstrates the ability for these commands to interact with multiple virtual objects. This advancement in eye-wear device interaction will facilitate the usability of eye-wear devices with virtual objects moving forward.

Publication Date

4-24-2019

Project Designation

Independent Research

Primary Advisor

Van Tam Nguyen

Primary Advisor's Department

Computer Science

Keywords

Stander Symposium project

Human-eyewear device interaction

Share

COinS