Presenter(s)

Skyler Barclay

Comments

3:00-4:15, Kennedy Union Ballroom

Download

Download Project (1.3 MB)

Description

Virtual reality (VR) has become popular in research due to its ability to present clear and customizable tasks. Infrared (IR) motion capture allows for the collection of full kinematic data, however the cost may not be feasible for most clinics. Vive trackers allow for integrated wearables and VR therapy at a lower cost. Our current VR motion capture system requires segment definitions from the IR motion capture system in order to build an accurate skeletal model. The aim for this study is to output kinematics from the raw VR motion capture data using a Bidirectional Long-Short-Term-Memory (BLSTM) algorithm trained with joint kinematics calculated from the IR motion capture system.IR and VR motion capture of the upper extremity was collected simultaneously, in Nexus and Brekel respectively, while participants played customized levels in Beat Saber. The participants were instructed to slice through the virtual blocks with a saber in the directed position, orientation, and correct arm. To determine if shoulder, elbow, and wrist joint kinematics can be predicted using raw VR motion capture data a person specific BLSTM algorithm (n=3) was trained in Python on IR joint kinematics from the first visit (lookback = 100) and tested on the participants’ second visit data.The BLSTM results found an average error of ±10° for the joints. Collecting known joint angle poses, filtering the input data, and fine tuning the algorithm hyperparameters should decrease the error further. This means one baseline IR capture could make it possible for clinics to predict upper extremity joint kinematics of a patient during these customizable Beat Saber therapy games, and possibly other motions, with only the VR equipment, Brekel, and Python. Additionally, the use of VR therapy allows for individualized and fun therapies with quantitative results to track progress.

Publication Date

4-23-2025

Project Designation

Graduate Research

Primary Advisor

Allison L. Kinney, Megan E. Reissman, Timothy Reissman

Primary Advisor's Department

Mechanical and Aerospace Engineering

Keywords

Stander Symposium, School of Engineering

Clinical Translation of Virtual Reality Motion Capture for Upper Extremity Therapy Using Machine Learning

Share

COinS