Ethan Matzek

Ethan Matzek

he/him

Computer Science MS Student Researcher with TARS Lab

Publications

You can also find my articles on my Google Scholar profile.

Conference Papers

VRmonic: A VR Piano Playing Form Trainer

Published in 2024 IEEE International Conference on Artificial Intelligence and eXtended and Virtual Reality (AIxVR), 1900

Improper hand form during practice and performance can lead to long-term injuries in musicians of all levels. We present a demonstration of VRmonic, a novel immersive VR-based piano trainer for learning correct hand form for piano by overlaying the playing form of an expert oracle. VRmonic consists of a library of 48 scales comprising of the major, and natural, harmonic, and melodic minor scales for each of the 12 notes. We obtain correct hand form by using an automatic approach to detect hand keypoints from recordings of an expert pianist collected using an overhead RGB-D mounted camera. During training, the user can use VRmonic to load in a prior recording of themselves playing a scale and compare their hand form against the expert oracle. During playback, we display virtual oracle and user hand overlap via a color change, controlled by our form tolerance threshold slider that allows the user to determine how aggressively VRmonic should check incorrect form. VRmonic enables everyday users to practice piano playing with continuous feedback while mitigating long-term physical injury.

Recommended citation: E. Matzek et al., "VRmonic: A VR Piano Playing Form Trainer," in 2024 IEEE International Conference on Artificial Intelligence and eXtended and Virtual Reality (AIxVR), Los Angeles, CA, USA, 2024, pp. 330-334, doi: 10.1109/AIxVR59861.2024.00056.
Download Paper

VitaMaze: A VR Exergame Driven using Feedback from Physiological Sensors

Published in 2025 IEEE International Conference on Artificial Intelligence and eXtended and Virtual Reality (AIxVR), 1900

Exercise is an integral part of a healthy lifestyle. However, circumstances can prevent individuals from having the motivation to exercise. We present a demonstration of VitaMaze, a novel integration of physiological sensing with virtual reality for exercise encouragement. VitaMaze is a maze-based application that uses heart rate and surface electromyography sensors attached to the players body to control avatar motion in game. We apply intelligent decision-making algorithms to decipher incoming data from the sensors and generate deterministic movement in real-time based upon the players perceived effort and strength. The player is timed throughout the game, and is incentivized to strive for faster times. Multiple layers of feedback are used to guide the players exertion, including visual and auditory feedback. Since cheating can greatly reduce the effectiveness of exergames, our application uses classification-based cheating mitigation that identifies cheating based on incoming signals and adapts the application to disincentivize cheating.

Recommended citation: E. Matzek, A. Megyeri, T. Yankee, N. K. Banerjee and S. Banerjee, "VitaMaze: A VR Exergame Driven using Feedback from Physiological Sensors," in 2025 IEEE International Conference on Artificial Intelligence and eXtended and Virtual Reality (AIxVR), Lisbon, Portugal, 2025, pp. 296-300, doi: 10.1109/AIxVR63409.2025.00057.
Download Paper