This paper was accepted on the Basis Fashions for the Mind and Physique Workshop at NeurIPS 2025.
Hand gesture classification utilizing high-quality structured information akin to movies, pictures, and hand skeletons is a well-explored downside in pc imaginative and prescient. Leveraging low-power, cost-effective biosignals, e.g. floor electromyography (sEMG), permits for steady gesture prediction on wearables. On this paper, we show that studying representations from weak-modality information which can be aligned with these from structured, high-quality information can enhance illustration high quality and allows zero-shot classification. Particularly, we suggest a Contrastive Pose-EMG Pre-training (CPEP) framework to align EMG and pose representations, the place we be taught an EMG encoder that produces high-quality and pose-informative representations. We assess the gesture classification efficiency of our mannequin via linear probing and zero-shot setups. Our mannequin outperforms emg2pose benchmark fashions by as much as 21% on in-distribution gesture classification and 72% on unseen (out-of-distribution) gesture classification.
- † College of Southern California
- ** Work completed whereas at Apple
