Hand gesture classification utilizing high-quality structured knowledge similar to movies, im-
ages, and hand skeletons is a well-explored drawback in pc imaginative and prescient. Alterna-
tively, leveraging low-power, cost-effective bio-signals, e.g., floor electromyo-
graphy (sEMG), permits for steady gesture prediction on wearable gadgets.
On this work, we purpose to reinforce EMG illustration high quality by aligning it with
embeddings obtained from structured, high-quality modalities that present richer
semantic steerage, in the end enabling zero-shot gesture generalization. Specif-
ically, we suggest EMBridge, a cross-modal illustration studying framework
that bridges the modality hole between EMG and pose. EMBridge learns high-
high quality EMG representations by introducing a Querying Transformer (Q-Former),
a masked pose reconstruction loss, and a community-aware comfortable contrastive learn-
ing goal that aligns the relative geometry of the embedding areas. We eval-
uate EMBridge on each in-distribution and unseen gesture classification duties and
exhibit constant efficiency positive aspects over all baselines. To one of the best of our
data, EMBridge is the primary cross-modal illustration studying framework
to realize zero-shot gesture classification from wearable EMG indicators, exhibiting
potential towards real-world gesture recognition on wearable gadgets.
- † College of Southern California
- ** Work completed whereas at Apple
