Hand Retargeting

Hand retargeting maps human hand keypoints (21 MediaPipe-format landmarks) to a 20-value joint-angle vector for Wuji Hand 2 and Wuji Hand — ready to send to the device. The algorithm is built into the SDK: install with pip install wuji-sdk numpy — numpy is used for keypoint/qpos arrays, and nothing else is required. The C SDK provides an equivalent API — see C SDK Reference — Retargeting.

Quick start

RetargetSession.for_hand(model, side) binds a session to a hand model with the built-in IK config. session.step(keypoints) accepts a (21, 3) float32 array and returns a (20,) float32 vector of joint angles — send the result to the hand as-is:

import numpy as np
from wuji_sdk import Handedness, HandModel, RetargetSession

session = RetargetSession.for_hand(HandModel.WujiHand2, side=Handedness.Right)

# keypoints: 21 MediaPipe landmarks in meters
keypoints = np.zeros((21, 3), dtype=np.float32)
# ... fill keypoints from camera / MediaPipe / VR

joint_angles = session.step(keypoints)   # numpy.ndarray (20,)

Call session.reset() to clear warm-start and low-pass-filter state when switching data sources (re-align timing, drop stale frames).

API reference

Create a session

Class method RetargetSession.for_hand(hand_model, side) builds a session bound to a hand model with the built-in config.

Retarget a frame

session.step(keypoints) runs retargeting on a single keypoint frame.

Returns np.ndarray of shape (20,) float32. Send to the hand directly.

Reset state

session.reset() clears warm-start and low-pass-filter state. Call when switching data sources or after a long pause before resuming.

Input format

step input follows the MediaPipe Hands landmark convention — 21 fixed-order points (wrist + 5 fingers × 4 joints), coordinates in meters. Any source producing this format works: camera + MediaPipe, the Wuji Glove hand_skeleton subscription, VR hand tracking, and so on.

Live teleoperation example

RetargetSession is a pure retargeting interface — teleoperation (driving a hand from a live keypoint stream) is application-level code built on top. For full examples, see examples/python/retargeting/ and examples/c/retargeting/ in the wuji-sdk repo (Wuji Glove input → retarget → drive Wuji Hand 2 / Wuji Hand).

Drain the keypoint stream to the latest frame before feeding step — taking the oldest queued frame causes latency to accumulate.

Choose the SDK User for Teleop

The hand model depends on the currently selected SDK user. Switch before or after connecting — either works. An already-connected glove picks up the new user's model on its own, with no reconnect:

• Default user — always runs on the SDK's built-in default hand URDF. Calibrating under the default user has no effect.
• Named user — runs on that user's calibrated hand model. When that user hasn't calibrated yet, it falls back to the built-in default URDF too.

The Python and C teleop examples (1.teleop_real.py and 1_teleop_real.c) behave the same: both list every SDK user before connecting the glove and prompt for a choice. Press Enter to keep the current user, and the example restores the previously selected user on exit. The menu doesn't show whether a user is calibrated — no public API resolves a user's hand model before connecting.

The built-in default URDF is currently more reliable than a per-user calibrated hand model, so start with the default user. If the built-in URDF doesn't track your hand well, create a named SDK user, switch to it, then calibrate the glove under that user. After calibration, pick that user from the example menu. See SDK User Management and Calibration for details.

Platform and limits

• Linux x86_64 and aarch64 only. Precompiled wheels are not distributed for macOS, Windows, or other architectures.
• The algorithm is built into the SDK — no extra download. The only runtime dependency is numpy (for keypoint/qpos arrays).