Hand Retargeting

Hand retargeting maps human hand keypoints (21 MediaPipe-format landmarks) to a 20-value joint-angle vector for Wuji Hand 2 and Wuji Hand — ready to send to the device. The algorithm is built into the SDK: install with pip install wuji-sdk numpy — numpy is used for keypoint/qpos arrays, and nothing else is required. The C SDK provides an equivalent API — see C SDK Reference — Retargeting.

Quick start

RetargetSession.for_hand(model, side) binds a session to a hand model with the built-in IK config. session.step(keypoints) accepts a (21, 3) float32 array and returns a (20,) float32 vector of joint angles — send the result to the hand as-is:

import numpy as np
from wuji_sdk import Handedness, HandModel, RetargetSession

session = RetargetSession.for_hand(HandModel.WujiHand2, side=Handedness.Right)

# keypoints: 21 MediaPipe landmarks in meters
keypoints = np.zeros((21, 3), dtype=np.float32)
# ... fill keypoints from camera / MediaPipe / VR

joint_angles = session.step(keypoints)   # numpy.ndarray (20,)

Call session.reset() to clear warm-start and low-pass-filter state when switching data sources (re-align timing, drop stale frames).

API reference

Create a session

Class method RetargetSession.for_hand(hand_model, side) builds a session bound to a hand model with the built-in config.

Retarget a frame

session.step(keypoints) runs retargeting on a single keypoint frame.

Returns np.ndarray of shape (20,) float32. Send to the hand directly.

Reset state

session.reset() clears warm-start and low-pass-filter state. Call when switching data sources or after a long pause before resuming.

Input format

step input follows the MediaPipe Hands landmark convention — 21 fixed-order points (wrist + 5 fingers × 4 joints), coordinates in meters. Any source producing this format works: camera + MediaPipe, the Wuji Glove hand_skeleton subscription, VR hand tracking, and so on.

Live teleoperation example

RetargetSession is a pure retargeting interface — teleoperation (driving a hand from a live keypoint stream) is application-level code built on top. For full examples, see the retargeting/ subdirectory under the wuji-sdk repo's examples/python/ (Wuji Glove input → retarget → drive Wuji Hand 2 / Wuji Hand).

Drain the keypoint stream to the latest frame before feeding step — taking the oldest queued frame causes latency to accumulate.

Run teleop under the default SDK user

Run teleop under the default SDK user: the default user runs the glove on the SDK's built-in default hand URDF, which is currently more reliable than per-user IK calibration. The 1.teleop_real.py example calls manager.switch_to_default_user() before connecting the glove and restores the previously selected user on exit.

If the built-in URDF doesn't track your hand well, create a named SDK user, switch to it, then calibrate the glove under that user — calibrating under the default user has no effect. See SDK User Management and Calibration for details.

Platform and limits

• Linux x86_64 and aarch64 only. Precompiled wheels are not distributed for macOS, Windows, or other architectures.
• The algorithm is built into the SDK — no extra download. The only runtime dependency is numpy (for keypoint/qpos arrays).