API Reference

Basic Usage

from wuji_retargeting import Retargeter

# Create retargeter from configuration file
retargeter = Retargeter.from_yaml("config/adaptive_analytical_avp.yaml", hand_side="right")

# Retarget: convert 21 keypoints to 20 joint angles
qpos = retargeter.retarget(raw_keypoints)  # (21, 3) -> (20,)

Input Format

Input data is MediaPipe format hand keypoints with shape (21, 3), containing 3D coordinates for 21 keypoints (unit: meters). Regardless of whether the source is Vision Pro, replay data, MP4 video, or Intel RealSense, the input pipeline ultimately normalizes the data into this format.

Keypoint index mapping:

Output Format

Output is a length-20 joint angle array (radians), corresponding to Wuji Hand's 20 joints:

Configuration File

optimizer:
  type: "AdaptiveOptimizerAnalytical"

retarget:
  # Huber loss thresholds
  huber_delta: 2.0             # Position Huber threshold (cm)
  huber_delta_dir: 0.5         # Direction Huber threshold

  # Loss weights
  w_pos: 1.0           # Fingertip position weight
  w_dir: 10.0          # Fingertip direction weight
  w_full_hand: 1.0     # Full hand weight

  # Regularization
  norm_delta: 0.04     # Velocity regularization weight

  # Scaling
  scaling: 1.0         # Global scaling

  # Per-finger segment scaling [PIP, DIP, TIP]
  segment_scaling:
    thumb:  [1.0, 1.0, 1.0]
    index:  [1.0, 1.03, 1.05]
    middle: [1.0, 1.0, 1.0]
    ring:   [1.0, 1.0, 1.0]
    pinky:  [1.05, 1.15, 1.15]

  # Pinch thresholds (cm)
  pinch_thresholds:
    index:  { d1: 2.0, d2: 4.0 }
    middle: { d1: 2.0, d2: 4.0 }
    ring:   { d1: 2.0, d2: 4.0 }
    pinky:  { d1: 2.0, d2: 4.0 }

  # Low-pass filter (0~1, lower is smoother)
  lp_alpha: 0.2

The example above reflects the default Vision Pro configuration style. For MediaPipe vision sources such as video and RealSense, use the dedicated adaptive_analytical_video.yaml configuration.

Configuration by Input Mode

Driving a Custom Dexterous Hand

Three keys under the optimizer block point the same retargeting algorithm at different hand models without code changes:

link_naming supports a prefix (for example l_ for the Wuji Hand 2 left hand) and the {finger} placeholder. Wuji Hand 2 uses anatomical names (r_wrist, r_index_finger_pip, ...), configured as:

optimizer:
  link_naming:
    prefix: "r_"
    palm:    "wrist"
    fingers: [thumb, index_finger, middle_finger, ring_finger, pinky]
    pip:     "{finger}_middle"
    dip:     "{finger}_distal"
    tip:     "{finger}_tip"
    link1:   "{finger}_proximal_abd"

When the URDF and MJCF declare joints in different orders (for example Wuji Hand 2 puts index first in the URDF), output qpos is automatically reordered by joint name to the MJCF or device order, so finger angles do not land on the wrong joint. If optimizer.mjcf_path is set but joint names cannot be aligned, the sim and real-hardware entry points fail loudly at startup instead of silently driving the wrong joints.

For video and realsense modes, the example scripts automatically switch to config/adaptive_analytical_video.yaml. This configuration typically:

• uses stronger smoothing and regularization to reduce jitter from vision input
• adjusts segment_scaling to compensate for finger-length distortion in monocular landmarks
• widens pinch_thresholds for better stability in noisy conditions
• adds mediapipe_rotation to compensate for systematic orientation offsets

video_input Parameters

adaptive_analytical_video.yaml also includes a video_input section for preprocessing video and RealSense input:

Parameter Descriptions

Hand Size Adaptation

If retargeting results are poor, adjust the following parameters:

Global Scaling: Adjust scaling parameter to match palm size

scaling: 1.1  # Increase for larger hands
scaling: 0.9  # Decrease for smaller hands

Finger Length: Adjust each finger segment's length ratio via segment_scaling

segment_scaling:
  index: [1.0, 1.05, 1.1]  # Index finger segment scaling: [PIP, DIP, TIP]