Wuji Hand
Wuji Glove
Wuji Studio
Wuji SDKQuick Start
Before running examples, make sure:
- You have completed Installation. If you encounter missing model files, check if submodules are initialized:
git submodule update --init --recursive - When using Vision Pro, install Tracking Streamer on your Apple Vision Pro (available from App Store or GitHub)
Simulation Demo
cd example
# Vision Pro real-time teleoperation (recommended), replace <vision-pro-ip> with actual IP (found in Tracking Streamer)
python teleop_sim.py --input visionpro --ip <vision-pro-ip> --hand left
# Playback pre-recorded data (when Vision Pro is unavailable)
python teleop_sim.py --play data/avp1.pkl --hand left
# Use an MP4 video as input
python teleop_sim.py --video <your-video.mp4> --hand right
python teleop_sim.py --video <your-video.mp4> --hand right --show-video
# Use Intel RealSense as live input
python teleop_sim.py --realsense --hand right
python teleop_sim.py --realsense --hand right --show-videoExpected results:
With Vision Pro real-time teleoperation, the simulation interface will display a simulated hand following the user's hand movements. With pre-recorded data playback, the result will look similar to the following demo:
When using MP4 or RealSense input, you can add --show-video to inspect the original image together with the MediaPipe landmark overlay for debugging.
Real Hardware Control
Run Examples
cd example
# Vision Pro real-time input (recommended, requires Apple Vision Pro configuration), replace <vision-pro-ip> with actual IP (found in Tracking Streamer)
python teleop_real.py --input visionpro --ip <vision-pro-ip> --hand right
# Playback pre-recorded data (when Vision Pro is unavailable)
python teleop_real.py --play data/avp1.pkl --hand right
# Use an MP4 video as input
python teleop_real.py --video <your-video.mp4> --hand right
python teleop_real.py --video <your-video.mp4> --hand right --show-video
# Use Intel RealSense as live input
python teleop_real.py --realsense --hand right
python teleop_real.py --realsense --hand right --show-videoLinux USB Permissions: Wuji Hand communicates via USB serial. Linux restricts serial device access for regular users by default. We recommend adding your user to the dialout group: sudo usermod -aG dialout $USER (requires logout/login). sudo chmod a+rw /dev/ttyUSB0 is only for temporary troubleshooting and resets after reboot.
Expected results:
With Vision Pro real-time teleoperation, the Wuji Hand will follow the user's hand movements. With pre-recorded data playback, the Wuji Hand motion trajectory will be similar to the expected results video in the Simulation Demo. MP4 video input is useful for offline validation and demos, while RealSense is useful for live input testing when Vision Pro is unavailable.
Record Input Data
When using Vision Pro, you can simultaneously record input data for later playback and debugging:
cd example
python teleop_real.py --input visionpro --recordWhen you use --video or --realsense, the example scripts automatically switch to config/adaptive_analytical_video.yaml. You can still override this behavior with --config.
Hand Model: Wuji Hand (Default) and Wuji Hand 2
The commands above drive the Wuji Hand (USB) by default — no extra flag needed. To drive Wuji Hand 2 (networked), pass --hand-model wh120 with a Wuji Hand 2 config. The Wuji Hand 2 config points the optimizer at the Wuji Hand 2 model via optimizer.urdf_path / mjcf_path / link_naming, so no code change is needed:
cd example
# Wuji Glove input -> Wuji Hand 2 (right hand)
python teleop_real.py --input wuji_glove --hand right --glove-sn <YOUR_SN> \
--config config/adaptive_analytical_wuji_glove_wh120_right.yaml
# Multiple Wuji Hand 2 devices online — select one by address:
python teleop_real.py --input wuji_glove --hand right --glove-sn <YOUR_SN> \
--wh120-ip <hand-ip>:50001 \
--config config/adaptive_analytical_wuji_glove_wh120_right.yamlWithout --hand-model, the model is inferred from the config: a filename containing wh120 or an optimizer.urdf_path that points to a Wuji Hand 2 model selects Wuji Hand 2, otherwise Wuji Hand.
When --wh120-ip is omitted, the script scans the network for Wuji Hand 2 devices. With one online it auto-connects. With several online it matches --hand left/right against each device's reported handedness.
Wuji Hand 2 and Wuji Hand firmware must match the installed wuji_sdk. Mismatched firmware fails fast at connect with a clear error.
Command Reference
teleop_sim.py (Simulation)
| Option | Default | Description |
|---|---|---|
--config | config/adaptive_analytical_avp.yaml | YAML configuration file |
--hand | left | Hand side (left/right) |
--input | - | Input type (visionpro/mediapipe_replay/video/realsense) |
--play FILE | - | Play recorded data (shortcut for --input mediapipe_replay) |
--video FILE | - | Use an MP4 file as input (shortcut for --input video) |
--realsense | - | Use Intel RealSense as input (shortcut for --input realsense) |
--show-video | - | Show the image with MediaPipe overlay, only used in video/realsense mode |
--ip | 192.168.50.127 | Vision Pro IP address |
--speed | 1.0 | Playback speed |
--no-loop | - | Disable loop playback |
--record | - | Record input data |
--output FILE | - | Recording output path |
teleop_real.py (Real Hardware)
| Option | Default | Description |
|---|---|---|
--config | config/adaptive_analytical_avp.yaml | YAML configuration file |
--hand | right | Hand side (left/right) |
--input | - | Input type (visionpro/mediapipe_replay/video/realsense) |
--play FILE | - | Play recorded data (shortcut for --input mediapipe_replay) |
--video FILE | - | Use an MP4 file as input (shortcut for --input video) |
--realsense | - | Use Intel RealSense as input (shortcut for --input realsense) |
--show-video | - | Show the image with MediaPipe overlay, only used in video/realsense mode |
--ip | 192.168.50.127 | Vision Pro IP address |
--speed | 1.0 | Playback speed |
--no-loop | - | Disable loop playback |
--record | - | Record input data |
--output FILE | - | Recording output path |
--hand-model | inferred from config | Hand model: wh110 (Wuji Hand, USB) or wh120 (Wuji Hand 2, networked) |
--hand-serial | - | Wuji Hand serial number |
--wh120-ip | auto-scan | Wuji Hand 2 network address, e.g. 192.168.1.111:50001 |
--kp | 3.0 | Wuji Hand 2 MIT controller kp |
--kd | 0.1 | Wuji Hand 2 MIT controller kd |
--current-limit | 1.5 | Wuji Hand 2 per-joint system current limit, amps |