Appendix

Algorithm Principles

Optimization Formula

The system uses AdaptiveOptimizerAnalytical optimizer with Huber loss + hand-written analytical gradients + NLopt SLSQP:

\begin{aligned} \min_{q} \quad & L(q) + \lambda \left\lVert q - q_{\mathrm{prev}} \right\rVert^2 \\ \text{s.t.} \quad & q_{\min} \le q \le q_{\max} \end{aligned}

Where $q_{\mathrm{prev}}$ is the joint angles from the previous frame, and $\lambda$ is norm_delta (velocity regularization weight).

Adaptive Blending

The algorithm automatically switches optimization strategies based on finger pinch state:

L = \sum_i \left[ \alpha_i L_{\mathrm{tip\_dir\_vec}, i} + (1 - \alpha_i) L_{\mathrm{full\_hand}, i} \right]

\alpha_i = \operatorname{clip}\!\left(\frac{d_2 - d_i}{d_2 - d_1},\; 0,\; 0.7\right)

$d_i$ : Distance from the thumb to the tip of finger $i$
$d_1$ , $d_2$ : Pinch thresholds (default: 2.0 cm, 4.0 cm)

TipDirVec Mode: Optimizes fingertip position and direction, suitable for fine pinch actions

FullHandVec Mode: Optimizes full hand pose, suitable for open and grasping actions

Troubleshooting

Q: pinocchio installation fails?

If installing from a PyPI mirror fails, use the official source. Note: The package name on PyPI is pin (not pinocchio):

pip install pin==3.8.0 -i https://pypi.org/simple

Q: MuJoCo window not displaying on macOS?

Use mjpython instead of python to run simulation scripts on macOS:

mjpython teleop_sim.py --play data/avp1.pkl --hand left

Q: Video mode says mediapipe or opencv-python is missing?

Install the video dependencies:

pip install -e ".[video]"

This will install the mediapipe and opencv-python packages required for video mode.

Q: RealSense mode cannot start or reports that pyrealsense2 is missing?

Install the RealSense extra dependencies:

pip install -e ".[realsense]"

Also make sure an Intel RealSense device is connected and not occupied by another application.

Q: RealSense reports device is busy?

This usually means the camera is already being used by another tool or process. Close the other application and try again.

Q: What should I do if --show-video causes lag?

--show-video is intended for debugging and adds extra rendering overhead. Disable it when you care more about runtime performance than visual inspection.

GitHub Repository: wuji-technology/wuji-retargeting
Wuji Hand SDK: wuji-technology/wujihandpy
Vision Pro Streaming: VisionProTeleop
Technical Support: support@wuji.tech

Algorithm Principles

Troubleshooting

Related Resources

On this page