> ## Documentation Index
> Fetch the complete documentation index at: https://docs.wuji.tech/llms.txt
> Use this file to discover all available pages before exploring further.

# Retargeting Sessions: Keypoints to Joint Angles

## Overview

Feed in a set of MediaPipe-format hand keypoints, get back a joint command you can send straight to a dexterous hand. This step connects to no device at all. This step performs computation only.

The keypoints can come from anywhere. A camera, a Wuji Glove, a recorded replay, VR headset hand tracking — as long as the array has the same shape and units, everything downstream stays identical. Run this example first to confirm retargeting works and to see the input and output shapes. When you then attach real hardware, there's far less ground to search.

Matching examples: `examples/python/retargeting/0.retarget_session.py` and `examples/c/retargeting/0_retarget_session.c`. The C interface maps one to one onto Python.

For the interface list, input format, and platform limits, see [Retargeting](/en/docs/wuji-sdk/retargeting). This page covers only how to run it and what the measurements show.

## Prerequisites

### Hardware

None. This example uses synthetic keypoints, so it neither scans for nor connects to a device.

### Software

| Item | Requirement |
| - | - |
| Operating system | Ubuntu 22.04 LTS or later |
| Architecture | Retargeting ships prebuilt wheels for Linux x86\_64 and Linux aarch64 only |
| Wuji SDK | v2026.7.21 or later. That release moved the retargeting interface to the package top level. On earlier versions you import from the `wuji_sdk.retargeting` submodule, and this example's imports don't apply |
| Python | 3.10 or later, plus `numpy` to hold the keypoint and joint-angle arrays |
| C toolchain | CMake 3.16 or later, plus the extracted SDK package, which supplies `include/` and `lib/libwuji_sdk_c.so` |

<Tabs>
  <Tab title="Python">
    ```bash theme={null}
    pip install wuji-sdk numpy
    ```
  </Tab>

  <Tab title="C">
    ```bash theme={null}
    cmake -S examples/c/retargeting -B build/retargeting \
      -DWUJI_SDK_INCLUDE_DIR="${SDK}/include" \
      -DWUJI_SDK_LIB="${SDK}/lib/libwuji_sdk_c.so"
    cmake --build build/retargeting --target 0_retarget_session
    ```
  </Tab>
</Tabs>

### Before You Start

Nothing. This example is the smallest starting point on the retargeting path. Get it running, then move on to real hardware.

## Data Flow

A single frame travels the path below. The session object keeps state across frames, so create it once and reuse it in your loop.

```mermaid theme={null}
flowchart TD
    SRC["keypoint source<br/>live tracking or replay"]
    KP["21 hand keypoints<br/>unit: meters"]
    SES["RetargetSession<br/>cross-frame solve and smoothing"]
    Q["20 joint angles<br/>unit: radians"]
    DOWN["downstream<br/>publish a control command or write a replay"]

    SRC -->|"one keypoint set per frame"| KP
    KP -->|"step()"| SES
    SES -->|"solve joint angles"| Q
    Q --> DOWN
```

Keypoints are in meters, joint commands in radians. The output is always `float32` with a fixed shape that doesn't vary with the number of online joints. `float64` input is accepted and converted internally.

## Runtime Handling

| Branch point | Value | Effect |
| - | - | - |
| Hand model | `HandModel.WujiHand2` or `HandModel.WujiHand` | Selects the internal tuning profile. You never name a config file path |
| Handedness | `Handedness.Right` or `Handedness.Left` | The example defaults to the right hand. Keypoints and target hand must be the same side, and mixing them gives mirrored results |
| Tracking gap | Whether you call `reset()` | Skip the reset and the previous segment's warm-start and smoothing state carry into the new one, so the first few frames jump |
| Degenerate keypoints | All zeros, or containing NaN or Inf | `step()` raises `WujiException: retargeting: invalid keypoints: contains NaN or Inf`. Confirm the keypoint source is valid before solving, and never pass an uninitialized array as a placeholder |
| Wrong shape | Element count other than 63 | Python raises `ValueError: cannot reshape array of size N into shape (63,)`. Python works with the same length-63 layout as C internally |
| C-side error | create or step returns something other than `WUJI_STATUS_OK` | Call `wuji_last_error()` for the reason. A session that was created still needs `free`, or it leaks |

The session is stateful. Reuse one session across a continuous tracking segment, reset between segments, and never create one per frame.

## Integration Steps

Three fixed steps in a fixed order. C adds an explicit free.

<Tabs>
  <Tab title="Python">
    ```python theme={null}
    from wuji_sdk import HandModel, Handedness, RetargetSession

    # 1. Create the session once per tracking segment
    session = RetargetSession.for_hand(
        HandModel.WujiHand2, side=Handedness.Right
    )

    # 2. Solve each frame. keypoints is (21, 3) float32 in meters
    qpos = session.step(keypoints)   # (20,) float32, radians

    # 3. Clear state after a tracking gap, then start the next segment
    session.reset()
    ```
  </Tab>

  <Tab title="C">
    ```c theme={null}
    #include "wuji_sdk.h"

    /* 1. Create the session once per tracking segment */
    struct WujiRetargetSession *session = NULL;
    WujiStatus st = wuji_retarget_session_create(
        WUJI_HAND_MODEL_WUJI_HAND2, WUJI_HANDEDNESS_RIGHT, &session);
    if (st != WUJI_STATUS_OK) {
        fprintf(stderr, "create failed (%d): %s\n", (int)st, wuji_last_error());
        return EXIT_FAILURE;
    }

    /* 2. Solve each frame. keypoints is a flat array of 63, row-major xyz, in meters */
    float qpos[20];   /* radians, firmware joint order */
    st = wuji_retarget_session_step(session, keypoints, qpos);

    /* 3. Clear state after a tracking gap */
    wuji_retarget_session_reset(session);

    /* 4. Free it when you're done */
    wuji_retarget_session_free(session);
    ```
  </Tab>
</Tabs>

The only real difference between the two sides is the keypoint memory layout. Python takes a `(21, 3)` two-dimensional array, C takes a flat array of 63 laying out x, y, and z for each point in landmark order.

## Running It

<Tabs>
  <Tab title="Python">
    ```bash theme={null}
    python examples/python/retargeting/0.retarget_session.py
    ```
  </Tab>

  <Tab title="C">
    ```bash theme={null}
    ./build/retargeting/0_retarget_session
    ```
  </Tab>
</Tabs>

Both sides print the same structure: the joint command's shape and dtype, then the 20 joint angles grouped by finger.

```text theme={null}
qpos shape : (20,), dtype float32
joint command (radians, finger-major thumb/index/middle/ring/pinky):
  thumb : ['+1.291', '+0.206', '+0.345', '+0.039']
  index : ['+0.219', '+0.654', '+0.202', '+0.087']
  middle: ['+0.290', '+0.477', '+0.004', '+0.001']
  ring  : ['+0.166', '+0.274', '-0.002', '-0.001']
  pinky : ['+0.061', '+0.130', '+0.029', '+0.039']
```

C skips the shape line and starts at the joint command:

```text theme={null}
joint command (radians, finger-major thumb/index/middle/ring/pinky):
  thumb : [+1.291, +0.206, +0.345, +0.039]
  index : [+0.219, +0.654, +0.202, +0.087]
  middle: [+0.290, +0.477, +0.004, +0.001]
  ring  : [+0.166, +0.274, -0.002, -0.001]
  pinky : [+0.061, +0.130, +0.029, +0.039]
```

These are actual results, measured on Wuji SDK v2026.8.31, Python 3.12, the C SDK for x86\_64-linux-gnu, and Linux x86\_64, with the example's built-in synthetic open right-hand pose as input.

For the same input, both interfaces produce **bit-identical** joint angles across all 20 values, because they share one solver implementation. Change the keypoints or the hand model and the numbers change, but the shape and grouping don't.

## Adapting It

### Swap In a Real Keypoint Source

Replace the example's `make_open_hand_keypoints()` with your source. As long as it produces `(21, 3)` keypoints in meters in MediaPipe landmark order, nothing downstream changes. The Wuji Glove's `hand_skeleton()` stream is already in this format, and [Retargeting](/en/docs/wuji-sdk/retargeting) shows how to connect it.

### Change Hand Model and Handedness

Targeting a Wuji Hand instead, change `HandModel.WujiHand2` to `HandModel.WujiHand`. For a left-hand assembly, change `Handedness.Right` to `Handedness.Left` and confirm the keypoints come from a left hand too.

### Move to a Real-Time Loop

This example solves one frame. In a real-time setting, call `step()` on a fixed period in a loop and handle the case where no new frame arrived. For the full loop, see [Retargeting](/en/docs/wuji-sdk/retargeting).

### Input Validity

Confirm the keypoints aren't degenerate before solving. An all-zero array, or one holding NaN or Inf, makes `step()` raise `invalid keypoints`. When tracking is lost, repeating the last valid frame is safer than sending zeros.

### Handling Tracking Gaps

The hand leaves the frame, the device disconnects, a replay cuts to a new segment — call `reset()` once after any of these. Skip it and the first few frames after recovery ease out of the old state, which looks like lag.

## Related

* [Retargeting](/en/docs/wuji-sdk/retargeting/): interface description and parameter meanings
* [Common Data Types](/en/docs/wuji-sdk/data-reference/): the structures behind keypoints and joint commands


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.