Replaying Data Offline

SDK Practice

Overview

This page uses Python to replay Wuji Glove emf_poses and imu_raw/palm. Data Recording produces an MCAP file. The replay code reads that file, rebuilds the frames, and calls compute(frame) for Offline Solving.

SDK source examples: Python and C.

Prerequisites

Hardware

None. This is the only page in the series that needs no device.

Software

pip install wuji-sdk
# add this to read MCAP recordings
pip install mcap

Before You Start

To replay real data, read Recording Data first. Running the example’s synthetic frames needs no preparation.

Data Flow

Channels the Pipeline Offers

The object returned by offline_pipeline() exposes these methods:

The pipeline also provides these read-only properties: sn, hand_side, urdf_source, and urdf_source_path.

Hand solving depends on the URDF. IMU fusion doesn’t. Swapping hand models changes only the first three methods.

Where the URDF Comes From

Create the pipeline with the capture device’s serial number and hand side. Check the serial number against the MCAP channel name. Get the hand side and model used during capture from the capture record. Raw frames in the recording don’t select a hand model automatically.

To reproduce the hand outputs from capture, pass the absolute path to the URDF used then, or select the same SDK user. Check pipeline.urdf_source and pipeline.urdf_source_path. An unreadable explicit path raises an error. If the managed model isn’t found, the SDK uses a built-in model and the results may differ. See Select and Verify the Hand Model for the lookup order.

Consistency With Online Results

This is the section worth reading.

Method

1. Connect a real glove and use TopicRecorder to capture emf_poses, hand_joint_angles, and hand_skeleton together for 4 seconds
2. Align the three streams by header.seq in the MCAP file, yielding 525 common frames
3. Rebuild the recorded emf_poses into EmfPoseArray and feed them to the offline pipeline
4. Subtract the offline results from the recorded online results, element by element

Results

Across the first 200 frames:

Joint angles are exactly identical. The 7-nanometer difference in skeleton positions comes from MCAP’s JSON serialization round-tripping a float, not from the algorithm.

Offline replay therefore reproduces online solving without loss. Tuning algorithms on recorded data, comparing hand models, and reproducing production issues all give trustworthy results.

Replay Code

This example reads emf_poses and imu_raw/palm from a recording and submits them to one pipeline in MCAP message order. Record both channels first. Get the serial number from /<serial-number>/emf_poses and confirm the hand side against the capture record. Use --urdf-path for the model used during capture. If you omit it, the SDK uses its normal model lookup rules.

Save this as replay_recording.py:

import argparse
import json
from mcap.reader import make_reader
from wuji_sdk import EmfPose, EmfPoseArray, FrameHeader, ImuData, Pose, Quaternion, Vector3F64, WujiGlove


def vector(data):
    return Vector3F64(data["x"], data["y"], data["z"])


def header(data):
    return FrameHeader(seq=data["seq"], timestamp_us=data["timestamp_us"], frame_id=data["frame_id"])


def emf_frame(data):
    return EmfPoseArray(
        header=header(data["header"]),
        poses=[
            EmfPose(
                pose=Pose(
                    position=pose["pose"]["position"],
                    orientation=Quaternion(**pose["pose"]["orientation"]),
                ),
                confidence=pose["confidence"],
            )
            for pose in data["poses"]
        ],
    )


parser = argparse.ArgumentParser()
parser.add_argument("mcap_path")
parser.add_argument("sn")
parser.add_argument("hand_side", choices=["left", "right"])
parser.add_argument("--urdf-path")
args = parser.parse_args()
pipeline = WujiGlove.offline_pipeline(sn=args.sn, hand_side=args.hand_side, urdf_path=args.urdf_path)
print(f"Model: {pipeline.urdf_source} {pipeline.urdf_source_path}")
angles_channel = pipeline.hand_joint_angles()
tips_channel = pipeline.tip_poses()
skeleton_channel = pipeline.hand_skeleton()
imu_channel = pipeline.imu_data_palm()
counts = {"emf_poses": 0, "imu_raw/palm": 0}
last_angles = last_tips = last_skeleton = last_fused = None
with open(args.mcap_path, "rb") as stream:
    for _, channel, message in make_reader(stream).iter_messages():
        if channel.topic == f"/{args.sn}/emf_poses":
            frame = emf_frame(json.loads(message.data))
            last_angles = angles_channel.compute(frame)
            last_tips = tips_channel.compute(frame)
            last_skeleton = skeleton_channel.compute(frame)
            counts["emf_poses"] += 1
        elif channel.topic == f"/{args.sn}/imu_raw/palm":
            data = json.loads(message.data)
            frame = ImuData.raw(
                header=header(data["header"]),
                angular_velocity=vector(data["angular_velocity"]),
                linear_acceleration=vector(data["linear_acceleration"]),
            )
            last_fused = imu_channel.compute(frame)
            counts["imu_raw/palm"] += 1
print(f"EMF frames: {counts['emf_poses']}, palm IMU frames: {counts['imu_raw/palm']}")
if not all(counts.values()):
    raise SystemExit("Missing expected recording channel")
print(f"Last EMF result: fingers={len(last_angles.fingers)}, "
      f"tips={len(last_tips.poses)}, skeleton joints={len(last_skeleton.joints)}")
q = last_fused.orientation
print(f"Last palm IMU result: orientation=({q.x:.6f}, {q.y:.6f}, {q.z:.6f}, {q.w:.6f})")

Install mcap, then replace the file path, serial number, and hand side with the values from your recording:

python -m pip install mcap
python replay_recording.py ./data/session.mcap DEVICE_SN left --urdf-path /absolute/path/to/hand.urdf

The output shows the model source, frame counts, hand output sizes from the last EMF frame, and the fused orientation from the last palm IMU frame. Both frame counts should exceed zero. Five fingers, five fingertip poses, and 21 skeleton joints confirm that the recorded EMF frames were solved. If capture used the built-in model, omit --urdf-path and check the reported Model. This example doesn’t write the results to a file. C applications can construct and submit the same frames as shown in the example. The application reads the MCAP file.

IMU Fusion

The fuser carries state and advances frame by frame. The example feeds a single frame, which shows nothing, so here is how it evolves over a continuous run.

At Rest

Feeding angular_velocity=(0,0,0) and linear_acceleration=(0,0,9.8) for 399 frames:

Orientation locks to the identity quaternion with no drift. When gravity aligns with the ideal Z axis, the fuser produces no correction.

Rotating

Switching to angular_velocity=(0, 0, 1.0) rad/s with timestamps 5 ms apart, 200 frames (1.0 second):

One rad/s for one second should give 1 rad, or 57.30°, and the measurement reads 57.0°. That 0.3° gap exists because the first frame has no predecessor to compute a time delta against, so it doesn’t integrate. Only 199 frames integrate: 199/200 x 57.30° = 57.01°, matching the measurement.

Two consequences. Timestamps must increase for real, because the fuser derives its integration step entirely from them. And the first frame always returns the initial orientation, so never judge anything by it.

Integration Steps

See the Python example for a runnable program. The snippets below show the key calls in order. The example includes argument parsing, signal handling, and error paths.

1. Create the Offline Pipeline

Use the serial number and hand side of the glove that captured the data. Reuse one pipeline for the whole replay. You don’t need to connect to a device.

from wuji_sdk import WujiGlove
pipeline = WujiGlove.offline_pipeline(sn=serial_number, hand_side="right")
print(pipeline.urdf_source, pipeline.urdf_source_path)

2. Prepare EMF Pose Frames

The Python example uses make_emf_frame() to create synthetic frames with five fingers. For recorded data, rebuild each EmfPoseArray from the poses, confidence values, and frame header captured at the same time. Keep the frames in capture order.

# See the Python example linked above for the complete make_emf_frame() function
frames = [
    make_emf_frame(seq=1, timestamp_us=0, curl=0.00),
    make_emf_frame(seq=2, timestamp_us=8333, curl=0.35),
]

3. Compute the Hand Outputs Frame by Frame

Pass each frame to the outputs you need. You can compute joint angles, fingertip poses, and the skeleton independently.

for frame in frames:
    angles = pipeline.hand_joint_angles().compute(frame)
    tips = pipeline.tip_poses().compute(frame)
    skeleton = pipeline.hand_skeleton().compute(frame)
    print(frame.header.seq, len(angles.fingers), len(skeleton.joints))

4. Compute IMU Fusion When Needed

IMU fusion takes separate input. The Python example uses make_raw_imu() to create a raw IMU frame. For recorded data, submit consecutive frames in capture order. The pipeline needs no explicit shutdown.

fused = pipeline.imu_data_palm().compute(
    make_raw_imu(seq=4, timestamp_us=25000)
)
print(fused.orientation)

Running It

python examples/python/wuji_glove/3.offline_pipeline.py

pipeline: sn=WujiGlove-DEMO-OFFLINE, hand_side=right
urdf: source=builtin_default, path=None
frame=1 fingers=5 tips=5 joints=21 thumb_conf=0.475 thumb_angles=[-0.565, -0.6, -0.6, -0.22, 0.629]
frame=2 fingers=5 tips=5 joints=21 thumb_conf=0.475 thumb_angles=[-0.699, -0.611, -0.714, -0.203, 0.934]
frame=3 fingers=5 tips=5 joints=21 thumb_conf=0.475 thumb_angles=[-0.731, -0.611, -0.734, -0.161, 1.087]
imu_palm seq=4 orientation=(0.000, -0.000, 0.000, 1.000) cov0=0.0

Measured with no device connected. The example uses a demo serial number that doesn’t exist, so urdf_source reads builtin_default.

The three frames raise curl from 0.00 to 0.70, simulating fingers closing. The thumb’s fifth angle rises from 0.629 to 1.087 accordingly, in the right direction. thumb_conf stays at 0.475 because confidence on synthetic data doesn’t vary with curl.

Usage Notes

Adapting It

Compare Two Hand Models

Run the same data through two pipelines, one on the default model and one on a calibrated model, then compare joint angles frame by frame to quantify what calibration buys:

default_pl = WujiGlove.offline_pipeline(sn="WG1KA00XXXXXXXXX", hand_side="right")
custom_pl = WujiGlove.offline_pipeline(
    sn="WG1KA00XXXXXXXXX", hand_side="right",
    urdf_path="/path/to/custom_hand.urdf",
)

Don’t Build a New Pipeline Per Frame

hand_joint_angles() returns a channel object on each call, and calling it repeatedly in a loop is fine. IMU fusion state, however, lives on the pipeline, so a new pipeline resets the fuser. Run a whole batch through one pipeline instance.

Use Synthetic Data for Regression Tests

It runs without hardware, which suits CI. The example’s hand-written three frames extend naturally into fixed-input, fixed-expected-output assertions.

Related

• Recording Data: produce the MCAP files this page replays
• Hand Model Calibration: generate a URDF that matches your hand
• Setting a Custom Hand Model: switch models in an online session
• Common Data Types: EmfPoseArray, ImuData, Pose, and related structures