Clarification on Expected EEF Pose Alignment in Camera Frame

[catburgg]

We are trying to visualize the robot end-effector pose in the camera coordinate frame.

Intrinsic used: head_intrinsic_params.json

Extrinsic used: head_extrinsic_params_aligned.json

URDF used: G1_120s.urdf

Here are two examples of our current result:

Could you please clarify:

Should the EEF pose be aligned with the gripper itself, or should it be aligned with the point currently shown in our visualization?

If the alignment is incorrect, we can also provide our transformation code for your reference.
Do you have any suggestions or modifications we should make?

[ModiShi]

How do you get the EEF pose? It shoud be read from the h5 file, /state/end, and it should be aligned with the gripper

[yanziwei]

Is it possible to share the code?

[zhouyc1006]

We are trying to visualize the robot end-effector pose in the camera coordinate frame.

Intrinsic used: head_intrinsic_params.json

Extrinsic used: head_extrinsic_params_aligned.json

URDF used: G1_120s.urdf

Here are two examples of our current result:

Could you please clarify:

Should the EEF pose be aligned with the gripper itself, or should it be aligned with the point currently shown in our visualization?

If the alignment is incorrect, we can also provide our transformation code for your reference. Do you have any suggestions or modifications we should make

We are trying to visualize the robot end-effector pose in the camera coordinate frame.

Intrinsic used: head_intrinsic_params.json

Extrinsic used: head_extrinsic_params_aligned.json

URDF used: G1_120s.urdf

Here are two examples of our current result:

Could you please clarify:

Should the EEF pose be aligned with the gripper itself, or should it be aligned with the point currently shown in our visualization?

If the alignment is incorrect, we can also provide our transformation code for your reference. Do you have any suggestions or modifications we should make?

Hi, have you solved this problem?

[AngchenXie]

We appear to have hit the same misalignment described in this issue, and after a systematic investigation we believe the left arm's recorded state does not match the published kinematic description — while the right arm matches to pixel level.

Setup: episode 839660 (task 619), G1_120s.urdf / robot.usda, head camera with the dataset's intrinsics/extrinsics. Per the guidance above, EEF pose read from /state/end in the h5.

What we verified, each independently:

• FK computed from /state/joint agrees with the recorded /state/end for the left arm — the dataset is internally self-consistent.
• Right arm: projecting FK / /state/end into the head camera lands on the visible right gripper to within a few pixels — confirming our camera model, units, and kinematics handling.
• Left arm: the same projection through the same camera lands hundreds of pixels away from the visible left gripper (analytically outside the frustum at this episode's static left-arm pose).
• Ruled out on our side: intrinsics/extrinsics (validated via right arm), proprio index→joint-name mapping, units (radians confirmed), shoulder-mount transforms (URDF and USD agree to 8 decimals, including the intentional Rz(180) L/R asymmetry), claw-tip link chain (USD vs URDF differ by only 0.93 mm ≈ <2 px), exhaustive reinterpretation of arm_l2 over ±180° (no value brings the projection on-frame), and multi-joint sign/offset fits (they fail held-out validation on episodes 843929 / 843368).
• Cross-episode: episodes with moving left arms (843929, 843368) show smaller but systematic left-arm misalignment (~40–140 px) whose magnitude varies with pose — consistent with a joint-level convention mismatch whose error grows with joint value.

Question: for the left arm specifically (arm_l2 and arm_l7 are our prime suspects) — do the recorded /state/joint values use the same sign convention and zero-reference as the published URDF/USD joint frames? Is there a firmware-level correction (sign flip, or an as-built vs. as-designed zero offset) applied in your stack that is not reflected in the published kinematic description?

Happy to share reproduction scripts and overlay visualizations.