Meta Quest 3 VR Teleop Specifications

Complete technical reference: UDP packet schema, field types and byte layout, end-to-end latency breakdown, key tunable parameters, and compatible arm specifications.

System Overview

Hardware & Runtime
Headset Meta Quest 3 (hand-tracking mode required)
Unity version 2022.3 LTS or later · XR Hands package · OpenXR plugin
Host runtime Python 3.10+ (Linux or macOS; no ROS required)
Network requirement Same LAN subnet; Wi-Fi 6 access point recommended
Right hand UDP port 8888
Left hand UDP port 8889 (bilateral control only; otherwise unused)
Packet rate (sender) ~50 Hz from Unity (Unity's fixed update loop)
Robot command rate 30 Hz default (configurable via CONTROL_HZ)
Primary SDK (Piper) piper_sdk · python-can · CAN over USB adapter

UDP Packet Schema

Each packet is a fixed-length 45-byte binary message. All multi-byte fields use little-endian byte order. Unknown bytes appended at the end are ignored by the receiver for forward compatibility.

Binary Packet Layout — 45 bytes total
Field Type Bytes Description
header uint8[4] 4 Magic bytes 0x52 0x43 0x54 0x50 — ASCII "RCTP". Packets with a wrong magic value are silently dropped.
timestamp float64 8 Unity Time.realtimeSinceStartup in seconds. Used for jitter measurement; not interpreted by the robot controller.
pos_x float32 4 End-effector X position in metres, VR world frame (right-handed, Y-up). Converted to mm robot frame by transform_position().
pos_y float32 4 End-effector Y position in metres, VR world frame.
pos_z float32 4 End-effector Z position in metres, VR world frame.
rot_x, rot_y, rot_z, rot_w float32 × 4 16 End-effector rotation as a unit quaternion in VR world frame. Converted to Euler (roll/pitch/yaw) by quat_to_euler() before passing to the robot SDK.
gripper float32 4 Gripper openness: 0.0 = fully closed, 1.0 = fully open. Derived from pinch strength via VRGripperController.cs.
flags uint8 1 Bit 0: tracking valid (1 = hand detected). Bit 1: emergency stop requested (1 = operator pressed Menu). Remaining bits reserved.

Byte offset summary: header 0–3, timestamp 4–11, pos_x 12–15, pos_y 16–19, pos_z 20–23, rot_x 24–27, rot_y 28–31, rot_z 32–35, rot_w 36–39, gripper 40–43, flags 44.

Python struct format string for unpacking:

import struct

PACKET_MAGIC = b'\x52\x43\x54\x50'  # "RCTP"
PACKET_FMT   = '<4sdfffffffff f B'
#              ^  ^ ^^^^^^^^^^ ^ ^
#              |  | pos xyz    | flags (uint8)
#              |  timestamp   gripper (float32)
#              header (4 bytes)   rot xyzw (float32×4)

def parse_packet(data: bytes) -> dict | None:
    if len(data) < struct.calcsize(PACKET_FMT):
        return None
    fields = struct.unpack_from(PACKET_FMT, data)
    header, ts, px, py, pz, rx, ry, rz, rw, gripper, flags = fields
    if header != PACKET_MAGIC:
        return None
    return {
        "timestamp": ts,
        "position": (px, py, pz),
        "rotation": (rx, ry, rz, rw),
        "gripper":  gripper,
        "valid":    bool(flags & 0x01),
        "estop":    bool(flags & 0x02),
    }

End-to-End Latency

Measured on a Wi-Fi 6 LAN with the AgileX Piper arm at 30 Hz control rate and 25% speed limit. Values are typical; actual latency depends on Wi-Fi conditions and robot servo rate.

~20 ms
Quest 3 Tracking Pipeline
Hand pose sensor fusion to Unity callback
<5 ms
UDP Transit
Wi-Fi 6 LAN; up to ~15 ms on older Wi-Fi
<2 ms
Python Parse & Queue
Struct unpack + queue insert
30–80 ms
Robot Trajectory Execution
Depends on speed limit and move distance
Typical Total End-to-End
Wi-Fi 6 LAN, 25% speed 50–120 ms
Wi-Fi 5 LAN, 25% speed 80–150 ms
Software e-stop response (Bit 1 path) 1 control cycle (~33 ms at 30 Hz)
Tracking-loss detection Immediate (flags byte in current packet)
Keep speed limits conservative to reduce latency feel. The dominant perceptual latency component is robot trajectory execution. A lower SPEED_PERCENT makes the system feel more predictable even though actual round-trip time is similar. Start at 25% and raise only after motion is fully validated.

Key Tunable Parameters

Unity Inspector (VRHandPoseSender)
positionOffset (m) (0, 0, 0.3) — Piper starting point
rotationOffset (deg) (0, 90, 0) — Piper starting point
scaleFactor 0.75 — scales hand motion to Piper workspace
Workspace X clamp (mm) ±400 (Piper); ±600 (xArm6)
Workspace Z clamp (mm) 50 – 700 (Piper); 0 – 900 (xArm6)
UDP target port 8888 (right hand) / 8889 (left hand)
Python Server (teleoperation_main.py)
CONTROL_HZ 30 Hz — robot command rate; match to servo rate
QUEUE_MAXSIZE 3 — frames buffered between receiver and controller threads
piper_controller.py (AgileX Piper)
SPEED_PERCENT 25% — maximum joint speed fraction; raise gradually
X_MIN / X_MAX (mm) −400 / +400
Y_MIN / Y_MAX (mm) −400 / +400
Z_MIN / Z_MAX (mm) 50 / 700
GRIPPER_MAX_UM (µm) 70 000 — 70 mm physical opening
SDK position units Micrometres (integer) — multiply mm values by 1 000
SDK orientation units Millidegrees (integer) — multiply degree values by 1 000
CAN interface can0 (default) — activate with can_activate.sh before running

Compatible Arms at SVRC

Arm DOF SDK / Interface Controller File Status
AgileX Piper 6 piper_sdk · CAN over USB piper_controller.py Production-ready
OpenArm 7 SocketCAN / ROS 2 MoveIt2 openarm_controller.py Beta
DK1 Bimanual Kit Dual 6 Ports 8888 + 8889 simultaneously dk1_controller.py Beta
VLAI L1 Dual 6 ROS 2 bridge · TCP/IP vlai_l1_controller.py In development

Software Requirements

Quest 3 / Unity Side
Unity 2022.3 LTS or later
XR Hands package com.unity.xr.hands ≥ 1.3
OpenXR plugin com.unity.xr.openxr ≥ 1.9
Meta XR SDK Optional but recommended for passthrough
Hand tracking mode Enabled in Quest system settings (Movement Tracking)
Control PC / Python Side
Python 3.10 or later (uses union type syntax)
piper_sdk pip install piper_sdk
python-can pip install python-can
CAN interface USB-to-CAN adapter; activated with can_activate.sh can0 1000000
Standard library only socket, struct, queue, threading, signal, time

Ready to Teleop?

Return to the setup guide to get your Quest 3 streaming to a robot arm, or read the full wiki for the complete developer reference.