手部追踪技术

Hand tracking is a technology that allows mixed reality (MR) systems to accurately detect and track the user's hand movements and finger positions in real time. By using depth sensors and computer vision algorithms, MR devices can identify the position, orientation, and articulation of each hand.手部追踪是一项允许混合现实（MR）系统实时精准检测并追踪用户手部动作与手指位置的技术。通过深度传感器与计算机视觉算法的结合，MR设备能够识别每只手的位置、方向及关节活动。

With hand tracking, users interact with virtual environments using natural gestures, such as pointing, pinching, or grabbing, mirroring how they would manipulate objects in the real world. This allows for a more intuitive and immersive user experience, reducing the learning curve and increasing the sense of presence.借助手部追踪技术，用户可通过自然手势（如指点、捏取或抓握）与虚拟环境交互，这种交互方式完全复现了现实世界中的物体操控逻辑。该技术不仅能显著降低学习门槛，更能通过直觉化操作大幅提升沉浸感。

The system continuously captures and interprets hand poses and movements in real time, providing a detailed model of the user's hand. Hand tracking supports a wide range of use cases, including user interface navigation, direct manipulation of virtual objects, and execution of gesture-based commands.系统持续实时捕捉并解析手部姿态与动作，构建精细的手部运动模型。其应用场景涵盖：用户界面导航、虚拟物体直接操控、手势指令执行等多样化需求。

UNIGINE provides built-in, native support for hand tracking based on the OpenXR Standard. This allows VR applications created with UNIGINE to track user's hands and fingers in real time. The native hand tracking system supports the following OpenXR Extensions:UNIGINE基于OpenXR 标准原生集成手部追踪功能，使开发的VR应用可实时追踪用户手部与手指动作。原生支持以下OpenXR 扩展协议：

• XR_EXT_hand_tracking - core extension for tracking hand joints and providing their positions and orientations.XR_EXT_hand_tracking：提供手部关节位置与方向数据的核心扩展
• XR_EXT_hand_interaction - enhances the basic tracking by providing interaction-specific hand data.XR_EXT_hand_interaction：增强交互专用手部数据支持
• XR_EXT_hand_joints_motion_range - offers additional control over the reported range of motion for joints, useful for improved realism and precision.XR_EXT_hand_joints_motion_range：提供关节活动范围的精细控制，提升真实感
• XR_EXT_hand_tracking_data_source - provides information about the source or method used for hand tracking data.XR_EXT_hand_tracking_data_source：标识手部追踪数据来源

This feature is integrated into the engine's VR system (no extra plugin needed) and supports any OpenXR - compatible device that offers hand tracking capabilities.该功能深度集成于引擎VR系统（无需额外插件），兼容所有支持手部追踪的OpenXR设备。

When a UNIGINE VR-based application with OpenXR backend starts, the engine checks for an available OpenXR runtime provided by the user's VR device or a platform. Then the engine automatically initializes native hand tracking and retrieves all the hand related data via the OpenXR API.当具有OpenXR后端的基于UNIGINE VR的应用程序启动时，引擎会检查用户的VR设备或平台提供的可用OpenXR运行时。然后引擎自动初始化本机手跟踪，并通过OpenXR API检索所有与手相关的数据。

To use hand tracking, make sure to launch the engine with the OpenXR backend:要使用手部追踪，请确保使用OpenXR后端启动引擎:

-vr_app openxr

Native hand tracking is only available in this mode. Other VR backends (such as OpenVR or Varjo Native SDK) do not support it and require external plugins to function. In this cases, hand tracking can still be enabled through the LeapMotion or Ultraleap plugins, but only if the necessary hardware is connected.原生手部追踪仅在此模式下可用。其他VR后端（如OpenVR或Varjo Native SDK）不支持该功能，需要外部插件才能运行。在这种情况下，仍可通过LeapMotion或Ultraleap插件启用手部追踪，但前提是连接了必要的硬件设备。

Note that some Varjo headsets (Varjo XR-3, VR-3, VR-2 Pro) include built-in Ultraleap sensors, but to access them, either the OpenXR runtime must be used or the Ultraleap plugin must be manually enabled in the application.请注意，部分Varjo头显（Varjo XR-3, VR-3, VR-2 Pro）内置了Ultraleap传感器，但要访问这些传感器，必须使用OpenXR运行时或在应用程序中手动启用Ultraleap插件。

OpenXRExt::init(): Extension "XR_EXT_hand_tracking" found
OpenXRExt::init(): Extension "XR_EXT_hand_interaction" found
OpenXRExt::init(): Extension "XR_EXT_hand_joints_motion_range" found
OpenXRExt::init(): Extension "XR_EXT_hand_tracking_data_source" found

All hand tracking data is spatially relative to the main camera (i.e. the player's HMD - head-mounted display). This means that hand positions follow the headset, when the user moves or rotates their head, the hands move accordingly in the same local tracking space. They are not positioned in world space independently of the HMD.所有手部追踪数据在空间上都相对于主摄像机（即玩家的HMD--头戴显示器）定位。这意味着手部位置会跟随头显移动，当用户移动或旋转头部时，手部会在同一局部追踪空间中相应移动。它们并非独立于HMD的世界空间定位。

注意

In the OpenXR Specification, tracked hand elements are referred to as joints. In UNIGINE, these elements are called bones, consistent with the VRBone class used in API. Conceptually, each bone corresponds to a tracked joint defined by XR_EXT_hand_tracking.在OpenXR规范中，被追踪的手部元素称为关节(joints)。在UNIGINE中，这些元素称为骨骼(bones)，与API中使用的VRBone类保持一致。从概念上讲，每个骨骼对应XR_EXT_hand_tracking定义的一个被追踪关节。

Each hand's skeletal model consists of 26 bones organized in a hierarchy under a single root. The bones include the Wrist, Palm, and Finger bones for all five fingers (thumb, index, middle, ring, little). The finger bones, except the tips, are named after the corresponding bone at the further end of the bone from the fingertips.每只手的骨骼模型由26块骨骼组成，这些骨骼在单一根节点下按层级排列。骨骼包括腕部(Wrist)、手掌(Palm)以及五根手指（拇指、食指、中指、无名指、小指）的指骨。除指尖外，其他指骨均以远离指尖方向的相应骨骼命名。

The Wrist is considered as the root of the hierarchy for the hand - it's the point where the hand would connect to the forearm. Under the Wrist, all finger bones are arranged.腕部被视为手部层级的根节点——这是手部与前臂连接的部位。所有指骨都排列在腕部下方。

The Palm bone is located at the center of the middle finger's metacarpal bone, representing a convenient central point (the middle of the palm area), which can be useful for certain calculations or attaching objects to the user's hand.手掌骨骼位于中指掌骨中心，代表一个便利的中心点（手掌中部区域），可用于特定计算或将物体附着到用户手上。

Unlike the other fingers, the thumb lacks an intermediate phalanx, resulting in one less bone - just like in a real human hand.与其他手指不同，拇指缺少中间指节，因此少一块骨骼——这与真人手部结构一致。

注意

The bone structure in the native OpenXR system differs from that used in the Ultraleap plugin.原生OpenXR系统的骨骼结构与Ultraleap插件使用的结构不同。

All the hand tracking data is given in UNIGINE’s native coordinate system, which is a right-handed Cartesian coordinate system: X and Y form a horizontal plane, Z axis points up. When looking at a hand from the back (palm facing downward), the axes of each bone are aligned as follows:所有手部追踪数据都采用UNIGINE原生坐标系，这是一个右手笛卡尔坐标系：X和Y轴构成水平面，Z轴指向上方。当从手背观察（掌心朝下）时，每块骨骼的轴对齐如下：

• X-axis points to the leftX轴指向左侧
• Y-axis points backwards, along the wristY轴沿腕部向后指向
• Z-axis points upward, out of the back of the handZ轴向上方伸出，垂直于手背

This consistent basis orientation applies to all bones in the hand.这种统一的基准方向适用于手部所有骨骼。

Hand tracking feature is accessible through a set of specialized classes in the engine's VR API. These classes provide a structured way to get hand and finger bone information, and to control debug visualization. The main classes involved are:手部追踪功能通过引擎VR API中的一组专用类访问。这些类提供了获取手部和指骨信息、控制调试可视化的结构化方式。主要涉及以下类：

• VRHandTracking - The manager class for the hand tracking system. Through VRHandTracking you can access the two tracked hands and configure global settings related to hand tracking visualization.VRHandTracking：手部追踪系统的管理类。通过VRHandTracking可访问两只被追踪的手，并配置与手部追踪可视化相关的全局设置。

• VRHand - This class represents a single hand (either left or right) being tracked. Each VRHand contains the set of bones (as VRBone objects) for that hand. The class also provides access to state information such as whether the hand is holding a controller or whether the controller is active. It also includes methods for querying individual bones and rendering a debug visualizer.VRHand：该类表示被追踪的单只手（左或右手）。每个VRHand包含该手的骨骼集合（作为VRBone对象）。该类还提供状态信息访问，如手部是否握持控制器或控制器是否激活。它还包括查询单个骨骼和渲染调试可视化器的方法。

• VRBone - This class represents a single bone in the hand’s skeletal model. Each VRBone provides the position and orientation of itself and additional information like its radius (approximate thickness) and velocity. It also includes a method for rendering a debug visualizer for the specific bone.VRBone：该类表示手部骨骼模型中的单块骨骼。每个VRBone提供自身位置和方向，以及半径（近似厚度）和速度等附加信息。它还包括为特定骨骼渲染调试可视化器的方法。

The system updates the VRBone transforms every frame in sync with the VR loop, so at any time you can query these classes to get the latest hand pose.系统每帧都会与VR循环同步更新VRBone变换，因此可随时查询这些类获取最新手部姿态。

The following console commands control the hand tracking debug visualizer. They can be entered into the Console (or set in configuration files or called via the API).以下控制台命令控制手部追踪调试可视化器。可通过控制台输入（或在配置文件中设置/通过API调用）这些命令。

With the introduction of native OpenXR hand tracking, UNIGINE supports a unified and preffered approach for hand and finger tracking. However, the LeapMotion and the Ultraleap plugins are available and functional for cases where OpenXR is not an option or specific hardware configurations (such as Varjo Native SDK or OpenVR) require them.随着原生OpenXR手部追踪的引入，UNIGINE现已支持标准化且推荐的手部与手指追踪方案。但对于无法采用OpenXR的特殊硬件配置（如Varjo Native SDK或OpenVR），LeapMotion和Ultraleap插件仍保持可用状态。

LeapMotion Plugin (Legacy)

LeapMotion插件（传统方案）#

Ultraleap Plugin (for Specific Use Cases)

Ultraleap插件（特定场景专用）#