Method for retrieving in-situ building information based on visual gaze

A method for retrieving in-situ building information based on visual gaze, executed through a mixed reality (MR) device worn by a user in a physical space, the MR device comprising an eye tracker, wherein the method comprises the following steps:

• Step 1, importing a Building Information Modeling (BIM) into a mixed reality development environment configured in Unity through the MR device, spatially aligning the BIM with a physical entity of a building component in the physical space, and locking the BIM in the physical space using spatial anchors technology to achieve linking and binding between digital information of the building component and the physical entity of the building component;
• Step 2, using the eye tracker to acquire user gaze position data in real time, and acquiring timestamp associated with the current user gaze position data to maintain data synchronization;
• Step 3, by monitoring the user gaze position data, using a floating-point variable to record a gaze stay duration of the user on a current gaze target;
• Step 4, when the gaze stay duration exceeds a set threshold, automatically identifying the building component gazed at by the user, and retrieving detailed information of the building component from the BIM, wherein the set threshold is a minimum gaze stay duration required to trigger interaction, and the detailed information includes material properties and size specifications;
• Step 5, displaying an information panel in the user's field of view through the MR device, and displaying the detailed information of the building component in real time via the information panel in situ of the building component in the physical space;
• wherein, the information panel corresponds one-to-one with the building component in the physical space;
• an initial state of each information panel is invisible, and the information panel is activated for display when the corresponding building component is gazed at;
• and an orientation of the information panel is automatically adjusted based on the gaze position data to ensure that it always faces the user's line of sight.