# US 11,899,204 B2 · Soft follow and pitch angle effects for VR/AR interface > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,899,204 B2 - Document ID: US11899204B2 - Status: Granted - Kind: B2 - Application number: 17/343,658 - Filed: 2021-06-09 - Published: 2022-12-15 - Granted: 2024-02-13 - Earliest claimed: 2021-06-09 - Assignees: Snap Inc. - Applicants: Mathieu Emmanuel Vignau - Inventors: Mathieu Emmanuel Vignau - Web page: https://patentgrep.com/patent/US11899204B2 ## CPC - G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking} - G02B 27/017: {Head mounted} - G02B 27/0172: {characterised by optical features} - G02B 27/0179: {Display position adjusting means not related to the information to be displayed} - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06F 3/012: {Head tracking input arrangements} - G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)} - G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors - G06T 15/20: Perspective computation - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - G02B 2027/014: {comprising information/image processing systems} - G02B 2027/0178: {Eyeglass type (eyeglass details G02C)} - G02B 2027/0181: {Adaptation to the pilot/driver} - G02B 2027/0187: {slaved to motion of at least a part of the body of the user, e.g. head, eye} - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract The user interface of an electronic eyewear device tracks a user's head movement to keep the user interface in front of the user's eyes. A position forward vector, a rotation forward vector, and an up vector are defined to enable the system to individually adjust the rates of translation and rotation of the user interface as the user's head moves through space. By disconnecting the translation and rotation of the user interface from the head forward vector, the rates of translation and rotational movement of the user interface are separated from the rate of movement of the user's head to enable a delay in user interface movement relative to head movement for a more realistic VR/AR 3D presentation. The pitch angle is monitored to maintain the user interface in front of the user's eyes in the correct orientation even at difficult pitch angles such as straight up or straight down. ## Claims ### Claim 1 An electronic eyewear device adapted to be worn on the head of a user, comprising: a frame; at least one optical assembly within the frame; at least one display located within the at least one optical assembly adapted to display a user interface; a memory that stores instructions; and a processor that executes the instructions to perform head tracking operations including: determining a pitch angle A of the electronic eyewear device with respect to ground; when the pitch angle A is in a first zone between a minimum pitch angle and a maximum pitch angle, setting an up vector for the user interface as equal to a world up vector relative to ground, when the pitch angle A is in a second zone in a first transition angle region between a minimum pitch angle and a first angle region relative to a real world sky or in a second transition angle region between a maximum pitch angle and a second angle region relative to a real world ground, setting an up vector for the user interface as equal to an interpolated vector between a world up vector relative to ground and a head up vector for a user's head, or when the pitch angle A is in a third zone in the first angle region relative to the real world sky or in the second angle region relative to the real world ground, setting an up vector for the user interface as equal to a head up vector for a user's head; and rendering the user interface in alignment with the up vector. ### Claim 2 (depends on 1) The electronic eyewear device of claim 1, wherein the processor executes the instructions to perform additional operations including: determining at least one of a rotation damping constant or a position damping constant for a given render frame rate of the user interface; calculating at least one of a rotation forward vector or a position forward vector for a position of the user interface for a current rendering frame, calculation of the rotation forward vector comprising calculating an angular linear interpolation of the rotation forward vector for a previous frame and a head forward vector of a current frame at a rotation damping specified by the rotation damping constant, and calculation of the position forward vector comprising calculating an angular linear interpolation of the position forward vector for the previous frame and a head forward vector of the current frame at a position damping specified by the position damping constant; and rendering the user interface to the display for the current rendering frame at a position of at least one of the calculated position forward vector or the calculated rotation forward vector, whereby at least one of translational or rotational movement of the user interface is delayed relative to movement of the user's head. ### Claim 3 (depends on 2) The electronic eyewear device of claim 2, wherein the processor executes the instructions to perform the calculating and rendering once per rendering frame of the user interface. ### Claim 4 (depends on 2) The electronic eyewear device of claim 2, wherein the processor executes the instructions to perform the determining of at least one of the rotation damping constant or the position damping constant for the given render frame rate of the user interface by reading a preset value or determining a value provided by a user for at least one of the rotation damping constant or the position damping constant for the given render frame rate. ### Claim 5 (depends on 2) The electronic eyewear device of claim 2, wherein the processor executes the instructions to perform additional operations including dynamically disabling delayed rendering of at least one of translational or rotational movement of the user interface relative to movement of the user's head when the render frame rate is below a threshold. ### Claim 6 (depends on 1) The electronic eyewear device of claim 1, wherein the processor executes the instructions to determine the minimum pitch angle and the maximum pitch angle by reading a value that is at least one of preset or provided by a user for at least one of the minimum pitch angle or the maximum pitch angle. ### Claim 7 (depends on 1) The electronic eyewear device of claim 1, wherein the processor executes the instructions to determine the minimum pitch angle, the maximum pitch angle, the first transition angle region, and the second transition angle region by determining a value that is at least one of preset or provided by a user for at least one of the minimum pitch angle, the maximum pitch angle, the first transition angle region, or the second transition angle region. ### Claim 8 A method of displaying a user interface on a display of an electronic eyewear device adapted to be worn on the head of a user, comprising: determining a pitch angle A of the electronic eyewear device with respect to ground; when the pitch angle A is in a first zone between a minimum pitch angle and a maximum pitch angle, setting an up vector for the user interface as equal to a world up vector relative to ground, when the pitch angle A is in a second zone in a first transition angle region between a minimum pitch angle and a first angle region relative to a real world sky or in a second transition angle region between a maximum pitch angle and a second angle region relative to a real world ground, setting an up vector for the user interface as equal to an interpolated vector between a world up vector relative to ground and a head up vector for a user's head, or when the pitch angle A is in a third zone in the first angle region relative to the real world sky or in the second angle region relative to the real world ground, setting an up vector for the user interface as equal to a head up vector for a user's head; and rendering the user interface in alignment with the up vector. ### Claim 9 (depends on 8) The method of claim 8, further comprising: determining at least one of a rotation damping constant or a position damping constant for a given render frame rate of the user interface; calculating at least one of a rotation forward vector or a position forward vector for a position of the user interface for a current rendering frame, calculation of the rotation forward vector comprising calculating an angular linear interpolation of the rotation forward vector for a previous frame and a head forward vector of a current frame at a rotation damping specified by the rotation damping constant, and calculation of the position forward vector comprising calculating an angular linear interpolation of the position forward vector for the previous frame and a head forward vector of the current frame at a position damping specified by the position damping constant; and rendering the user interface to the display for the current rendering frame at a position of at least one of the calculated position forward vector or the calculated rotation forward vector, whereby at least one of translational or rotational movement of the user interface is delayed relative to movement of the user's head. ### Claim 10 (depends on 9) The method of claim 9, wherein the calculating and rendering are performed once per rendering frame of the user interface. ### Claim 11 (depends on 9) The method of claim 9, wherein a value for at least one of the rotation damping constant or the position damping constant for the given render frame rate is preset or is provided by a user. ### Claim 12 (depends on 9) The method of claim 9, further comprising dynamically disabling delayed rendering of at least one of translational or rotational movement of the user interface relative to movement of the user's head when the render frame rate is below a threshold. ### Claim 13 (depends on 8) The method of claim 8, further comprising determining the minimum pitch angle and the maximum pitch angle by reading a value that is at least one of preset or provided by a user for at least one of the minimum pitch angle or the maximum pitch angle. ### Claim 14 (depends on 8) The method of claim 8, further comprising determining the minimum pitch angle, the maximum pitch angle, the first transition angle region, and the second transition angle region by determining a value that is at least one of preset or provided by a user for at least one of the minimum pitch angle, the maximum pitch angle, the first transition angle region, or the second transition angle region. ### Claim 15 A non-transitory computer-readable storage medium that stores instructions that when executed by at least one processor cause the processor to display a user interface on a display of an electronic eyewear device adapted to be worn on the head of a user by performing operations comprising: determining a pitch angle A of the electronic eyewear device with respect to ground; when the pitch angle A is in a first zone between a minimum pitch angle and a maximum pitch angle, setting an up vector for the user interface as equal to a world up vector relative to ground, when the pitch angle A is in a second zone in a first transition angle region between a minimum pitch angle and a first angle region relative to a real world sky or in a second transition angle region between a maximum pitch angle and a second angle region relative to a real world ground, setting an up vector for the user interface as equal to an interpolated vector between a world up vector relative to ground and a head up vector for a user's head, or when the pitch angle A is in a third zone in the first angle region relative to the real world sky or in the second angle region relative to the real world ground, setting an up vector for the user interface as equal to a head up vector for a user's head; and rendering the user interface in alignment with the up vector. ### Claim 16 (depends on 15) The medium of claim 15, further comprising instructions that when executed by the at least one processor causes the processor to perform operations comprising: determining at least one of a rotation damping constant or a position damping constant for a given render frame rate of the user interface; calculating at least one of a rotation forward vector or a position forward vector for a position of the user interface for a current rendering frame, calculation of the rotation forward vector comprising calculating an angular linear interpolation of the rotation forward vector for a previous frame and a head forward vector of a current frame at a rotation damping specified by the rotation damping constant, and calculation of the position forward vector comprising calculating an angular linear interpolation of the position forward vector for the previous frame and a head forward vector of the current frame at a position damping specified by the position damping constant; and rendering the user interface to the display for the current rendering frame at a position of at least one of the calculated position forward vector or the calculated rotation forward vector, whereby at least one of translational or rotational movement of the user interface is delayed relative to movement of the user's head. ### Claim 17 (depends on 16) The medium of claim 16, wherein a value for at least one of the rotation damping constant or the position damping constant for the given render frame rate is preset or is provided by a user. ### Claim 18 (depends on 16) The medium of claim 16, further comprising instructions that when executed by the at least one processor causes the processor to perform operations comprising dynamically disabling delayed rendering of at least one of translational or rotational movement of the user interface relative to movement of the user's head when the render frame rate is below a threshold. ### Claim 19 (depends on 15) The medium of claim 15, further comprising instructions that when executed by the at least one processor causes the processor to perform operations comprising determining the minimum pitch angle and the maximum pitch angle by reading a value that is at least one of preset or provided by a user for at least one of the minimum pitch angle or the maximum pitch angle. ### Claim 20 (depends on 15) The medium of claim 15, further comprising instructions that when executed by the at least one processor causes the processor to perform operations comprising determining the minimum pitch angle, the maximum pitch angle, the first transition angle region, and the second transition angle region by determining a value that is at least one of preset or provided by a user for at least one of the minimum pitch angle, the maximum pitch angle, the first transition angle region, or the second transition angle region.