# US 10,768,695 B2 · Artificial reality system having adaptive degrees of freedom (DOF) selection > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,768,695 B2 - Document ID: US10768695B2 - Status: Granted - Kind: B2 - Application number: 16/264,805 - Filed: 2019-02-01 - Published: 2020-08-06 - Granted: 2020-09-08 - Earliest claimed: 2019-02-01 - Assignees: Facebook Technologies, LLC - Applicants: Facebook Technologies, LLC - Inventors: Ananth Ranganathan; Andrew Melim; Ioannis Pavlidis - Web page: https://patentgrep.com/patent/US10768695B2 ## CPC - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06F 3/012: {Head tracking input arrangements} - A63F 13/212: using sensors worn by the player, e.g. for measuring heart beat or leg activity - A63F 13/40: Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment - G02B 27/017: {Head mounted} - G06F 1/3206: Monitoring of events, devices or parameters that trigger a change in power modality - G06F 1/3234: Power saving characterised by the action undertaken - G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)} - A63F 13/217: using environment-related information, i.e. information generated otherwise than by the player, e.g. ambient temperature or humidity - A63F 13/358: Adapting the game course according to the network or server load, e.g. for reducing latency due to different connection speeds between clients - A63F 13/428: involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes - G06F 1/3265: {Power saving in display device} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS ## Abstract An artificial reality system is described that implements adaptive degrees-of-freedom (DOF) selection when tracking frames of reference and rendering artificial reality content. In one example, the artificial reality system comprises a head mounted display (HMD) that outputs rendered artificial reality content. A performance monitor determines one or more performance indicators associated with the artificial reality system. A degree-of-freedom (DOF) selector applies one or more policies to the performance indicators to select between a first mode in which a pose tracker computes one or more poses of the HMD within the 3D environment using 6DOF and a second mode in which the pose tracker computes the one or more poses using 3DOF. The pose tracker computes the one or more poses for the HMD within the 3D environment in accordance with the selected mode. A rendering engine renders the content for the artificial reality application based on the computed pose. ## Claims ### Claim 1 An artificial reality system comprising: a head-mounted display (HMD) configured to output artificial reality content; a pose tracker configured to compute one or more poses of the HMD within a three-dimensional (3D) environment; a performance monitor, executing on one or more processors, configured to determine one or more performance indicators associated with the artificial reality system, wherein the one or more performance indicators comprise at least one of an eye tracking quality, a network performance or latency, a computing resource usage, or a measure of jitter; a degree-of-freedom (DOF) selector configured to: apply one or more policies to the performance indicators to determine that a transition condition has been satisfied; and select, based on the determination that the transition condition has been satisfied, between a first mode in which the pose tracker is configured to compute the one or more poses of the HMD using six degrees-of-freedom (6DOF) and a second mode in which the pose tracker is configured to compute the one or more poses using three degrees-of-freedom (3DOF); and a rendering engine configured to render the artificial reality content based on the computed pose. ### Claim 2 (depends on 1) The artificial reality system of claim 1, wherein, to compute the one or more poses of the HMD using 6DOF, the pose tracker is further configured to compute one or more poses of the HMD with respect to both rotational movement and translational movement by the HMD, wherein, to compute the one or more poses of the HMD using 3DOF, the pose tracker is further configured to compute the one or more poses of the HMD with respect to rotational movement by the HMD but not translational movement by the HMD, wherein the rotational movement by the HMD comprises rotational movement about at least one of a vertical axis of the HMD, a transverse axis of the HMD, or a longitudinal axis of the HMD, and wherein the translational movement by the HMD comprises translational movement along at least one of the vertical axis of the HMD, the transverse axis of the HMD, or the longitudinal axis of the HMD. ### Claim 3 (depends on 1) The artificial reality system of claim 1, wherein, to apply the one or more policies to the performance indicators to select between the first mode and the second mode, the DOF selector is further configured to: apply the one or more policies to a quality level of image tracking determined by the performance monitor to determine that the transition condition has been satisfied; and select, based on the determination that the transition condition has been satisfied, between the first mode and the second mode. ### Claim 4 (depends on 1) The artificial reality system of claim 1, wherein, to apply the one or more policies to the performance indicators to select between the first mode and the second mode, the DOF selector is configured to: apply the one or more policies to the eye tracking quality determined by the performance monitor to determine that the transition condition has been satisfied; and select, based on the determination that the transition condition has been satisfied, between the first mode and the second mode. ### Claim 5 (depends on 1) The artificial reality system of claim 1, wherein, to apply the one or more policies to the performance indicators to select between the first mode and the second mode, the DOF selector is configured to: apply the one or more policies to a level of an environmental brightness determined by the performance monitor to determine that the transition condition has been satisfied; and select, based on the determination that the transition condition has been satisfied, between the first mode and the second mode. ### Claim 6 (depends on 1) The artificial reality system of claim 1, wherein, to apply the one or more policies to the performance indicators to select between the first mode and the second mode, the DOF selector is further configured to: apply the one or more policies to a number of features currently being tracked by the artificial reality system to determine that the transition condition has been satisfied; and select, based on the determination that the transition condition has been satisfied, between the first mode and the second mode. ### Claim 7 (depends on 1) The artificial reality system of claim 1, wherein, to apply the one or more policies to the performance indicators to select between the first mode and the second mode, the DOF selector is further configured to: apply the one or more policies to an image texture of a real-world feature determined by the performance monitor to determine that the transition condition has been satisfied; and select, based on the determination that the transition condition has been satisfied, between the first mode and the second mode. ### Claim 8 (depends on 1) The artificial reality system of claim 1, wherein, to apply the one or more policies to the performance indicators to select between the first mode and the second mode, the DOF selector is configured to: apply the one or more policies to the network performance or latency determined by the performance monitor to determine that the transition condition has been satisfied; and select, based on the determination that the transition condition has been satisfied, between the first mode and the second mode. ### Claim 9 (depends on 1) The artificial reality system of claim 1, wherein, to apply the one or more policies to the performance indicators to select between the first mode and the second mode, the DOF selector is configured to: apply the one or more policies to the computing resource usage determined by the performance monitor to determine that the transition condition has been satisfied; and select, based on the determination that the transition condition has been satisfied, between the first mode and the second mode. ### Claim 10 (depends on 1) The artificial reality system of claim 1, wherein, to apply the one or more policies to the performance indicators to select between the first mode and the second mode, the DOF selector is configured to: apply the one or more policies to the measure of jitter determined by the performance monitor to determine that the transition condition has been satisfied; and select, based on the determination that the transition condition has been satisfied, between the first mode and the second mode. ### Claim 11 (depends on 1) The artificial reality system of claim 1, wherein the HMD comprises the performance monitor, the DOF selector, the pose tracker, and the rendering engine. ### Claim 12 (depends on 1) The artificial reality system of claim 1, further comprising a console device, wherein the console device comprises the performance monitor, the DOF selector, the pose tracker, and the rendering engine. ### Claim 13 (depends on 1) The artificial reality system of claim 1, wherein to apply the one or more policies to the performance indicators to select between the first mode in which the pose tracker computes the one or more poses of the HMD within the 3D environment using 6DOF and the second mode in which the pose tracker computes the one or more poses using 3DOF, the DOF selector is configured to apply the one or more policies to the performance indicators to switch from selecting the first mode in which the pose tracker computes the one or more poses of the HMD within the 3D environment using 6DOF to selecting the second mode in which the pose tracker computes the one or more poses of the HMD within the 3D environment using 3DOF, and wherein the pose tracker is further configured to switch from computing the one or more poses for the HMD within the 3D environment using 6DOF to computing the one or more poses for the HMD within the 3D environment using 3DOF. ### Claim 14 (depends on 1) The artificial reality system of claim 1, wherein to apply the one or more policies to the performance indicators to select between the first mode in which the pose tracker computes the one or more poses of the HMD within the 3D using 6DOF and the second mode in which the pose tracker computes the one or more poses using 3DOF, the DOF selector is configured to apply the one or more policies to the performance indicators to switch from selecting the second mode in which the pose tracker computes the one or more poses of the HMD within the 3D environment using 3DOF to selecting the first mode in which the pose tracker computes the one or more poses of the HMD within the 3D environment using 6DOF, and wherein the pose tracker is further configured to switch from computing the one or more poses for the HMD within the 3D environment using 3DOF to computing the one or more poses for the HMD within the 3D environment using 6DOF. ### Claim 15 A method comprising: determining one or more performance indicators associated with an artificial reality system having at least one head mounted display (HMD), wherein the one or more performance indicators comprise at least one of an eye tracking quality, a network performance or latency, a computing resource usage, or a measure of jitter; applying one or more policies to the performance indicators to determine that a transition condition has been satisfied; and selecting, based on the determination that the transition condition has been satisfied, between a first mode in which one or more poses of the HMD within a three dimensional (3D) environment are computed using six degrees-of-freedom (6DOF) and a second mode in which the one or more poses are computed using three degrees-of-freedom (3DOF); computing the one or more poses for the HMD within the 3D environment in accordance with the selected mode; rendering artificial reality content based on the computed one or more poses; and outputting, by the HMD, the rendered artificial reality content. ### Claim 16 (depends on 15) The method of claim 15, wherein computing the one or more poses of the HMD using 6DOF comprises computing one or more poses of the HMD with respect to both rotational movement and translational movement by the HMD, wherein computing the one or more poses of the HMD using 3DOF comprises computing the one or more poses of the HMD with respect to rotational movement by the HMD but not translational movement by the HMD, wherein the rotational movement by the HMD comprises rotational movement about at least one of a vertical axis of the HMD, a transverse axis of the HMD, or a longitudinal axis of the HMD, and wherein the translational movement by the HMD comprises translational movement along at least one of the vertical axis of the HMD, the transverse axis of the HMD, or the longitudinal axis of the HMD. ### Claim 17 (depends on 15) The method of claim 15, wherein applying the one or more policies to the performance indicators to select between the first mode and the second mode further comprises: applying the one or more policies to a quality level of image tracking to determine that the transition condition has been satisfied; and selecting, based on the determination that the transition condition has been satisfied, between the first mode and the second mode. ### Claim 18 (depends on 15) The method of claim 15, wherein applying the one or more policies to the performance indicators to select between the first mode and the second mode comprises: applying the one or more policies to the eye tracking quality to determine that the transition condition has been satisfied; and selecting, based on the determination that the transition condition has been satisfied, between the first mode and the second mode. ### Claim 19 (depends on 15) The method of claim 15, wherein applying the one or more policies to the performance indicators to select between the first mode and the second mode comprises: applying the one or more policies to the computing resource usage to determine that the transition condition has been satisfied; and selecting, based on the determination that the transition condition has been satisfied, between the first mode and the second mode. ### Claim 20 A non-transitory, computer-readable medium comprising instructions that, when executed, cause one or more processors to: determine one or more performance indicators associated with an artificial reality system having at least one head mounted display (HMD), wherein the one or more performance indicators comprise at least one of an eye tracking quality, a network performance or latency, a computing resource usage, or a measure of jitter; apply one or more policies to the performance indicators to determine that a transition condition has been satisfied; select, based on the determination that the transition condition has been satisfied, between a first mode in which one or more poses of the HMD within a three dimensional (3D) environment are computed using six degrees-of-freedom (6DOF) and a second mode in which the one or more poses are computed using three degrees-of-freedom (3DOF); compute the one or more poses for the HMD within the 3D environment in accordance with the selected mode; render artificial reality content based on the computed one or more poses; and output the rendered artificial reality content.