# US 12,001,615 B2 · Integration of a two-dimensional input device into a three-dimensional computing environment > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,001,615 B2 - Document ID: US12001615B2 - Status: Granted - Kind: B2 - Application number: 18/150,156 - Filed: 2023-01-04 - Published: 2023-05-11 - Granted: 2024-06-04 - Earliest claimed: 2021-10-14 - Assignees: Autodesk, Inc. - Applicants: Autodesk, Inc. - Inventors: Qian Zhou; Fraser Anderson; George Fitzmaurice - Web page: https://patentgrep.com/patent/US12001615B2 ## CPC - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors - G02B 27/0172: {characterised by optical features} - G02B 30/50: the image being built up from image elements distributed over a three-dimensional [3D] volume, e.g. voxels - G06F 3/03543: {Mice or pucks (G06F3/03541 takes precedence)} - G06F 3/038: Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry - G06T 19/00: Manipulating three-dimensional [3D] models or images for computer graphics - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - G02B 2027/0138: {comprising image capture systems, e.g. camera} - G02B 2027/014: {comprising information/image processing systems} - G02B 2027/0178: {Eyeglass type (eyeglass details G02C)} - G02B 2027/0187: {slaved to motion of at least a part of the body of the user, e.g. head, eye} - G06T 2200/24: involving graphical user interfaces [GUIs] - G06T 2219/012: Dimensioning, tolerancing - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A workstation enables operation of a 2D input device with a 3D interface. A cursor position engine determines the 3D position of a cursor controlled by the 2D input device as the cursor moves within a 3D scene displayed on a 3D display. The cursor position engine determines the 3D position of the cursor for a current frame of the 3D scene based on a current user viewpoint, a current mouse movement, a CD gain value, a Voronoi diagram, and an interpolation algorithm, such as the Laplacian algorithm. A CD gain engine computes CD gain optimized for the 2D input device operating with the 3D interface. The CD gain engine determines the CD gain based on specifications for the 2D input device and the 3D display. The techniques performed by the cursor position engine and the techniques performed by the CD gain engine can be performed separately or in conjunction. ## Claims ### Claim 1 A computer-implemented method for determining the position of a cursor within a three-dimensional (3D) scene, the method comprising: determining a minimum object size that is displayable by a 3D display; determining a range of control-display (CD) gain values based on the minimum object size and at least one specification of the 3D display; selecting a CD gain value within the range of CD gain values; and determining a 3D position of the cursor within the 3D scene based on the CD gain value. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, wherein the minimum object size is determined based on the at least one specification of the 3D display. ### Claim 3 (depends on 1) The computer-implemented method of claim 1, wherein determining the range of CD gain values is further based on at least one specification a two-dimensional (2D) input device. ### Claim 4 (depends on 1) The computer-implemented method of claim 1, wherein the at least one specification of the 3D display comprises a display resolution the 3D display. ### Claim 5 (depends on 1) The computer-implemented method of claim 1, wherein the 3D scene is displayed on the 3D display. ### Claim 6 (depends on 1) The computer-implemented method of claim 1, wherein the cursor is controlled by a two-dimensional (2D) input device. ### Claim 7 (depends on 1) The computer-implemented method of claim 1, wherein determining the minimum object size comprises determining a minimum angular size of a graphic object displayed within the 3D display that is discernible. ### Claim 8 (depends on 1) The computer-implemented method of claim 1, wherein determining the minimum object size comprises determining a minimum angular size of a character object displayed within the 3D display that is legible. ### Claim 9 (depends on 1) The computer-implemented method of claim 1, wherein determining the range of CD gain values comprises determining a minimum CD gain value based on the at least one specification of the 3D display. ### Claim 10 (depends on 1) The computer-implemented method of claim 1, wherein determining the range of CD gain values comprises determining a minimum CD gain value based on a ratio between a field of view of the 3D display and a physical operating range of a two-dimensional (2D) input device. ### Claim 11 (depends on 1) The computer-implemented method of claim 1, wherein determining the range of CD gain values comprises determining a device-precision CD gain based on a resolution of the 2D input device. ### Claim 12 (depends on 11) The computer-implemented method of claim 11, wherein determining the range of CD gain values comprises determining a hand-precision CD gain based on a minimal input movement performed by a human hand. ### Claim 13 (depends on 12) The computer-implemented method of claim 12, wherein determining the range of CD gain values comprises setting a maximum CD gain value equal to a lesser of the device-precision CD gain and the hand-precision CD gain. ### Claim 14 One or more non-transitory computer-readable media including instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of: determining a minimum object size that is displayable by a three-dimensional (3D) display; determining a range of control-display (CD) gain values based on the minimum object size; selecting a CD gain value within the range of CD gain values; and determining a 3D position of a cursor within a 3D scene based on the CD gain value. ### Claim 15 (depends on 14) The one or more non-transitory computer-readable media of claim 14, wherein determining the range of CD gain values comprises determining a minimum CD gain value based on at least one specification of the 3D display. ### Claim 16 (depends on 14) The one or more non-transitory computer-readable media of claim 14, wherein determining the range of CD gain values comprises determining a minimum CD gain value based on a ratio between a field of view of the 3D display and a physical operating range of a two-dimensional (2D) input device. ### Claim 17 (depends on 14) The one or more non-transitory computer-readable media of claim 14, wherein selecting the CD gain value within the range of CD gain values comprises selecting a CD gain value approximately in a middle of the range of CD gain values. ### Claim 18 (depends on 14) The one or more non-transitory computer-readable media of claim 14, wherein the 3D display comprises a virtual-reality (VR) headset, and the 3D scene comprises a VR scene that includes one or more virtual objects. ### Claim 19 (depends on 14) The one or more non-transitory computer-readable media of claim 14, wherein the 3D display comprises an augmented-reality (AR) headset, and the 3D scene comprises an AR scene that includes at least one virtual object and at least one real-world object. ### Claim 20 A system, comprising: one or more memories that include instructions; and one or more processors that are coupled to the memory and, when executing the instructions, are configured to perform the steps of: determining a minimum object size that is displayable by a three-dimensional (3D) display; determining a control-display (CD) gain value based on the minimum object size; determining a 3D position of a cursor within a 3D scene based on the CD gain value.