# US 12,008,152 B1 · Distance determination for mixed reality interaction > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,008,152 B1 - Document ID: US12008152B1 - Status: Granted - Kind: B1 - Application number: 17/565,321 - Filed: 2021-12-29 - Granted: 2024-06-11 - Earliest claimed: 2020-12-31 - Assignees: Snap Inc. - Applicants: Snap Inc. - Inventors: Benjamin Lucas - Web page: https://patentgrep.com/patent/US12008152B1 ## CPC - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors - G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object - G06F 3/0482: Interaction with lists of selectable items, e.g. menus - G06F 3/04847: Interaction techniques to control parameter settings, e.g. interaction with sliders or dials - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Systems, methods, and computer readable media that determines distances for mixed reality interaction, where the methods include determining a first position of a point of a surface and rendering a virtual reality (VR) interactive item comprising a VR interactive control. The methods further include tracking a control indicator controlled by the user by determining a first plurality of positions of the control indicator and activating the VR interactive control in response to detecting the control indicator controlled by the user transgressing a first threshold distance from the VR interactive control. The methods further include determining a closest position of the first plurality of positions to the point based on the first position, determining the point of the surface to have a second position based on the determined closest position plus a constant for the control indicator, and associating a second threshold with the point of the surface. ## Claims ### Claim 1 A method comprising: determining, by one or more processors, a first position of a point of a surface based on a distance from a right camera and a left camera of a mobile device being equal to: (R*X0)/(2*tan((Θ1/2)+C)*(x2−x1)), where R is an optical distance between an optical axis of the left camera and an optical axis of the right camera, X0 is a width of the point in pixels of an image sensor of the left camera, x2 is a number of pixels from the optical axis of an image sensor of the right camera, x1 is a number of pixels from the optical axis of the left camera, Θ1 is a view angle of the left camera and the right camera, and C is a compensation for distortions of the mobile device; rendering on a display, of the mobile device of a user, a virtual reality (VR) interactive item comprising a VR interactive control, wherein the VR interactive item and the VR interactive control are rendered to appear to the user to be on the surface at the first position of the surface; tracking a control indicator controlled by the user by determining a first plurality of positions of the control indicator; activating the VR interactive control in response to detecting the control indicator controlled by the user transgressing a first threshold distance from the VR interactive control of the VR interactive item; rendering on the display an indication that the VR interactive control is activated in response to activating the VR interactive control; determining a closest position of the first plurality of positions to the point based on the first position; determining the point of the surface to have a second position based on the determined closest position plus a constant for the control indicator; and associating a second threshold distance, less than the first threshold distance, with the point of the surface. ### Claim 2 (depends on 1) The method of claim 1 further comprising: deactivating the VR interactive control in response to detecting the control indicator no longer transgressing the first threshold distance. ### Claim 3 (depends on 2) The method of claim 2 further comprising: reactivating the VR interactive control in response to detecting the control indicator transgresses the second threshold distance. ### Claim 4 (depends on 3) The method of claim 3 wherein the closest position is a closest first position and wherein the method further comprises: tracking the control indicator by determining a second plurality of positions of the control indicator; determining a closest second position of the second plurality of positions to the second position of the point; determining the point of the surface to have a third position based on the determined closest second position plus the constant for the control indicator; and associating a third threshold distance, less than the second threshold distance, with the point of the surface. ### Claim 5 (depends on 1) The method of claim 1 further comprising: rendering on the display the VR interactive item comprising the VR interactive control, wherein the VR interactive item and the VR interactive control are rendered to appear to the user to be on the surface at the second position of the surface; activating the VR interactive control in response to detecting the control indicator transgressing the second threshold distance from the VR interactive control of the VR interactive item; and deactivating the VR interactive control in response to detecting the control indicator not transgressing the second threshold distance. ### Claim 6 (depends on 1) The method of claim 1 wherein the point is a first point and wherein the method further comprises: determining a plurality of closest positions of the first plurality of positions to a plurality of positions of a line along the surface associated with the VR interactive item; determining new positions for the plurality of positions based on a corresponding closest position of the plurality of closest positions plus the constant for the control indicator; and associating the second threshold distance with the plurality of positions. ### Claim 7 (depends on 6) The method of claim 6 wherein the line along the surface is a slider and the VR interactive control is a virtual button, a virtual icon, a virtual object, a virtual image, or a virtual item. ### Claim 8 (depends on 1) The method of claim 1 further comprising: adjusting C based on a difference between the first position and the second position. ### Claim 9 (depends on 1) The method of claim 1 wherein the point is an area of the surface. ### Claim 10 (depends on 1) The method of claim 1 further comprising: adjusting a compensation in an equation for determining the distance to the point based on the first position and the second position. ### Claim 11 (depends on 1) The method of claim 1 further comprising: determining a second plurality of positions of the surface; determining a plurality of closest positions of the first plurality of positions to the second plurality of positions; determining new positions for the second plurality of positions based on a corresponding closest position of the plurality of closest positions and the constant for the control indicator; and associating the second threshold distance with each of the second plurality of positions. ### Claim 12 (depends on 1) The method of claim 1 wherein the VR interactive item is a menu of items, the VR interactive control is a menu item, and the control indicator is a control indicator. ### Claim 13 (depends on 1) The method of claim 1 further comprising: accessing an activation of the VR interactive control; rendering the VR interactive item to be highlighted; rendering the VR interactive item to move simultaneously with the control indicator along a predetermined line associated with the VR interactive item; accessing a deactivation of the VR interactive control; and rendering the VR interactive item to not be highlighted. ### Claim 14 (depends on 1) The method of claim 1 wherein the first threshold distance is a value between 0.5 centimeters and 2 centimeters and the second threshold distance is a value between 0.5 centimeters and 0.01 centimeters. ### Claim 15 (depends on 1) The method of claim 1 wherein the control indicator is a finger of the user, a pen, a pointer, a wand, or a stick, and wherein the constant for the control indicator is a value between 0.1 centimeters and two centimeters. ### Claim 16 (depends on 15) The method of claim 15 further comprising: adjusting a position of the surface based on the determined closest position plus a constant for the control indicator. ### Claim 17 A system comprising: a processor; and a memory storing instructions that, when executed by the processor, configure the system to perform operations comprising: determining, by one or more processors, a first position of a point of a surface based on a distance from a right camera and a left camera of a mobile device being equal to: (R*X0)/(2*tan((Θ1/2)+C)*(x2−x1)), where R is a distance between an optical axis of the left camera and an optical axis of the right camera, X0 is a width of the point in pixels of an image sensor of the left camera, x2 is a number of pixels from the optical axis of an image sensor of the right camera, x1 is a number of pixels from the optical axis of the left camera, Θ1 is a view angle of the left camera and the right camera, and C is a compensation for distortions of the mobile device; rendering on a display, of the mobile device of a user, a virtual reality (VR) interactive item comprising a VR interactive control, wherein the VR interactive item and the VR interactive control are rendered to appear to the user to be on the surface at the first position of the surface; tracking a control indicator controlled by the user by determining a first plurality of positions of the control indicator; activating the VR interactive control in response to detecting the control indicator controlled by the user transgressing a first threshold distance from the VR interactive control; determining a closest position of the first plurality of positions to the point based on the first position; determining the point of the surface to have a second position based on the determined closest position plus a constant for the control indicator; and associating a second threshold distance, less than the first threshold distance, with the point of the surface. ### Claim 18 (depends on 17) The system of claim 17, wherein the operations further comprise: rendering on the display the VR interactive item comprising the VR interactive control, wherein the VR interactive item is rendered to appear to the user to be on the surface with the VR interactive control at the second position of the surface; activating the VR interactive control in response to detecting the control indicator transgressing the second threshold distance from the VR interactive control; and deactivating the VR interactive control in response to detecting the control indicator not transgressing the second threshold distance. ### Claim 19 A non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by a computer, cause the computer to perform operations comprising: determining, by one or more processors, a first position of a point of a surface based on a distance from a right camera and a left camera of a mobile device being equal to: (R*X0)/(2*tan((Θ1/2)+C)*(x2−x1)), where R is a distance between an optical axis of the left camera and an optical axis of the right camera, X0 is a width of the point in pixels of an image sensor of the left camera, x2 is a number of pixels from the optical axis of an image sensor of the right camera, x1 is a number of pixels from the optical axis of the left camera, Θ1 is a view angle of the left camera and the right camera, and C is a compensation for distortions of the mobile device; rendering on a display, of the mobile device of a user, a virtual reality (VR) interactive item comprising a VR interactive control, wherein the VR interactive item and the VR interactive control are rendered to appear to the user to be on the surface at the first position of the surface; tracking a control indicator controlled by the user by determining a first plurality of positions of the control indicator; activating the VR interactive control in response to detecting the control indicator controlled by the user transgressing a first threshold distance from the VR interactive control; determining a closest position of the first plurality of positions to the point based on the first position; determining the point of the surface to have a second position based on the determined closest position plus a constant for the control indicator; and associating a second threshold distance, less than the first threshold distance, with the point of the surface.