# US 2021/0082084 A1 · Virtual object translation > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0082084 A1 - Document ID: US20210082084A1 - Status: Published application - Kind: A1 - Application number: 16/573,907 - Filed: 2019-09-17 - Published: 2021-03-18 - Earliest claimed: 2019-09-17 - Assignees: Adobe Inc. - Applicants: Adobe Inc. - Inventors: Michael Baldus - Web page: https://patentgrep.com/patent/US20210082084A1 ## CPC - G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object - G06T 3/20: Linear translation of whole images or parts thereof, e.g. panning - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) - G06F 3/04845: for image manipulation, e.g. dragging, rotation, expansion or change of colour - G06F 3/0486: Drag-and-drop - G06F 3/04883: for inputting data by handwriting, e.g. gesture or text - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts - G06F 3/03543: {Mice or pucks (G06F3/03541 takes precedence)} - G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback} - G06T 2200/24: involving graphical user interfaces [GUIs] - G06T 2219/2004: Aligning objects, relative positioning of parts - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract Techniques are disclosed for translating a virtual object within a displayed scene in a digital environment in a realistic and intuitive manner. The realistic and intuitive translation is achieved due to new techniques that are used for determining locations of the virtual object in the digital scene as the virtual object is being translated responsive to user input. In some embodiments, a control plane is constructed based upon a point of interaction with a virtual object and an initial surface in the scene on which the virtual object is located. The constructed control plane is then used to determine locations in the scene for locating and displaying the virtual object as the virtual object is translated from its start position to an end position in the scene. ## Claims ### Claim 1 A method comprising: receiving, by a computing system, information indicating that a user is translating a virtual object from an initial location in a scene displayed on an output device; identifying, by the computing system, a point on the virtual object based upon the received information; identifying, by the computing system, an initial surface in the scene on which the virtual object is located; constructing, by the computing system, a control plane based upon the identified point on the virtual object and the identified initial surface; using the constructed control plane to determine, by the computing system, a new location for positioning the virtual object in the scene, where the new location is different from the initial location; and updating, by the computing system, the scene displayed on the output device by displaying the virtual object in the determined new location. ### Claim 2 (depends on 1) The method of claim 1, wherein constructing the control plane comprises generating the control plane such that the control plane is parallel to the initial surface and passes through the identified point on the virtual object. ### Claim 3 (depends on 2) The method of claim 2, wherein constructing the control plane further comprises computing a normal to the initial surface for use in generating the control plane to be parallel to the initial surface. ### Claim 4 (depends on 1) The method of claim 1, wherein determining the new location for positioning the virtual object comprises: detecting, by the computing system, a motion event corresponding to user input; based on the motion event, identifying, by the computing system, a point of interaction on a screen of the computing system; generating, by the computing system, a pick ray originating from a camera location and passing through both the point of interaction on the screen and the control plane; identifying, by the computing system, a point on the control plane where the pick ray intersects the control plane; and setting, by the computing system, the new location for positioning the virtual object based on the point on the control plane where the pick ray intersects the control plane. ### Claim 5 (depends on 4) The method of claim 4, further comprising: determining, by the computing system, an offset between the identified point on the virtual object and the new location for positioning the virtual object; and determining, by the computing system, a translated location for the virtual object based on a sum of the identified point on the virtual object and the offset, such that a same region of the virtual object coincides with the translated location as coincided with the identified point on the virtual object. ### Claim 6 (depends on 1) The method of claim 1, wherein identifying the initial surface on which the virtual object is located comprises identifying a horizontal plane using a combination of real-world scene information gathered by a camera of the computing system and acceleration information gathered by a sensor of the computing system. ### Claim 7 (depends on 1) The method of claim 1, wherein: the scene is displayed via a screen of the output device; and identifying the point on the virtual object based on the received information comprises: determining, by the computing system, a point on the screen used by the user to interact with the virtual object; casting, by the computing system, a pick ray into the scene, where the pick ray originates from a camera location and passes through the point on the screen; determining, by the computing system, a point on the virtual object where the pick ray intersects the virtual object; and setting, by the computing system, the point on the virtual object where the pick ray intersects the virtual object as the identified point on the virtual object. ### Claim 8 (depends on 1) The method of claim 1, wherein the information is generated based on one or more of user interaction with a touchscreen, user interaction with a mouse, and/or a voice command. ### Claim 9 (depends on 1) The method of claim 1, wherein the displayed scene is an Augmented Reality (AR) scene wherein the virtual object is displayed in the scene along with real-world content captured using a camera. ### Claim 10 A computing system comprising: a memory; a processor; the memory including instructions, which when executed by the processor, cause the processor to perform steps comprising: receiving information indicating that a user is translating a virtual object from an initial location in a scene displayed on an output device; identifying a point on the virtual object based upon the received information; identifying an initial surface in the scene on which the virtual object is located; constructing a control plane based upon the identified point on the virtual object and the identified initial surface; and using the constructed control plane to determine a new location for positioning the virtual object in the scene, where the new location is different from the initial location. ### Claim 11 (depends on 10) The computing system of claim 10, wherein constructing the control plane comprises generating the control plane such that the control plane is parallel to the initial surface and passes through the identified point on the virtual object. ### Claim 12 (depends on 11) The computing system of claim 11, wherein constructing the control plane further comprises computing a normal to the initial surface for use in generating the control plane to be parallel to the initial surface. ### Claim 13 (depends on 10) The computing system of claim 10, wherein determining the new location for positioning the virtual object comprises: detecting a motion event corresponding to user input; based on the motion event, identifying a point of interaction on a screen of the computing system; generating a pick ray originating from a camera location and passing through both the point of interaction on the screen of the computing system and the control plane; identifying a point on the control plane where the pick ray intersects the control plane; and setting the new location for positioning the virtual object based on the point on the control plane where the pick ray intersects the control plane. ### Claim 14 (depends on 13) The computing system of claim 13, the steps further comprising: determining an offset between the identified point on the virtual object and the point of interaction on the screen; and determining a translated location for the virtual object based on a sum of the identified point on the virtual object and the offset, such that a same region of the virtual object coincides with the translated location as coincided with the identified point on the virtual object. ### Claim 15 (depends on 10) The computing system of claim 10, wherein identifying the initial surface on which the virtual object is located comprises identifying a horizontal plane using a combination of real-world scene information gathered by a camera of the computing system and acceleration information gathered by a sensor of the computing system. ### Claim 16 (depends on 10) The computing system of claim 10, wherein the steps further comprise updating the scene displayed on the output device by displaying the virtual object in the determined new location. ### Claim 17 A non-transitory computer-readable medium having instructions stored thereon, the instructions executable by a processing device to perform operations comprising: receive information indicating that a user is translating a virtual object from an initial location in a scene displayed on an output device; identify a point on the virtual object based upon the received information; identify an initial surface in the scene on which the virtual object is located; construct a control plane based upon the identified point on the virtual object and the identified initial surface by generating the control plane such that the control plane is parallel to the initial surface and passes through the identified point on the virtual object; use the constructed control plane to determine a new location for positioning the virtual object in the scene, where the new location is different from the initial location; and update the scene displayed on the output device by displaying the virtual object in the determined new location. ### Claim 18 (depends on 17) The medium of claim 17, wherein determining the new location for positioning the virtual object comprises: detecting a motion event corresponding to user input; based on the motion event, identifying a point of interaction on a screen of the computing system; generating a pick ray originating from a camera location and passing through both the point of interaction on the screen and the control plane; identifying a point on the control plane where the pick ray intersects the control plane; and setting the new location for positioning the virtual object based on the point on the control plane where the pick ray intersects the control plane. ### Claim 19 (depends on 18) The medium of claim 18, the operations further comprising: determining an offset between the identified point on the virtual object and the point of interaction on the screen; and determining a translated location for the virtual object based on a sum of the identified point on the virtual object and the offset, such that a same region of the virtual object coincides with the translated location as coincided with the identified point on the virtual object. ### Claim 20 (depends on 17) The medium of claim 17, wherein constructing the control plane further comprises computing a normal to the initial surface for use in generating the control plane to be parallel to the initial surface.