# US 2021/0174084 A1 · Method for human-machine interaction and apparatus for the same > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0174084 A1 - Document ID: US20210174084A1 - Status: Published application - Kind: A1 - Application number: 16/728,827 - Filed: 2019-12-27 - Published: 2021-06-10 - Earliest claimed: 2019-12-09 - Assignees: Electronics and Telecommunications Research Institute - Applicants: Electronics and Telecommunications Research Institute - Inventors: Joo-Haeng LEE - Web page: https://patentgrep.com/patent/US20210174084A1 ## CPC - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)} - G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object - G06F 18/285: {Selection of pattern recognition techniques, e.g. of classifiers in a multi-classifier system} - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts - G06V 10/764: using classification, e.g. of video objects - G06V 10/7788: {the supervisor being a human, e.g. interactive learning with a human teacher} - G06V 10/87: {using selection of the recognition techniques, e.g. of a classifier in a multiple classifier system} - G06V 20/20: in augmented reality scenes - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract Disclosed herein are a method for human-machine interaction and an apparatus for the same. The method includes receiving object identification input for identifying an object related to the task to be dictated to a machine through the I/O interface of a user device that displays a 3D space; displaying an object identification visual interface, corresponding to the object identified within the space recognized by the machine, on the user device in an augmented-reality manner; receiving position identification input for identifying a position in the 3D space related to the task; displaying a position identification visual interface, corresponding to the position identified within the space recognized by the machine, on the user device in an augmented-reality manner; and receiving information related to the result of the task performed through the machine. ## Claims ### Claim 1 A method for human-machine interaction, comprising: receiving object identification input for identifying an object related to a task to be dictated to a machine through an I/O interface of a user device that displays a 3D space; displaying an object identification visual interface, corresponding to the object identified within a space recognized by the machine, on the user device in an augmented-reality manner; receiving position identification input for identifying a position in the 3D space related to the task; displaying a position identification visual interface, corresponding to the position identified within the space recognized by the machine, on the user device in an augmented-reality manner; and receiving information related to a result of the task performed through the machine. ### Claim 2 (depends on 1) The method of claim 1, wherein the object identification visual interface includes posture information displayed within a preset area from a center of the identified object. ### Claim 3 (depends on 2) The method of claim 2, wherein the posture information includes information about an x-axis, a y-axis, and a z-axis corresponding to a 3D posture of the object. ### Claim 4 (depends on 2) The method of claim 2, wherein the position identification visual interface includes peripheral information displayed within a preset area from the identified position and corresponding to a peripheral object located at the position. ### Claim 5 (depends on 4) The method of claim 4, wherein the peripheral information is generated based on properties of the peripheral object. ### Claim 6 (depends on 5) The method of claim 5, wherein the properties of the peripheral object include one or more of a posture and a material of the peripheral object. ### Claim 7 (depends on 4) The method of claim 4, wherein the position is identified differently depending on a type of the identified object. ### Claim 8 (depends on 4) The method of claim 4, wherein the peripheral information is generated differently depending on a type of the identified object. ### Claim 9 (depends on 4) The method of claim 4, further comprising: displaying task visual feedback related to the task, which is identified based on the object and the position, on the user device in an augmented-reality manner. ### Claim 10 (depends on 1) The method of claim 1, wherein the object identification visual interface includes information about a path along which the identified object is able to move within the 3D space. ### Claim 11 A user device, comprising: one or more processors; and executable memory for storing at least one program executed by the one or more processors, wherein the at least one program is configured to: receive object identification input for identifying an object related to a task to be dictated to a machine through an I/O interface that displays a 3D space, display an object identification visual interface corresponding to the object identified within a space recognized by the machine in an augmented-reality manner, receive position identification input for identifying a position in the 3D space related to the task, and display a position identification visual interface corresponding to the position identified within the space recognized by the machine in an augmented-reality manner. ### Claim 12 (depends on 11) The user device of claim 11, wherein the object identification visual interface includes posture information displayed within a preset area from a center of the identified object. ### Claim 13 (depends on 12) The user device of claim 12, wherein the position identification visual interface includes peripheral information displayed within a preset area from the identified position and corresponding to a peripheral object located at the position. ### Claim 14 (depends on 13) The user device of claim 13, wherein the peripheral information is generated based on properties of the peripheral object. ### Claim 15 (depends on 14) The user device of claim 14, wherein the properties of the peripheral object include one or more of a posture and a material of the peripheral object. ### Claim 16 (depends on 13) The user device of claim 13, wherein the at least one program displays task visual feedback related to the task identified based on the object and the position in an augmented-reality manner. ### Claim 17 A machine device, comprising: one or more processors; and executable memory for storing at least one program executed by the one or more processors, wherein the at least one program is configured to: receive object identification input for identifying an object related to a task to be dictated by a user device, generate information about an object identification visual interface corresponding to the object identified within a space in response to the object identification input, receive position identification input for identifying a position within the space from the user device, generate information about a position identification visual interface corresponding to the position identified within the space in response to the position identification input, identify the task based on the object and the position, and perform an operation corresponding to the identified task. ### Claim 18 (depends on 17) The machine device of claim 17, wherein the object identification visual interface includes posture information displayed within a preset area from a center of the identified object. ### Claim 19 (depends on 18) The machine device of claim 18, wherein the position identification visual interface includes peripheral information displayed within a preset area from the identified position and corresponding to a peripheral object located at the position. ### Claim 20 (depends on 19) The machine device of claim 19, wherein the position is identified differently depending on a type of the identified object.