# US 10,768,892 B2 · Device and method for performing functions > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,768,892 B2 - Document ID: US10768892B2 - Status: Granted - Kind: B2 - Application number: 16/127,943 - Filed: 2018-09-11 - Published: 2019-01-10 - Granted: 2020-09-08 - Earliest claimed: 2014-07-31 - Assignees: Samsung Electronics Co., Ltd. - Applicants: Samsung Electronics Co., Ltd. - Inventors: Han-min Bang; Hyok-sung Choi - Web page: https://patentgrep.com/patent/US10768892B2 ## CPC - G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback} - G06F 3/0481: based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance - G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) - G06F 3/04883: for inputting data by handwriting, e.g. gesture or text - G06F 9/451: Execution arrangements for user interfaces - G06F 9/5044: {considering hardware capabilities} - G06F 9/5055: {considering software capabilities, i.e. software resources associated or available to the machine} - H04M 1/724: User interfaces specially adapted for cordless or mobile telephones - G06F 2203/0381: Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer - H04M 1/7243: with interactive means for internal management of messages - H04M 2250/74: with voice recognition means - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08) ## Abstract Provided is a device including a display, an audio inputter, and a controller. The display displays at least one screen page of an application that is being executed. The audio inputter receives a voice command of a user. The controller performs an operation corresponding to the voice command by using screen page transition information for transition between application screen pages corresponding to the voice command, which is obtained from information about user interface (UI) elements included in the application screen pages of the application. Each of the UI elements performs a predetermined function when selected by the user. ## Claims ### Claim 1 A device comprising: a communicator configured to receive service information from at least two Internet of Things (IoT) devices; a memory configured to store the service information received from the at least two IoT devices; and an inputter configured to receive a user input comprising a command phrase that requests performance of at least two predetermined functions; and a controller configured to: analyze the user input and the service information of the at least two IoT devices stored in the memory; determine the at least two predetermined functions based on the analyzing of the user input and the service information of the at least two IoT devices, select at least two IoT devices for performing each of the at least two predetermined functions based on the determined at least two predetermined functions, transmit a first execution command to a first IoT device among the selected at least two IoT devices, and transmit a second execution command to a second IoT device among the selected at least two IoT devices, wherein the first execution command comprises information related to execution of a first predetermined function performed by the first IoT device and information related to transmitting result data of the execution of the first predetermined function to the second IoT device as an input parameter of a second predetermined function performed by the second IoT device, wherein the second execution command comprises information related to execution of the second predetermined function performed by the second IoT device and information related to receiving the result data of the execution the first predetermined function as an input parameter of the second predetermined function, and wherein the service information received from the at least two IoT devices comprises a list of predetermined functions provided by the at least two IoT devices. ### Claim 2 (depends on 1) The device of claim 1, wherein the service information comprises at least one of predetermined functions provided by the at least two IoT devices, device identification information, sensor data collected by a sensor and operation information of the at least two IoT devices. ### Claim 3 (depends on 1) The device of claim 1, wherein the communicator is further configured to provide the user input to a server and receive text data generated by analyzing the user input from the server. ### Claim 4 (depends on 1) The device of claim 1, wherein the user input is at least one of a voice input or a text input. ### Claim 5 (depends on 1) The device of claim 1, wherein the communicator is further configured to receive a periodically updated list of predetermined functions from the at least two IoT devices. ### Claim 6 (depends on 1) The device of claim 1, wherein the communicator is further configured to connect to the at least two IoT devices through a communication link. ### Claim 7 A method of performing a function by a device, the method comprising: receiving service information from at least two Internet of Things (IoT) devices; storing the service information received from the at least two IoT devices into a memory; receiving a user input comprising a command phrase that requests performance of at least two predetermined functions; analyzing the user input and the service information of the at least two IoT devices stored in the memory; determining the at least two predetermined functions based on the analyzing of the user input and the service information of the at least two IoT devices; selecting at least two IoT devices for performing each of the at least two predetermined functions based on the determined at least two predetermined functions; transmitting a first execution command to a first IoT device among the selected at least two IoT devices; and transmitting a second execution command to a second IoT device among the selected at least two IoT devices, wherein the first execution command comprises information related to execution of a first predetermined function performed by the first IoT device and information related to transmitting result data of the execution of the first predetermined function to the second IoT device as an input parameter of a second predetermined function performed by the second IoT device, wherein the second execution command comprises information related to execution of the second predetermined function performed by the second IoT device and information related to receiving the result data of the execution the first predetermined function as an input parameter of the second predetermined function, wherein the service information received from the at least two IoT devices comprises a list of predetermined functions provided by the at least two IoT devices. ### Claim 8 (depends on 7) The method of claim 7, wherein the service information comprises at least one of predetermined functions provided by the at least two IoT devices, device identification information, sensor data collected by a sensor and operation information of the at least two IoT devices. ### Claim 9 (depends on 7) The method of claim 7, further comprising providing the user input to a server and receiving text data generated by analyzing the user input from the server. ### Claim 10 (depends on 7) The method of claim 7, wherein the user input is at least one of a voice input or a text input. ### Claim 11 (depends on 7) The method of claim 7, wherein the receiving of the service information from the at least two IoT devices comprises receiving a periodically updated list of predetermined function from the at least two IoT devices. ### Claim 12 (depends on 7) The method of claim 7, further comprising connecting to the at least two IoT devices through a communication link. ### Claim 13 (depends on 7) A non-transitory computer-readable recording medium having recorded thereon a program for performing the method of claim 7.