# US 11,443,745 B2 · Apparatus control device, apparatus control system, apparatus control method, and apparatus control program > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,443,745 B2 - Document ID: US11443745B2 - Status: Granted - Kind: B2 - Application number: 17/076,532 - Filed: 2020-10-21 - Published: 2021-02-04 - Granted: 2022-09-13 - Earliest claimed: 2018-06-15 - Assignees: Mitsubishi Electric Corporation - Applicants: Mitsubishi Electric Corporation - Inventors: Masato Hirai; Kenshiro Kitamura; Miho Ishikawa; Daisuke Iizawa - Web page: https://patentgrep.com/patent/US11443745B2 ## CPC - G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue - G06F 3/16: Sound input; Sound output (speech processing G10L) - G06F 13/00: Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units (interface circuits for specific input/output devices G06F3/00 {; multiprogram control therefor G06F9/46}; multiprocessor systems G06F15/16 ) - G10L 15/02: Feature extraction for speech recognition; Selection of recognition unit - G10L 15/083: {Recognition networks (G10L15/142, G10L15/16 take precedence)} - G10L 15/1815: {Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning} - H04Q 9/00: Arrangements in telecontrol or telemetry systems for selectively calling a substation from a main station, in which substation desired apparatus is selected for applying a control signal thereto or for obtaining measured values therefrom - G10L 2015/223: {Execution procedure of a spoken command} - G10L 2015/228: {of application context} - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04Q: SELECTING (switches, relays, selectors H01H; wireless communication networks H04W) ## Abstract Included are: an apparatus function information acquiring unit for acquiring apparatus function information in which a target apparatus and one or more target functions to be executed by the target apparatus, which are determined on the basis of uttered speech, are associated with each other; a procedure determining unit for determining one or more manual operations for executing the one or more target functions and an order of the one or more manual operations on the basis of the apparatus function information acquired by the apparatus function information acquiring unit; and an operation command transmission controlling unit for sequentially transmitting, to the target apparatus, operation commands for outputting operation response output control information corresponding to each of the one or more manual operations in accordance with the order of the one or more manual operations determined by the procedure determining unit. ## Claims ### Claim 1 An apparatus control device for controlling an apparatus on a basis of uttered speech, the apparatus control device comprising: processing circuitry performing a process of: acquiring apparatus function information in which a target apparatus and one or more target functions to be executed by the target apparatus, which are determined on a basis of the uttered speech, are associated with each other; determining one or more manual operations for executing the one or more target functions and an order of the one or more manual operations on a basis of the apparatus function information acquired; and sequentially transmitting, to the target apparatus, operation commands for outputting operation response output control information corresponding to each of the one or more manual operations for each of the one or more manual operations in accordance with the order of the one or more manual operations determined. ### Claim 2 (depends on 1) The apparatus control device according to claim 1, wherein, in a case where the one or more manual operations include multiple manual operations having mutually a same content and multiple manual operations having mutually different content, the process sets mutually different transmission intervals for a transmission interval of operation commands at which the operation commands corresponding to the multiple manual operations having mutually the same content are transmitted successively and a transmission interval of operation commands at which the operation commands corresponding to the multiple manual operations having the mutually different content are transmitted successively. ### Claim 3 (depends on 2) The apparatus control device according to claim 2, wherein the transmission interval of the operation commands at which the operation commands corresponding to the multiple manual operations having mutually the same content are transmitted successively is shorter than the transmission interval of the operation commands at which the operation commands corresponding to the multiple manual operations having the mutually different content are transmitted successively. ### Claim 4 (depends on 1) The apparatus control device according to claim 1, the process further comprising: acquiring information of a status of the target apparatus, wherein the process determines the one or more operations for executing the one or more target functions and the order of the one or more operations on a basis of the information of the status acquired and the apparatus function information. ### Claim 5 (depends on 1) The apparatus control device according to claim 1, the process further comprising: a procedure data in which, for each of the apparatus, one or more functions of the apparatus and one or more manual operations for executing the one or more functions are associated with each other. ### Claim 6 (depends on 1) The apparatus control device according to claim 1, the process further comprising: acquiring the uttered speech; recognizing content of utterance on a basis of the uttered speech acquired; and determining the target apparatus specified by the uttered speech and the one or more target functions to be executed by the target apparatus on a basis of the content of utterance recognized and outputting the apparatus function information in which the target apparatus and the one or more target functions are associated with each other, wherein the process acquires the apparatus function information output. ### Claim 7 (depends on 1) An apparatus comprising: the apparatus control device according to claim 1; and second processing circuitry performing a second process of: acquiring the operation command transmitted and outputting the operation response output control information each time the operation command is acquired. ### Claim 8 (depends on 7) The apparatus according to claim 7, the second process further comprising: outputting sound in accordance with the operation response output control information output. ### Claim 9 (depends on 8) The apparatus according to claim 8, wherein the operation command includes operation information indicating the one or more operations, and the second process outputs the operation response output control information for changing sound to be output depending on the operation information. ### Claim 10 (depends on 7) The apparatus according to claim 7, the second process further comprising: displaying information based on the operation response output control information output. ### Claim 11 (depends on 10) The apparatus according to claim 10, wherein the operation command includes operation information indicating the one or more operations, and the second process outputs the operation response output control information for changing information to be displayed depending on the operation information. ### Claim 12 An apparatus control system comprising: an apparatus control device comprising: processing circuitry performing a process of: acquiring apparatus function information in which a target apparatus and one or more target functions to be executed by the target apparatus, which are determined on a basis of uttered speech, are associated with each other; and determining one or more manual operations for executing the one or more target functions and an order of the one or more manual operations on a basis of the apparatus function information acquired; and an apparatus that is controlled, the apparatus comprising: second circuitry performing a second process of: sequentially outputting operation commands for outputting operation response output control information corresponding to each of the one or more manual operations for each of the one or more manual operations in accordance with an order of the one or more manual operations determined; and acquiring the operation commands output and outputting the operation response output control information each time one of the operation commands is acquired. ### Claim 13 An apparatus control method for controlling an apparatus on a basis of uttered speech, the apparatus control method comprising: acquiring apparatus function information in which a target apparatus and one or more target functions to be executed by the target apparatus, which are determined on a basis of the uttered speech, are associated with each other; determining one or more manual operations for executing the one or more target functions and an order of the one or more manual operations on a basis of the apparatus function information acquired; and sequentially transmitting operation commands for outputting operation response output control information corresponding to each of the one or more manual operations for each of the one or more manual operations in accordance with the order of the one or more manual operations determined. ### Claim 14 A tangible non-transitory storage medium storing an apparatus control program for controlling an apparatus on a basis of uttered speech, the apparatus control program for causing a computer to execute a process of: acquiring apparatus function information in which a target apparatus and one or more target functions to be executed by the target apparatus, which are determined on a basis of the uttered speech, are associated with each other; determining one or more manual operations for executing the one or more target functions and an order of the one or more manual operations on a basis of the apparatus function information acquired; and sequentially transmitting, to the target apparatus, operation commands for outputting operation response output control information corresponding to each of the one or more manual operations for each of the one or more manual operations in accordance with the order of the one or more manual operations determined.