# US 2023/0274205 A1 · Multi computational agent performance of tasks

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0274205 A1
- Document ID: US20230274205A1
- Status: Published application
- Kind: A1
- Application number: 18/135,579
- Filed: 2023-04-17
- Published: 2023-08-31
- Earliest claimed: 2016-10-03
- Assignees: Google Inc.
- Applicants: Google LLC
- Inventors: Robert Stets; Valerie Nygaard; Bogdan Caprita; Bradley M. Abrams; Jason Brant Douglas
- Web page: https://patentgrep.com/patent/US20230274205A1

## CPC

- G10L 13/00: Speech synthesis; Text to speech systems
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G06F 9/5027: {the resource being a machine, e.g. CPUs, Servers, Terminals}
- G06F 16/951: Indexing; Web crawling techniques
- G06F 16/953: Querying, e.g. by the use of web search engines
- G06Q 10/063112: {Skill-based matching of a person or a group to a task}
- G06Q 30/016: After-sales
- G06Q 30/0282: Rating or review of business operators or products
- G06Q 30/0623: {by investigating goods or services}
- G06Q 30/0631: {Recommending goods or services}
- G10L 13/04: Details of speech synthesis systems, e.g. synthesiser structure or memory management
- G10L 15/08: Speech classification or search
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L 15/26: Speech to text systems (G10L15/08 takes precedence)
- G06F 2209/5017: Task decomposition
- G10L 13/08: Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
- G10L 2015/088: {Word spotting}
- G10L 2015/223: {Execution procedure of a spoken command}
- Y02D 10/00: Energy efficient computing, e.g. low power processors, power management or thermal management
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- Y02D: CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE

## Abstract

An example method includes receiving, by one or more processors, a representation of an utterance spoken at a computing device; identifying, by a first computational agent from a plurality of computational agents and based on the utterance, a multi-element task to be performed, wherein the plurality of computational agents includes one or more first party computational agents and a plurality of third-party computational agents; and performing, by the first computational agent, a first sub-set of elements of the multi-element task, wherein performing the first sub-set of elements comprises selecting a second computational agent from the plurality of computational agents to perform a second sub-set of elements of the multi-element task.

## Claims

### Claim 1

A method implemented by one or more processors comprising:
receiving a spoken utterance from a user of a computing device;
determining, based on processing the spoken utterance, that the spoken utterance includes a task to be performed;
determining, based on processing the spoken utterance, whether the spoken utterance includes a given trigger word or phrase for a given computational agent from among a plurality of disparate computational agents, the plurality of disparate computational agents including a first party computational agent and a plurality of third party computational agents;
in response to determining that the given trigger word or phrase is associated with the first party computational agent:
selecting the first party computational agent to perform the task; and
causing the first party computational agent to perform the task; and
in response to determining that the given trigger word or phrase is associated with a given third party computational agent from among the plurality of third party computational agents:
selecting the given third party computational agent to perform the task; and
causing the given third party computational agent to perform the task.

### Claim 2 (depends on 1)

The method of claim 1, wherein each of the plurality of disparate computing agents are associated with a corresponding trigger word or phrase.

### Claim 3 (depends on 2)

The method of claim 2, wherein determining whether the spoken utterance includes the given trigger word or phrase for the given computational agent from among the plurality of disparate computational agents comprises:
accessing an index that includes the corresponding trigger word or phrase for each of the plurality of disparate computing agents; and
determining, based on accessing the index, that the spoken utterance includes the given trigger word or phrase for the given computational agent

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
in response to determining that the given trigger word or phrase is not associated with the first party computational agent:
determining that the given trigger word or phrase is associated with the given third party computational agent from among the plurality of third party computational agents.

### Claim 5 (depends on 4)

The method of claim 4, further comprising:
in response to determining that the given trigger word or phrase is not associated with the first party computational agent and in response to determining that the given trigger word or phrase is not associated with any given third party computational agent from among the plurality of third party computational agents:
selecting, based on corresponding capability levels associated with each of the plurality of computational agents, the given computational agent from among a plurality of disparate computational agents; and
causing the given computational agent to perform the task.

### Claim 6 (depends on 5)

The method of claim 5, wherein the given computational agent is the first party computational agent.

### Claim 7 (depends on 5)

The method of claim 5, wherein the given computational agent is one of the plurality of third party computational agents.

### Claim 8 (depends on 1)

The method of claim 1, further comprising:
subsequent to causing the first party computational agent to perform the task or subsequent to causing the given third party computational agent to perform the task:
causing a result of performance of the task to be provided for presentation to the user of the computing device.

### Claim 9

A computing device comprising:
one or more hardware processors; and
memory comprising instructions that, when executed by the one or more hardware processors, cause the one or more hardware processors to:
receive a spoken utterance from a user of a computing device;
determine, based on processing the spoken utterance, that the spoken utterance includes a task to be performed;
determine, based on processing the spoken utterance, whether the spoken utterance includes a given trigger word or phrase for a given computational agent and from among a plurality of disparate computational agents, the plurality of disparate computational agents including a first party computational agent and a plurality of third party computational agents;
in response to determining that the given trigger word or phrase is associated with the first party computational agent:
select the first party computational agent to perform the task; and
cause the first party computational agent to perform the task; and
in response to determining that the given trigger word or phrase is associated with a given third party computational agent from among the plurality of third party computational agents:
select the given third party computational agent to perform the task; and
cause the given third party computational agent to perform the task.

### Claim 10 (depends on 9)

The computing device of claim 9, wherein each of the plurality of disparate computing agents are associated with a corresponding trigger word or phrase.

### Claim 11 (depends on 10)

The computing device of claim 10, wherein the instructions to determine whether the spoken utterance includes the given trigger word or phrase for the given computational agent from among the plurality of disparate computational agents comprise instructions to:
access an index that includes the corresponding trigger word or phrase for each of the plurality of disparate computing agents; and
determine, based on accessing the index, that the spoken utterance includes the given trigger word or phrase for the given computational agent

### Claim 12 (depends on 9)

The computing device of claim 9, wherein the instructions further comprise instructions to:
in response to determining that the given trigger word or phrase is not associated with the first party computational agent:
determine that the given trigger word or phrase is associated with the given third party computational agent from among the plurality of third party computational agents.

### Claim 13 (depends on 12)

The computing device of claim 12, wherein the instructions further comprise instructions to:
in response to determining that the given trigger word or phrase is not associated with the first party computational agent and in response to determining that the given trigger word or phrase is not associated with any given third party computational agent from among the plurality of third party computational agents:
select, based on corresponding capability levels associated with each of the plurality of computational agents, the given computational agent from among a plurality of disparate computational agents; and
cause the given computational agent to perform the task.

### Claim 14 (depends on 13)

The computing device of claim 13, wherein the given computational agent is the first party computational agent.

### Claim 15 (depends on 13)

The computing device of claim 13, wherein the given computational agent is one of the plurality of third party computational agents.

### Claim 16 (depends on 1)

The method of claim 1, wherein the instructions further comprise instructions to:
subsequent to causing the first party computational agent to perform the task or subsequent to causing the given third party computational agent to perform the task:
cause a result of performance of the task to be provided for presentation to the user of the computing device.

### Claim 17

A non-transitory computer-readable storage medium storing instructions that, when executed by one or more hardware processors, cause the one or more hardware processors to perform operations, the operations comprising:
receiving a spoken utterance from a user of a computing device;
determining, based on processing the spoken utterance, that the spoken utterance includes a task to be performed;
determining, based on processing the spoken utterance, whether the spoken utterance includes a given trigger word or phrase for a given computational agent and from among a plurality of disparate computational agents, the plurality of disparate computational agents including a first party computational agent and a plurality of third party computational agents;
in response to determining that the given trigger word or phrase is associated with the first party computational agent:
selecting the first party computational agent to perform the task; and
causing the first party computational agent to perform the task; and
in response to determining that the given trigger word or phrase is associated with a given third party computational agent from among the plurality of third party computational agents:
selecting the given third party computational agent to perform the task; and
causing the given third party computational agent to perform the task.
