# US 2024/0203414 A1 · Talk back from actions in applications

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0203414 A1
- Document ID: US20240203414A1
- Status: Published application
- Kind: A1
- Application number: 18/393,047
- Filed: 2023-12-21
- Published: 2024-06-20
- Earliest claimed: 2017-05-18
- Assignees: Peloton Interactive, Inc.
- Applicants: Peloton Interactive, Inc.
- Inventors: Mark ROBINSON; Matan Levi; Kiran Bindhu Hemaraj; Rajat Mukherjee
- Web page: https://patentgrep.com/patent/US20240203414A1

## CPC

- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G10L 15/1815: {Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning}
- G10L 15/30: Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
- G10L 13/04: Details of speech synthesis systems, e.g. synthesiser structure or memory management
- G10L 2015/223: {Execution procedure of a spoken command}
- G10L 2015/228: {of application context}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Embodiments of the present invention provide systems, methods, and computer storage media directed to providing talk back automation for applications installed on a mobile device. To do so actions (e.g., talk back features) can be created, via the digital assistant, by recording a series of events that are typically provided by a user of the mobile device when manually invoking the desired action. At a desired state, the user may select an object that represents the output of the application. The recording embodies the action and can be associated with a series of verbal commands that the user would typically announce to the digital assistant when an invocation of the action is desired. In response, the object is verbally communicated to the user via the digital assistant, a different digital assistant, or even another device. Alternatively, the object may be communicated to the same application or another application as input.

## Claims

### Claim 1

A non-transitory computer storage medium storing computer-useable instructions that, when used by at least one processor, cause the at least one processor to:
determine that a received command corresponds to an action associated with an application, the action including a workflow of the application and a talk back object that corresponds to a graphical user interface (GUI) object presented via the application at a specific state of the workflow;
emulate the workflow in response to the determination that the received command corresponds to the action; and
decode, at the specific state of the emulated workflow, the talk back object to provide at least one output that corresponds to the talk back object, the at least one output being different than the GUI object.
