# US 2019/0057688 A1 · Passive Word Detection with Sound Effects

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0057688 A1
- Document ID: US20190057688A1
- Status: Published application
- Kind: A1
- Application number: 15/677,518
- Filed: 2017-08-15
- Published: 2019-02-21
- Earliest claimed: 2017-08-15
- Assignees: Sony Interactive Entertainment Inc.
- Applicants: Sony Interactive Entertainment Inc.
- Inventors: Glenn Black
- Web page: https://patentgrep.com/patent/US20190057688A1

## CPC

- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G06F 3/162: {Interface to dedicated audio devices, e.g. audio drivers, interface to CODECs}
- G06F 3/165: {Management of the audio stream, e.g. setting of volume, audio stream path}
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G06F 16/24: Querying
- G06F 16/634: {Query by example, e.g. query by humming}
- G06N 20/00: Machine learning
- G10L 15/063: {Training}
- G10L 15/26: Speech to text systems (G10L15/08 takes precedence)
- G10L 17/00: Speaker identification or verification techniques
- G10L 2015/088: {Word spotting}
- G10L 2015/225: {Feedback of the input speech}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A system, method, and processor-readable storage medium for providing sound effects are disclosed. An example method comprises receiving an audio signal associated with a speech of a user, performing speech recognition on the audio signal to identify one or more recognized words, identifying at least one trigger word among the one or more recognized words, and providing to the user at least one sound effect associated with the at least one trigger word. The speech recognition can be implemented by a trained machine-learning system. The sound effects can be provided to the user and optionally to other users, for example, via a network gaming environment. The sound effects can also be adjusted or selected based on a conversational context, user identification, and/or audio characteristics of the received audio signal. The system for providing sound effects can include a smart speaker, mobile device, game console, television device, and the like.

## Claims

### Claim 1

A method for providing sound effects, the method comprising:
receiving an audio signal associated with a speech of a user;
performing speech recognition on the audio signal to identify one or more recognized words;
identifying at least one trigger word among the one or more recognized words;
determining at least one audio characteristic of the audio signal;
selecting at least one sound effect associated with the at least one trigger word based on the determined at least one audio characteristic; and
providing to the user the at least one sound effect associated with the at least one trigger word.

### Claim 2 (depends on 1)

The method of claim 1, in which the at least one trigger word is predetermined by the user.

### Claim 3 (depends on 1)

The method of claim 1, in which the at least one trigger word is predetermined by a system administrator.

### Claim 4 (depends on 1)

The method of claim 1, in which the identification of the at least one trigger word among the one or more recognized words comprises:
for each of the recognized words, dynamically searching a database for the at least one trigger word.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
determining a conversational context based on the one or more recognized words, in which the identification of the at least one trigger word among the one or more recognized words further depends on the conversational context.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
maintaining a machine-learning system configured to perform the speech recognition on the audio signal to identify the one or more recognized words and identify the at least one trigger word among the one or more recognized words;
obtaining historical user data associated with the user or comparable users; and
training the machine-learning system based on the historical user data to improve accuracy of the speech recognition and the identification of the at least one trigger word.

### Claim 7 (depends on 6)

The method of claim 6, in which the machine-learning system is further trained based on the historical user data to identify the user based on the one or more recognized words.

### Claim 8 (depends on 6)

The method of claim 6, in which the machine-learning system is further trained to determine a conversational context based on the one or more recognized words.

### Claim 9 (depends on 1)

The method of claim 1, in which the at least one sound effect associated with the at least one trigger word is provided to the user less than one second after receipt of the audio signal.

### Claim 10 (depends on 1)

The method of claim 1, further comprising:
performing voice recognition on the audio signal to determine identity of the user, in which the user is one of a plurality of users;
accessing predetermined settings associated with the identity of the user, in which the identification of the at least one trigger word among the one or more recognized words is further based on the settings associated with the identity of the user; and
providing to the user at least one sound effect associated with the at least one trigger word in a manner prescribed by the settings associated with the identity of the user.

### Claim 11 (depends on 1)

The method of claim 1, in which the audio signal associated with the user is received within a multiplayer computerized game environment, in which:
the user and at least one second user are players within the multiplayer computerized game environment,
the audio signal associated with the user is audible to the at least one second user via the multiplayer computerized game environment; and
the at least one sound effect associated with the at least one trigger word is provided to the user and to the at least one second user of the multiplayer computerized game environment.

### Claim 12 (depends on 11)

The method of claim 11, further comprising:
substituting a portion of the audio signal, which is associated with the at least one trigger word, with the at least one sound effect such that the at least one second user of the multiplayer computerized game environment is provided with a modified audio signal, the modified audio signal comprising a remaining part of the audio signal and the at least one sound effect appearing instead of the portion of the audio signal, which is associated with the at least one trigger word.

### Claim 13 (depends on 1)

The method of claim 1, further comprising:
adjusting at least one audio parameter of the at least one sound effect in accordance with the at least one audio characteristic of the audio signal.

### Claim 14 (depends on 13)

The method of claim 13, in which the at least one audio characteristic of the audio signal includes a tonality or a loudness of the speech of the user.

### Claim 15 (depends on 13)

The method of claim 13, in which the at least one audio parameter of the sound effect includes a frequency or an amplitude.

### Claim 16 (depends on 1)

The method of claim 1,
wherein the identifying of the at least one trigger word among the one or more recognized words further depends on the at least one audio characteristic of the audio signal.

### Claim 17 (depends on 1)

The method of claim 1, further comprising:
causing at least one electronic device of a second user to play the at least one sound effect associated with the at least one trigger word.

### Claim 18 (depends on 1)

The method of claim 1, in which the at least one sound effect further includes at least one visual effect.

### Claim 19

A system for providing sound effects, the system comprising at least one processor and a memory storing processor-executable codes, in which the at least one processor is configured to implement the following operations upon executing the processor-executable codes:
receiving an audio signal associated with a speech of a user;
performing speech recognition on the audio signal to identify one or more recognized words;
identifying at least one trigger word among the one or more recognized words;
determining at least one audio characteristic of the audio signal;
selecting at least one sound effect associated with the at least one trigger word based on the determined at least one audio characteristic; and
providing to the user the at least one sound effect associated with the at least one trigger word.

### Claim 20

A non-transitory processor-readable medium having instructions stored thereon, which when executed by one or more processors, cause the one or more processors to implement a method for providing sound effects, the method comprising:
receiving an audio signal associated with a speech of a user;
performing speech recognition on the audio signal to identify one or more recognized words;
identifying at least one trigger word among the one or more recognized words;
determining at least one audio characteristic of the audio signal;
selecting at least one sound effect associated with the at least one trigger word based on the determined at least one audio characteristic; and
providing to the user the at least one sound effect associated with the at least one trigger word.
