# US 2020/0266789 A1 · Ambient sound activated device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0266789 A1
- Document ID: US20200266789A1
- Status: Published application
- Kind: A1
- Application number: 16/867,732
- Filed: 2020-05-06
- Published: 2020-08-20
- Earliest claimed: 2018-07-24
- Assignees: Sony Interactive Entertainment Inc.
- Applicants: Sony Interactive Entertainment Inc.
- Inventors: Takeo Matsukawa
- Web page: https://patentgrep.com/patent/US20200266789A1

## CPC

- H03G 3/32: the control being dependent upon ambient noise level or sound level
- H03G 3/02: Manually-operated control {(H03G3/001 and H03G3/002 take precedence)}
- H03G 5/025: {Equalizers; Volume or gain control in limited frequency bands}
- H03G 5/165: {Equalizers; Volume or gain control in limited frequency bands}
- H04R 1/1091: {Details not provided for in groups H04R1/1008 - H04R1/1083}
- H04R 5/033: Headphones for stereophonic communication {(details thereof, e.g. relating to batteries, cables or control elements H04R1/10)}
- H04S 3/004: {For headphones}
- H04R 3/005: {for combining the signals of two or more microphones (specially adapted for hearing aids H04R25/407)}
- H04R 5/04: Circuit arrangements, {e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments (combinations of amplifiers H03F3/68; stereophonic systems H04S)}
- H04R 2205/022: Plurality of transducers corresponding to a plurality of sound channels in each earpiece of headphones or in a single enclosure
- H04R 2420/01: Input selection or mixing for amplifiers or loudspeakers
- H04R 2430/03: Synergistic effects of band splitting and sub-band processing
- H04S 7/304: {For headphones}
- H04S 2400/01: Multi-channel, i.e. more than two input channels, sound reproduction with two speakers wherein the multi-channel information is substantially preserved
- H04S 2400/11: Positioning of individual sound objects, e.g. moving airplane, within a sound field (H04S2420/13 takes precedence)
- H04S 2420/07: Synergistic effects of band splitting and sub-band processing
- H03G: CONTROL OF AMPLIFICATION
- H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K)
- H04S: STEREOPHONIC SYSTEMS

## Abstract

In device having at least one microphone and one or more speakers, environmental sound may be recorded using the microphone, classified and mixed with source media sound to produce a mixed sound depending on the classification. The mixed sound may then be played over the one or more speakers.

## Claims

### Claim 1

A method for sound detection and generation, the method comprising:
recording environmental sound using one or more microphones;
classifying a source of the recorded environmental sound;
determining whether to mix the recorded environmental sound based on the classification of the source;
mixing the recorded environmental sound with source media sound to produce a mixed sound; and
playing the mixed sound over one or more speakers.

### Claim 2 (depends on 1)

The method of claim 1 wherein the one or more microphones includes two or more microphones and the one or more speakers include two or more speakers.

### Claim 3 (depends on 2)

The method of claim 2 wherein mixing the environmental sound with source media sound includes generating a multi-channel sound that includes the ambient room sounds.

### Claim 4 (depends on 3)

The method of claim 3 wherein the multi-channel sound includes five sound channels.

### Claim 5 (depends on 4)

The method of claim 4 wherein the two or more speakers include five or more speakers.

### Claim 6 (depends on 1)

The method of claim 1 wherein the source media sound includes sound generated by a television system, home theater system, stereo system, digital video recorder, video cassette recorder, video game console, portable music or video player or handheld video game device.

### Claim 7 (depends on 1)

The method of claim 1 wherein the at least one microphone is configured to detect ambient noise.

### Claim 8 (depends on 1)

The method of claim 1, further comprising performing noise reduction on the environmental sound after it has been recorded and before mixing it with the source media sound.

### Claim 9 (depends on 1)

The method of claim 1 wherein classifying the source of the recorded environmental sounds includes converting the recorded environmental sound into digital data representing the recorded environmental sound.

### Claim 10 (depends on 9)

The method of claim 9 wherein classifying the source of the recorded environmental sounds includes providing the digital data representing the recorded environmental sound to a neural network trained to predict a class for the source of the recorded environmental sound wherein the classes are sound source types.

### Claim 11 (depends on 10)

The method of claim 10 wherein classifying the source of the recorded environmental sound further includes using coherence based sound-detection technology to calculate the direction-of-arrival of the environmental sound recorded by one or more microphones and wherein mixing retains the direction-of-arrival of the environmental sound.

### Claim 12 (depends on 10)

The method of claim 10 further comprises training the neural network to classify the source of the recorded environmental sound based on a user input.

### Claim 13 (depends on 1)

The method of claim 1 wherein determining whether to mix the recorded environmental sound comprises looking up in a look-up table a user preference for mixing in, the classification of the source of the recorded environmental sound.

### Claim 14 (depends on 1)

The method of claim 1 wherein determining whether to mix the recorded environmental sound comprises weighting a user input and the classification of the source of the recorded environmental sound.

### Claim 15 (depends on 13)

The method of claim 13 wherein determining whether to mix the recorded environmental sound includes providing the classification of the source of the recorded environmental sound and a user input to an untrained neural network and training the neural network to weight classifications based on the user input and wherein the recorded environmental sound is mixed in based on the applied weight.

### Claim 16 (depends on 1)

The method of claim 1 further comprises applying filters to the recorded environmental sound based on the classification of the source of the recorded environmental sound before mixing.

### Claim 17 (depends on 1)

The method of claim 1 wherein the mixing the recorded environmental sound further comprises adding a sound cue based on the classification of the source of the recorded environmental sound.

### Claim 18

An audio system, comprising:
one or more microphones and one or more speakers,
a processor coupled to the one or more microphones and the one or more speakers;
a memory coupled to the processor;
a set of processor-executable instructions embodied in the memory, wherein the instructions are configured, when executed by the processor to implement a method for sound detection and generation, wherein the method comprises:
recording environmental sound using the one or more microphones;
classifying a source of the recorded environmental sound;
determining to mix the recorded environmental sound based on the classification of the source;
mixing the environmental sound with source media sound to produce a mixed sound; and
playing the mixed sound over the one or more speakers.

### Claim 19 (depends on 18)

The system of claim 18 wherein mixing the environmental sound with source media sound include multi-channel sound that includes the ambient room sounds.

### Claim 20 (depends on 18)

The system of claim 18 wherein the multi-channel sound includes five sound channels.

### Claim 21 (depends on 18)

The system of claim 18 wherein the processor is located on a console device, the system further comprising a wireless transceiver on the console device coupled to the processor, a wireless transmitter coupled to the one or more microphones, and a wireless receiver coupled to the one or more speakers.

### Claim 22 (depends on 18)

The method of claim 18 wherein classifying the source of the environmental sounds includes converting the environmental sound into digital data representing the environmental sound.

### Claim 23 (depends on 22)

The system of claim 22 wherein classifying the source of the environmental sounds includes providing the digital data representing the environmental sound to a neural network trained to predict a class for the source of the environmental sound wherein the classes are sound source types.

### Claim 24 (depends on 23)

The system of claim 23 wherein classifying the source of the environmental sound further includes using coherence based sound-detection technology to calculate the direction-of-arrival of the environmental sound recorded by the at least one microphone and wherein mixing retains the direction-of-arrival of the environmental sound.

### Claim 25 (depends on 23)

The system of claim 23, further comprising training the neural network to classify the source of the environmental sound based on a user input.

### Claim 26 (depends on 18)

The system of claim 18 wherein determining whether to mix the environmental sound comprises looking up in a look-up table a user preference for mixing in, the classification of the source of the environmental sound.

### Claim 27 (depends on 18)

The system of claim 18 wherein determining whether to mix the environmental sound comprises weighting a user input and the classification of the source of the environmental sound.

### Claim 28 (depends on 18)

The system of claim 18 where determining whether to mix the environmental sound includes providing the classification of the source of the environmental sound and a user input to an untrained neural network and training the neural network to weight classifications based on the user input and wherein the environmental sound is mixed in based on the applied weight.

### Claim 29 (depends on 18)

The system of claim 18 further comprises applying filters to the environmental sound based on the classification of the source of the environmental sound before mixing.
