# US 2023/0403426 A1 · System and method for incorporating audio into audiovisual content

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0403426 A1
- Document ID: US20230403426A1
- Status: Published application
- Kind: A1
- Application number: 17/746,873
- Filed: 2022-05-17
- Published: 2023-12-14
- Earliest claimed: 2022-05-17
- Assignees: Meta Platforms, Inc.
- Applicants: Meta Platforms, Inc.
- Inventors: Andrew Lovitt; Salvael Ortega Estrada
- Web page: https://patentgrep.com/patent/US20230403426A1

## CPC

- G11B 27/031: Electronic editing of digitised analogue information signals, e.g. audio or video signals
- H04N 21/4307: {Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen}
- G06F 16/433: {using audio data}
- G06F 16/438: Presentation of query results
- G11B 27/10: Indexing; Addressing; Timing or synchronising; Measuring tape travel
- H04N 21/4394: {involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams (arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58)}
- H04N 21/8455: {involving pointers to the content, e.g. pointers to the I-frames of the video stream}
- H04N 21/8456: {by decomposing the content in the time domain, e.g. in time segments}
- G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

In some embodiments, a method includes detecting an incorporation attribute of audiovisual content; analyzing an audio library to determine an audio file that maps to the incorporation attribute; selecting the audio file from the audio library that maps to the incorporation attribute; incorporating the selected audio file into the audiovisual content to generate egocentric audiovisual content; and providing the egocentric audiovisual content to a user for audiovisual consumption. In some embodiments of the method, the incorporation attribute is at least one of a mood of a target of the audiovisual content, an activity of the target of the audiovisual content, and a background of the audiovisual content.

## Claims

### Claim 1

A method, comprising:
detecting an incorporation attribute of audiovisual content;
analyzing an audio library to determine an audio file that maps to the incorporation attribute;
selecting the audio file from the audio library that maps to the incorporation attribute;
incorporating the selected audio file into the audiovisual content to generate egocentric audiovisual content; and
providing the egocentric audiovisual content to a user for audiovisual consumption.

### Claim 2 (depends on 1)

The method of claim 1, wherein:
the incorporation attribute is at least one of a mood of a target of the audiovisual content, an activity of the target of the audiovisual content, and a background of the audiovisual content.

### Claim 3 (depends on 2)

The method of claim 2, wherein:
the audiovisual content includes video content of a video stream and audio content of an audio stream.

### Claim 4 (depends on 3)

The method of claim 3, further comprising:
synchronizing the selected audio file to the audiovisual content based upon an audio file marker placed in the audio file.

### Claim 5 (depends on 4)

The method of claim 4, further comprising:
mapping audio files in the audio library to at least one of an activity type, a mood type, a background type, and a sponsorship type.

### Claim 6 (depends on 5)

The method of claim 5, wherein:
an audio configuration unit selects the audio file based upon the mapping of the audio file to the incorporation attribute.

### Claim 7 (depends on 6)

The method of claim 6, wherein:
an audio library analysis unit analyzes the audio library to determine which audio file in the audio library maps to the incorporation attribute of the audiovisual content.

### Claim 8 (depends on 7)

The method of claim 7, wherein:
synchronizing the audio file to a video stream of the audiovisual content is dependent upon a video stream timing marker being placed in the video stream of the audiovisual content.

### Claim 9 (depends on 8)

The method of claim 8, wherein:
synchronizing the audio file to the video stream is dependent upon an audio stream time marker being placed in the audio stream.

### Claim 10

A system, comprising:
a video recorder;
an audio recorder coupled to the video recorder; and
a controller coupled to the video recorder and the audio recorder, wherein the controller is configured to:
detect an incorporation attribute of audiovisual content;
analyze an audio library to determine an audio file that maps to the incorporation attribute;
select the audio file from the audio library that maps to the incorporation attribute;
incorporate the selected audio file into the audiovisual content to generate egocentric audiovisual content; and
provide the egocentric audiovisual content to a user for audiovisual consumption.

### Claim 11 (depends on 10)

The system of claim 10, wherein:
the incorporation attribute is at least one of a mood of a target of the audiovisual content, an activity of the target of the audiovisual content, and a background of the audiovisual content.

### Claim 12 (depends on 11)

The system of claim 11, wherein:
the audiovisual content includes video content of a video stream and audio content of an audio stream.

### Claim 13 (depends on 12)

The system of claim 12, wherein:
the selected audio file is synchronized to the audiovisual content based upon an audio file marker placed in the audio file.

### Claim 14 (depends on 13)

The system of claim 13, wherein:
audio files in the audio library are mapped to at least one of an activity type, a mood type, a background type, and a sponsorship type.

### Claim 15 (depends on 14)

The system of claim 14, wherein:
an audio configuration unit selects the audio file based upon the mapping of the audio file to the incorporation attribute.

### Claim 16 (depends on 15)

The system of claim 15, wherein:
an audio library analysis unit analyzes the audio library to determine which audio file in the audio library maps to the incorporation attribute of the audiovisual content.

### Claim 17 (depends on 16)

The system of claim 16, wherein:
the audio file is synchronized to a video stream of the audiovisual content using a video stream timing marker that is placed in the video stream of the audiovisual content.

### Claim 18 (depends on 16)

The system of claim 16, wherein:
the audio file that is synchronized to a video stream of the audiovisual content using an audio stream time marker that is placed in the audio stream of the audiovisual content.

### Claim 19

A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:
detecting an incorporation attribute of audiovisual content;
analyzing an audio library to determine a first audio file that maps to the incorporation attribute of the audiovisual content;
selecting the first audio file from the audio library that maps to the incorporation attribute;
incorporating the selected first audio file into the audiovisual content to generate egocentric audiovisual content;
uploading the egocentric audiovisual content to a social network for view by a viewer of the social network; and
modifying the egocentric audiovisual content to include a second audio file selected using preferences of the viewer of the egocentric audiovisual content.

### Claim 20 (depends on 19)

The non-transitory computer-readable medium of claim 19, wherein:
the second audio file selected by the viewer of the egocentric audiovisual content replaces the first audio file selected from the audio library that maps to the incorporation attribute.
