US 12,542,145 B1Granted

Customized dialogue audio de-mixing and adjustment

AssigneeAmazon Technologies, Inc.

InventorsAbhishek Yanamandra, Daniel Stephen ReMine, Rohith Mysore Vijaya Kumar, Sarah Elizabeth Norred, Avijit Vajpayee

Life of this patent

Estimating the term…

patentgrep

Loading patent…

Abstract

Systems and techniques for automatic audio adjustments to video and/or audio files based on user preferences. The systems and techniques receive data and information to determine content to identify and adjust or remove within the audio data such as a filter list of words or rating data. The systems and techniques further determine the timestamps for the instances within the audio/video file. The audio/video file is de-mixed by a machine learning model into multiple components, for example vocal and non-vocal components. An audio adjustment is made based on the content identified before re-mixing to present the adjusted content to a user.

Claims16

Dependent claims

A system comprising:

one or more processors;
and
a non-transitory computer-readable medium having instructions stored thereon that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
determining, based on a user input indicating a type of content to remove, a target word to identify within first audio data associated with a video file;
determining a timestamp of the first audio data and the video file associated with the target word by aligning text data associated with the first audio data and identifying the target word within the text data;
de-mixing the first audio data into a vocal component and a non-vocal component by inputting the first audio data and the text data into a cross-modal machine learning separator trained to separate audio data into two components based on sources of the first audio data;
applying a first audio adjustment to the vocal component at the timestamp to produce an adjusted vocal component, wherein applying the audio adjustment comprises muting the vocal component at the timestamp to remove the target word;
applying a second audio adjustment to the non-vocal component, wherein the second audio adjustment comprises applying negative gain to the non-vocal component;
generating second audio data by combining the adjusted vocal component and the adjusted non-vocal component;
and
presenting the second audio data with the video file at a user device.

8 more elements

A non-transitory computer-readable medium having instructions stored thereon that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

determining content to identify within first audio data;
determining a first non-vocal component and a second speech component of the first audio data, the second speech component associated with the content;
applying first audio adjustment to the first component to produce an adjusted first component, wherein the first component comprises non-vocal components and the second component comprises vocal components, and wherein applying the audio adjustment comprises applying negative gain to the non-vocal components;
applying a second audio adjustment to the second speech component to produce an adjusted second component, wherein applying the second audio adjustment comprises muting a vocal track of the second speech component at the timestamp to remove a target phrase included within the content;
generating second audio data by combining the adjusted first component and the adjusted second component;
and
causing presentation of the second audio data at a display of a user device.

4 more elements

Dependencies are inferred from the claim text.