Back to results
US 12,646,322 B2Granted
Video-driven spatial audio enhancement
AssigneeTencent Technology (Shenzhen) Company Limited
InventorQireng Liang
Life of this patent
Estimating the term…
Abstract
A data processing method includes acquiring video frame data including one or more video frames and audio data of a video, and determining position attribute information of a target object in the acquired one or more video frames, the target object being associated with the audio data. The method also includes acquiring a channel encoding parameter associated with the position attribute information, and performing azimuth enhancement processing on the audio data according to the channel encoding parameter to obtain enhanced audio data. Apparatus and non-transitory computer-readable storage medium counterpart embodiments are also contemplated.
Claims18
A data processing method, comprising:
- acquiring video frame data including one or more video frames and audio data of a video;
- determining, by processing circuitry of a data processing apparatus, position attribute information of a target object in the acquired one or more video frames, the target object being associated with the audio data;
- and
- acquiring, from a parameter mapping table that stores a mapping relationship between predefined channel encoding parameters associated with ear positions of a user and relative positions of the target object with respect to the ear positions of the user, a set of channel encoding parameters including a first channel encoding parameter associated with a first one of the ear positions of the user and a second channel encoding parameter associated with a second one of the ear positions of the user for different audio channels corresponding to the position attribute information;
- performing azimuth enhancement processing on the audio data based on the set of channel encoding parameters, the azimuth enhancement processing comprising:
- processing the audio data based on the first channel encoding parameter to obtain first enhanced audio data;
- processing the audio data based on the second channel encoding parameter to obtain second enhanced audio data;
- and
- generating enhanced audio data from the first enhanced audio data and the second enhanced audio data, the first enhanced audio data being louder than the second enhanced audio data to the user based on the position attribute information of the target object indicating that the target object is closer to the first one of the ear positions of the user than the second one of the ear positions of the user.
6 more elements
A data processing apparatus, comprising:
- processing circuitry configured to:
- acquire video frame data including one or more video frames and audio data of a video;
- determine position attribute information of a target object in the acquired one or more video frames, the target object being associated with the audio data;
- and
- acquire, from a parameter mapping table that stores a mapping relationship between predefined channel encoding parameters associated with ear positions of a user and relative positions of the target object with respect to the ear positions of the user, a set of channel encoding parameters including a first channel encoding parameter associated with a first one of the ear positions of the user and a second channel encoding parameter associated with a second one of the ear positions of the user for different audio channels corresponding to the position attribute information;
- perform azimuth enhancement processing on the audio data based on the set of channel encoding parameters, the azimuth enhancement processing comprising:
- processing the audio data based on the first channel encoding parameter to obtain first enhanced audio data;
- processing the audio data based on the second channel encoding parameter to obtain second enhanced audio data;
- and
- generating enhanced audio data from the first enhanced audio data and the second enhanced audio data, the first enhanced audio data being louder than the second enhanced audio data to the user based on the position attribute information of the target object indicating that the target object is closer to the first one of the ear positions of the user than the second one of the ear positions of the user.
7 more elements
A non-transitory computer-readable storage medium storing computer-readable instructions thereon, which, when executed by a computer, cause the computer to perform a data processing method comprising:
- acquiring video frame data including one or more video frames and audio data of a video;
- determining position attribute information of a target object in the acquired one or more video frames, the target object being associated with the audio data;
- and
- acquiring, from a parameter mapping table that stores a mapping relationship between predefined channel encoding parameters associated with ear positions of a user and relative positions of the target object with respect to the ear positions of the user, a set of channel encoding parameters including a first channel encoding parameter associated with a first one of the ear positions of the user and a second channel encoding parameter associated with a second one of the ear positions of the user for different audio channels corresponding to the position attribute information;
- performing azimuth enhancement processing on the audio data based on the set of channel encoding parameters, the azimuth enhancement processing comprising:
- processing the audio data based on the first channel encoding parameter to obtain first enhanced audio data;
- processing the audio data based on the second channel encoding parameter to obtain second enhanced audio data;
- and
- generating enhanced audio data from the first enhanced audio data and the second enhanced audio data, the first enhanced audio data being louder than the second enhanced audio data to the user based on the position attribute information of the target object indicating that the target object is closer to the first one of the ear positions of the user than the second one of the ear positions of the user.
6 more elements
Dependencies are inferred from the claim text.