Virtual-musical-instrument-based audio processing method and apparatus, electronic device, computer-readable storage medium, and computer program product
AssigneeTencent Technology (Shenzhen) Company Limited
InventorWeihang Wang
Life of this patent
Estimating the term…
Abstract
A virtual-musical-instrument-based audio processing method is provided. In the method, a video is played. A virtual musical instrument is displayed in the video when the virtual musical instrument is matched with at least one musical instrument graphic element in the video. Played audio of the virtual musical instrument is outputted according to interactions with the at least one musical instrument graphic element matched with the virtual musical instrument in the video. Apparatus and non-transitory computer-readable storage medium counterpart embodiments are also contemplated.
Claims20
A virtual-musical-instrument-based audio processing method, the method comprising:
- displaying, on a display, a video based on an original video captured by a first camera;
- displaying a first component of a virtual musical instrument in the video when the first component is matched with at least one musical instrument graphic element in the original video;
- displaying a second component of the virtual musical instrument in the video based on an object overlapping the at least one musical instrument graphic element in the original video;
- determining, by processing circuitry, an initial distance between the first component and the second component along an observation direction of the original video, the initial distance corresponding to an initial volume of the virtual musical instrument;
- obtaining a real-time distance between the first component and the second component along the observation direction;
- determining, by the processing circuitry, a real-time volume of the virtual musical instrument based on the real-time distance and a mapping relationship, one or more coefficients of the mapping relationship being determined based on the initial distance and the initial volume;
- and
- outputting played audio of the virtual musical instrument together with the video based on the real-time volume.
A virtual-musical-instrument-based audio processing apparatus, comprising:
- processing circuitry configured to:
- display, on a display, a video based on an original video captured by a first camera;
- display a first component of a virtual musical instrument in the video when the first component is matched with at least one musical instrument graphic element in the original video;
- display a second component of the virtual musical instrument in the video based on an object overlapping the at least one musical instrument graphic element in the original video;
- determine an initial distance between the first component and the second component along an observation direction of the original video, the initial distance corresponding to an initial volume of the virtual musical instrument;
- obtain a real-time distance between the first component and the second component along the observation direction;
- determine a real-time volume of the virtual musical instrument based on the real-time distance and a mapping relationship, one or more coefficients of the mapping relationship being determined based on the initial distance and the initial volume;
- and
- output played audio of the virtual musical instrument together with the video based on the real-time volume.
A non-transitory computer-readable storage medium, storing instructions which when executed by a processor cause the processor to perform:
- displaying, on a display, a video based on an original video captured by a first camera;
- displaying a first component of a virtual musical instrument in the video when the first component is matched with at least one musical instrument graphic element in the video;
- displaying a second component of the virtual musical instrument in the video based on an object overlapping the at least one musical instrument graphic element in the original video;
- determining an initial distance between the first component and the second component along an observation direction of the original video, the initial distance corresponding to an initial volume of the virtual musical instrument;
- obtaining a real-time distance between the first component and the second component along the observation direction;
- determining a real-time volume of the virtual musical instrument based on the real-time distance and a mapping relationship, one or more coefficients of the mapping relationship being determined based on the initial distance and the initial volume;
- and
- outputting played audio of the virtual musical instrument together with the video based on the real-time volume.
Dependencies are inferred from the claim text.