# US 12,014,723 B2 · Information processing method, information processing device, and program > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,014,723 B2 - Document ID: US12014723B2 - Status: Granted - Kind: B2 - Application number: 17/119,426 - Filed: 2020-12-11 - Published: 2021-04-01 - Granted: 2024-06-18 - Earliest claimed: 2018-06-15 - Assignees: Yamaha Corporation - Applicants: Yamaha Corporation - Inventors: Makoto Tachibana; Motoki Ogasawara - Web page: https://patentgrep.com/patent/US12014723B2 ## CPC - G10L 13/0335: {Pitch control} - G10L 13/047: Architecture of speech synthesisers - G06T 11/23: using straight lines or curves - G06T 11/26: Drawing of charts or graphs - G10H 1/0008: {Associated control or indicating means} - G10H 7/008: {Means for controlling the transition from one tone waveform to another} - G10H 2220/126: for graphical editing of individual notes, parts or phrases represented as variable length segments on a 2D or 3D representation, e.g. graphical edition of musical collage, remix files or pianoroll representations of MIDI-like files - G10H 2250/455: Gensound singing voices, i.e. generation of human voices for musical applications, vocal singing sounds or intelligible words at a desired pitch or with desired vocal effects, e.g. by phoneme synthesis - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G10H: ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE ## Abstract An information processing method is realized by a computer and includes generating a first characteristic transition which is a transition of acoustic characteristics, in accordance with an instruction from a user, generating a second characteristic transition which is a transition of acoustic characteristics of voice that is pronounced in a specific pronunciation style selected from a plurality of pronunciation styles, and generating a combined characteristic transition which is a transition of the acoustic characteristics of synthesized voice by combining the first characteristic transition and the second characteristic transition. ## Claims ### Claim 1 An information processing method realized by a computer, the information processing method comprising: generating a first characteristic transition which is a transition of an acoustic characteristic, in accordance with an instruction from a user; generating a second characteristic transition which is a transition of an acoustic characteristic of voice that is pronounced in a specific pronunciation style selected from a plurality of pronunciation styles; setting, in accordance with an instruction from the user, a first weighted value for the first characteristic transition and a second weighted value for the second characteristic transition, and calculating a weighted sum of the first characteristic transition and the second characteristic transition by using the first weighted value and the second weighted value; and generating a combined characteristic transition which is a transition of an acoustic characteristic of synthesized voice, by combining the first characteristic transition and the second characteristic transition using the weighted sum, the acoustic characteristic for the first characteristic transition, the acoustic characteristic of the voice for the second characteristic transition, and the acoustic characteristic of the synthesized voice for the combined characteristic transition being the same acoustic characteristic. ### Claim 2 (depends on 1) The information processing method according to claim 1, wherein the first characteristic transition is generated such that the first characteristic transition represents a line drawing drawn by the user. ### Claim 3 (depends on 1) The information processing method according to claim 1, wherein the second characteristic transition is generated by using a transition estimation model corresponding to the specific pronunciation style from a plurality of transition estimation models respectively corresponding to the plurality of pronunciation styles. ### Claim 4 (depends on 1) The information processing method according to claim 1, wherein the second characteristic transition is generated by generating, as the second characteristic transition, a transition of an acoustic characteristic of an expression sample corresponding to the specific pronunciation style from a plurality of expression samples representing voices respectively corresponding to the plurality of pronunciation styles. ### Claim 5 (depends on 1) The information processing method according to claim 1, wherein in the generation of the second characteristic transition, an expression sample corresponding to a note is selected from among a plurality of expression samples representing voices by using an expression selection model corresponding to the specific pronunciation style from a plurality of expression selection models, to generate the second characteristic transition in accordance with a transition of characteristic of the expression sample. ### Claim 6 (depends on 1) The information processing method according to claim 1, wherein in response to change of the specific pronunciation style from a first pronunciation style to a second pronunciation style, the second characteristic transition is changed from a characteristic transition corresponding to the first pronunciation style to a characteristic transition corresponding to the second pronunciation style, while the first characteristic transition is maintained. ### Claim 7 (depends on 1) The information processing method according to claim 1, wherein the combined characteristic transition is generated in accordance with an adjustment parameter that is set in accordance with an instruction from the user. ### Claim 8 An information processing device comprising: an electronic controller including at least one processor, the electronic controller being configured to generate a first characteristic transition which is a transition of an acoustic characteristic, in accordance with an instruction from a user, generate a second characteristic transition which is a transition of an acoustic characteristic of voice that is pronounced in a specific pronunciation style selected from a plurality of pronunciation styles, set, in accordance with an instruction from the user, a first weighted value for the first characteristic transition and a second weighted value for the second characteristic transition, and calculate a weighted sum of the first characteristic transition and the second characteristic transition by using the first weighted value and the second weighted value, and generate a combined characteristic transition which is a transition of an acoustic characteristic of synthesized voice, by combining the first characteristic transition and the second characteristic transition using the weighted sum, the acoustic characteristic for the first characteristic transition, the acoustic characteristic of the voice for the second characteristic transition, and the acoustic characteristic of the synthesized voice for the combined characteristic transition being the same acoustic characteristic. ### Claim 9 (depends on 8) The information processing device according to claim 8, wherein the electronic controller generates the first characteristic transition that represents a line drawing drawn by the user. ### Claim 10 (depends on 8) The information processing device according to claim 8, wherein the electronic controller uses a transition estimation model corresponding to the specific pronunciation style from a plurality of transition estimation models respectively corresponding to the plurality of pronunciation styles to generate the second characteristic transition. ### Claim 11 (depends on 8) The information processing device according to claim 8, wherein the electronic controller generates, as the second characteristic transition, a transition of an acoustic characteristic of an expression sample corresponding to the specific pronunciation style from a plurality of expression samples representing voices respectively corresponding to the plurality of pronunciation styles. ### Claim 12 (depends on 8) The information processing device according to claim 8, wherein the electronic controller selects an expression sample corresponding to a note from a plurality of expression samples representing voices by using an expression selection model corresponding to the specific pronunciation style from a plurality of expression selection models, to generate the second characteristic transition in accordance with a transition of characteristic of the expression sample. ### Claim 13 (depends on 8) The information processing device according to claim 8, wherein in response to change of the specific pronunciation style from a first pronunciation style to a second pronunciation style, the electronic controller changes the second characteristic transition from a characteristic transition corresponding to the first pronunciation style to a characteristic transition corresponding to the second pronunciation style, while maintaining the first characteristic transition. ### Claim 14 (depends on 8) The information processing device according to claim 8, wherein the electronic controller generates the combined characteristic transition corresponding to an adjustment parameter that is set in accordance with an instruction from the user. ### Claim 15 A non-transitory computer-readable medium storing a program that causes a computer to execute a process, the process comprising: generating a first characteristic transition which is a transition of an acoustic characteristic, in accordance with an instruction from a user; generating a second characteristic transition which is a transition of an acoustic characteristic of voice that is pronounced in a specific pronunciation style; setting, in accordance with an instruction from the user, a first weighted value for the first characteristic transition and a second weighted value for the second characteristic transition, and calculating a weighted sum of the first characteristic transition and the second characteristic transition by using the first weighted value and the second weighted value; and generating a combined characteristic transition which is a transition of an acoustic characteristic of synthesized voice, by combining the first characteristic transition and the second characteristic transition using the weighted sum, the acoustic characteristic for the first characteristic transition, the acoustic characteristic of the voice for the second characteristic transition, and the acoustic characteristic of the synthesized voice for the combined characteristic transition being the same acoustic characteristic. ### Claim 16 (depends on 1) The information processing method according to claim 1, wherein the generating of the second characteristic transition is performed by generating a base transition that is a transition of a voice acoustic characteristic corresponding to notes specified by musical score data, generating a relative transition that corresponds to the specific pronunciation style and that is a transition of acoustic characteristic-relative values relative to the base transition, and combining the base transition and the relative transition to generate the second characteristic transition. ### Claim 17 (depends on 8) The information processing device according to claim 8, wherein to generate the second characteristic transition, the electronic controller is configured to generate a base transition that is a transition of a voice acoustic characteristic corresponding to notes specified by musical score data, generate a relative transition that corresponds to the specific pronunciation style and that is a transition of acoustic characteristic-relative values relative to the base transition, and combine the base transition and the relative transition to generate the second characteristic transition. ### Claim 18 (depends on 1) The information processing method according to claim 1, wherein the same acoustic characteristic is pitch or volume.