# US 2026/0097312 A1 · Live voice synthesis with voice mixing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0097312 A1
- Document ID: US20260097312A1
- Status: Published application
- Kind: A1
- Application number: 18/907,329
- Filed: 2024-10-04
- Published: 2026-04-09
- Earliest claimed: 2024-10-04
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Mastafa Hamza FOUFA; Corentin Alexandre BRAUGE
- Web page: https://patentgrep.com/patent/US20260097312A1

## CPC

- A63F 13/63: by the player, e.g. authoring using a level editor
- A63F 13/215: comprising means for detecting acoustic signals, e.g. using a microphone
- A63F 13/54: involving acoustic signals, e.g. for simulating revolutions per minute [RPM] dependent engine sounds in a driving game or reverberation against a virtual wall
- A63F 13/67: adaptively or by learning from player actions, e.g. skill level adjustment or by storing successful combat sequences for re-use
- A63F 13/87: Communicating with other players during game play, e.g. by e-mail or chat
- G10L 13/033: Voice editing, e.g. manipulating the voice of the synthesiser
- G10L 13/047: Architecture of speech synthesisers
- G10L 21/013: Adapting to target pitch
- G10L 2021/0135: {Voice conversion or morphing}
- A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

Systems, devices, methods, and machine-readable media configured to provide voice inference in a video game are provided. A video game system can include an encoder configured to generate a first encoding representative of physical characteristics of a specified entity, a similarity operator configured to determine similarity values between (i) corresponding stored encodings of multiple characters, the stored encodings representative of physical characteristics of respective characters of the multiple characters and (ii) the first encoding, identify a selected character from the multiple characters based on the similarity values, and provide an identifier of the selected character, a voice database configured to provide audio or a spectrogram of the selected character, and a video game configured to provide the audio of a player-selected character in a voice of the selected character.

## Claims

### Claim 1

A video game system comprising:
an encoder configured to generate a first encoding representative of physical characteristics of a specified entity;
a similarity operator configured to:
determine similarity values between (i) corresponding stored encodings of multiple characters, the stored encodings representative of physical characteristics of respective characters of the multiple characters and (ii) the first encoding;
identify a selected character from the multiple characters based on the similarity values; and
provide an identifier of the selected character;
a voice database configured to provide audio or a spectrogram of the selected character; and
a video game configured to provide the audio of a player-selected character in a voice of the selected character.

### Claim 2 (depends on 1)

The video game system of claim 1, wherein the physical characteristics represent physical attributes of respective characters in the video game and the entity is a player of the video game.

### Claim 3 (depends on 2)

The video game system of claim 2, wherein the encoder generates the first encoding based on an image of the player.

### Claim 4 (depends on 2)

The video game system of claim 2, further comprising:
a physical characteristic selection interface of the video game configured to present physical characteristics to the player and receive, from the player, the physical characteristics of the entity.

### Claim 5 (depends on 2)

The video game system of claim 2, further comprising:
a voice transform model trained to receive spectrograms of multiple, player-selected characters and generate a composite spectrogram that is a mixture of the received spectrograms.

### Claim 6 (depends on 5)

The video game system of claim 5, wherein the received spectrograms include a spectrogram of the selected character and the selected character is associated with physical characteristic most similar to the entity.

### Claim 7 (depends on 5)

The video game system of claim 5, wherein the received spectrograms include a spectrogram of audio from the player.

### Claim 8 (depends on 5)

The video game system of claim 5, wherein the voice transform model includes a sequence-to-sequence model is trained to convert the received spectrograms directly into the composite spectrogram.

### Claim 9

A method comprising:
generating, by an encoder model, a first encoding representative of physical characteristics of a specified entity;
determining, by a similarity operator, similarity values between (i) corresponding stored encodings of multiple characters, the stored encodings representative of physical characteristics of respective characters of the multiple characters and (ii) the first encoding;
identifying, by the similarity operator, a selected character from the multiple characters based on the similarity values;
providing an identifier of the selected character;
retrieving, by a voice database and based on the identifier, audio or a spectrogram of the selected character; and
providing, by a video game and based on the audio or the spectrogram of the character, audio of a player-selected character in a voice of the selected character.

### Claim 10 (depends on 9)

The method of claim 9, wherein the entity is a player of the video game.

### Claim 11 (depends on 10)

The method of claim 10, further comprising:
receiving, by the encoder model, an image of the player and wherein the encoder model generates the first encoding based on the image of the player.

### Claim 12 (depends on 10)

The method of claim 10, further comprising:
presenting, by a physical characteristic selection interface of the video game, physical characteristics; and
receiving, from the player, the physical characteristics of the entity.

### Claim 13 (depends on 10)

The method of claim 10, further comprising:
receiving, by a voice transform model, spectrograms of multiple, player-selected characters; and
generating, by the voice transform model a composite spectrogram that is a mixture of the received spectrograms.

### Claim 14 (depends on 13)

The method of claim 13, wherein the received spectrograms include a spectrogram of the selected character and the selected character is associated with physical characteristic most similar to the entity.

### Claim 15 (depends on 14)

The method of claim 14, wherein the received spectrograms include a spectrogram of audio from the player.

### Claim 16 (depends on 13)

The method of claim 13, wherein the voice transform model includes a sequence-to-sequence model trained to convert the received spectrograms directly into the composite spectrogram.

### Claim 17

A non-transitory machine-readable medium including instructions that, when executed by a machine, cause the machine to perform operations for voice inference in a video game, the operations comprising:
receiving, from an encoder model, a first encoding representative of physical characteristics of a player of the video game;
determining similarity values between (i) corresponding stored encodings of multiple characters, the stored encodings representative of physical characteristics of respective characters of the multiple characters and (ii) the first encoding;
identifying a selected character of the multiple characters, based on the similarity values, corresponding to a character with character physical characteristics that are most similar to physical characteristics of the player;
providing an identifier of the selected character;
retrieving, by a voice database, audio or a spectrogram of the selected character; and
providing, by the video game and based on the audio or the spectrogram of the character, audio of a player-selected character in a voice of the selected character.

### Claim 18 (depends on 17)

The non-transitory machine-readable medium of claim 17, wherein the operations further comprise:
presenting, by a physical characteristic selection interface of the video game, physical characteristics; and
receiving, from the player and by the physical characteristic selection interface, the physical characteristics of the player.

### Claim 19 (depends on 17)

The non-transitory machine-readable medium of claim 17, wherein the operations further comprise:
receiving, by a voice transform model, spectrograms of multiple, player-selected characters including a spectrogram of the selected character, the selected character associated with physical characteristics most similar to the physical characteristics of the player; and
generating, by the voice transform model, a composite spectrogram that is a mixture of the received spectrograms.

### Claim 20 (depends on 19)

The non-transitory machine-readable medium of claim 19, wherein the voice transform model includes a sequence-to-sequence model trained to convert the received spectrograms directly into the composite spectrogram.
