# US 10,482,887 B1 · Machine learning model assisted enhancement of audio and/or visual communications

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,482,887 B1
- Document ID: US10482887B1
- Status: Granted
- Kind: B1
- Application number: 15/925,451
- Filed: 2018-03-19
- Granted: 2019-11-19
- Earliest claimed: 2018-03-19
- Assignees: Amazon Technologies, Inc.
- Applicants: Amazon Technologies, Inc.
- Inventors: Madhav Jha; Edo Liberty
- Web page: https://patentgrep.com/patent/US10482887B1

## CPC

- G10L 19/0018: {Speech coding using phonetic or linguistical decoding of the source; Reconstruction using text-to-speech synthesis}
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/047: Probabilistic or stochastic networks
- G06N 3/08: Learning methods
- G06N 20/00: Machine learning
- G10L 13/00: Speech synthesis; Text to speech systems
- G10L 15/16: using artificial neural networks
- G10L 25/30: using neural networks
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Techniques for using machine learning models to approximate a user in a communication are described. For example, a method of initiating a communication link with an edge device to exchange audio data; receiving a compressed audio data from the edge device; re-encoding the received compressed audio data using a re-encoding machine learning model to approximate a voice; and outputting the approximated voice is detailed.

## Claims

### Claim 1

A computer-implemented method comprising:
initiating a communication link between a first and a second edge device for the first and seconds edge devices to transmit and receive at least compressed audio data;
determining a bandwidth of the communication link, the determined bandwidth to be used to set an amount of compression of audio data to be performed by the first edge device;
transmitting a re-encoding machine learning model corresponding to a voice of a user of the first edge device to the second edge device, the second edge device to utilize the re-encoding machine learning model of the user of the first edge device to re-encoding at least the compressed audio data transmitted over the initiated communication link to approximate the voice of the user of the first edge device; and
passing compressed audio data from the first edge device to the second edge device using the initiated communication link.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein the re-encoding machine learning model is a text to speech model comprising a plurality of layers.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, wherein the compressed audio data is an output of an intermediate layer of a compression machine learning model.

### Claim 4

A computer-implemented method comprising:
initiating a communication link with an edge device to exchange audio data;
receiving a compressed audio data from the edge device;
re-encoding the received compressed audio data using a re-encoding machine learning model to approximate a voice; and
outputting the approximated voice.

### Claim 5 (depends on 4)

The computer-implemented method of claim 4, further comprising:
determining a bandwidth of the communication link, the determined bandwidth to be used to set an amount of compression of audio data to be performed by the edge device.

### Claim 6 (depends on 4)

The computer-implemented method of claim 4, further comprising:
receiving raw input audio data;
compressing the received raw input audio data into compressed raw audio data using a compression machine learning model using a text to speech machine learning model comprising a plurality of layers.

### Claim 7 (depends on 6)

The computer-implemented method of claim 6, further comprising:
training the compression machine learning model using the raw input audio data.

### Claim 8 (depends on 6)

The computer-implemented method of claim 6, wherein the compression machine learning model is a recurrent neural network (RNN).

### Claim 9 (depends on 8)

The computer-implemented method of claim 8, wherein at least one of the hidden layers of the RNN utilizes long short-term memory (LSTM).

### Claim 10 (depends on 8)

The computer-implemented method of claim 8, wherein the compressed raw audio data is an output of intermediate layer of the RNN and includes audio data and metadata.

### Claim 11 (depends on 4)

The computer-implemented method of claim 4, further comprising:
sending audio data that has not been compressed using a compression machine learning model.

### Claim 12 (depends on 4)

The computer-implemented method of claim 4, wherein the re-encoding machine learning model corresponds to a voice of a user of the edge device.

### Claim 13 (depends on 4)

The computer-implemented method of claim 4, further comprising:
receiving the re-encoding machine learning model from the edge device.

### Claim 14 (depends on 4)

The computer-implemented method of claim 4, further comprising:
receiving the re-encoding machine learning model from an intermediate device.

### Claim 15 (depends on 4)

The computer-implemented method of claim 4, wherein the compressed audio data is text.

### Claim 16 (depends on 4)

The computer-implemented method of claim 4, wherein the compressed audio data is an output of an intermediate layer of a recurrent neural network (RNN) of the edge device and includes audio data and metadata.

### Claim 17

A system comprising:
a web services provider server to initiation communications between a first and a second edge device; and
the first edge device including hardware execution resources to execute instructions stored in memory, the instructions upon execution to cause:
a receipt of a compressed audio data from the second edge device,
a re-encode of the received compressed audio data using a re-encoding machine learning model to approximate a voice; and
a generation of the approximated voice.

### Claim 18 (depends on 17)

The system of claim 17, wherein the web services provider is further to determine a bandwidth of the communication link, the determined bandwidth to be used to set an amount of compression of audio data to be performed by the second edge device.

### Claim 19 (depends on 17)

The system of claim 17, wherein the instructions are further to cause:
a receipt of raw input audio data; and
a compression of the received raw input audio data into compressed raw audio data using a compression machine learning model using a text to speech machine learning model comprising a plurality of layers.

### Claim 20 (depends on 19)

The system of claim 19, wherein the compressed raw audio data is an output of an intermediate layer of a recurrent neural network (RNN).