# US 10,311,454 B2 · Customer interaction and experience system using emotional-semantic computing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,311,454 B2
- Document ID: US10311454B2
- Status: Granted
- Kind: B2
- Application number: 16/163,482
- Filed: 2018-10-17
- Published: 2019-02-14
- Granted: 2019-06-04
- Earliest claimed: 2017-06-22
- Assignees: NewVoiceMedia, Ltd.
- Applicants: NewVoiceMedia Ltd.
- Inventors: Alan McCord
- Web page: https://patentgrep.com/patent/US10311454B2

## CPC

- G06Q 30/0201: Market modelling; Market analysis; Collecting market data
- G06F 16/24578: {using ranking}
- G06F 16/248: Presentation of query results
- G06F 16/9535: Search customisation based on user profiles and personalisation
- G06F 40/237: Lexical tools
- G06F 40/242: Dictionaries
- G06F 40/30: Semantic analysis
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/09: Supervised learning
- G06N 3/092: Reinforcement learning
- G06N 20/00: Machine learning
- G10L 15/1815: {Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning}
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L 15/26: Speech to text systems (G10L15/08 takes precedence)
- G10L 25/63: for estimating an emotional state
- G06F 18/23: Clustering techniques
- G06N 3/006: based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
- G06N 3/045: Combinations of networks
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

A system and method for customer interaction and experience enhancement which automatically gathers direct and indirect customer communications about products and services, converts them to text where necessary, and analyzes the communications for sentiment and emotional content, and scores and displays the information in a manner conducive to making business decisions based on the customer sentiment and emotion, such as making changes to products or services, troubleshooting customer service interactions, and better marketing.

## Claims

### Claim 1

A system for enhancing customer relations through automated analysis of the sentiment and emotion contained in direct and indirect text-based communications, comprising:
a deep web search engine comprising at least a plurality of programming instructions stored in a memory and operating on a processor of a computer, wherein the programmable instructions, when operating on the processor, cause the processor to:
gather indirect customer communications about products and services from online sources; and
an automated speech recognition engine comprising at least a plurality of programming instructions stored in a memory and operating on a processor of a computer, wherein the programmable instructions, when operating on the processor, cause the processor to:
receive audio data from direct and indirect customer audio or video communications;
automatically recognize the speech contained in the audio data;
generate a textual transcription of the speech contained in the audio data; and
a sentiment and emotion analyzer comprising at least a plurality of programming instructions stored in a memory and operating on a processor of a computer, wherein the programmable instructions, when operating on the processor, cause the processor to:
receive the text of direct and indirect customer communications from the deep web search engine, the automated speech recognition engine, emails from customers, and live chats with customers;
create a corpus of words and phrases from current and prior text of direct and indirect customer communications by parsing and organizing natural language words and phrases and stores that information as a corpus;
create a lexicon by assigning business-domain specific vector dimensions using machine learning algorithms to the words and phrases contained in the corpus, providing information regarding the meaning including sentiment and emotion, and other context of the words and phrases in the corpus, wherein the meanings of words and phrases are represented by the vector;
analyze incoming text for sentiment and emotions being expressed by the customer using the corpus and lexicon by assigning probability distributions to the potential sentiments and emotional content of the text using dilated convolutional artificial neural network to generate lower-dimensional dense vectors;
output the analysis to a scoring and display engine; and
a scoring and display engine comprising at least a plurality of programming instructions stored in a memory and operating on a processor of a computer, wherein the programmable instructions, when operating on the processor, cause the processor to:
receive analyses of sentiment and emotion in direct and indirect customer communications from the sentiment and emotion analyzer;
cumulatively assign similarity scores to products or services of interest based on the incoming analyses that includes relative distance of the vectors; and
display the results of the scoring in textual, tabular, or graphical format to make business decisions.

### Claim 2 (depends on 1)

The system of claim 1 wherein vector clustering is used to segment similar content.

### Claim 3 (depends on 1)

The system of claim 1 wherein linear transformation is used to translate between the language of business-specific domains.

### Claim 4 (depends on 1)

The system of claim 1 wherein vector algebra is used to find content needed to achieve target emotional-semantic content.

### Claim 5 (depends on 1)

The system of claim 1 wherein vector calculus is used to understand and predict time evolution in emotion and meaning by calculating rates of change of vectors and time dynamics.

### Claim 6

A method for enhancing customer relations through automated analysis of the sentiment and emotion contained in text-based communications, comprising:
a deep web search engine, an automated speech recognition engine, a sentiment and emotion analyzer, and a scoring and display engine comprising at least a plurality of programming instructions stored in a memory and operating on a processor of a computer, wherein the programmable instructions, when operating on the processor, automatically cause the processor to perform the steps of:
(a) gathering indirect customer communications about products and services from online sources;
(b) gathering direct customer communications about products and services including audio data and video data;
(c) converting any audio data in the customer communications to text using automated speech recognition to generate a textual transcription of the speech contained in the audio data;
(d) creating a corpus of words and phrases from the gathered and converted text of direct and indirect customer communications by parsing and organizing natural language words and phrases and stores that information as a corpus;
(e) creating a lexicon by assigning business-domain specific vector dimensions using machine learning algorithms to the words and phrases contained in the corpus, associating meaning of the words and phrases including sentiment and emotion depending on context, wherein the meanings of words and phrases are represented by the vector;
(f) analyzing incoming text for sentiment and emotions being expressed by the customer using the corpus and lexicon by assigning probability distributions to the potential sentiments and emotional content of the text using dilated convolutional artificial neural network to generate lower-dimensional dense vectors;
(g) receiving the analyses of the sentiment and emotion in direct and indirect customer communications from the sentiment and emotion analyzer;
(h) outputting the analysis to a scoring and display engine; and
(i) displaying the results of the scoring in a textual, tabular, or graphical format to make business decisions.

### Claim 7 (depends on 6)

The method of claim 6 comprising the additional step of using vector clustering to segment similar content.

### Claim 8 (depends on 6)

The method of claim 6 comprising the additional step of using linear transformation to translate between the language of business-specific domains.

### Claim 9 (depends on 6)

The method of claim 6 comprising the additional step of using vector algebra to find content needed to achieve target emotional-semantic content.

### Claim 10 (depends on 6)

The method of claim 6 comprising the additional step of using vector calculus to understand and predict time evolution in emotion and meaning by calculating rates of change of vectors and time dynamics.