# US 10,769,648 B2 · Automated business reviews based on patron sentiment

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,769,648 B2
- Document ID: US10769648B2
- Status: Granted
- Kind: B2
- Application number: 16/601,020
- Filed: 2019-10-14
- Published: 2020-02-06
- Granted: 2020-09-08
- Earliest claimed: 2018-02-13
- Assignees: Capital One Services, LLC
- Applicants: Capital One Services, LLC
- Inventors: Steve Blanchet; Zviad Aznaurashvili; Hannes Jouhikainen; Timur Sherif
- Web page: https://patentgrep.com/patent/US10769648B2

## CPC

- G06Q 30/0201: Market modelling; Market analysis; Collecting market data
- G06Q 30/02: Marketing; Price estimation or determination; Fundraising
- G06Q 30/0282: Rating or review of business operators or products
- G06Q 50/12: Hotels or restaurants
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

Embodiments disclosed herein generally relate to a method and system of determining an overall sentiment of a facility. A computing system receives a video stream, including a plurality of frames, of one or more patrons in a facility over a first time period. The video stream includes data indicative of a sentiment of each of the one or more patrons. The computing system parses the plurality of frames to determine the sentiment of the patron based at least on audio and visual cues of the patron captured in the video stream during the first time period. The computing system aggregates one or more sentiments corresponding to the one or more patrons in a data set indicative of an overall sentiment of the facility. The computing system generates a sentiment value corresponding to the overall sentiment of the facility. The computing system outputs the overall sentiment of the facility.

## Claims

### Claim 1

A method, comprising:
receiving a first video stream comprising one or more patrons in a facility and data indicative of a sentiment of each of the one or more patrons from a camera positioned in the facility;
receiving a second video stream comprising a subset of the one or more patrons in the facility and data indicative of a sentiment of each patron in the subset of the one or more patrons from a mobile device of a first patron of the one or more patrons;
for each patron in the first video stream and the second video stream, parsing the first video stream and the second video stream to determine the sentiment of the patron based at least on audio and visual cues of the patron captured in the first video stream and the second video stream;
generating a sentiment value corresponding to the determined sentiment of each patron in the first video stream and the second video stream, the generated sentiment value corresponding to an overall sentiment for the facility; and
outputting the overall sentiment of the facility based on the generated sentiment value.

### Claim 2 (depends on 1)

The method of claim 1, wherein the sentiment of the patron comprises an attitude the patron is conveying toward the facility.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
receiving a third video stream, the third video stream comprising a second one or more patrons in the facility over a second time period succeeding a first time period, the third video stream comprising data indicative of each of the one or more patrons over the second time period;
for each patron in the third video stream, parsing the third video stream to determine a second sentiment of the patron in the third video stream based at least on audio and visual cues of the patron captured in the third video stream during the second time period;
aggregating one or more second sentiments corresponding to the one or more patrons in a second data set indicative of a second overall sentiment of the facility over the second time period;
appending the second data set to an overall data set that includes at least a data set of one or more sentiments over the first time period;
generating an updated sentiment value corresponding to the overall sentiment of the facility spanning the first time period and the second overall sentiment of the facility spanning the second time period, based on the appended data set; and
outputting the updated sentiment of the facility based on the generated updated sentiment value.

### Claim 4 (depends on 1)

The method of claim 1, wherein parsing the first video stream and the second video stream to determine the sentiment of the patron based at least on the audio and visual cues of the patron captured in the first video stream and the second video stream, comprises:
identifying one or more words spoken by evaluating one or more lip movements of the patron.

### Claim 5 (depends on 1)

The method of claim 1, wherein parsing the first video stream and the second video stream to determine the sentiment of the patron based at least on the audio and visual cues of the patron captured in the first video stream and the second video stream, comprises:
identifying one or more facial expressions of the patron by evaluating one or more facial movements of the patron.

### Claim 6 (depends on 1)

The method of claim 1, wherein receiving the first video stream comprising the one or more patrons in the facility and the data indicative of the sentiment of each of the one or more patrons from the camera positioned in the facility comprises:
receiving the first video stream corresponding to a bounded location in the facility.

### Claim 7 (depends on 1)

The method of claim 1, wherein parsing the first video stream and the second video stream to determine the sentiment of the patron based at least on the audio and visual cues of the patron captured in the first video stream and the second video stream, comprises:
identifying one or more key terms uttered by the patron, relating to one or more dimensions of the facility.

### Claim 8

A system, comprising:
a processor; and
a memory having programming instructions stored thereon, which, when executed by the processor, performs one or more operations comprising:
receiving a first video stream comprising one or more patrons in a facility and data indicative of a sentiment of each of the one or more patrons from a camera positioned in the facility;
receiving a second video stream comprising a subset of the one or more patrons in the facility and data indicative of a sentiment of each patron in the subset of the one or more patrons from a mobile device of a first patron of the one or more patrons;
for each patron in the first video stream and the second video stream, parsing the first video stream and the second video stream to determine the sentiment of the patron based at least on audio and visual cues of the patron captured in the first video stream and the second video stream;
generating a sentiment value corresponding to the determined sentiment of each patron in the first video stream and the second video stream, the generated sentiment value corresponding to an overall sentiment for the facility; and
outputting the overall sentiment of the facility based on the generated sentiment value.

### Claim 9 (depends on 8)

The system of claim 8, wherein the sentiment of the patron comprises an attitude the patron is conveying toward the facility.

### Claim 10 (depends on 8)

The system of claim 8, wherein the one or more operation further comprise:
receiving a third video stream, the third video stream comprising a second one or more patrons in the facility over a second time period succeeding a first time period, the third video stream comprising data indicative of each of the one or more patrons over the second time period;
for each patron in the third video stream, parsing the third video stream to determine a second sentiment of the patron in the third video stream based at least on audio and visual cues of the patron captured in the third video stream during the second time period;
aggregating one or more second sentiments corresponding to the one or more patrons in a second data set indicative of a second overall sentiment of the facility over the second time period;
appending the second data set to an overall data set that includes at least a data set of one or more sentiments over the first time period;
generating an updated sentiment value corresponding to the overall sentiment of the facility spanning the first time period and the second overall sentiment of the facility spanning the second time period, based on the appended data set; and
outputting the updated sentiment of the facility based on the generated updated sentiment value.

### Claim 11 (depends on 8)

The system of claim 8, wherein parsing the first video stream and the second video stream to determine the sentiment of the patron based at least on the audio and visual cues of the patron captured in the first video stream and the second video stream, comprises:
identifying one or more words spoken by evaluating one or more lip movements of the patron.

### Claim 12 (depends on 8)

The system of claim 8, wherein parsing the first video stream and the second video stream to determine the sentiment of the patron based at least on the audio and visual cues of the patron captured in the first video stream and the second video stream, comprises:
identifying one or more facial expressions of the patron by evaluating one or more facial movements of the patron.

### Claim 13 (depends on 8)

The system of claim 8, wherein receiving the first video stream comprising the one or more patrons in the facility and the data indicative of the sentiment of each of the one or more patrons from the camera positioned in the facility comprises:
receiving the first video stream corresponding to a bounded location in the facility.

### Claim 14 (depends on 8)

The system of claim 8, wherein parsing the first video stream and the second video stream to determine the sentiment of the patron based at least on the audio and visual cues of the patron captured in the first video stream and the second video stream, comprises:
identifying one or more key terms uttered by the patron, relating to one or more dimensions of the facility.

### Claim 15

A non-transitory computer readable medium including one or more sequences of instructions that, when executed by one or more processors, causes:
receiving a first video stream comprising one or more patrons in a facility and data indicative of a sentiment of each of the one or more patrons from a camera positioned in the facility;
receiving a second video stream comprising a subset of the one or more patrons in the facility and data indicative of a sentiment of each patron in the subset of the one or more patrons from a mobile device of a first patron of the one or more patrons;
for each patron in the first video stream and the second video stream, parsing the first video stream and the second video stream to determine the sentiment of the patron based at least on audio and visual cues of the patron captured in the first video stream and the second video stream;
generating a sentiment value corresponding to the determined sentiment of each patron in the first video stream and the second video stream, the generated sentiment value corresponding to an overall sentiment for the facility; and
outputting the overall sentiment of the facility based on the generated sentiment value.

### Claim 16 (depends on 15)

The non-transitory computer readable medium of claim 15, wherein the sentiment of the patron comprises an attitude the patron is conveying toward the facility.

### Claim 17 (depends on 15)

The non-transitory computer readable medium of claim 15, further comprising:
receiving a third video stream, the third video stream comprising a second one or more patrons in the facility over a second time period succeeding a first time period, the third video stream comprising data indicative of each of the one or more patrons over the second time period;
for each patron in the third video stream, parsing the third video stream to determine a second sentiment of the patron in the third video stream based at least on audio and visual cues of the patron captured in the third video stream during the second time period;
aggregating one or more second sentiments corresponding to the one or more patrons in a second data set indicative of a second overall sentiment of the facility over the second time period;
appending the second data set to an overall data set that includes at least a data set of one or more sentiments over the first time period;
generating an updated sentiment value corresponding to the overall sentiment of the facility spanning the first time period and the second overall sentiment of the facility spanning the second time period, based on the appended data set; and
outputting the updated sentiment of the facility based on the generated updated sentiment value.

### Claim 18 (depends on 15)

The non-transitory computer readable medium of claim 15, wherein parsing the first video stream and the second video stream to determine the sentiment of the patron based at least on the audio and visual cues of the patron captured in the first video stream and the second video stream, comprises:
identifying one or more words spoken by evaluating one or more lip movements of the patron.

### Claim 19 (depends on 15)

The non-transitory computer readable medium of claim 15, wherein parsing the first video stream and the second video stream to determine the sentiment of the patron based at least on the audio and visual cues of the patron captured in the first video stream and the second video stream, comprises:
identifying one or more facial expressions of the patron by evaluating one or more facial movements of the patron.

### Claim 20 (depends on 15)

The non-transitory computer readable medium of claim 15, wherein receiving the first video stream comprising the one or more patrons in the facility and the data indicative of the sentiment of each of the one or more patrons from the camera positioned in the facility comprises:
receiving the first video stream corresponding to a bounded location in the facility.