# US 12,223,524 B1 · Systems and methods for selecting product placement locations and products

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,223,524 B1
- Document ID: US12223524B1
- Status: Granted
- Kind: B1
- Application number: 17/847,775
- Filed: 2022-06-23
- Granted: 2025-02-11
- Earliest claimed: 2022-06-23
- Assignees: Amazon Technologies, Inc.
- Applicants: Amazon Technologies, Inc.
- Inventors: Mauricio Alejandro Flores Rios; Han-Kai Hsu; Yujia Chen; Linda Liu; Yash Chaturvedi
- Web page: https://patentgrep.com/patent/US12223524B1

## CPC

- G06Q 30/0242: Determining effectiveness of advertisements
- G06V 20/49: {Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes}
- H04N 21/812: {involving advertisement data (advertising per se G06Q30/02)}
- H04N 21/8146: {involving graphical data, e.g. 3D object, 2D graphics}
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Systems, methods, and computer-readable media are disclosed for determining virtual product placement opportunities in a media content and determining product candidates for virtual insertion into the media content. The product placement system may determine shot segments from the media content and for each shot segment may determine candidate product placement locations. The product placement system may determine contextual information from the shot segments and from the contextual information determine candidate products suitable for the product placement locations. The product placement system may determine total screen time for each product placement opportunity as well as quality of each opportunity. For each product and product placement opportunity, the product placement system may determine an expected revenue and a projected insertion cost.

## Claims

### Claim 1

A method comprising:
determining, by a computer processor, first, second, and third shot segments of a video;
determining a first segment quality value of the first shot segment, a second segment quality value of the second shot segment, and a third quality value for the third shot segment;
determining the third quality value does not satisfy a quality threshold value;
analyzing the first shot segment and the second shot segment using a first model trained to determine a first product location corresponding to a first position for placement within the first shot segment and a second product location corresponding to a second position for placement within the second shot segment, wherein the first model is a machine learning computer vision model trained to determine locations in shot segments of the digital media for placement of virtual products;
determining the first product location is the same as the second product location;
determining a first product size and a second product size, the first product size based on the first product location and the second product size based on the second product location;
analyzing the first shot segment and the second shot segment using a second model to determine first contextual data corresponding to the first shot segment and second contextual data corresponding to the second shot segment, wherein the second model is a machine learning model trained to automatically determine contextual data in shot segments of the digital media;
determining a first product and a second product based on the first product location, the second product location, the first contextual data, and the second contextual data;
determining a first contextual value corresponding to the first product and a second contextual value corresponding to the second product, each of the first contextual value and the second contextual value based on the first contextual data and second contextual data;
determining the first contextual value exceeds the second contextual value;
determining a first time period corresponding to the first shot segment and the second shot segment;
determining a first revenue value based on the first product, the first product location, the first product size, the second product location, and the first time period; and
determining a first cost value corresponding to digitally incorporating the first product into the first product location and the second product location.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
determining a third product based on the first contextual data and second contextual data;
determining a second revenue value based on the third product, the first product location, the second product location, and the first time period; and
determining a second cost value corresponding to digitally incorporating the third product into the first product location and the second product location.

### Claim 3 (depends on 2)

The method of claim 2, further comprising:
determining to select the first product based on at least the first contextual value, the first revenue value, and the first cost value.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
determining a third product location corresponding to the first shot segment;
determining a first location quality value for the first product location;
determining a second location quality value for the second product location;
determining a third location quality value for the third product location; and
determining the first location quality value and the second location quality value each exceed the third location quality value.

### Claim 5

A method comprising:
determining, by a computer processor, a first shot segment and a second shot segment of digital media;
analyzing the first shot segment and the second shot segment using a first model to determine a first product location corresponding to a first position for placement within the first shot segment and a second product location corresponding to a second position for placement within the second shot segment, wherein the first model is a machine learning computer vision model trained to determine locations in shot segments of the digital media for placement of virtual products;
determining the first product location is the same as the second product location;
determining a first product size and a second product size, the first product size based on the first product location and the second product size based on the second product location;
analyzing the first shot segment and the second shot segment using a second model to determine first contextual data corresponding to the first shot segment and second contextual data corresponding the second shot segment, wherein the second model is machine learning model trained to automatically determine contextual data in shot segments of the digital media;
determining a first product and a second product based on the first product location, the first contextual data, and the second contextual data;
determining a first revenue value corresponding to the first product and based on the first product, the first product size, and the first product location;
determining a second revenue value corresponding to the second product and based on the second product, the second product size, and the first product location; and
determining the first revenue value exceeds the second revenue value.

### Claim 6 (depends on 5)

The method of claim 5, further comprising:
determining a third shot segment of the digital media;
determining a first segment quality value of the first shot segment, a second segment quality value of the second shot segment, and a third quality value of the third shot segment; and
determining the third quality value does not satisfy a quality threshold value.

### Claim 7 (depends on 5)

The method of claim 5, further comprising:
determining a first contextual value corresponding to the first product and a second contextual value corresponding to the second product, each of the first contextual value and the second contextual value based on the first contextual data and the second contextual data; and
determining the first contextual value exceeds the second contextual value.

### Claim 8 (depends on 7)

The method of claim 7, wherein the first contextual value includes at least one of setting data, actor data, object data, or dimensional data.

### Claim 9 (depends on 5)

The method of claim 5, further comprising:
determining a first cost value corresponding to digitally incorporating the first product into the first product location; and
determining to select the first product based on at least the first revenue value and the first cost value.

### Claim 10 (depends on 5)

The method of claim 5, wherein the machine learning computer vision model is an occlusion model.

### Claim 11 (depends on 5)

The method of claim 5, further comprising determining a first location quality value corresponding to the first product location, wherein the first location quality value is based on a proximity to a center position of the first shot segment and the second shot segment and the first product size.

### Claim 12

A system comprising:
memory configured to store computer-executable instructions, and
at least one computer processor configured to access memory and execute the computer-executable instructions to:
determine a first shot segment and a second shot segment of digital media;
analyze the first shot segment and the second shot segment using a first model to determine a first product location corresponding to a first position for placement within the first shot segment and a second product location corresponding to a second position for placement within the second shot segment, wherein the first model is a machine learning computer vision model trained to determine locations in shot segments of the digital media for placement of virtual products;
determine the first product location is the same as the second product location;
determining a first product size and a second product size, the first product size based on the first product location and the second product size based on the second product location;
analyzing the first shot segment and the second shot segment using a second model to determine first contextual data corresponding to the first shot segment and second contextual data corresponding to the second shot segment, wherein the second model is a machine learning model trained to automatically determine contextual data in shot segments of the digital media;
determine a first product and a second product based on the first product location, the first contextual data, and the second contextual data;
determine a first revenue value corresponding to the first product and based on the first product and the first product location;
determine a second revenue value corresponding to the second product and based on the second product and the first product location; and
determine the first revenue exceeds the second revenue.

### Claim 13 (depends on 12)

The system of claim 12, wherein the at least one computer processor is further configured to access the memory and execute the computer-executable instructions to:
determine a third shot segment of the digital media;
determine a first segment quality value of the first shot segment, a second segment quality value of the second shot segment, and a third quality value of the third shot segment; and
determine the third quality value does not satisfy a quality threshold value.

### Claim 14 (depends on 12)

The system of claim 12, wherein the at least one computer processor is further configured to access the memory and execute the computer-executable instructions to:
determine a first contextual value corresponding to the first product and a second contextual value corresponding to the second product, each of the first contextual value and the second contextual value based on the first contextual data and the second contextual data; and
determine the first contextual value exceeds the second contextual value.

### Claim 15 (depends on 12)

The system of claim 12, wherein the first contextual value includes at least one of setting data, actor data, object data, or dimensional data.

### Claim 16 (depends on 12)

The system of claim 12, wherein the at least one computer processor is further configured to access the memory and execute the computer-executable instructions to:
determine a first cost value corresponding to digitally incorporating the first product into the first product location; and
determine to select the first product based on at least the first revenue value and the first cost value.

### Claim 17 (depends on 12)

The system of claim 12, wherein the machine learning computer vision model is an occlusion model.

### Claim 18 (depends on 12)

The system of claim 12, wherein the at least one computer processor is further configured to access the memory and execute the computer-executable instructions to determine a first location quality value corresponding to the first product location, wherein the first location quality value is based on a proximity to a center position of the first shot segment and the second shot segment and the first product size.