# US 2022/0021888 A1 · Systems and methods to encode regions-of-interest based on video content detection

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0021888 A1
- Document ID: US20220021888A1
- Status: Published application
- Kind: A1
- Application number: 16/931,019
- Filed: 2020-07-16
- Published: 2022-01-20
- Earliest claimed: 2020-07-16
- Assignees: Alibaba Group Holding Limited
- Applicants: Alibaba Group Holding Limited
- Inventors: Minghai QIN; Yen-kuang CHEN; Tae Meon BAE; Guanlin WU; Yuanwei FANG; Sicheng LI
- Web page: https://patentgrep.com/patent/US20220021888A1

## CPC

- H04N 19/167: Position within a video image, e.g. region of interest [ROI]
- A63F 13/355: Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client
- A63F 13/52: involving aspects of the displayed game scene
- A63F 13/822: Strategy games; Role-playing games (A63F13/825, A63F13/828 take precedence)
- G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI]
- G06V 20/41: {Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49)}
- G06V 20/46: {Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames}
- H04N 19/115: Selection of the code volume for a coding unit prior to coding
- H04N 19/124: Quantisation
- H04N 19/136: Incoming video signal characteristics or properties
- H04N 19/146: Data rate or code amount at the encoder output
- H04N 19/147: according to rate distortion criteria (rate-distortion as a criterion for motion estimation H04N19/567)
- H04N 19/17: the unit being an image region, e.g. an object
- H04N 19/172: the region being a picture, frame or field
- H04N 19/177: the unit being a group of pictures [GOP]
- A63F 13/80: Special adaptations for executing a specific game genre or game mode
- G06V 10/82: using neural networks
- G06V 2201/10: Recognition assisted with metadata
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Video coding techniques including variable bitrate encoding based on regions-of-interest (ROIs) and the type of the video content, the type of sets of frames of the video content, the type of scenes of the video content, or the like.

## Claims

### Claim 1

A video processing unit comprising:
a content detection engine configured to receive input video content and to determine a content genre of the input video content or content genre of one or more portions of the input video content;
a region-of-interest (ROI) generator configured to receive an indication of the content genre of the input video content or one or more portions of the input video content and to select one or more predetermined regions of corresponding video frames of the input video content or one or more portions of the input video content as one or more ROIs based on the content genre of the input video content or content genre of the one or more portions of the input video content;
a rate controller configured to receive the one or more ROIs and to determine a first encoder rate for the one or more ROIs and a second encoder rate for one or more non-ROIs of the corresponding video frames; and
a video encoder configured to receive the input video content, the one or more ROIs and the one or more non-ROIs, and the first and second encoder rates, and to generate a compressed bit stream of the input video content using the first encoder rate for the one or more ROIs and the second encoder rate for the one or more non-ROIs.

### Claim 2 (depends on 1)

The video processing unit of claim 1, wherein the content detection engine is configured to determine the content genre from metadata of the input video content.

### Claim 3 (depends on 1)

The video processing unit of claim 1, wherein content detection engine is configured to determine the content genre using one or more artificial intelligence models.

### Claim 4 (depends on 1)

The video processing unit of claim 1, wherein the one or more ROIs can be in a predetermined location.

### Claim 5 (depends on 1)

The video processing unit of claim 1, wherein the one or more ROIs can be a predetermined size.

### Claim 6 (depends on 1)

The video processing unit of claim 1, wherein the one or more ROIs can be positioned relative to one or more given features of the input video content.

### Claim 7 (depends on 1)

The video processing unit of claim 1, wherein the first encoding rate is greater than the second encoding rate.

### Claim 8 (depends on 1)

The video processing unit of claim 1, wherein the input video content comprises video game content.

### Claim 9 (depends on 8)

The video processing unit of claim 8, wherein:
a first content genre comprises a first person perspective action video game; and
a second content genre comprises a strategy video game.

### Claim 10

A video processing unit comprising:
a content detection engine including,
a frame sampler configured to sample sets of frames or scenes of input video content; and
a scene classifier configured to determine a content genre of each set of frames or each scene;
a region-of-interest (ROI) generator configured to receive an indication of the content genre of each set of frames or each scene of the input video content and to select one or more predetermined regions of a corresponding video frame as one or more ROIs based on the determined content genre; and
a rate controller configured to receive indications of the one or more ROIs and to determine a first encoder rate for the one or more ROIs and a second encoder rate for one or more non-ROIs.

### Claim 11 (depends on 10)

The video processing unit of claim 10; further comprising:
a video encoder configured to generate a compressed bit stream based on the first encoder rate for the one or more ROIs and the second encoder rate for the one or more non-ROIs of corresponding sets of frames or scenes.

### Claim 12 (depends on 10)

The video processing unit of claim 10, wherein the first encoding rate is greater than the second encoding rate.

### Claim 13 (depends on 10)

The video processing unit of claim 10, wherein:
the one or more ROIs are in a predetermined location for a first content genre; and
the one or more ROIs are in a variable location for a second content genre.

### Claim 14 (depends on 13)

The video processing unit of claim 13, wherein the variable location comprises a position relative to one or more given features in the input video content.

### Claim 15 (depends on 13)

The video processing unit of claim 13, wherein:
the first content genre includes first person perspective action video games; and
the second content genre comprises strategy video games.

### Claim 16

A method of video processing comprising:
detecting a content genre of a given portion of video content;
generating one or more regions-of-interest (ROIs) for the given portion of the video content based on the content genre of the given portion of the video content;
determining a first encoding rate for the one or more ROIs and a second encoding rate for one or more non-ROIs of the given portion of the video content; and
encoding the one or more ROIs at the first encoding rate and the one or more non-ROIs at the second encoding rate of the given portion of the video content to generate a compressed bitstream.

### Claim 17 (depends on 16)

The method according to claim 16, wherein the first encoding rate is greater than the second encoding rate.

### Claim 18 (depends on 16)

The method according to claim 16, further comprising:
streaming the compressed bitstream.

### Claim 19 (depends on 16)

The method according to claim 16, wherein the video content comprises video game content.

### Claim 20 (depends on 19)

The method according to claim 19, wherein:
a first content genre of video content includes a first person perspective action video game; and
a second content genre of video content includes a strategy video game.

### Claim 21 (depends on 20)

The method according to claim 20, wherein:
the one or more ROIs are in a predetermined location for a first content genre; and
the one or more ROIs are in a location relative to one or more given features in the video content for a second content genre.
