# US 12,003,760 B2 · Adaptive foveated encoder and global motion predictor

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,003,760 B2
- Document ID: US12003760B2
- Status: Granted
- Kind: B2
- Application number: 18/345,880
- Filed: 2023-06-30
- Published: 2023-12-28
- Granted: 2024-06-04
- Earliest claimed: 2021-11-15
- Assignees: Intel Corporation
- Applicants: Intel Corporation
- Inventors: Yunbiao Lin; Changliang Wang; Ce Wang; Yongfa Zhou; Bo Zhao; Ping Liu; Jianwei Yang; Zhan Lou; Yu Yang; Yating Wang; Wenyi Tang; Bo Qiu
- Web page: https://patentgrep.com/patent/US12003760B2

## CPC

- H04N 19/527: Global motion vector estimation
- G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking}
- G02B 27/017: {Head mounted}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/012: {Head tracking input arrangements}
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
- H04N 19/00: Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N 19/154: Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion (use of rate-distortion criteria H04N19/147)
- H04N 19/162: User input
- H04N 19/167: Position within a video image, e.g. region of interest [ROI]
- H04N 19/176: the region being a block, e.g. a macroblock
- H04N 19/189: characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding
- H04N 19/52: by predictive encoding
- H04N 21/234345: {the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment}
- H04N 21/234354: {by altering signal-to-noise ratio parameters, e.g. requantization}
- H04N 21/42202: {environmental sensors, e.g. for detecting temperature, luminosity, pressure, earthquakes}
- H04N 21/44218: {Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV programme (methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45)}
- H04N 21/4728: for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region
- H04N 21/6587: Control parameters, e.g. trick play commands, viewpoint selection
- G02B 2027/0138: {comprising image capture systems, e.g. camera}
- G02B 2027/0187: {slaved to motion of at least a part of the body of the user, e.g. head, eye}
- H04N 13/161: Encoding, multiplexing or demultiplexing different image signal components (for multi-view video sequence encoding H04N19/597)
- H04N 19/30: using hierarchical techniques, e.g. scalability (H04N19/63 takes precedence)
- H04N 19/46: Embedding additional information in the video signal during the compression process (H04N19/517, H04N19/68, H04N19/70 take precedence)
- H04N 19/50: using predictive coding (H04N19/61 takes precedence)
- H04N 19/85: using pre-processing or post-processing specially adapted for video compression
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

An embodiment of an adaptive video encoder may include technology to determine headset-related information including at least one of focus-related information and motion-related information, and determine one or more video encode parameters based on the headset-related information. Other embodiments are disclosed and claimed.

## Claims

### Claim 1

A semiconductor package apparatus, comprising:
a substrate; and
logic coupled to the substrate, wherein the logic is at least partly implemented in one or more of configurable logic and fixed-functionality hardware logic, the logic to:
obtain encoding statistics from a previous encoded frame, wherein the encoding statistics include a fovea area quantization parameter (FVQP), and wherein the FVQP is based on a frame level quantization parameter (FRQP) and a delta parameter;
determine whether the FVQP exceeds a threshold;
apply encoding settings to a current frame that are the same as encoding settings for the previous encoded frame, when the FVQP does not exceed the threshold;
modify the encoding settings for the previous encoded frame and apply the modified encoding settings to the current frame, when the FVQP exceeds the threshold; and
encode the current frame with the applied encoding settings.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein to modify the encoding settings for the previous encoded frame includes to determine, based on the FRQP, whether there is room to adjust the delta parameter.

### Claim 3 (depends on 2)

The apparatus of claim 2, wherein the encoding settings for the previous encoded frame are to be modified based on adjusting the delta parameter when there is room to adjust the delta parameter.

### Claim 4 (depends on 2)

The apparatus of claim 2, wherein the encoding settings for the previous encoded frame are to be modified based on applying a new group of picture (GOP) size when there is no room to adjust the delta parameter.

### Claim 5 (depends on 4)

The apparatus of claim 4, wherein the new GOP size is to be determined based on incrementing a GOP size from the previous frame.

### Claim 6 (depends on 5)

The apparatus of claim 5, wherein the new GOP size is to be constrained to a maximum GOP size threshold.

### Claim 7 (depends on 1)

The apparatus of claim 1, further comprising a hardware encoder to encode the current frame.

### Claim 8 (depends on 7)

The apparatus of claim 7, further comprising a virtual reality renderer to render the current frame.

### Claim 9

At least one non-transitory computer readable medium, comprising a set of instructions which, when executed by a computing device, cause the computing device to:
obtain encoding statistics from a previous encoded frame, wherein the encoding statistics include a fovea area quantization parameter (FVQP), and wherein the FVQP is based on a frame level quantization parameter (FRQP) and a delta parameter;
determine whether the FVQP exceeds a threshold;
apply encoding settings to a current frame that are the same as encoding settings for the previous encoded frame, when the FVQP does not exceed the threshold;
modify the encoding settings for the previous encoded frame and apply the modified encoding settings to the current frame, when the FVQP exceeds the threshold; and
encode the current frame with the applied encoding settings.

### Claim 10 (depends on 9)

The at least one non-transitory computer readable medium of claim 9, wherein to modify the encoding settings for the previous encoded frame includes to determine, based on the FRQP, whether there is room to adjust the delta parameter.

### Claim 11 (depends on 10)

The at least one non-transitory computer readable medium of claim 10, wherein the encoding settings for the previous encoded frame are to be modified based on adjusting the delta parameter when there is room to adjust the delta parameter.

### Claim 12 (depends on 10)

The at least one non-transitory computer readable medium of claim 10, wherein the encoding settings for the previous encoded frame are to be modified based on applying a new group of picture (GOP) size when there is no room to adjust the delta parameter.

### Claim 13 (depends on 12)

The at least one non-transitory computer readable medium of claim 12, wherein the new GOP size is to be determined based on incrementing a GOP size from the previous frame.

### Claim 14 (depends on 13)

The at least one non-transitory computer readable medium of claim 13, wherein the new GOP size is to be constrained to a maximum GOP size threshold.

### Claim 15

A method of adaptive encoding, comprising:
obtaining encoding statistics from a previous encoded frame, wherein the encoding statistics include a fovea area quantization parameter (FVQP), and wherein the FVQP is based on a frame level quantization parameter (FRQP) and a delta parameter;
determining whether the FVQP exceeds a threshold;
applying encoding settings to a current frame that are the same as encoding settings for the previous encoded frame, when the FVQP does not exceed the threshold;
modifying the encoding settings for the previous encoded frame and applying the modified encoding settings to the current frame, when the FVQP exceeds the threshold; and
encoding the current frame with the applied encoding settings.

### Claim 16 (depends on 15)

The method of claim 15, wherein modifying the encoding settings for the previous encoded frame includes determining, based on the FRQP, whether there is room to adjust the delta parameter.

### Claim 17 (depends on 16)

The method of claim 16, wherein the encoding settings for the previous encoded frame are modified based on adjusting the delta parameter when there is room to adjust the delta parameter.

### Claim 18 (depends on 16)

The method of claim 16, wherein the encoding settings for the previous encoded frame are modified based on applying a new group of picture (GOP) size when there is no room to adjust the delta parameter.

### Claim 19 (depends on 18)

The method of claim 18, wherein the new GOP size is determined based on incrementing a GOP size from the previous frame.

### Claim 20 (depends on 19)

The method of claim 19, wherein the new GOP size is constrained to a maximum GOP size threshold.