# US 2022/0044386 A1 · Hardware friendly fixed-point approximations of video quality metrics

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0044386 A1
- Document ID: US20220044386A1
- Status: Published application
- Kind: A1
- Application number: 17/394,270
- Filed: 2021-08-04
- Published: 2022-02-10
- Earliest claimed: 2020-08-05
- Applicants: Facebook, Inc.
- Inventors: Deepa Palamadai Sundar; Xing Cindy Chen; Hsiao-Chiang Chuang; Visalakshi Vaduganathan; Harikrishna Madadi Reddy
- Web page: https://patentgrep.com/patent/US20220044386A1

## CPC

- G06T 7/0002: {Inspection of images, e.g. flaw detection}
- H04N 19/423: characterised by memory arrangements (H04N19/433 takes precedence)
- G06F 12/0875: with dedicated cache, e.g. instruction or stack
- G06F 12/0897: with two or more cache hierarchy levels (with multilevel cache hierarchies G06F12/0811)
- G06F 17/17: Function evaluation by approximation methods, e.g. inter- or extrapolation, smoothing, least mean square method ({G06F17/18 takes precedence } ; interpolation for numerical control G05B19/18)
- H04N 19/103: Selection of coding mode or of prediction mode
- H04N 19/117: Filters, e.g. for pre-processing or post-processing (sub-band filter banks H04N19/635)
- H04N 19/147: according to rate distortion criteria (rate-distortion as a criterion for motion estimation H04N19/567)
- H04N 19/154: Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion (use of rate-distortion criteria H04N19/147)
- H04N 19/176: the region being a block, e.g. a macroblock
- H04N 19/182: the unit being a pixel
- H04N 19/36: Scalability techniques involving formatting the layers as a function of picture distortion after decoding, e.g. signal-to-noise [SNR] scalability
- H04N 19/40: using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
- H04N 19/523: with sub-pixel accuracy
- H04N 21/44008: {involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)}
- G06T 2207/10016: Video; Image sequence
- G06T 2207/30168: Image quality inspection
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A scalable hardware accelerator configured to compute video quality metrics is disclosed. In some embodiments, an accelerator for video quality metrics comprises an application-specific integrated circuit that includes an interface configured to receive pixel data of a frame of a video being analyzed for quality metric determination and a kernel configured to compute a video quality metric for the received pixel data using a fixed-point hardware approximation of a floating-point based algorithm associated with the video quality metric.

## Claims

### Claim 1

A system, comprising:
an interface of an application-specific integrated circuit configured to receive pixel data of a frame of a video being analyzed for quality metric determination; and
a kernel of the application-specific integrated circuit configured to compute a video quality metric for the received pixel data using a fixed-point hardware approximation of a floating-point based algorithm associated with the video quality metric.

### Claim 2 (depends on 1)

The system of claim 1, wherein the fixed-point hardware approximation comprises an approximation of a non-linear function.

### Claim 3 (depends on 2)

The system of claim 2, wherein the approximation of the non-linear function comprises a piecewise linear function.

### Claim 4 (depends on 3)

The system of claim 3, wherein the piecewise linear function is uniformly or non-uniformly sampled.

### Claim 5 (depends on 2)

The system of claim 2, wherein the approximation of the non-linear function comprises a k-th order polynomial.

### Claim 6 (depends on 1)

The system of claim 1, wherein the fixed-point hardware approximation comprises an approximation of a logarithmic function.

### Claim 7 (depends on 6)

The system of claim 6, wherein an input argument of the approximation of the logarithmic function is scaled to a value comprising a prescribed range.

### Claim 8 (depends on 6)

The system of claim 6, wherein the approximation of the logarithmic function comprises a piecewise linear function with more segments in a region closer to an input argument having a value of one than a region closer to a value of two.

### Claim 9 (depends on 1)

The system of claim 1, wherein the fixed-point hardware approximation preserves numerical stability of the algorithm by using corresponding thresholds for one or more associated parameters of the algorithm.

### Claim 10 (depends on 1)

The system of claim 1, wherein the fixed-point hardware approximation replaces a negative value of a parameter of the algorithm with a non-negative value, wherein the parameter has a mathematical property of being non-negative.

### Claim 11 (depends on 10)

The system of claim 10, wherein the non-negative value is zero.

### Claim 12 (depends on 1)

The system of claim 1, wherein the video quality metric comprises a no reference metric or a full reference metric.

### Claim 13 (depends on 1)

The system of claim 1, wherein the video quality metric comprises a perception-based video quality metric.

### Claim 14 (depends on 13)

The system of claim 13, wherein the perception-based video quality metric comprises one or more of: Structural Similarity Index Measure (SSIM), Multi-Scale SSIM (MS-SSIM), Visual Information Fidelity (VIF), Video Multimethod Assessment Fusion (VMAF), and Detail Loss Metric (DLM).

### Claim 15

A method, comprising:
configuring an interface of an application-specific integrated circuit to receive pixel data of a frame of a video being analyzed for quality metric determination; and
configuring a kernel of the application-specific integrated circuit to compute a video quality metric for the received pixel data using a fixed-point hardware approximation of a floating-point based algorithm associated with the video quality metric.

### Claim 16 (depends on 15)

The method of claim 15, wherein the fixed-point hardware approximation comprises an approximation of a non-linear function.

### Claim 17 (depends on 15)

The method of claim 15, wherein the fixed-point hardware approximation preserves numerical stability of the algorithm by using corresponding thresholds for one or more associated parameters of the algorithm.

### Claim 18

A computer program product embodied in a non-transitory computer readable medium and comprising computer instructions for:
configuring an interface of an application-specific integrated circuit to receive pixel data of a frame of a video being analyzed for quality metric determination; and
configuring a kernel of the application-specific integrated circuit to compute a video quality metric for the received pixel data using a fixed-point hardware approximation of a floating-point based algorithm associated with the video quality metric.

### Claim 19 (depends on 18)

The computer program product of claim 18, wherein the fixed-point hardware approximation comprises an approximation of a non-linear function.

### Claim 20 (depends on 18)

The computer program product of claim 18, wherein the fixed-point hardware approximation preserves numerical stability of the algorithm by using corresponding thresholds for one or more associated parameters of the algorithm.
