# US 11,798,128 B2 · Robust frame size error detection and recovery mechanism to minimize frame loss for camera input sub-systems

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,798,128 B2
- Document ID: US11798128B2
- Status: Granted
- Kind: B2
- Application number: 16/745,589
- Filed: 2020-01-17
- Published: 2021-07-08
- Granted: 2023-10-24
- Earliest claimed: 2020-01-02
- Assignees: Texas Instruments Incorporated
- Applicants: Texas Instruments Incorporated
- Inventors: Brian Okchon Chae; Niraj Nandan; Anthony Joseph Lell; Mihir Mody
- Web page: https://patentgrep.com/patent/US11798128B2

## CPC

- G06T 3/4023: based on decimating pixels or lines of pixels; based on inserting pixels or lines of pixels
- G06V 10/32: Normalisation of the pattern dimensions
- G06V 10/96: Management of image or video recognition tasks
- G06V 10/993: {Evaluation of the quality of the acquired pattern}
- H04N 1/00005: {relating to image data}
- H04N 1/00021: {Picture signal circuits}
- H04N 1/00034: {Measuring, i.e. determining a quantity by comparison with a standard (H04N1/00031 takes precedence)}
- H04N 1/00082: {Adjusting or controlling (interrupting an operation H04N1/00915; inhibiting an operation H04N1/00925)}
- G06T 2207/10016: Video; Image sequence
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

An image data frame is received from an external source. An error concealment operation is performed on the received image data frame in response to determining that a first frame size of the received image data frame is erroneous. The first frame size of the image data frame is determined to be erroneous based on at least one frame synchronization signal associated with the image data frame. An image processing operation is performed on the received image data frame on which the error concealment operation has been performed, thereby enabling an image processing module to perform the image processing operation without entering into a deadlock state and thereby prevent a host processor from having to execute hardware resets of deadlocked modules.

## Claims

### Claim 1

An image processing device, comprising:
an error handling circuit configured to:
receive an image dataframe;
determine that a size of the image data frame is erroneous based on a first marker indicating a start of the image data frame and further based on a second marker indicating an end of the image data frame; and
perform an error concealment operation on the image data frame in response to determining that the size of the image data frame is erroneous, the error concealment operation including discarding data of a remainder of the image data frame being received by the error handling circuit, and
when the image data frame is determined to be undersized, determine that pixel data of at least one of a plurality of lines of the image data frame is missing, and in response, generate dummy data for the missing pixel data, and
when the image data frame is determined to be oversized, generate dummy data to complete the image data frame; and
an image processing circuit that is communicatively coupled to receive, from the error handling circuit, and perform an image processing operation on, the image data frame on which the error concealment operation has been performed.

### Claim 2 (depends on 1)

The image processing device according to claim 1, wherein, in performing the error concealment operation when the image data frame is determined to be undersized and pixel data of at least one of a plurality of lines of the image data frame is determined to be missing, the error handling circuit is further configured to:
generate the dummy data for the missing pixel data so that the size of the image data frame becomes equal to a reference size.

### Claim 3 (depends on 2)

The image processing device according to claim 2, wherein the error handling circuit generates the dummy data by repeating data of a last received good pixel of the image data frame.

### Claim 4 (depends on 1)

The image processing device according to claim 1, wherein to determine that the image data frame is oversized, the error handling circuit is further configured to determine that the size of the image frame is larger than a reference size, and in response to determining that the size of the image frame is larger than the reference size, the error handling circuit is further configured to:
generate the dummy data to complete the image data frame so that the size of the completed image data frame becomes equal to the reference size.

### Claim 5 (depends on 4)

The image processing device according to claim 4, wherein to determine that the size of the image data frame is larger than the reference size, the error handling circuit is configured to determine that a number of pixels for which data is received by the error handling circuit for at least one of a plurality of lines of the image data frame is greater than a reference number of pixels per line for the image data frame.

### Claim 6 (depends on 4)

The image processing device according to claim 4, wherein to determine that the size of the image data frame is larger than the reference size, the error handling circuit is configured to determine that a number of lines for which data is received by the error handling circuit for the image data frame is greater than a reference number of lines of the image data frame.

### Claim 7 (depends on 1)

The image processing device according to claim 1, wherein the error handling circuit is further configured to detect a frame sync signal protocol violation for the image data frame based on at least one of the first marker, the second marker, or a third marker,
wherein the first marker comprises a vertical-start signal that is active while receiving data of a first pixel of the image data frame by the error handling circuit,
wherein the second marker comprises a vertical-end signal that is active while receiving data of a last pixel of the image data frame by the error handling circuit, and
wherein the third marker comprises:
a horizontal-start signal that is active while receiving data of a first pixel of each of a plurality of lines of the image data frame by the error handling circuit; or
a horizontal-end signal that is active while receiving data of a last pixel of each of the plurality of lines of the image data frame by the error handling circuit.

### Claim 8 (depends on 7)

The image processing device according to claim 7, wherein the error handling circuit is further configured to interrupt a host processor in response to detecting the frame sync signal protocol violation for the image data frame.

### Claim 9 (depends on 1)

The image processing device according to claim 1, further comprising an image data receiver circuit communicatively coupled to the error handling circuit, wherein the image data receiver circuit is configured to receive a continuous stream of image data frames, and wherein the error handling circuit is configured to perform the error concealment operation for each image data frame of the continuous stream in response to determining that a size of the corresponding image data frame is erroneous.

### Claim 10 (depends on 1)

The image processing device according to claim 1, further comprising an error detection circuit that is communicatively coupled to the error handling circuit,
wherein the size of the image data frame is a first size, and
wherein the error detection circuit is configured to determine whether a second fame-size of the image data frame is erroneous based on a reference size.

### Claim 11 (depends on 10)

The image processing device according to claim 10, wherein the error detection circuit is further configured to:
determine whether the second size of the image data frame is larger than the reference size; and
responsive to determining that the second size is larger than the reference size, discard extra pixel data of one or more lines of the image data frame so that the second size becomes equal to the reference size.

### Claim 12 (depends on 1)

The image processing device according to claim 1, wherein the error handling circuit is further configured to interrupt a host processor in response to determining that the size of the image data frame is erroneous.

### Claim 13 (depends on 12)

The image processing device according to claim 12, wherein the error handling circuit is further configured to perform the error concealment operation for each of a plurality of lines of the image data frame, one line at a time.

### Claim 14 (depends on 13)

The image processing device according to claim 13, wherein the error handling circuit is further configured to report error information to the host processor, wherein the error information includes at least one of error location information and frame confidence value information.

### Claim 15

An image processing method, comprising:
receiving an image data frame;
determining that a size of the image data frame is erroneous based on a first marker indicating a start of the image data frame and further based on a second marker indicating an end of the image data frame;
performing an error concealment operation on the image data frame in response to determining that the size of the image data frame is erroneous, wherein the error concealment operation includes:
discarding data of a remainder of the image data frame being received by the error handling circuit, and
when the image frame is determined to be undersized and pixel data of at least one of a plurality of lines of the image data frame is determined to be missing, generating dummy data for the missing pixel data, and
when the image data frame is determined to be oversized, generating dummy data to complete the image data frame; and
performing an image processing operation on the image data frame on which the error concealment operation has been performed.

### Claim 16 (depends on 15)

The image processing method according to claim 15, wherein generating the dummy data for the missing pixel data includes:
generating the dummy data for the missing pixel data so that the size of the image data frame becomes equal to a reference size.

### Claim 17 (depends on 15)

The image processing method according to claim 15, wherein generating the dummy data to complete the image data frame includes:
generating the dummy data to complete the image data frame so that the size of the completed frame becomes equal to the reference size.

### Claim 18 (depends on 15)

The image processing method according to claim 15, further comprising:
detecting a frame sync signal protocol violation for the image data frame based on at least one of the first marker, the second marker, or a third marker,
wherein the first marker comprises a vertical-start signal that is active while receiving data of a first pixel of the image data frame,
wherein the second marker comprises a vertical-end signal that is active while receiving data of a last pixel of the image data frame,
wherein the third marker comprises:
a horizontal-start signal that is active while receiving data of a first pixel of each of a plurality of lines of the image data frame; or
a horizontal-end signal that is active while receiving data of a last pixel of each of the plurality of lines of the image data frame.

### Claim 19 (depends on 1)

The image processing method according to claim 1, wherein the size of the image data frame is a first size, the method further comprising determining whether a second size of the image data frame is erroneous based on a reference size,
wherein the second size of the image data frame is measured by a first module of an image processing device that is upstream of a second module of the image processing device that performs the error concealment operation.

### Claim 20 (depends on 19)

The image processing method according to claim 19, further comprising:
determining whether the second size of the image data frame is larger than the reference sine;
responsive to determining that the second size is larger than the reference size, discarding, by the first module, extra pixel data of one or more lines of the image data frame so that the second size becomes equal to the reference size; and
transmitting the image data frame with the discarded extra pixel data to the second module.

### Claim 21 (depends on 15)

The image processing method according to claim 15, further comprising interrupting a host processor in response to determining that the size of the image data frame is erroneous.

### Claim 22

A non-transitory computer readable medium comprising instructions stored thereon that, when executed by one or more processors, cause the one or more processors to:
receive an image data frame;
determine that a size of the image data frame is erroneous based on a first marker indicating a start of the image data frame and further based on a second marker indicating an end of the image data frame;
perform an error concealment operation on the image data frame in response to determining that the size of the image data frame is erroneous, wherein, in performing the error concealment operation, the one or more processors:
discard data of a remainder of the image data frame being received by the error handling circuit, and
when the image frame is determined to be undersized and pixel data of at least one of a plurality of lines of the image data frame is determined to be missing, generate dummy data for the missing pixel data, and
when the image data frame is determined to be oversized, generate dummy data to complete the image data frame, and
perform an image processing operation on the image data frame on which the error concealment operation has been performed.
