# US 2026/0073473 A1 · System and method for maximization of image resolution and monocular depth estimation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0073473 A1
- Document ID: US20260073473A1
- Status: Published application
- Kind: A1
- Application number: 18/829,840
- Filed: 2024-09-10
- Published: 2026-03-12
- Earliest claimed: 2024-09-10
- Assignees: GM Global Technology Operations LLC
- Applicants: GM Global Technology Operations LLC
- Inventors: Chao-Hung Lin; Sai Vishnu Aluru; Alexander Lesnick
- Web page: https://patentgrep.com/patent/US20260073473A1

## CPC

- G06T 3/4007: based on interpolation, e.g. bilinear interpolation (image demosaicing G06T3/4015; edge-driven or edge-based scaling G06T3/403)
- G06T 3/403: Edge-driven scaling; Edge-based scaling
- G06T 7/13: Edge detection
- G06T 7/50: Depth or shape recovery
- H04N 25/704: Pixels specially adapted for focusing, e.g. phase difference pixel sets
- G06T 2207/10024: Color image
- G06T 2207/30252: Vehicle exterior; Vicinity of vehicle
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A method includes receiving image data captured by a sensor system indicating an object in an object field of a vehicle, the sensor system including a plurality of micro-lenses and a pixel, each micro-lens of the plurality of micro-lenses corresponding to a sub-pixel of the pixel, each sub-pixel of the pixel having a plurality of phase-pixels. The method also includes identifying a respective phase ratio of the pixel, each of the sub-pixels of the pixel, and each phase-pixel of the plurality of phase-pixels, and identifying, based on the respective phase ratios of each of the phase-pixels, a depth of the object in the object field. The method also includes estimating an edge of the object, rearranging the sub-pixels of the pixel to generate a transformed image file of the object, and generating, for output to a viewing stack and a perception stack, the transformed image file.

## Claims

### Claim 1

A computer-implemented method when executed on data processing hardware causes the data processing hardware to perform operations comprising:
receiving image data captured by a sensor system of a vehicle and indicating an object in an object field of the vehicle, the sensor system comprising a plurality of micro-lenses and a pixel, each micro-lens of the plurality of micro-lenses corresponding to a sub-pixel of the pixel, each sub-pixel of the pixel having a plurality of phase-pixels;
identifying a respective phase ratio of:
the pixel;
each of the sub-pixels of the pixel; and
each phase-pixel of the plurality of phase-pixels;
identifying, based on the respective phase ratios of each of the phase-pixels, a depth of the object in the object field of the vehicle;
estimating an edge of the object in the object field of the vehicle;
rearranging the phase-pixels of the pixel to generate a transformed image file of the object; and
generating, for output to a viewing stack and a perception stack, the transformed image file of the object.

### Claim 2 (depends on 1)

The method of claim 1, wherein the operations further comprise:
identifying spatial information of the plurality of the phase-pixels;
receiving sensor gain ratios captured by the sensor system of the vehicle; and
interpolating the respective phase ratios of the pixel, the sub-pixels, and the phase-pixels based on the spatial information from the plurality of phase-pixels and the sensor gain ratios to generate an image of the object.

### Claim 3 (depends on 1)

The method of claim 1, wherein estimating the edge of the object in the object field comprises refining the edge of the object based on the depth of the object in the object field.

### Claim 4 (depends on 1)

The method of claim 1, wherein the operations further comprise receiving a-priori information of a camera lens of the sensor system.

### Claim 5 (depends on 4)

The method of claim 4, wherein identifying the depth of the object in the object field of the vehicle is further based on the a-priori information of the camera lens of the sensor system.

### Claim 6 (depends on 1)

The method of claim 1, wherein the operations further comprise receiving a color filter array from a data store in communication with the vehicle.

### Claim 7 (depends on 6)

The method of claim 6, wherein rearranging the phase-pixels of the pixel to generate the transformed image file of the object is based on the received color filter array.

### Claim 8 (depends on 1)

The method of claim 1, wherein the viewing stack includes image processing to render the transformed image file in a display of the vehicle.

### Claim 9 (depends on 1)

The method of claim 1, wherein operations further comprise:
performing a canonical transformation on the image data; and
performing a de-canonical camera transformation of the transformed image file.

### Claim 10 (depends on 1)

The method of claim 1, wherein the operations further comprise determining, based on the depth of the object in the object field of the vehicle, whether the object includes a real object in front of a windshield of the vehicle or a ghost image reflected by the windshield of the vehicle.

### Claim 11

A system comprising:
data processing hardware; and
memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations comprising:
receiving image data captured by a sensor system of a vehicle and indicating an object in an object field of the vehicle, the sensor system comprising a plurality of micro-lenses and a pixel, each micro-lens of the plurality of micro-lenses corresponding to a sub-pixel of the pixel, each sub-pixel of the pixel having a plurality of phase-pixels;
identifying a respective phase ratio of:
the pixel;
each of the sub-pixels of the pixel; and
each phase-pixel of the plurality of phase-pixels;
identifying, based on the respective phase ratios of each of the phase-pixels, a depth of the object in the object field of the vehicle;
estimating an edge of the object in the object field of the vehicle;
rearranging the phase-pixels of the pixel to generate a transformed image file of the object; and
generating, for output to a viewing stack and a perception stack, the transformed image file of the object.

### Claim 12 (depends on 11)

The system of claim 11, wherein the operations further comprise:
identifying spatial information of the plurality of the phase-pixels;
receiving sensor gain ratios captured by the sensor system of the vehicle; and
interpolating the respective phase ratios of the pixel, the sub-pixels, and the phase-pixels based on the spatial information from the plurality of phase-pixels and the sensor gain ratios to generate an image of the object.

### Claim 13 (depends on 11)

The system of claim 11, wherein estimating the edge of the object in the object field comprises refining the edge of the object based on the depth of the object in the object field.

### Claim 14 (depends on 11)

The system of claim 11, wherein the operations further comprise receiving a-priori information of a camera lens of the sensor system.

### Claim 15 (depends on 14)

The system of claim 14, wherein identifying the depth of the object in the object field of the vehicle is further based on the a-priori information of the camera lens of the sensor system.

### Claim 16 (depends on 11)

The system of claim 11, wherein the operations further comprise receiving a color filter array from a data store in communication with the vehicle.

### Claim 17 (depends on 16)

The system of claim 16, wherein rearranging the phase-pixels of the pixel to generate the transformed image file of the object is based on the received color filter array.

### Claim 18 (depends on 11)

The system of claim 11, wherein the viewing stack includes image processing to render the transformed image file in a display of the vehicle.

### Claim 19 (depends on 11)

The system of claim 11, wherein operations further comprise:
performing a canonical transformation on the image data; and
performing a de-canonical camera transformation of the transformed image file.

### Claim 20 (depends on 11)

The system of claim 11, wherein the operations further comprise determining, based on the depth of the object in the object field of the vehicle, whether the object includes a real object in front of a windshield of the vehicle or a ghost image reflected by the windshield of the vehicle.
