# US 2019/0174058 A1 · Super resolution and color motion artifact correction in a pulsed color imaging system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0174058 A1
- Document ID: US20190174058A1
- Status: Published application
- Kind: A1
- Application number: 16/272,992
- Filed: 2019-02-11
- Published: 2019-06-06
- Earliest claimed: 2013-03-15
- Assignees: DePuy Synthes Products, Inc.
- Applicants: DePuy Synthes Products, Inc.
- Inventors: John Richardson; Laurent Blanquart
- Web page: https://patentgrep.com/patent/US20190174058A1

## CPC

- H04N 23/951: by using two or more images to influence resolution, frame rate or aspect ratio
- G06T 3/4007: based on interpolation, e.g. bilinear interpolation (image demosaicing G06T3/4015; edge-driven or edge-based scaling G06T3/403)
- G06T 3/4053: based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
- H04N 23/12: with one sensor only
- H04N 23/56: provided with illuminating means
- H04N 23/60: Control of cameras or camera modules
- H04N 23/74: by influencing the scene brightness using illuminating means
- H04N 23/843: {Demosaicing, e.g. interpolating colour pixel values}
- H04N 13/239: using two two-dimensional [2D] image sensors having a relative position equal to or related to the interocular distance (H04N13/243 takes precedence)
- H04N 23/555: {for picking-up images in sites, inaccessible due to their dimensions or hazardous conditions, e.g. endoscopes or borescopes}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

The disclosure extends to methods, systems, and computer program products for producing an image in light deficient environments and associated structures, methods and features. The features of the systems and methods described herein may include providing improved resolution and color reproduction.

## Claims

### Claim 1

A digital imaging method for use with an endo scope in ambient light deficient environments comprising:
actuating an emitter to emit a pulse of a wavelength of electromagnetic radiation to cause illumination within the light deficient environment;
wherein said pulse is within a first wavelength range that comprises a first portion of electromagnetic spectrum;
pulsing said emitter at a predetermined interval;
sensing reflected electromagnetic radiation from said pulse with a pixel array;
wherein said pixel array is actuated at a sensing interval that corresponds to the pulse interval of said emitter;
detecting motion of objects being imaged;
increasing resolution for the pixel array by compensating for the detected motion; and
creating a stream of images by combining a plurality of sensed reflected electromagnetic energies into a frame.

### Claim 2 (depends on 1)

The method of claim 1, wherein increasing resolution comprises:
sensing luminescence of a plurality of neighboring pixels to gather luminance data;
bilinear interpolating the luminance data into a first upscaled data set;
bicubic interpolating the luminance data into a second upscaled data set;
creating a baseline with no interpolation of the luminance data into a third upscaled data set.

### Claim 3 (depends on 1)

The method of claim 1, wherein said sensing process comprises sensing luminance and linear sums of luminance plus chrominance in adjacent images in a stream of images forming a video stream.

### Claim 4 (depends on 3)

The method of claim 3, further comprising indexing frames within the video stream with a rotating frame index.

### Claim 5 (depends on 4)

The method of claim 4, wherein the rotating frame index comprises four counts.

### Claim 6 (depends on 5)

The method of claim 5, further comprising reconstructing the video stream by combining luminance and chrominance data from a prior indexed frame.

### Claim 7 (depends on 5)

The method of claim 5, further comprising reconstructing the video stream by combining luminance and chrominance data from a prior indexed frame and following indexed frame.

### Claim 8 (depends on 5)

The method of claim 5, further comprising reconstructing a frame for luminance and two frames for chrominance.

### Claim 9 (depends on 5)

The method of claim 5, further comprising reconstructing the video stream by combining luminance and chrominance data from a plurality of prior indexed frames and a plurality of latter indexed frames for increased resolution and accuracy.

### Claim 10 (depends on 2)

The method of claim 2, wherein the first upscaled data set is used for block matching.

### Claim 11 (depends on 2)

The method of claim 2, wherein the second upscaled data set is used for fall back pixel data.

### Claim 12 (depends on 2)

The method of claim 2, wherein the third upscaled data set forms the baseline for the resolution-enhanced data set.

### Claim 13 (depends on 1)

The method of claim 1, further comprising segmenting data created by the pixel array into segments of pixels and nearest neighbors.

### Claim 14 (depends on 13)

The method of claim 13, further comprising shifting each segment of pixels in the x direction and comparing with a neighboring frame at the same resolution, in order to determine motion of an object being imaged in the x direction.

### Claim 15 (depends on 14)

The method of claim 14, further comprising shifting each segment of pixels in the x direction in sub-pixel increments and comparing to the first up-scaled data set for greater precision of motion detection in the x direction.

### Claim 16 (depends on 14)

The method of claim 14, further comprising shifting each segment of pixels in the y direction and comparing with a neighboring frame at the same resolution, in order to determine motion of an object being imaged in the y direction.

### Claim 17 (depends on 16)

The method of claim 16, further comprising shifting each segment of pixels in the y direction in sub-pixel increments and comparing to the first up-scaled data set for greater precision of motion detection in the y direction.

### Claim 18 (depends on 16)

The method of claim 16, further comprising determining the vector of the motion of the object by combining the x and y motion of the of the object being imaged.

### Claim 19 (depends on 18)

The method of claim 18, further comprising estimating motion to combine data from multiple luminance frames into a single, higher resolution luminance frame.

### Claim 20 (depends on 19)

The method of claim 19, wherein said process is repeated for every frame containing luminance data in a continuous sequence.

### Claim at position 21

21-48. (canceled)
