# US 2022/0092753 A1 · Image processing systems and methods of using the same

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0092753 A1
- Document ID: US20220092753A1
- Status: Published application
- Kind: A1
- Application number: 17/479,388
- Filed: 2021-09-20
- Published: 2022-03-24
- Earliest claimed: 2020-09-22
- Assignees: Boston Scientific Scimed, Inc.
- Applicants: Boston Scientific Scimed, Inc.
- Inventors: George Wilfred DUVAL; Kirsten VIERING; Louis J. BARBATO; Gang HU
- Web page: https://patentgrep.com/patent/US20220092753A1

## CPC

- G06T 5/40: using histogram techniques
- G06T 5/90: Dynamic range modification of images or parts thereof
- A61B 1/05: characterised by the image sensor, e.g. camera, being in the distal end portion
- A61B 1/0676: {at distal tip of an endoscope}
- G06T 3/40: Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T 5/50: using two or more images, e.g. averaging or subtraction
- H04N 23/56: provided with illuminating means
- H04N 23/80: Camera processing pipelines; Components thereof
- H04N 23/843: {Demosaicing, e.g. interpolating colour pixel values}
- G06T 2207/10016: Video; Image sequence
- G06T 2207/10068: Endoscopic image
- G06T 2207/20021: Dividing image into blocks, subimages or windows
- G06T 2207/30028: Colon; Small intestine
- H04N 23/555: {for picking-up images in sites, inaccessible due to their dimensions or hazardous conditions, e.g. endoscopes or borescopes}
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A method is provided for enhancing video images in a medical device. The method includes receiving a first image frame and a second image frame from an image sensor. First image sub-blocks are generated by dividing the first image frame. Second image sub-blocks are generated by dividing the second image frame based on the first image sub-blocks. Histogram data of the first image sub-blocks is generated. Histogram data of the second image sub-blocks is generated based on the histogram data of the first image sub-blocks. A histogram enhanced image frame is generated based on the histogram data of the second image sub-blocks. A video image stream is generated based on the histogram enhanced image frame.

## Claims

### Claim 1

A method of enhancing video images in a medical device, the method comprising:
receiving a first image frame and a second image frame from an image sensor;
generating first image sub-blocks by dividing the first image frame;
generating second image sub-blocks by dividing the second image frame based on the first image sub-blocks;
generating histogram data of the first image sub-blocks;
generating histogram data of the second image sub-blocks based on the histogram data of the first image sub-blocks;
generating a histogram enhanced image frame based on the histogram data of the second image sub-blocks; and
generating a video image stream based on the histogram enhanced image frame.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
performing one or more iterations of sizing each of the first image sub-blocks; and
determining a size of each of the first image sub-blocks based on the one or more iterations of the first image sub-blocks.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
determining a size of each of the second image sub-blocks based on one or more iterations of sizing each of the second image sub-blocks and/or a size of each of the first image sub-blocks.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
generating the histogram data of the first image sub-blocks by:
generating a histogram for each of the first image sub-blocks; and
determining a clipping level of each histogram of the first image sub-blocks.

### Claim 5 (depends on 4)

The method of claim 4, wherein the clipping level of each histogram of the first image sub-blocks is determined based on one or more iterations of clipping each histogram of the first image sub-blocks.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
generating the histogram data of the second image sub-blocks by:
generating a histogram for each of the second image sub-blocks; and
determining a clipping level of each histogram of the second image sub-blocks.

### Claim 7 (depends on 6)

The method of claim 6, wherein the histogram for each of the second image sub-blocks is generated at least based on a cumulative distribution function.

### Claim 8 (depends on 6)

The method of claim 6, wherein the clipping level of each histogram of the second image sub-blocks is determined based on one or more iterations of clipping each histogram of the second image sub-blocks and/or a clipping level of each histogram of the first image sub-blocks.

### Claim 9 (depends on 6)

The method of claim 6, wherein generating the histogram data of the second image sub-blocks further comprises:
distributing clipped data of each histogram of the second image sub-blocks to the histogram for each of the second image sub-blocks.

### Claim 10 (depends on 1)

The method of claim 1, wherein generating the histogram enhanced image frame comprises interpolating the histogram data of the second image sub-blocks.

### Claim 11 (depends on 1)

The method of claim 1, further comprising:
receiving a plurality of image frames from the image sensor; and
generating sub-blocks in each of the plurality of image frames.

### Claim 12 (depends on 11)

The method of claim 11, wherein the plurality of image frames is a least three frames.

### Claim 13 (depends on 11)

The method of claim 11, further comprising:
generating a histogram for each sub-block of the plurality of image frames; and
determining a clipping level of each sub-block of the plurality of image frames based on an average histogram data of the plurality of image frames.

### Claim 14 (depends on 11)

The method of claim 11, further comprising:
generating the histogram data of the second image sub-blocks by:
generating a histogram for each of the second image sub-blocks; and
determining a clipping level of each histogram of the second image sub-blocks.

### Claim 15 (depends on 14)

The method of claim 14, wherein the clipping level of each histogram of the second image sub-blocks is determined based on one or more iterations of clipping each histogram of the second image sub-blocks and/or an average histogram data of the plurality of image frames.

### Claim 16

A medical device, comprising:
a shaft;
an image sensor coupled to a distal end of the shaft;
at least one illumination device coupled to the distal end of the shaft;
one or more computer readable media storing instructions performing image processing to enhance video images; and
one or more processors configured to execute the instructions to perform the image processing comprising:
receiving a first image frame and a second image frame from the image sensor;
generating first image sub-blocks by dividing the first image frame;
generating second image sub-blocks of the second image frame based on the first image sub-blocks;
generating histogram data of the first image sub-blocks;
generating histogram data of the second image sub-blocks based on the histogram data of the first image sub-blocks;
generating a histogram enhanced image frame based on the histogram data of the second image sub-blocks; and
generating a video image stream based on the histogram enhanced image frame.

### Claim 17 (depends on 16)

The device of claim 16, wherein generating the second image sub-blocks comprises:
determining a size of each of the second image sub-blocks based on one or more iterations of sizing each of the second image sub-blocks and/or a size of each of the first image sub-blocks.

### Claim 18 (depends on 16)

The device of claim 16, wherein generating the histogram data of the second image sub-blocks comprises:
determining a clipping level of each histogram of the second image sub-blocks is based on one or more iterations of clipping each histogram of the second image sub-blocks and/or a clipping level of each histogram of the first image sub-blocks.

### Claim 19 (depends on 16)

The device of claim 16, wherein generating the histogram data of the second image sub-blocks comprises:
determining wherein a clipping level of each histogram of the second image sub-blocks is based on one or more iterations of clipping each histogram of the second image sub-blocks and/or an average histogram data of a plurality of image frames.

### Claim 20

A non-transitory computer-readable medium storing instructions for enhancing video images, the instructions, when executed by one or more processors, causing the one or more processors to perform operations comprising:
receiving a first image frame and a second image frame from an image sensor;
generating first image sub-blocks by dividing the first image frame;
generating second image sub-blocks of the second image frame based on the first image sub-blocks;
generating histogram data of the first image sub-blocks;
generating histogram data of the second image sub-blocks based on the histogram data of the first image sub-blocks;
generating a histogram enhanced image frame based on the histogram data of the second image sub-blocks; and
generating a video image stream based on the histogram enhanced image frame.
