# US 2019/0356867 A1 · Image rotation using software for endoscopic applications

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0356867 A1
- Document ID: US20190356867A1
- Status: Published application
- Kind: A1
- Application number: 16/509,297
- Filed: 2019-07-11
- Published: 2019-11-21
- Earliest claimed: 2013-03-15
- Assignees: DePuy Synthes Products, Inc.
- Applicants: DePuy Synthes Products, Inc.
- Inventors: John Richardson; Laurent Blanquart; Jeremiah D. Henley; Donald M. Wichern
- Web page: https://patentgrep.com/patent/US20190356867A1

## CPC

- H04N 5/2628: {Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation}
- A61B 1/00006: {of control signals}
- A61B 1/000095: {for image enhancement}
- A61B 1/00045: {Display arrangement}
- A61B 1/05: characterised by the image sensor, e.g. camera, being in the distal end portion
- H04N 23/50: Constructional details
- H04N 23/54: Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
- H04N 23/6812: {based on additional sensors, e.g. acceleration sensors}
- H04N 23/555: {for picking-up images in sites, inaccessible due to their dimensions or hazardous conditions, e.g. endoscopes or borescopes}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)

## Abstract

The disclosure extends to endoscopic devices and systems for image correction of a rotating sensor. The disclosure allows for the distal image sensor to rotate as the user rotates the lumen with respect to a fixed handpiece. The system includes an angle sensor located at the junction of the rotating lumen and the fixed handpiece. Periodic measurements of angle are used in the system's image processing chain in order to effect suitable software image rotations, thereby providing a final displayed image or video stream with the desired orientation.

## Claims

### Claim 1

An endoscopic device comprising:
a hand-piece;
a lumen comprising a distal tip;
an image sensor disposed within the lumen for providing visualization of an area, wherein the image sensor is located near the distal tip of the lumen;
wherein the lumen is rotatable about an axis of the endoscope and with respect to the hand-piece; and
an image signal processing pipeline that is configured to rotate images to maintain a constant image horizon for a user on a display.

### Claim 2 (depends on 1)

The endoscopic device of claim 1, wherein the device further comprises an angle sensor for detecting an angle of rotation of the hand-piece relative to the lumen, wherein the angle sensor is a rotation-detecting Hall-effect sensor.

### Claim 3 (depends on 2)

The endoscopic device of claim 2, wherein the rotation-detecting Hall-effect sensor is located in the hand-piece.

### Claim 4 (depends on 2)

The endoscopic device of claim 2, wherein the device further comprises a diametrically-polarized magnetic annulus and wherein the rotation-detecting Hall-effect sensor produces a voltage that is used to detect an angle of the diametrically-polarized, magnetic annulus.

### Claim 5 (depends on 1)

The endoscopic device of claim 1, wherein the device further comprises an angle sensor for detecting an angle of rotation of the hand-piece relative to the lumen, wherein the angle sensor produces a voltage that is used to detect the angle of rotation of the hand-piece relative to the lumen.

### Claim 6 (depends on 1)

The endoscopic device of claim 1, wherein the device further comprises an angle sensor for detecting an angle of rotation of the hand-piece relative to the lumen, wherein the angle sensor is a potentiometer.

### Claim 7 (depends on 6)

The endoscopic device of claim 6, wherein the potentiometer comprises a carbon filament, wherein the carbon filament of the potentiometer is disposed on said lumen.

### Claim 8 (depends on 1)

The endoscopic device of claim 1, wherein the device further comprises an angle sensor for detecting an angle of rotation of the hand-piece relative to the lumen, wherein the angle sensor comprises a light source and a photo diode that rotate relative to a gradient disc.

### Claim 9 (depends on 8)

The endoscopic device of claim 8, wherein said photo diode detects the electromagnetic energy from said light source that is reflected by said gradient disc.

### Claim 10 (depends on 1)

The endoscopic device of claim 1, wherein said image sensor incorporates a two-dimensional array of pixels capable of detecting electromagnetic radiation.

### Claim 11 (depends on 1)

The endoscopic device of claim 1, wherein the image signal processing pipeline performs image rotation transformations upon images captured by the image sensor.

### Claim 12 (depends on 11)

The endoscopic device of claim 11, wherein said image rotation transformation comprises taking each initial pixel's integer x,y coordinates and transforming them to final, real number pixel coordinates by applying a rotation kernel.

### Claim 13 (depends on 12)

The endoscopic device of claim 12, wherein said image rotation transformation further comprises truncating the final, real number pixel coordinates to integer values, then assigning values to blank pixels in a final image using the values of nearby, filled pixels.

### Claim 14 (depends on 13)

The endoscopic device of claim 13, wherein said assignment is performed using bilinear interpolation.

### Claim 15 (depends on 13)

The endoscopic device of claim 13, wherein said assignment is performed using bicubic interpolation.

### Claim 16 (depends on 13)

The endoscopic device of claim 13, wherein said assignment is performed using nearest neighbor substitution.

### Claim 17 (depends on 11)

The endoscopic device of claim 11, wherein said image rotation transformation comprises taking each final pixel's integer x,y coordinates and transforming them to initial real number x,y coordinates by applying an inverse rotation kernel.

### Claim 18 (depends on 17)

The endoscopic device of claim 17, wherein said image rotation transformation further comprises estimating a pixel value at the initial real number x,y coordinates, using data from one or more closest integer coordinate locations.

### Claim 19 (depends on 18)

The endoscopic device of claim 18, wherein said estimation is performed using nearest neighbor substitution.

### Claim 20 (depends on 18)

The endoscopic device of claim 18, wherein said estimation is performed using bilinear interpolation.

### Claim 21 (depends on 18)

The endoscopic device of claim 18, wherein said estimation is performed using bicubic interpolation.

### Claim 22

An endoscopic system comprising:
an endoscope unit comprising:
a hand-piece;
a lumen comprising a distal tip;
an image sensor disposed within the lumen for providing visualization of an area, wherein the image sensor is located near the distal tip of the lumen;
wherein said image sensor incorporates a two dimensional array of pixels capable of detecting electromagnetic radiation;
wherein the lumen is rotatable about an axis of the endoscope and with respect to the hand-piece; and
an image signal processing pipeline performs rotation transformations upon images captured by the image sensor, wherein the image signal processing pipeline is configured to rotate the images to maintain a constant image horizon for a user on a display.

### Claim 23

A method of rotating an image in an endoscopic application comprising:
providing an endoscope unit comprising:
a hand-piece;
a lumen comprising a distal tip;
an image sensor disposed within the lumen for providing visualization of an area, wherein the image sensor is located near the distal tip of the lumen;
wherein said image sensor incorporates a two dimensional array of pixels capable of detecting electromagnetic radiation;
detecting an angle of rotation of the hand-piece relative to the lumen using an angle sensor, wherein the lumen is rotatable about an axis of the endoscope and with respect to the hand-piece; and
performing image rotation transformations upon images captured by the image sensor using an image signal processing pipeline that is configured to rotate the images to maintain a constant image horizon for a user on a display.

### Claim 24 (depends on 23)

The method of claim 23, wherein the step of performing image rotation transformation comprises taking each initial pixel's integer x,y coordinates and transforming them to final, real number pixel coordinates by applying a rotation kernel.

### Claim 25 (depends on 24)

The method of claim 24, wherein said image rotation transformation further comprises truncating the final, real number pixel coordinates to integer values, then assigning values to blank pixels in a final image using the values of nearby, filled pixels.

### Claim 26 (depends on 25)

The method of claim 25, wherein said assignment is performed using bilinear interpolation.

### Claim 27 (depends on 25)

The method of claim 25, wherein said assignment is performed using bicubic interpolation.

### Claim 28 (depends on 25)

The method of claim 25, wherein said assignment is performed using nearest neighbor substitution.

### Claim 29 (depends on 23)

The method of claim 23, wherein the step of performing image rotation transformation comprises taking each final pixel's integer x,y coordinates and transforming them to initial real number x,y coordinates by applying an inverse rotation kernel.

### Claim 30 (depends on 29)

The method of claim 29, wherein said image rotation transformation further comprises estimating a pixel value at the initial real number x,y coordinates, using the data from one or more closest integer coordinate locations.

### Claim 31 (depends on 29)

The method of claim 29, wherein said estimation is performed using nearest neighbor substitution.

### Claim 32 (depends on 29)

The method of claim 29, wherein said estimation is performed using bilinear interpolation.

### Claim 33 (depends on 29)

The method of claim 29, wherein said estimation is performed using bicubic interpolation.

### Claim 34 (depends on 23)

The method of claim 23, wherein the image signal processing pipeline is configured to rotate the images counter to the angle of rotation detected by the angle sensor to maintain the constant image horizon for the user on the display.

### Claim 35 (depends on 1)

The endoscopic device of claim 1, wherein the device further comprises an angle sensor for detecting an angle of rotation of the hand-piece relative to the lumen, wherein the image signal processing pipeline is configured to rotate the images counter to the angle of rotation detected by the angle sensor to maintain the constant image horizon for the user on the display.
