# US 12,003,880 B2 · Image rotation using software for endoscopic applications > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,003,880 B2 - Document ID: US12003880B2 - Status: Granted - Kind: B2 - Application number: 16/509,297 - Filed: 2019-07-11 - Published: 2019-11-21 - Granted: 2024-06-04 - Earliest claimed: 2013-03-15 - Assignees: DePuy Synthes Products, Inc. - Applicants: DePuy Synthes Products, Inc. - Inventors: John Richardson; Laurent Blanquart; Jeremiah D. Henley; Donald M. Wichern - Web page: https://patentgrep.com/patent/US12003880B2 ## CPC - H04N 5/2628: {Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation} - A61B 1/00006: {of control signals} - A61B 1/000095: {for image enhancement} - A61B 1/00045: {Display arrangement} - A61B 1/05: characterised by the image sensor, e.g. camera, being in the distal end portion - H04N 23/50: Constructional details - H04N 23/54: Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils - H04N 23/6812: {based on additional sensors, e.g. acceleration sensors} - H04N 23/555: {for picking-up images in sites, inaccessible due to their dimensions or hazardous conditions, e.g. endoscopes or borescopes} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) ## Abstract An endoscopic device and system for image correction of a rotating sensor includes a hand-piece, a lumen comprising a distal tip, and an image sensor disposed within the lumen for providing visualization of an area. The image sensor is located near the distal tip of the lumen. The lumen is rotatable about an axis of the endoscope and with respect to the hand-piece. The endoscopic device and system further includes an image signal processing pipeline that is configured to rotate images to maintain a constant image horizon for a user on a display. ## Claims ### Claim 1 An endoscopic device comprising: a hand-piece; an endoscope tube comprising a proximal end and a distal end, wherein the proximal end of the endoscope tube is rotatably connected to the hand-piece such that the proximal end and the distal end of the endoscope tube are rotatable about an axis of the endoscope tube and with respect to the hand-piece; an image sensor disposed within the endoscope tube for providing visualization of an area, wherein the image sensor is located near the distal end of the endoscope tube; a sensor for detecting rotation of the proximal end of endoscope tube relative to the hand piece; and an image signal processing pipeline that is configured to rotate images counter to a direction of rotation of the proximal end of the endoscope tube relative to the hand piece to maintain a constant image horizon for a user on a display. ### Claim 2 (depends on 1) The endoscopic device of claim 1, wherein the sensor comprises an angle sensor for detecting an angle of rotation of the hand-piece relative to the proximal end of the endoscope tube, wherein the angle sensor is a rotation-detecting Hall-effect sensor. ### Claim 3 (depends on 2) The endoscopic device of claim 2, wherein the rotation-detecting Hall-effect sensor is located in the hand-piece. ### Claim 4 (depends on 2) The endoscopic device of claim 2, wherein the device further comprises a diametrically-polarized magnetic annulus and wherein the rotation-detecting Hall-effect sensor produces a voltage that is used to detect an angle of the diametrically-polarized, magnetic annulus. ### Claim 5 (depends on 1) The endoscopic device of claim 1, wherein the sensor comprises an angle sensor for detecting an angle of rotation of the hand-piece relative to the proximal end of the endoscope tube, wherein the angle sensor produces a voltage that is used to detect the angle of rotation of the hand-piece relative to the proximal end of the endoscope tube. ### Claim 6 (depends on 1) The endoscopic device of claim 1, wherein the sensor comprises an angle sensor for detecting an angle of rotation of the hand-piece relative to the proximal end of the endoscope tube, wherein the angle sensor is a potentiometer. ### Claim 7 (depends on 6) The endoscopic device of claim 6, wherein the potentiometer comprises a carbon filament, wherein the carbon filament of the potentiometer is disposed on said proximal end of the endoscope tube. ### Claim 8 (depends on 1) The endoscopic device of claim 1, wherein the sensor comprises an angle sensor for detecting an angle of rotation of the hand-piece relative to the proximal end of the endoscope tube, wherein the angle sensor comprises a light source and a photo diode that rotate relative to a gradient disc. ### Claim 9 (depends on 8) The endoscopic device of claim 8, wherein said photo diode detects the electromagnetic energy from said light source that is reflected by said gradient disc. ### Claim 10 (depends on 1) The endoscopic device of claim 1, wherein said image sensor incorporates a two-dimensional array of pixels capable of detecting electromagnetic radiation. ### Claim 11 (depends on 1) The endoscopic device of claim 1, wherein the image signal processing pipeline performs image rotation transformations upon images captured by the image sensor; and wherein said image rotation transformation comprises taking each initial pixel's integer x,y coordinates and transforming them to final, real number pixel coordinates by applying a rotation kernel. ### Claim 12 (depends on 11) The endoscopic device of claim 11, wherein said image rotation transformation further comprises truncating the final, real number pixel coordinates to integer values, then assigning values to blank pixels in a final image using the values of nearby, filled pixels. ### Claim 13 (depends on 12) The endoscopic device of claim 12, wherein said assignment is performed using bilinear interpolation. ### Claim 14 (depends on 12) The endoscopic device of claim 12, wherein said assignment is performed using bicubic interpolation. ### Claim 15 (depends on 13) The endoscopic device of claim 13, wherein said assignment is performed using nearest neighbor substitution. ### Claim 16 (depends on 1) The endoscopic device of claim 1, wherein the image signal processing pipeline performs image rotation transformations upon images captured by the image sensor; and wherein said image rotation transformation comprises taking each final pixel's integer x,y coordinates and transforming them to initial real number x,y coordinates by applying an inverse rotation kernel. ### Claim 17 (depends on 16) The endoscopic device of claim 16, wherein said image rotation transformation further comprises estimating a pixel value at the initial real number x,y coordinates, using data from one or more closest integer coordinate locations. ### Claim 18 (depends on 17) The endoscopic device of claim 17, wherein said estimation is performed using nearest neighbor substitution. ### Claim 19 (depends on 17) The endoscopic device of claim 17, wherein said estimation is performed using bilinear interpolation. ### Claim 20 (depends on 17) The endoscopic device of claim 17, wherein said estimation is performed using bicubic interpolation. ### Claim 21 An endoscopic system comprising: an endoscope unit comprising: a hand-piece; an endoscope tube comprising a proximal end and a distal end, wherein the proximal end of the endoscope tube is rotatably connected to the hand-piece such that the proximal end and the distal end of the endoscope tube are rotatable about an axis of the endoscope tube and with respect to the hand-piece; an image sensor disposed within the endoscope tube for providing visualization of an area, wherein the image sensor is located near the distal end of the endoscope tube and the image sensor incorporates a two dimensional array of pixels capable of detecting electromagnetic radiation; a sensor for detecting rotation of the proximal end of the endoscope tube relative to the hand piece; and an image signal processing pipeline performs rotation transformations upon images captured by the image sensor, wherein the image signal processing pipeline is configured to rotate the images counter to a direction of rotation of the proximal end of the endoscope tube relative to the hand piece to maintain a constant image horizon for a user on a display. ### Claim 22 A method of rotating an image in an endoscopic application comprising: providing an endoscope unit comprising: a hand-piece; an endoscope tube comprising a proximal end and a distal end, wherein the proximal end of the endoscope tube is rotatably connected to the hand-piece such that the proximal end and the distal end of the endoscope tube are rotatable about an axis of the endoscope tube and with respect to the hand-piece; an image sensor disposed within the endoscope tube for providing visualization of an area, wherein the image sensor is located near the distal end of the endoscope tube and incorporates a two dimensional array of pixels capable of detecting electromagnetic radiation; and a sensor for detecting rotation of the proximal end of endoscope tube relative to the hand piece; detecting an angle of rotation of the hand-piece relative to the proximal end of the endoscope tube using the sensor; and performing image rotation transformations upon images captured by the image sensor using an image signal processing pipeline that is configured to rotate the images counter to a direction of rotation of the proximal end of the endoscope tube relative to the hand piece to maintain a constant image horizon for a user on a display. ### Claim 23 (depends on 22) The method of claim 22, wherein the step of performing image rotation transformation comprises taking each initial pixel's integer x,y coordinates and transforming them to final, real number pixel coordinates by applying a rotation kernel. ### Claim 24 (depends on 23) The method of claim 23, wherein said image rotation transformation further comprises truncating the final, real number pixel coordinates to integer values, then assigning values to blank pixels in a final image using the values of nearby, filled pixels. ### Claim 25 (depends on 24) The method of claim 24, wherein said assignment is performed using bilinear interpolation. ### Claim 26 (depends on 24) The method of claim 24, wherein said assignment is performed using bicubic interpolation. ### Claim 27 (depends on 24) The method of claim 24, wherein said assignment is performed using nearest neighbor substitution. ### Claim 28 (depends on 22) The method of claim 22, wherein the step of performing image rotation transformation comprises taking each final pixel's integer x,y coordinates and transforming them to initial real number x,y coordinates by applying an inverse rotation kernel. ### Claim 29 (depends on 28) The method of claim 28, wherein said image rotation transformation further comprises estimating a pixel value at the initial real number x,y coordinates, using the data from one or more closest integer coordinate locations. ### Claim 30 (depends on 28) The method of claim 28, wherein said estimation is performed using nearest neighbor substitution. ### Claim 31 (depends on 28) The method of claim 28, wherein said estimation is performed using bilinear interpolation. ### Claim 32 (depends on 28) The method of claim 28, wherein said estimation is performed using bicubic interpolation. ### Claim 33 (depends on 1) The endoscopic device of claim 1, wherein the sensor for detecting rotation of the proximal end of endoscope tube relative to the hand piece comprises a first portion and a second portion, wherein the first portion senses angular position of the second portion as the second portion rotates relative to the first portion; wherein the first portion is fixed to one of the proximal end of the endoscope tube and the hand-piece; and wherein the second portion is fixed to the other one of the proximal end and the hand-piece to which the first portion is not attached, such that as the proximal end rotates relative to the hand-piece, the first portion rotates relative to the second portion and detects the angle of rotation of the second portion, thereby detecting rotation of the proximal end of the endoscope tube relative to the hand-piece. ### Claim 34 (depends on 1) The endoscopic device of claim 1, wherein the endoscope tube is a rigid endoscope tube such that a rotation angle of the proximal end relative to the hand-piece indicates a rotation angle of the distal end relative to the hand-piece. ### Claim 35 (depends on 1) The endoscopic device of claim 1, wherein digital angle information indicating an angle of rotation between the proximal end of the endoscope tube relative to the hand piece is made available to the image processing pipeline where it is sampled periodically and quantized to a predetermined degree increment; and wherein image rotation is performed by the image processing pipeline when a same quantized angle is observed consistently over a predetermined number of samples.