Back to results
US 12,183,017 B2Granted
Capsule endoscope image three-dimensional reconstruction method, electronic device, and readable storage medium
AssigneesAnkon Technologies Co., Ltd, Anx Anx IP Holding Pte. Ltd.
InventorsHui Liu, Wenjin Yuan, Hang Zhang, Hao Zhang
Life of this patent
Estimating the term…
Abstract
A method of three-dimensional reconstruction of capsule endoscope images is described. The method combines two algorithms for obtaining image depth information values to improve the calculation accuracy of image depth values, and thus to increase the image three-dimensional reconstruction rate and improve the image identification precision.
Claims9
A three-dimensional reconstruction method of a capsule endoscope image, comprising:
- obtaining a first image and a second image synchronously through two cameras arranged side by side;
- matching the first image with the second image to obtain corresponding stable homonymy points, wherein the stable homonymy points are two pixels with a unique matching relationship in the first image and the second image after being processed by the same rule;
- calculating a first depth information value corresponding to each pair of the stable homonymy points;
- calculating a second depth information value corresponding to each pixel in a comparison image that is one of the first image and the second image;
- obtaining a unique depth information value corresponding to each pixel in the comparison image based on the first depth information value and the second depth information value matched by each stable homonymy point;
- and
- obtaining a three-dimensional spatial coordinate point of each pixel in the comparison image mapped to the camera coordinate system based on the unique depth information value, and mapping the attributes of each pixel in the comparison image to the corresponding three-dimensional spatial coordinate point to complete the three-dimensional image reconstruction,
- wherein obtaining the unique depth information value corresponding to each pixel in the comparison image based on the first depth information value and the second depth information value matched by each stable homonymy point comprises:
- obtaining the first depth information value and the second depth information value mutually matching based on the pixels in the comparison images corresponding to the stable homonymy points, and obtaining a depth residual between the first depth information value and the second depth information value;
- then, d(xm1, ym1)=|depth (xm1, ym1)−depth (xm2, ym2)|;
- wherein, (xm1, ym1) represents a pixel coordinate value in the comparison image corresponding to any stable homonymy point;
- d(xm1, ym1) represents the depth residual corresponding to the stable homonymy point with the coordinate value (xm1, ym1) in the comparison image, depth (xm1, ym1) represents the first depth information value corresponding to the stable homonymy point with the coordinate value (xm1, ym1) in the comparison image, and depth (xm2, ym2) represents the second depth information value corresponding to the stable homonymy point with the coordinate value (xm1, ym1) in the comparison image;
- performing interpolation calculation based on the obtained depth residual of each stable homonymy point, to obtain the depth residuals of all pixels in the comparison image;
- obtaining the unique depth information value corresponding to each pixel in the comparison image based on the depth residual of each pixel in the comparison image and the corresponding second depth information value;
- then, Z(xm, ym)=depth (xm, ym)+d(xm, ym);
- wherein, (xm, ym) represents the coordinate value of any pixel in the comparison image;
- d(xm, ym) represents the depth residual corresponding to the pixel with the coordinate value (xm, ym) in the comparison image;
- depth (xm, ym) represents the secondary depth information value corresponding to the pixel with the coordinate value (xm, ym) in the comparison image;
- and
- Z(xm, ym) represents the unique depth information value corresponding to the pixel with the coordinate value (xm, ym) in the comparison image.
17 more elements
An electronic device, comprising a memory and a processor, wherein the memory stores a computer program that runs on the processor, and the processor executes the program to implement steps of a three-dimensional reconstruction method of a capsule endoscope image, wherein the method comprises:
- obtaining a first image and a second image synchronously through two cameras arranged side by side;
- matching the first image with the second image to obtain corresponding stable homonymy points, wherein the stable homonymy points are two pixels with a unique matching relationship in the first image and the second image after being processed by the same rule;
- calculating a first depth information value corresponding to each pair of the stable homonymy points;
- calculating a second depth information value corresponding to each pixel in a comparison image that is one of the first image and the second image;
- obtaining a unique depth information value corresponding to each pixel in the comparison image based on the first depth information value and the second depth information value matched by each stable homonymy point;
- and
- obtaining a three-dimensional spatial coordinate point of each pixel in the comparison image mapped to the camera coordinate system based on the unique depth information value, and mapping the attributes of each pixel in the comparison image to the corresponding three-dimensional spatial coordinate point to complete the three-dimensional image reconstruction, wherein obtaining the unique depth information value corresponding to each pixel in the comparison image based on the first depth information value and the second depth information value matched by each stable homonymy point comprises:
- obtaining the first depth information value and the second depth information value mutually matching based on the pixels in the comparison images corresponding to the stable homonymy points, and obtaining a depth residual between the first depth information value and the second depth information value;
- then, d(xm1, ym1)=|depth (xm1, ym1)−depth (xm2, ym2)|;
- wherein, (xm1, ym1) represents a pixel coordinate value in the comparison image corresponding to any stable homonymy point;
- d(xm1, ym1) represents the depth residual corresponding to the stable homonymy point with the coordinate value (xm1, ym1) in the comparison image, depth (xm1, ym1) represents the first depth information value corresponding to the stable homonymy point with the coordinate value (xm1, ym1) in the comparison image, and depth (xm2, ym2) represents the second depth information value corresponding to the stable homonymy point with the coordinate value (xm1, ym1) in the comparison image;
- performing interpolation calculation based on the obtained depth residual of each stable homonymy point, to obtain the depth residuals of all pixels in the comparison image;
- obtaining the unique depth information value corresponding to each pixel in the comparison image based on the depth residual of each pixel in the comparison image and the corresponding second depth information value;
- then, Z(xm, ym)=depth (xm, ym)+d(xm, ym);
- wherein, (xm, ym) represents the coordinate value of any pixel in the comparison image;
- d(xm, ym) represents the depth residual corresponding to the pixel with the coordinate value (xm, ym) in the comparison image;
- depth (xm, ym) represents the secondary depth information value corresponding to the pixel with the coordinate value (xm, ym) in the comparison image;
- and
- Z(xm, ym) represents the unique depth information value corresponding to the pixel with the coordinate value (xm, ym) in the comparison image.
16 more elements
A computer-readable storage medium having stored thereon a computer program that, when executed by a processor, implements steps of a three-dimensional reconstruction method of a capsule endoscope image, wherein the method comprises:
- obtaining a first image and a second image synchronously through two cameras arranged side by side;
- matching the first image with the second image to obtain corresponding stable homonymy points, wherein the stable homonymy points are two pixels with a unique matching relationship in the first image and the second image after being processed by the same rule;
- calculating a first depth information value corresponding to each pair of the stable homonymy points;
- calculating a second depth information value corresponding to each pixel in a comparison image that is one of the first image and the second image;
- obtaining a unique depth information value corresponding to each pixel in the comparison image based on the first depth information value and the second depth information value matched by each stable homonymy point;
- and
- obtaining a three-dimensional spatial coordinate point of each pixel in the comparison image mapped to the camera coordinate system based on the unique depth information value, and mapping the attributes of each pixel in the comparison image to the corresponding three-dimensional spatial coordinate point to complete the three-dimensional image reconstruction,
- wherein obtaining the unique depth information value corresponding to each pixel in the comparison image based on the first depth information value and the second depth information value matched by each stable homonymy point comprises:
- obtaining the first depth information value and the second depth information value mutually matching based on the pixels in the comparison images corresponding to the stable homonymy points, and obtaining a depth residual between the first depth information value and the second depth information value;
- then, d(xm1, ym1)=|depth (xm1, ym1)−depth (xm2, ym2)|;
- wherein, (xm1, ym1) represents a pixel coordinate value in the comparison image corresponding to any stable homonymy point;
- d(xm1, ym1) represents the depth residual corresponding to the stable homonymy point with the coordinate value (xm1, ym1) in the comparison image, depth (xm1, ym1) represents the first depth information value corresponding to the stable homonymy point with the coordinate value (xm1, ym1) in the comparison image, and depth (xm2, ym2) represents the second depth information value corresponding to the stable homonymy point with the coordinate value (xm1, ym1) in the comparison image;
- performing interpolation calculation based on the obtained depth residual of each stable homonymy point, to obtain the depth residuals of all pixels in the comparison image;
- obtaining the unique depth information value corresponding to each pixel in the comparison image based on the depth residual of each pixel in the comparison image and the corresponding second depth information value;
- then, Z(xm, ym)=depth (xm, ym)+d(xm, ym);
- wherein, (xm, ym) represents the coordinate value of any pixel in the comparison image;
- d(xm, ym) represents the depth residual corresponding to the pixel with the coordinate value (xm, ym) in the comparison image;
- depth (xm, ym) represents the secondary depth information value corresponding to the pixel with the coordinate value (xm, ym) in the comparison image;
- and
- Z(xm, ym) represents the unique depth information value corresponding to the pixel with the coordinate value (xm, ym) in the comparison image.
17 more elements
Dependencies are inferred from the claim text.