# US 2018/0025501 A1 · Image processing apparatus, image processing method, and, non-transitory computer readable medium > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2018/0025501 A1 - Document ID: US20180025501A1 - Status: Published application - Kind: A1 - Application number: 15/648,924 - Filed: 2017-07-13 - Published: 2018-01-25 - Earliest claimed: 2016-07-19 - Assignees: Canon Kabushiki Kaisha - Applicants: Canon Kabushiki Kaisha - Inventors: Takaaki Endo; Kiyohide Satoh - Web page: https://patentgrep.com/patent/US20180025501A1 ## CPC - G06T 7/32: using correlation-based methods - G06T 3/153: using elastic snapping - G06T 2207/10028: Range image; Depth image; 3D point clouds - G06T 2207/10072: Tomographic images - G06T 2207/20092: Interactive image processing based on input by user - G06T 2207/20101: Interactive definition of point of interest, landmark or seed - G06T 2207/30068: Mammography; Breast - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract An image processing apparatus comprising: an image obtaining unit obtaining a first image and a second image different from the first image; a deformation information obtaining unit obtaining deformation information representing deformation between the first image and the second image; a deformed image generating unit generating a first deformed image of the first image deformed in accordance with the second image and a second deformed image of the second image deformed in accordance with the first image; and a display controlling unit displaying the first image, the second image, an image based on the first deformed image, and an image based on the second deformed image is used. ## Claims ### Claim 1 An image processing apparatus comprising: an image obtaining unit configured to obtain a first image and a second image different from the first image; a deformation information obtaining unit configured to obtain deformation information representing deformation between the first image and the second image; a deformed image generating unit configured to generate a first deformed image of the first image deformed in accordance with the second image and a second deformed image of the second image deformed in accordance with the first image, on the basis of the deformation information; and a display controlling unit configured to control display, on a displaying unit, the first image, the second image, an image based on the first deformed image, and an image based on the second deformed image. ### Claim 2 (depends on 1) The image processing apparatus according to claim 1, wherein the image based on the first deformed image is the first deformed image itself, an image in which the first deformed image is superimposed on the second image, or a differential image of the second image and the first deformed image, and the image based on the second deformed image is the second deformed image itself, an image in which the second deformed image is superimposed on the first image, or a differential image of the first image and the second deformed image. ### Claim 3 (depends on 1) The image processing apparatus according to claim 1, wherein the display controlling unit is configured to display the first image, the second image, the image based on the first deformed image, and the image based on the second deformed image on the displaying unit simultaneously or in a switching manner by a predetermined number at a time. ### Claim 4 (depends on 1) The image processing apparatus according to claim 1, wherein the deformation information shows a correspondence in a shape of an object between the first image and the second image. ### Claim 5 (depends on 4) The image processing apparatus according to claim 4, wherein the deformation information is information on a deformation field including displacement vectors from the positions of the individual unit regions of the second image to the corresponding positions on the first image. ### Claim 6 (depends on 4) The image processing apparatus according to claim 4, wherein the deformed image generating unit is configured to generate the first deformed image by deforming the object in the first image in accordance with a shape of the object in the second image and generate the second deformed image by deformation of the object in the second image in accordance with a shape of the object in the first image, based on the deformation information. ### Claim 7 (depends on 1) The image processing apparatus according to claim 1, wherein the display controlling unit is configured to arrange the first image, the second image, the image based on the first deformed image, and the image based on the second deformed image in a 2×2 tile-like manner in the displaying unit. ### Claim 8 (depends on 7) The image processing apparatus according to claim 7, wherein the display controlling unit is configured to arrange the first image and the second image diagonally in the displaying unit. ### Claim 9 (depends on 1) The image processing apparatus according to claim 1, wherein the first image and the second image are a three-dimensional image, and the display controlling unit is configured to display, on the displaying unit, a cross-sectional image of a corresponding cross-section of the first image, the second image, the image based on the first deformed image, and the image based on the second deformed image. ### Claim 10 (depends on 9) The image processing apparatus according to claim 9, wherein the display controlling unit is configured to control a position to arrange the first image, the second image, the image based on the first deformed image, and the image based on the second deformed image on the displaying unit, in accordance with at least any of an orientation of the cross-sectional image for the object and the deformation information. ### Claim 11 (depends on 9) The image processing apparatus according to claim 9, further comprising an operating unit configured to receive an input from a user. ### Claim 12 (depends on 11) The image processing apparatus according to claim 11, wherein the deformed image generating unit is configured to change a position of the cross-sectional image, based on information input from the operating unit. ### Claim 13 (depends on 11) The image processing apparatus according to claim 11, wherein the deformed image generating unit is configured to obtain a target position in the first image and a corresponding position to the target position in the second image and generate the first deformed image and the second deformed image using information on the target position and the corresponding position, based on information input from the operating unit. ### Claim 14 (depends on 13) The image processing apparatus according to claim 13, wherein the display controlling unit is configured to superimpose information showing the target position on the first image and the image based on the second deformed image and displays the same, and, superimpose information showing the corresponding position on the second image and the image based on the first deformed image and displays the same. ### Claim 15 (depends on 9) The image processing apparatus according to claim 9, wherein the display controlling unit is configured to switch between a plurality of display modes in which number and combination of images displayed simultaneously on the displaying unit differ, based on information input from the operating unit. ### Claim 16 (depends on 1) The image processing apparatus according to claim 1, wherein the first image and the second image are different in at least imaging modality or imaging time. ### Claim 17 An image processing method comprising: a step of obtaining a first image and a second image different from the first image; a step of obtaining deformation information representing deformation between the first image and the second image; a step of generating a first deformed image of the first image deformed in accordance with the second image and a second deformed image of the second image deformed in accordance with the first image, on the basis of the deformation information; and a step of controlling display of the first image, the second image, an image based on the first deformed image, and an image based on the second deformed image. ### Claim 18 A non-transitory computer readable medium storing a program causing a computer to execute an image processing method, the method comprising: a step of obtaining a first image and a second image different from the first image; a step of obtaining deformation information representing deformation between the first image and the second image; a step of generating a first deformed image of the first image deformed in accordance with the second image and a second deformed image of the second image deformed in accordance with the first image, on the basis of the deformation information; and a step of controlling display of the first image, the second image, an image based on the first deformed image, and an image based on the second deformed image. ### Claim 19 An image processing apparatus comprising: an image obtaining unit configured to obtain a plurality of images different from each other; a deformation information obtaining unit configured to obtain deformation information representing deformation between the plurality of images; a deformed image generating unit configured to generate a plurality of deformed images of the respective plurality of images deformed in accordance with other images, based on the deformation information; and a display controlling unit configured to control display, on a displaying unit, of the plurality of images and an image based on the plurality of deformed images. ### Claim 20 (depends on 19) The image processing apparatus according to claim 19, wherein the display controlling unit is configured to display, in the displaying unit, an i-th image in a diagonal position (i,i) of a tile-like arrangement, and, display an image based on a deformed image of the i-th image deformed in accordance with a j-th image in a position (i,j) of the tile-like arrangement.