# US 2016/0239950 A1 · Image processing apparatus, image processing method, and storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2016/0239950 A1
- Document ID: US20160239950A1
- Status: Published application
- Kind: A1
- Application number: 15/016,453
- Filed: 2016-02-05
- Published: 2016-08-18
- Earliest claimed: 2015-02-13
- Assignees: Canon Kabushiki Kaisha
- Applicants: Canon Kabushiki Kaisha
- Inventors: Kazuhiro Miyasa; Ryo Ishikawa
- Web page: https://patentgrep.com/patent/US20160239950A1

## CPC

- G06T 7/337: {involving reference images or patches}
- G06V 10/44: Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V 10/462: {Salient features, e.g. scale invariant feature transforms [SIFT]}
- G06V 10/757: {Matching configurations of points or features}
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

An image processing apparatus includes: a feature point obtaining unit configured to obtain priority feature points that are registered preferentially and nonpriority feature points that have priority orders lower than those of the priority feature points to be registered; a weight setting unit configured to set a weight of each of the nonpriority feature points based on an evaluation value that evaluates a characteristic of the nonpriority feature point, and set a weight of each of the priority feature point, that has a value larger than the weight of the nonpriority feature point; and a registration unit configured to register the first image and the second image based on the priority feature points, the weights of the priority feature points, the nonpriority feature points, and the weights of the nonpriority feature points.

## Claims

### Claim 1

An image processing apparatus comprising:
a feature point obtaining unit configured to obtain, among feature points obtained from at least one of a first image and a second image, priority feature points that are registered preferentially and nonpriority feature points that have priority orders lower than those of the priority feature points to be registered;
a weight setting unit configured to set a weight of each of the nonpriority feature points based on an evaluation value that evaluates a characteristic of the nonpriority feature point, and set a weight of each of the priority feature point, that has a value larger than the weight of the nonpriority feature point; and
a registration unit configured to register the first image and the second image based on the priority feature points, the weights of the priority feature points, the nonpriority feature points, and the weights of the nonpriority feature points,
wherein the feature point obtaining unit, the weight setting unit, and the registration unit are implemented using a processor and a program stored in a memory.

### Claim 2 (depends on 1)

The apparatus according to claim 1, wherein each of a set of the priority feature points and a set of the nonpriority feature points has a one-to-one correspondence between the first image and the second image.

### Claim 3 (depends on 1)

The apparatus according to claim 1, further comprising:
a matching degree calculation unit configured to calculate, as the evaluation value of each of the nonpriority feature points, a matching degree indicating a degree of matching to deformation information between the first image and the second image defined by other feature points including the priority feature points and other nonpriority feature points,
wherein the matching degree calculation unit is implemented using the processor and the program stored in the memory.

### Claim 4 (depends on 3)

The apparatus according to claim 3, wherein the matching degree calculation unit calculates the matching degree based on information of a displacement field generated based on positions of the priority feature points and the nonpriority feature points.

### Claim 5 (depends on 4)

The apparatus according to claim 4, wherein
the matching degree calculation unit obtains a first displacement field between the first image and the second image based on the positions of the priority feature points and nonpriority feature points except a nonpriority feature point of interest, and obtains a second displacement field between the first image and the second image based on the positions of the priority feature points and nonpriority feature points including the nonpriority feature point of interest, and
the matching degree calculation unit calculates the matching degree of the nonpriority feature point of interest based on a difference between the first displacement field and the second displacement field.

### Claim 6 (depends on 2)

The apparatus according to claim 2, further comprising:
a reliability degree calculation unit configured to calculate, as the evaluation value of each of the nonpriority feature points, an association reliability degree between the first image and second image with respect to each of the nonpriority feature points,
wherein the reliability degree calculation unit is implemented using the processor and the program stored in the memory.

### Claim 7 (depends on 1)

The apparatus according to claim 1, further comprising:
an influence degree calculation unit configured to calculate, as the evaluation value of each of the nonpriority feature points, an influence degree of each of the nonpriority feature points on the priority feature point based on a relative positional relationship between the priority feature point and the nonpriority feature point,
wherein the influence degree calculation unit is implemented using the processor and the program stored in the memory.

### Claim 8 (depends on 7)

The apparatus according to claim 7, wherein the feature point obtaining unit determines whether registration is unexecuted, by referring to determination information representing whether the registration has been executed, and
if the registration has been executed, the influence degree calculation unit calculates an influence degree of each of the nonpriority feature points on priority feature points newly added after the registration.

### Claim 9 (depends on 7)

The apparatus according to claim 7, wherein the weight setting unit sets a weight of the nonpriority feature point by comparing the influence degree with a threshold.

### Claim 10 (depends on 9)

The apparatus according to claim 9, wherein
if the influence degree is not lower than the threshold, the weight setting unit sets the weight of the nonpriority feature point to zero, and
if the influence degree is lower than the threshold, the weight setting unit sets a weight used in the registration as the weight of the nonpriority feature point.

### Claim 11 (depends on 1)

The apparatus according to claim 1, further comprising:
a data obtaining unit configured to obtain the first image and the second image; and
a feature point extraction unit configured to extract feature points in the first image and the second image by performing image processing for the first image and the second image,
wherein the feature point obtaining unit sets, as the nonpriority feature points, the feature points extracted by the feature point extraction unit, and
wherein the data obtaining unit and the feature point extraction unit are implemented using the processor and the program stored in the memory.

### Claim 12 (depends on 1)

The apparatus according to claim 1, further comprising:
a display control unit configured to cause a display unit to display a registration result of the registration unit; and
a designation unit configured to designate a feature point in an image displayed on the display unit,
wherein the display control unit causes the display unit to display a cross-sectional image of the first image and a cross-sectional image of the second image side by side as the registration result,
the designation unit designates a set of corresponding feature points as input feature points in the cross-sectional image of the first image and the cross-sectional image of the second image, and
the display control unit causes the display unit to display a selection input portion for selecting whether to set the input feature points as priority feature points or nonpriority feature points, and
wherein the display control unit and the designation unit are implemented using the processor and the program stored in the memory.

### Claim 13 (depends on 12)

The apparatus according to claim 12, wherein the feature point obtaining unit sets the priority feature points and the nonpriority feature points in accordance with selection of the section input portion.

### Claim 14 (depends on 12)

The apparatus according to claim 12, wherein the display control unit causes the display unit to display the priority feature points and the nonpriority feature points in different display formats in the cross-sectional image of the first image and the cross-sectional image of the second image.

### Claim 15 (depends on 12)

The apparatus according to claim 12, wherein
the display control unit causes the display unit to display a priority setting portion in which a priority indicating a priority order for matching the feature points is settable by numerical information, and
the feature point obtaining unit sets the priority feature points and the nonpriority feature points in accordance with setting of the priority setting portion.

### Claim 16

An image processing apparatus comprising:
a feature point obtaining unit configured to obtain, among feature points obtained from at least one of a first image and a second image, priority feature points that are registered preferentially and nonpriority feature points that have priority orders lower than those of the priority feature points to be registered;
a weight setting unit configured to set a weight of each of the nonpriority feature points based on an evaluation value that evaluates a characteristic of the nonpriority feature point, and set a weight of each of the priority feature points, that has a value larger than the weight of the nonpriority feature point; and
a registration unit configured to register the first image and the second image based on the priority feature points, the weights of the priority feature points, the nonpriority feature points, and the weights of the nonpriority feature points.

### Claim 17

An image processing apparatus comprising:
an obtaining unit configured to obtain feature points from at least one of a first image and a second image;
a display control unit configured to cause a display unit to display a selection input portion for selecting whether to set each of the feature points as a priority feature point that is registered preferentially or a nonpriority feature point that has a priority order lower than that of the priority feature point to be registered; and
a registration unit configured to register the first image and the second image using the feature points and information selected in the selection input portion.

### Claim 18

An image processing method comprising:
obtaining, among feature points obtained from at least one of a first image and a second image, priority feature points that are registered preferentially and nonpriority feature points that have priority orders lower than those of the priority feature points to be registered;
setting a weight of each of the nonpriority feature points based on an evaluation value that evaluates a characteristic of the nonpriority feature point, and setting a weight of each of the priority feature points, that has a value larger than the weight of the nonpriority feature point; and
registering the first image and the second image based on the priority feature points, the weights of the priority feature points, the nonpriority feature points, and the weights of the nonpriority feature points.

### Claim 19

An image processing method comprising:
obtaining feature points from at least one of a first image and a second image;
causing a display unit to display a selection input portion for selecting whether to set each of the feature points as a priority feature point that is registered preferentially or a nonpriority feature point that has a priority order lower than that of the priority feature point to be registered; and
registering the first image and the second image using the feature points and information selected in the selection input portion.

### Claim 20

A computer-readable storage medium storing a program for causing a computer to function as each unit of an image processing apparatus, the image processing apparatus comprising:
a feature point obtaining unit configured to obtain, among feature points obtained from at least one of a first image and a second image, priority feature points that are registered preferentially and nonpriority feature points that have priority orders lower than those of the priority feature points to be registered;
a weight setting unit configured to set a weight of each of the nonpriority feature points based on an evaluation value that evaluates a characteristic of the nonpriority feature point, and set a weight of each of the priority feature points, that has a value larger than the weight of the nonpriority feature point; and
a registration unit configured to register the first image and the second image based on the priority feature points, the weights of the priority feature points, the nonpriority feature points, and the weights of the nonpriority feature points.
