# US 2022/0044057 A1 · Device-to-image registration method, apparatus, and storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0044057 A1
- Document ID: US20220044057A1
- Status: Published application
- Kind: A1
- Application number: 17/377,191
- Filed: 2021-07-15
- Published: 2022-02-10
- Earliest claimed: 2020-08-06
- Assignees: Canon U.S.A., Inc.
- Applicants: Canon U.S.A., Inc.
- Inventors: Santosh Narayan Ganesan; Barret Daniels
- Web page: https://patentgrep.com/patent/US20220044057A1

## CPC

- G06V 10/225: {based on a marking or identifier characterising the area}
- G06F 18/22: Matching criteria, e.g. proximity measures
- G06V 10/245: {by locating a pattern; Special marks for positioning}
- G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI]
- G06V 10/40: Extraction of image or video features
- G06V 2201/03: Recognition of patterns in medical or anatomical images
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A device to image registration method includes obtaining image data of a device, the device including a plurality of fiducials arranged as a fiducial frame on the device; detecting fiducial objects within the image data; cropping the image data to generate cropped image data; applying a feature enhancement to the cropped image data to enhance the fiducial objects; generating a candidate list of candidate objects based on the feature enhancement, and determining a common plane based on at least three points in the candidate list; determining a representative point for each candidate object; determining outlier candidate objects based on the common plane; determining resultant objects by extracting the outlier candidate objects from the candidate list; and registering the resultant objects to the device by matching the representative points of the resultant objects with a model of the fiducial frame.

## Claims

### Claim 1

A device to image registration method comprising:
obtaining image data of a device, the device including a plurality of fiducials arranged as a fiducial frame on the device;
detecting fiducial objects within the image data;
cropping the image data to generate cropped image data;
applying a feature enhancement to the cropped image data to enhance the fiducial objects;
generating a candidate list of candidate objects based on the feature enhancement, and determining a common plane based on at least three points in the candidate list;
determining a representative point for each candidate object;
determining outlier candidate objects based on the common plane;
determining resultant objects by extracting the outlier candidate objects from the candidate list; and
registering the resultant objects to the device by matching the representative points of the resultant objects with a model of the fiducial frame.

### Claim 2 (depends on 1)

The device to image registration method according to claim 1, wherein the representative points are determined by transforming stock model points to a user defined origin, and generating a candidate image points list.

### Claim 3 (depends on 2)

The device to image registration method according to claim 2, further comprising:
determining a common plane from a subset of the image points list; and
determining whether other points on the image points list fit the plane.

### Claim 4 (depends on 3)

The device to image registration method according to claim 3, further comprising:
determining a best fit among the candidate image points list and a center of the model by choosing k number of points that best fit the plane for a given number of candidate fiducials (N); and
matching remaining candidate points on the plane.

### Claim 5 (depends on 2)

The device to image registration method according to claim 2, further comprising:
finding a best fitting rotation angle between the stock model points and the resultant objects;
computing a transform between the stock model points and the resultant objects; and
registering the transform.

### Claim 6 (depends on 5)

The device to image registration method according to claim 5, wherein the best fitting rotation angle is determined between the stock model points and the resultant objects.

### Claim 7 (depends on 7)

The device to image registration method according to claim 7, wherein the transform is determined or calculated between the stock model points and the resultant objects, and registration takes place.

### Claim 8 (depends on 1)

The device to image registration method according to claim 1,
wherein the fiducials are arranged in a substantially planar arrangement.

### Claim 9 (depends on 1)

The device to image registration method according to claim 1,
wherein the fiducials are arranged as ring independent and asymmetric.

### Claim 10 (depends on 1)

The device to image registration method according to claim 1,
wherein the fiducials are arranged in a ring shape, and the fiducial objects are registered by rotating the ring shape until the representative points of the fiducial markers match the model of the frame.

### Claim 11 (depends on 1)

The device to image registration method according to claim 1, further comprising masking the image data.

### Claim 12 (depends on 1)

The device to image registration method according to claim 1, wherein the feature enhancement comprises shape data of the fiducials.

### Claim 13 (depends on 1)

The device to image registration method according to claim 1, further comprising applying feature extraction to the cropped image data to extract the fiducial objects by generating candidate fiducials.

### Claim 14 (depends on 1)

The device to image registration method according to claim 1, wherein the feature extraction extracts the fiducial objects by generating candidate fiducials based on the feature enhancement by applying a derivative-based operation to the cropped image data.

### Claim 15 (depends on 1)

The device to image registration method according to claim 1, further comprising rotating the device, and applying point to point matching by checking each combination of point orientation.

### Claim 16 (depends on 1)

The device to image registration method according to claim 1, further comprising adjusting the device to guide a needle to a target.

### Claim 17

A device to image registration apparatus comprising:
at least one processor configured to perform
obtaining image data of a device, the device including a plurality of fiducials arranged as a fiducial frame on the device;
detecting fiducial objects within the image data;
cropping the image data to generate cropped image data;
applying a feature enhancement to the cropped image data to enhance the fiducial objects;
generating a candidate list of candidate objects based on the feature enhancement, and determining a common plane based on at least three points in the candidate list;
determining a representative point for each candidate object;
determining outlier candidate objects based on the common plane;
determining resultant objects by extracting the outlier candidate objects from the candidate list; and
registering the resultant objects to the device by matching the representative points of the resultant objects with a model of the fiducial frame.

### Claim 18 (depends on 17)

The device to image registration apparatus according to claim 17, wherein the representative points are determined by transforming stock model points to a user defined origin, and generating a candidate image points list.

### Claim 19 (depends on 18)

The device to image registration apparatus according to claim 18, wherein the at least one processor is further configured to perform:
determining a common plane from a subset of the image points list; and
determining whether other points on the image points list fit the plane.

### Claim 20 (depends on 19)

The device to image registration apparatus according to claim 19, wherein the at least one processor is further configured to perform:
determining a best fit among the candidate image points list and a center of the model by choosing k number of points that best fit the plane for a given number of candidate fiducials (N); and
matching remaining candidate points on the plane.

### Claim 21

A non-transitory storage medium storing a program for causing a computer to execute a method of registration comprising:
obtaining image data of a device, the device including a plurality of fiducials arranged as a fiducial frame on the device;
detecting fiducial objects within the image data;
cropping the image data to generate cropped image data;
applying a feature enhancement to the cropped image data to enhance the fiducial objects;
generating a candidate list of candidate objects based on the feature enhancement, and determining a common plane based on at least three points in the candidate list;
determining a representative point for each candidate object;
determining outlier candidate objects based on the common plane;
determining resultant objects by extracting the outlier candidate objects from the candidate list; and
registering the resultant objects to the device by matching the representative points of the resultant objects with a model of the fiducial frame.

### Claim 22 (depends on 21)

The storage medium according to claim 21, wherein the representative points are determined by transforming stock model points to a user defined origin, and generating a candidate image points list.

### Claim 23 (depends on 22)

The storage medium according to claim 22, further comprising:
determining a common plane from a subset of the image points list; and
determining whether other points on the image points list fit the plane.

### Claim 24 (depends on 23)

The storage medium according to claim 23, further comprising:
determining a best fit among the candidate image points list and a center of the model by choosing k number of points that best fit the plane for a given number of candidate fiducials (N); and
matching remaining candidate points on the plane.
