# US 2024/0005603 A1 · Three-dimensional shape data generation method and information processing apparatus

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0005603 A1
- Document ID: US20240005603A1
- Status: Published application
- Kind: A1
- Application number: 18/212,222
- Filed: 2023-06-21
- Published: 2024-01-04
- Earliest claimed: 2022-06-30
- Assignees: Fujitsu Limited; Japan Medical Device Corporation; UT-Heart Inc.
- Applicants: Fujitsu Limited; Japan Medical Device Corporation; UT-Heart Inc.
- Inventors: Jinse Shimo; Kazuo Minami; Shuichi Chiba; Yoshihiko Tokumaru; Takafumi Okano; Seiryo Sugiura; Jun-ichi Okada; Toshiaki Hisada; Takumi Washio; Masahiro Watanabe; Yasutomo Shimizu
- Web page: https://patentgrep.com/patent/US20240005603A1

## CPC

- A61B 8/0883: {for diagnosis of the heart}
- G06T 17/10: Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes
- A61B 5/0044: {for the heart}
- A61B 5/1072: {measuring distances on the body, e.g. measuring length, height or thickness (A61B5/1076 takes precedence)}
- A61B 5/1075: {for measuring dimensions by non-invasive methods, e.g. for determining thickness of tissue layer (A61B8/0858 takes precedence)}
- A61B 5/28: for electrocardiography [ECG]
- A61B 5/33: specially adapted for cooperation with other devices
- A61B 5/367: Electrophysiological study [EPS], e.g. electrical activation mapping or electro-anatomical mapping
- A61B 5/6823: {Trunk, e.g., chest, back, abdomen, hip}
- A61B 5/743: {Displaying an image simultaneously with additional graphical information, e.g. symbols, charts, function plots}
- A61B 6/503: {for diagnosis of the heart}
- A61B 6/5247: {combining images from an ionising-radiation diagnostic technique and a non-ionising radiation diagnostic technique, e.g. X-ray and ultrasound}
- A61B 8/4416: {related to combined acquisition of different diagnostic modalities, e.g. combination of ultrasound and X-ray acquisitions}
- A61B 8/5223: {for extracting a diagnostic or physiological parameter from medical diagnostic data (for algorithms to analyse biomedical images G06T7/0012)}
- G06T 7/13: Edge detection
- G06T 7/50: Depth or shape recovery
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06T 7/75: {involving models}
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- G16H 10/60: for patient-specific data, e.g. for electronic patient records
- G16H 15/00: ICT specially adapted for medical reports, e.g. generation or transmission thereof
- G16H 20/40: relating to mechanical, radiation or invasive therapies, e.g. surgery, laser therapy, dialysis or acupuncture
- G16H 30/20: for handling medical images, e.g. DICOM, HL7 or PACS
- G16H 30/40: for processing medical images, e.g. editing
- G16H 40/67: for remote operation
- G16H 50/50: for simulation or modelling of medical disorders
- A61B 5/066: {Superposing sensor position on an image of the patient, e.g. obtained by ultrasound or x-ray imaging}
- G06T 2207/10116: X-ray image
- G06T 2207/10132: Ultrasound image
- G06T 2207/30048: Heart; Cardiac
- G06T 2210/41: Medical
- G06T 2219/2016: Rotation, translation, scaling
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

An information processing apparatus generates heart shape data indicating a three-dimensional shape including an internal structure of a heart of a subject based on echo image data indicating a cross-sectional image of the heart. The information processing apparatus then generates thorax shape data indicating a three-dimensional shape of a thorax of the subject based on X-ray image data indicating an X-ray image of the thorax of the subject. The information processing apparatus then decides, based on an image of the heart appearing in the X-ray image, a position and an orientation of the three-dimensional shape of the heart indicated by the heart shape data within the three-dimensional shape of the thorax indicated by the thorax shape data.

## Claims

### Claim 1

A non-transitory computer-readable recording medium storing therein a computer program that causes a computer to execute a process comprising:
generating heart shape data indicating a three-dimensional shape including an internal structure of a heart of a subject based on echo image data indicating a cross-sectional image of the heart;
generating thorax shape data indicating a three-dimensional shape of a thorax of the subject based on X-ray image data indicating an X-ray image of the thorax of the subject; and
deciding, based on an image of the heart appearing in the X-ray image, a position and an orientation of the three-dimensional shape of the heart indicated by the heart shape data within the three-dimensional shape of the thorax indicated by the thorax shape data.

### Claim 2 (depends on 1)

The non-transitory computer-readable recording medium according to claim 1, wherein the generating of the thorax shape data includes extracting an outer edge of the thorax based on the X-ray image and generating the three-dimensional shape of the thorax based on the outer edge that has been extracted and an average thorax shape that has been statistically obtained.

### Claim 3 (depends on 2)

The non-transitory computer-readable recording medium according to claim 2, wherein
the generating of the thorax shape data includes:
generating a similar thorax shape by enlarging or reducing the average thorax shape based on the outer edge of the thorax; and
correcting a ratio between a horizontal width and a vertical width of the similar thorax shape based on a ratio between a horizontal width of the thorax indicated in a first X-ray image taken from a front and a vertical width of the thorax indicated in a second X-ray image taken from a side.

### Claim 4 (depends on 3)

The non-transitory computer-readable recording medium according to claim 3, wherein
the generating of the thorax shape data includes:
specifying a center line of the thorax when viewed from the side based on the second X-ray image; and
deciding a position in a three-dimensional space of a corrected thorax shape that has been obtained by the correcting of the ratio between the horizontal width and the vertical width of the similar thorax shape based on the center line.

### Claim 5 (depends on 1)

The non-transitory computer-readable recording medium according to claim 1, wherein the deciding of the position and the orientation of the three-dimensional shape of the heart includes deciding a position of a characteristic part in the three-dimensional shape of the heart within the three-dimensional shape of the thorax based on a position of the characteristic part of the heart inside the thorax in the X-ray image.

### Claim 6 (depends on 5)

The non-transitory computer-readable recording medium according to claim 5, wherein
the deciding of the position and the orientation of the three-dimensional shape of the heart includes:
rotating the three-dimensional shape of the heart around a plurality of rotational axes that pass through the characteristic part in the three-dimensional shape of the heart; and
deciding the orientation of the three-dimensional shape of the heart within the three-dimensional shape of the thorax based on a result of comparing an image obtained by projecting the three-dimensional shape of the heart after the rotating and the image of the heart appearing in the X-ray image.

### Claim 7 (depends on 6)

The non-transitory computer-readable recording medium according to claim 6, wherein
the deciding of the position and the orientation of the three-dimensional shape of the heart includes:
generating a plurality of rotation patterns in which a ratio of rotational angles around each of the plurality of rotational axes and a rotational angle around an axial vector whose direction of rotation is specified by the ratio are designated;
rotating the three-dimensional shape of the heart according to each of the plurality of rotation patterns; and
deciding the orientation of the three-dimensional shape of the heart within the three-dimensional shape of the thorax based on the result of comparing an image obtained by projecting the three-dimensional shape of the heart after the rotating and the image of the heart appearing in the X-ray image.

### Claim 8 (depends on 1)

The non-transitory computer-readable recording medium according to claim 1, wherein
the process further includes:
placing the X-ray image inside the three-dimensional space in which the three-dimensional shape of the thorax is indicated; and
converting placed positions of electrodes for electrocardiogram measurement that have been designated on the X-ray image into positions on a surface of the three-dimensional shape of the thorax.

### Claim 9 (depends on 1)

The non-transitory computer-readable recording medium according to claim 1, wherein
the generating of the heart shape data includes generating the heart shape data that represents the three-dimensional shape of the heart with a first surface model,
the generating of the thorax shape data includes generating the thorax shape data that represents the three-dimensional shape of the thorax with a second surface model, and
the process further includes placing, upon deciding the position and the orientation of the three-dimensional shape of the heart, the first surface model inside the second surface model with the position and the orientation that have been decided and generating, based on the first surface model and the second surface model, computational grid data that represent the three-dimensional shape of the thorax incorporating the heart with a solid model.

### Claim 10

A non-transitory computer-readable recording medium storing therein a computer program that causes a computer to execute a process comprising:
extracting an outer edge of a thorax of a subject based on an X-ray image of the thorax indicated by X-ray image data of the thorax; and
generating thorax shape data indicating a three-dimensional shape of the thorax based on the outer edge that has been extracted and an average thorax shape that has been statistically obtained.

### Claim 11

A three-dimensional shape data generation method comprising:
generating, by a processor, heart shape data indicating a three-dimensional shape including an internal structure of a heart of a subject based on echo image data indicating a cross-sectional image of the heart;
generating, by the processor, thorax shape data indicating a three-dimensional shape of a thorax of the subject based on X-ray image data indicating an X-ray image of the thorax of the subject; and
deciding, by the processor and based on an image of the heart appearing in the X-ray image, a position and an orientation of the three-dimensional shape of the heart indicated by the heart shape data within the three-dimensional shape of the thorax indicated by the thorax shape data.

### Claim 12

An information processing apparatus comprising:
a memory; and
a processor coupled to the memory and the processor configured to:
generate heart shape data indicating a three-dimensional shape including an internal structure of a heart of a subject based on echo image data indicating a cross-sectional image of the heart;
generate thorax shape data indicating a three-dimensional shape of a thorax of the subject based on X-ray image data indicating an X-ray image of the thorax of the subject; and
decide, based on an image of the heart appearing in the X-ray image, a position and an orientation of the three-dimensional shape of the heart indicated by the heart shape data within the three-dimensional shape of the thorax indicated by the thorax shape data.
