# US 9,384,528 B2 · Image annotation using a haptic plane

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 9,384,528 B2
- Document ID: US9384528B2
- Status: Granted
- Kind: B2
- Application number: 14/120,519
- Filed: 2014-05-28
- Published: 2015-12-03
- Granted: 2016-07-05
- Earliest claimed: 2014-05-28
- Assignees: EchoPixel, Inc.
- Applicants: EchoPixel, Inc.
- Inventors: Sergio Aguirre-Valencia; William Johnsen; Anthony Chen
- Web page: https://patentgrep.com/patent/US9384528B2

## CPC

- G16H 30/40: for processing medical images, e.g. editing
- G06F 3/016: {Input arrangements with force or tactile feedback as computer generated output to the user}
- G06F 40/169: Annotation, e.g. comment data or footnotes
- G06T 3/20: Linear translation of whole images or parts thereof, e.g. panning
- G06T 3/60: Rotation of whole images or parts thereof
- G06T 19/00: Manipulating three-dimensional [3D] models or images for computer graphics
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- G16H 30/20: for handling medical images, e.g. DICOM, HL7 or PACS
- G06T 2207/30004: Biomedical image processing
- G06T 2210/41: Medical
- G06T 2219/004: Annotating, labelling
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

During an analysis technique, a three-dimensional (3D) image of a portion of an individual is transformed to facilitate accurate determination of detailed annotation of an anatomical structure (such as: specifying the size of the anatomical structure based on annotation markers, an orientation of the anatomical structure, a direction of the anatomical structure and/or a location of the anatomical structure). In particular, in response to receiving information specifying a two-dimensional (2D) plane having an angular position in the 3D image, the 3D image is translated and rotated from an initial position and orientation so that the 2D plane is presented in an orientation parallel to a reference 2D plane of a display. Then, after information specifying the detailed annotation in the 2D plane is received, the 3D image is translated and rotated back to the initial position and orientation.

## Claims

### Claim 1

A computer-implemented method for facilitating detailed annotation of an anatomical structure, the method comprising:
providing, on a display, a three-dimensional (3D) image of a portion of an individual, wherein the 3D image has an initial position and orientation;
receiving information specifying a two-dimensional (2D) plane in the 3D image, wherein the 2D plane has an arbitrary angular position relative to the initial orientation;
using the computer, translating and rotating the 3D image so that the 2D plane is presented in a reference 2D plane of the display with an orientation parallel to an orientation of the reference 2D plane, wherein, prior to the translating and the rotating, the angular position is different than that of the reference 2D plane and is different from a predefined orientation of slices in the 3D image;
receiving information specifying the detailed annotation in the 2D plane, wherein the detailed annotation includes at least one of: a size of the anatomical structure based on annotation markers, an orientation of the anatomical structure, a direction of the anatomical structure and a location of the anatomical structure; and
after the annotation is complete, translating and rotating the 3D image back to the initial position and orientation.

### Claim 2 (depends on 1)

The method of claim 1, wherein the 3D image includes a stereoscopic image.

### Claim 3 (depends on 1)

The method of claim 1, wherein the information specifying the detailed annotation is received from an interaction tool.

### Claim 4 (depends on 1)

The method of claim 1, wherein the information specifying the detailed annotation corresponds to haptic interaction between a digit of a user and the display.

### Claim 5 (depends on 1)

The method of claim 1, wherein the method further comprises, prior to translating and rotating the 3D image back to the initial orientation, receiving a command indicating that the annotation is complete.

### Claim 6 (depends on 1)

The method of claim 1, wherein the 2D plane is positioned at a zero parallax position so that 3D information in the 2D plane is perceived as 2D information.

### Claim 7 (depends on 1)

The method of claim 1, wherein the angular position is at an oblique angle relative to a symmetry axis of the individual.

### Claim 8 (depends on 1)

The method of claim 1, wherein the 3D image corresponds to one of: a cross section of a volume, and a multiplanar-reconstruction image.

### Claim 9

A computer-program product for use in conjunction with a computer system, the computer-program product comprising a non-transitory computer-readable storage medium and a computer-program mechanism embedded therein to facilitate detailed annotation of an anatomical structure, the computer-program mechanism including:
instructions for providing, on a display, a three-dimensional (3D) image of a portion of an individual, wherein the 3D image has an initial position and orientation;
instructions for receiving information specifying a two-dimensional (2D) plane in the 3D image, wherein the 2D plane has an arbitrary angular position relative to the initial orientation;
instructions for translating and rotating the 3D image so that the 2D plane is presented in a reference 2D plane of the display with an orientation parallel to an orientation of the reference 2D plane, wherein, prior to executing the instructions for translating and the instructions for rotating, the angular position is different than that of the reference 2D plane and is different from a predefined orientation of slices in the 3D image;
instructions for receiving information specifying detailed annotation in the 2D plane, wherein the detailed annotation includes at least one of: a size of the anatomical structure based on annotation markers, an orientation of the anatomical structure, a direction of the anatomical structure and a location of the anatomical structure; and
instructions for translating and rotating the 3D image back to the initial position and orientation after the annotation is complete.

### Claim 10 (depends on 9)

The computer-program product of claim 9, wherein the 3D image includes a stereoscopic image.

### Claim 11 (depends on 9)

The computer-program product of claim 9, wherein the information specifying the detailed annotation is received from an interaction tool.

### Claim 12 (depends on 9)

The computer-program product of claim 9, wherein the information specifying the detailed annotation corresponds to haptic interaction between a digit of a user and the display.

### Claim 13 (depends on 9)

The computer-program product of claim 9, wherein the computer-program mechanism further comprises, prior to the instructions for rotating the 3D image back to the initial orientation, instructions for receiving a command indicating that the annotation is complete.

### Claim 14 (depends on 9)

The computer-program product of claim 9, wherein the 2D plane is positioned at a zero parallax position so that 3D information in the 2D plane is perceived as 2D information.

### Claim 15 (depends on 9)

The computer-program product of claim 9, wherein the angular position is at an oblique angle relative to a symmetry axis of the individual.

### Claim 16 (depends on 9)

The computer-program product of claim 9, wherein the 3D image corresponds to one of: a cross section of a volume, and a multiplanar-reconstruction image.

### Claim 17

A computer system, comprising:
a processor;
memory; and
a program module, wherein the program module is stored in the memory and configurable to be executed by the processor to facilitate detailed annotation of an anatomical structure, the program module including:
instructions for providing, on a display, a three-dimensional (3D) image of a portion of an individual, wherein the 3D image has an initial position and orientation;
instructions for receiving information specifying a two-dimensional (2D) plane in the 3D image, wherein the 2D plane has an arbitrary angular position relative to the initial orientation;
instructions for translating and rotating the 3D image so that the 2D plane is presented in a reference 2D plane of the display with an orientation parallel to an orientation of the reference 2D plane, wherein, prior to the instructions for translating and the instructions for rotating, the angular position is different than that of the reference 2D plane and is different from a predefined orientation of slices in the 3D image;
instructions for receiving information specifying detailed annotation in the 2D plane, wherein the detailed annotation includes at least one of: a size of the anatomical structure based on annotation markers, an orientation of the anatomical structure, a direction of the anatomical structure and a location of the anatomical structure; and
instructions for translating and rotating the 3D image back to the initial position and orientation after the annotation is complete.

### Claim 18 (depends on 17)

The computer system of claim 17, wherein the 2D plane is positioned at a zero parallax position so that 3D information in the 2D plane is perceived as 2D information.

### Claim 19 (depends on 17)

The computer system of claim 17, wherein the angular position is at an oblique angle relative to a symmetry axis of the individual.

### Claim 20 (depends on 17)

The computer system of claim 17, wherein the 3D image corresponds to one of: a cross section of a volume, and a multiplanar-reconstruction image.
