# US 2022/0277451 A1 · Systems, methods and apparatuses for visualization of imaging data

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0277451 A1
- Document ID: US20220277451A1
- Status: Published application
- Kind: A1
- Application number: 17/632,007
- Filed: 2020-07-31
- Published: 2022-09-01
- Earliest claimed: 2019-08-01
- Assignees: Perimeter Medical Imaging, Inc.
- Applicants: Perimeter Medical Imaging Inc.
- Inventors: Vladimir PEKAR; David REMPEL
- Web page: https://patentgrep.com/patent/US20220277451A1

## CPC

- G16H 30/40: for processing medical images, e.g. editing
- G06F 3/0481: based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F 3/04845: for image manipulation, e.g. dragging, rotation, expansion or change of colour
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/09: Supervised learning
- G06N 20/00: Machine learning
- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 5/77: Retouching; Inpainting; Scratch removal
- G06T 7/0012: {Biomedical image inspection}
- G06T 7/0014: {using an image reference approach}
- G06T 11/60: Creating or editing images; Combining images with text
- G16H 30/20: for handling medical images, e.g. DICOM, HL7 or PACS
- G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06T 2200/24: involving graphical user interfaces [GUIs]
- G06T 2207/10024: Color image
- G06T 2207/10101: Optical tomography; Optical coherence tomography [OCT]
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30024: Cell structures in vitro; Tissue sections in vitro
- G06T 2207/30096: Tumor; Lesion
- G06T 2207/30204: Marker
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Medical imaging data can include imaging data of tissue portions, such as, for example, tissue samples. A processor is configured to receive imaging data obtained from imaging tissue. The imaging data includes a two-dimensional scan of tissue, such as, for example, a B-scan. The processor is configured to analyze the imaging data, and to identify imaging data associated with suspicious areas. The processor is configured to generate a visual interface that displays the imaging data associated with the suspicious areas in a stitched view. The processor, via the visual interface, enables a user to navigate to locations in the two-dimensional scan that correspond to imaging data associated with suspicious areas that the user determines to be positive for a characteristic (e.g., cancer).

## Claims

### Claim 1

An apparatus, comprising:
a display configured to display a user interface;
a memory; and
a processor operatively coupled to the display and the memory, the processor configured to:
receive image data of a tissue sample;
identify, using a machine learning algorithm trained using a set of images to identify regions of abnormality in the image data, a set of patches of the image data, each patch from the of patches including at least one feature associated with an abnormality;
generate a consolidated view of the set of patches in which the set of patches are arranged according to a predefined layout; and
display the consolidated view of the set of patches on the user interface; and
wherein each patch from the set of patches is associated with a spatial location in the tissue sample and the processor is configured to display markings in the consolidated view to associate patches from the set of patches that have proximate spatial locations with one another.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein the image data includes a set of two-dimensional scans of the tissue sample,
the processor further configured to process the set of two-dimensional scans of the tissue sample into a plurality of patches each having the same predefined dimensions, each patch from the plurality of patches being a different portion of a two-dimensional scan from the set of two-dimensional scans,
the set of patches being patches from the plurality of patches that include at least one feature associated with an abnormality.

### Claim 3

(canceled)

### Claim 4 (depends on 1)

The apparatus of claim 1, wherein the processor is configured to display (1) a first marking having a first color proximate to each patch from a first subset of the set of patches and (2) a second marking having a second color different from the first color proximate to each patch from a second subset of the set of patches,
each patch from the first subset of patches having a spatial location that is adjacent to at least one other patch from the first subset of patches, and each patch from the second subset of patches having a spatial location that is adjacent to at least one other patch from the second subset of patches.

### Claim 5 (depends on 1)

The apparatus of claim 1, wherein the processor is configured to display the consolidated view of the set of patches in a first area of the user interface,
the processor further configured to display a perspective view of the tissue sample in a second area of the user interface such that the perspective view of the tissue sample is displayed together with at least a portion of the consolidated view of the set of patches.

### Claim 6 (depends on 5)

The apparatus of claim 5, wherein the processor is further configured to display a two-dimensional view of the tissue sample at a predefined depth in a third area of the user interface such that the two-dimensional view of the tissue sample is displayed together with the perspective view and at least a portion of the consolidated view of the set of patches.

### Claim 7 (depends on 6)

The apparatus of claim 6, wherein each patch from the set of patches is associated with a spatial location in the tissue sample,
the processor further configured to display a first marking proximate to a patch from the set of patches in the consolidated view and a second marking in at least one of the perspective view or the two-dimensional view that is indicative of the spatial location of the patch, the first and second markings sharing a common characteristic.

### Claim 8 (depends on 7)

The apparatus of claim 7, wherein the first and second markings have the same color.

### Claim 9 (depends on 6)

The apparatus of claim 6, wherein the processor is configured to, in response to receiving an input indicating a different predefined depth, display a two-dimensional view of the tissue sample at the different predefined depth.

### Claim 10 (depends on 1)

The apparatus of claim 1, wherein the processor is configured to, in response to detecting a selection of a patch from the set of patches, display a two-dimensional scan of the tissue sample that includes the patch selected from the set of patches.

### Claim 11 (depends on 10)

The apparatus of claim 10, wherein the processor is further configured to display a marking proximate to a portion of the two-dimensional scan that corresponds to a location of the patch selected from the set of patches.

### Claim 12 (depends on 1)

The apparatus of claim 1, wherein the processor is further configured to, in response to receiving an input indicating that a patch from the set of patches is not associated with an abnormality:
remove the patch from the set of patches; and
after removing the patch from the set of patches, generate an updated consolidated view of the set of patches.

### Claim 13 (depends on 1)

The apparatus of claim 1, further comprising an imaging device configured to obtain the image data,
the processor operatively coupled to the imaging device.

### Claim 14

An apparatus, comprising:
a memory; and
a processor operatively coupled to the memory, the processor configured to:
receive a set of two-dimensional image scans of a three-dimensional tissue sample;
process the set of two-dimensional image scans to produce a set of patches each having the same dimensions, each patch from the set of patches including image data from a different portion of the set of two-dimensional image scans;
identify, using a machine learning algorithm trained using a set of images to identify regions of abnormality in the image data, a subset of patches (1) from the set of patches and (2) including at least one feature associated with an abnormality; and
generate a consolidated view of the subset of patches in which the subset of patches are arranged according to a predefined layout, and wherein each patch from the set of patches is associated with a spatial location in the tissue sample, the subset of patches arranged in the consolidated view such that patches from the subset of patches having spatial locations proximate to one another are arranged adjacent to one another.

### Claim 15

(canceled)

### Claim 16 (depends on 14)

The apparatus of claim 14, wherein the consolidated view of the subset of patches includes information associated with each patch from the subject of patches, the information including at least one of: a ductal carcinoma in situ (DCIS) score, a confidence value associated with the DCIS score, spatial location information.

### Claim 17 (depends on 14)

The apparatus of claim 14, wherein the processor is further configured to, in response to receiving an input indicating that a patch from the subset of patches is not associated with an abnormality:
remove the patch from the subset of patches; and
after removing the patch from the subset of patches, generate an updated consolidated view of the subset of patches.

### Claim 18

A method, comprising:
receiving image data of a tissue sample;
identifying, using a machine learning algorithm trained using a set of images to identify regions of abnormality in the image data, a set of patches of the image data, each patch from the of patches including at least one feature associated with an abnormality;
generating a consolidated view of the set of patches in which the set of patches are arranged according to a predefined layout; and
displaying the consolidated view of the set of patches on a user interface and wherein each patch from the set of patches is associated with a spatial location in the tissue sample, the method further comprising: displaying markings in the consolidated view to associate patches from the set of patches that have proximate spatial locations with one another.

### Claim 19

(canceled)

### Claim 20 (depends on 18)

The method of claim 18, wherein the consolidated view of the set of patches is displayed in a first area of the user interface, the method further comprising:
displaying in one or more second areas of the user interface at least one of: a perspective view of the tissue sample, or a two-dimensional view of the tissue sample at a predefined depth.

### Claim 21 (depends on 14)

The apparatus of claim 14, wherein the processor is configured to display the consolidated view of the set of patches in a first area of the user interface, the processor further configured to display a perspective view of the tissue sample in a second area of the user interface such that the perspective view of the tissue sample is displayed together with at least a portion of the consolidated view of the set of patches.

### Claim 22 (depends on 14)

The apparatus of claim 14, wherein the processor is further configured to display a two-dimensional view of the tissue sample at a predefined depth in a third area of the user interface such that the two-dimensional view of the tissue sample is displayed together with the perspective view and at least a portion of the consolidated view of the set of patches.

### Claim 23 (depends on 14)

The apparatus of claim 14, wherein each patch from the set of patches is associated with a spatial location in the tissue sample, the processor further configured to display a first marking proximate to a patch from the set of patches in the consolidated view and a second marking in at least one of the perspective view or the two-dimensional view that is indicative of the spatial location of the patch, the first and second markings sharing a common characteristic.
