# US 12,014,830 B2 · System and method for automation of surgical pathology processes using artificial intelligence

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,014,830 B2
- Document ID: US12014830B2
- Status: Granted
- Kind: B2
- Application number: 17/722,567
- Filed: 2022-04-18
- Published: 2022-11-24
- Granted: 2024-06-18
- Earliest claimed: 2021-04-18
- Assignees: Mary Hitchcock Memorial Hospital, for itself and on behalf of Dartmouth-Hitchcock Clinic
- Applicants: Mary Hitchcock Memorial Hospital, for itself and on behalf of Dartmouth-Hitchcock Clinic
- Inventors: Matthew LeBoeuf; Joshua J. Levy; Louis J. Vaickus
- Web page: https://patentgrep.com/patent/US12014830B2

## CPC

- G06V 20/70: Labelling scene content, e.g. deriving syntactic or semantic representations
- G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems
- G06V 10/82: using neural networks
- G16H 15/00: ICT specially adapted for medical reports, e.g. generation or transmission thereof
- G16H 30/40: for processing medical images, e.g. editing
- G16H 70/60: relating to pathologies
- G06V 2201/03: Recognition of patterns in medical or anatomical images
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

This invention provides a system for automation of a pathology process, which includes a processor having trained artificial intelligence (AI) modules operating in association therewith, adapted to receive image data from camera images of whole tissue acquired by a camera assembly and whole slide images (WSIs) of inked and segmented tissue samples. A mask produces image results for tissue with holes and free of holes, and a filter provides filtered image results to the AI modules, detecting tumors and macroarchitecture features. A quality assessment process produces quality score outputs for tumors and macroarchitecture features. A report generator provides reports with one or more parameters to a user via an interface. More particularly, the report generator automatically creates a pathology report, having a written description and pictorial diagram relative to the gross images of the tissue integrating the outputs of the AI modules used to analyze the whole slide digital images.

## Claims

### Claim 1

A system for automation of a pathology process comprising;
a processor having a neural network, having trained artificial intelligence (AI) modules operating in association therewith, adapted to receive image data from images of whole tissue acquired by an camera assembly and whole slide images (WSIs) of inked and segmented tissue samples;
a mask that produces image results for tissue with holes and free of holes;
a filter that provides filtered image results to the AI modules, which thereby detects tumors and macroarchitecture features;
a quality assessment process that produces quality score outputs for tumors and macroarchitecture features;
a report generator that provides reports with one or more parameters to a user via an interface;
a tissue determination process that determines a size and a description of tissue automatically based upon characteristics in the images acquired by the camera assembly;
a tissue grossing and inking process that automatically generates a grossing and inking scheme based upon a user input of desired percentage of tissue margins analyzed and the tissue size; and
an ink detection and orientation process that operates on the filtered images of the tissue samples and delivers results thereof to the report generator, and
wherein the macroarchitecture features are defined by holes, fat, edges, dermis and epidermis information.

### Claim 2 (depends on 1)

The system as set forth in claim 1 wherein the AI modules include a tumor inflammation convolutional neural network (CNN), a macroarchitecture hole CNN, a tumor detection graphical neural network (GNN) and a macroarchitecture detection GNN.

### Claim 3 (depends on 1)

The system as set forth in claim 1 wherein the report generator receives results from the quality assessment process, and wherein the quality assessment process acts on results from a tumor detection GNN and a macroarchitecture detection GNN.

### Claim 4 (depends on 1)

The system as set forth in claim 1, further comprising, a visual generation and 3D stitching process that provides results to the report generator.

### Claim 5 (depends on 1)

The system as set forth in claim 1, wherein the report generator automatically creates a pathology report consisting of a written description and pictorial diagram relative to the images of the tissue.

### Claim 6 (depends on 2)

The system as set forth in claim 2 wherein the filters include at least one of a Sobel filter and a gradient-based filter.

### Claim 7 (depends on 3)

The system as set forth in claim 3 wherein the report generator automatically creates a pathology report consisting of a written description and pictorial diagram relative to the images of the tissue.

### Claim 8 (depends on 4)

The system as set forth in claim 4 wherein the visual generation process receives information from a cell nuclei detection process.

### Claim 9 (depends on 8)

The system as set forth in claim 8 wherein the report generator automatically creates a pathology report consisting of a written description and pictorial diagram relative to the images of the tissue.

### Claim 10

A method for automation of a pathology process comprising the steps of:
providing a processor with a neural network having artificial intelligence (AI) modules operating in association therewith, which receive image data from camera images of whole tissue acquired by an camera assembly and whole slide images (WSIs) of inked and segmented tissue samples;
producing image results for tissue with holes and free of holes with a mask;
providing filtered image results to the AI modules, and detecting tumors and macroarchitecture features therewith;
generating quality score outputs for tumors and macroarchitecture features;
generating a report with one or more parameters to a user via an interface;
determining a size and a description of tissue automatically based upon characteristics in the images acquired by the camera assembly;
automatically generating a grossing and inking scheme based upon a user input of desired percentage of tissue margins analyzed and the tissue size; and
operating an ink detection and orientation process on the filtered images of the tissue samples that delivers results thereof to the report generator, and
wherein the macroarchitecture features are defined by holes, fat, edges, dermis and epidermis information.

### Claim 11 (depends on 10)

The method as set forth in claim 10, wherein the step of generating the report includes automatically creating a pathology report consisting of a written description and pictorial diagram relative to the images of the tissue.

### Claim 12 (depends on 10)

The system as set forth in claim 10, further comprising, defining the AI modules to include a tumor inflammation convolutional neural network (CNN), a macroarchitecture hole CNN, a tumor detection graphical neural network (GNN) and a macroarchitecture detection GNN.

### Claim 13 (depends on 10)

The method as set forth in claim 10, further comprising, wherein the step of generating includes receiving results from a quality assessment process acting on results from a tumor detection GNN and a macroarchitecture detection GNN.

### Claim 14 (depends on 10)

The method as set forth in claim 10, further comprising, wherein the step of generating includes operating a visual generation and 3D stitching process that provides results.

### Claim 15 (depends on 12)

The method as set forth in claim 12, wherein the step of providing filtered image results includes operating at least one of a Sobel filter and a gradient-based filter.

### Claim 16 (depends on 14)

The method as set forth in claim 14, further comprising, receiving by the visual generation process, information from a cell nuclei detection process.

### Claim 17 (depends on 16)

The method as set forth in claim 16, wherein the step of generating the report includes automatically creating a pathology report consisting of a written description and pictorial diagram relative to the images of the tissue.