# US 12,182,529 B2 · Multi-task adapters and task similarity for efficient extraction of pathologies from medical reports

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,182,529 B2
- Document ID: US12182529B2
- Status: Granted
- Kind: B2
- Application number: 17/992,212
- Filed: 2022-11-22
- Published: 2023-05-25
- Granted: 2024-12-31
- Earliest claimed: 2021-11-24
- Assignees: Covera Health
- Applicants: Covera Health
- Inventors: Arijit Sehanobish; Anasuya Das; Benjamin Odry
- Web page: https://patentgrep.com/patent/US12182529B2

## CPC

- G06F 40/58: Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
- G16H 30/40: for processing medical images, e.g. editing
- G06F 40/117: Tagging; Marking up (details of markup languages G06F40/143); Designating a block; Setting of attributes (style sheets, e.g. eXtensible Stylesheet Language Transformation [XSLT], G06F40/154)
- G06F 40/295: Named entity recognition
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/09: Supervised learning
- G06N 3/096: Transfer learning
- G06T 7/0012: {Biomedical image inspection}
- G16H 10/60: for patient-specific data, e.g. for electronic patient records
- G16H 40/63: for local operation
- G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems
- G16H 50/70: for mining of medical data, e.g. analysing previous cases of other patients
- G06T 2207/10072: Tomographic images
- G06T 2207/10116: X-ray image
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G16H 70/60: relating to pathologies
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Described are techniques for processing text data. A process can include obtaining an input text data indicative of pathologies associated with a corresponding radiological image. A BERT-based machine learning network can be used to generate a plurality of location tags, each location tag associated with a sentence of input text data and indicative of an anatomical location in the corresponding radiological image. A plurality of sentence groups can be generated using the input text data and the plurality of location tags, each sentence group including sentences of input text data that are associated with a location tag indicative of the same anatomical location. A multi-task learning (MTL)-based machine learning network can be used to generate a plurality of sets of features, each set of features generated based on a particular sentence group and indicative of pathology severity predictions determined for the anatomical location associated with the particular sentence group.

## Claims

### Claim 1

An apparatus for processing text data, the apparatus comprising:
at least one memory; and
at least one processor coupled to the at least one memory, the at least one processor configured to:
obtain an input text data, the input text data indicative of one or more pathologies associated with a corresponding radiological image, wherein the one or more pathologies are included in a plurality of pathologies;
generate, using a Bidirectional Encoder Representations from Transformers (BERT)-based machine learning network, a plurality of location tags associated with the input text data, wherein each respective location tag of the plurality of location tags is associated with a sentence included in the input text data and is indicative of an anatomical location;
generate a plurality of sentence groups based on the input text data and the plurality of location tags, wherein each sentence group of the plurality of sentence groups includes one or more sentences that are included in the input text data and are associated with a respective location tag indicative of a same respective anatomical location; and
generate, using a multi-task learning (MTL)-based machine learning network, a plurality of sets of features, wherein each set of features of the plurality of sets of features:
is generated based on a particular sentence group of the plurality of sentence groups; and
is indicative of a plurality of pathology severity predictions determined for the respective anatomical location associated with the particular sentence group.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein the BERT-based machine learning network includes:
a plurality of BERT output layers; and
one or more Adapter machine learning networks, wherein each Adapter machine learning network is provided between a pair of BERT output layers included in the plurality of BERT output layers.

### Claim 3 (depends on 2)

The apparatus of claim 2, wherein each BERT output layer of the plurality of BERT output layers is associated with at least one Adapter machine learning network.

### Claim 4 (depends on 2)

The apparatus of claim 2, wherein the same one or more Adapter machine learning networks are utilized to generate the plurality of pathology severity predictions for each pathology of the plurality of pathologies.

### Claim 5 (depends on 2)

The apparatus of claim 2, wherein:
the plurality of BERT output layers are associated with a first set of learned weights; and
each Adapter machine learning network comprises a multi-layer feedforward network associated with a second set of learned weights, wherein the second set of learned weights are trained based on freezing the first set of learned weights associated with the plurality of BERT output layers.

### Claim 6 (depends on 1)

The apparatus of claim 1, wherein the plurality of sets of features includes a pathology severity prediction for each pathology of the plurality of pathologies at each respective anatomical location.

### Claim 7 (depends on 1)

The apparatus of claim 1, wherein each set of features:
is associated with a different sentence group of the plurality of sentence groups;
is associated with a different respective anatomical location of a plurality of anatomical locations in the corresponding radiological image.

### Claim 8 (depends on 6)

The apparatus of claim 6, wherein each set of features of the plurality of sets of features is generated using a respective classifier head included in the MTL-based machine learning network.

### Claim 9 (depends on 1)

The apparatus of claim 1, wherein the MTL-based machine learning network comprises a second BERT-based machine learning network.

### Claim 10 (depends on 1)

The apparatus of claim 1, wherein each respective location tag is associated with a sentence included in the input text data and is indicative of an anatomical location in the corresponding radiological image.

### Claim 11 (depends on 10)

The apparatus of claim 10, wherein the corresponding radiological image is a cervical spine radiological image and each respective location tag is indicative of a particular motion segment of the cervical spine.

### Claim 12 (depends on 1)

The apparatus of claim 1, wherein the MTL-based machine learning network includes a plurality of jointly trained classifier heads, each jointly trained classifier head of the plurality of jointly trained classifier heads associated with a particular pathology of the plurality of pathologies.

### Claim 13 (depends on 12)

The apparatus of claim 12, wherein the plurality of pathology severity predictions determined for each respective anatomical location are generated based on:
analyzing the particular sentence group associated with each respective anatomical location using the plurality of jointly trained classifier heads; and
determining, for each respective anatomical location, a pathology severity prediction for each pathology of the plurality of pathologies, wherein each jointly trained classifier head of the plurality of jointly trained classifier heads generates a pathology severity prediction associated with a different pathology.

### Claim 14 (depends on 12)

The apparatus of claim 12, wherein:
each jointly trained classifier head is used to generate a respective sub-set of features associated with the particular pathology of the plurality of pathologies.

### Claim 15 (depends on 1)

The apparatus of claim 1, wherein the input text data comprises a radiological report text indicative of one or more clinical findings associated with the corresponding radiological image.

### Claim 16

A method for processing text data, comprising:
obtaining an input text data indicative of one or more pathologies associated with a corresponding radiological image, wherein the one or more pathologies are included in a plurality of pathologies;
generating, using a Bidirectional Encoder Representations from Transformers (BERT)-based machine learning network, a plurality of location tags associated with the input text data, wherein each respective location tag of the plurality of location tags is associated with a sentence included in the input text data and is indicative of an anatomical location of a plurality of anatomical locations in the corresponding radiological image;
generating a plurality of sentence groups based on the input text data and the plurality of location tags, wherein each sentence group of the plurality of sentence groups includes one or more sentences that are included in the input text data and are associated with a respective location tag indicative of a same respective anatomical location; and
generating, using a multi-task learning (MTL)-based machine learning network, a plurality of sets of features, wherein each set of features of the plurality of sets of features:
is generated based on a particular sentence group of the plurality of sentence groups; and
is indicative of a plurality of pathology severity predictions determined for the respective anatomical location associated with the particular sentence group.

### Claim 17 (depends on 16)

The method of claim 16, wherein the BERT-based machine learning network includes:
a plurality of BERT output layers; and
one or more Adapter machine learning networks, wherein each Adapter machine learning network is provided between a pair of BERT output layers included in the plurality of BERT output layers.

### Claim 18 (depends on 16)

The method of claim 16, wherein:
the plurality of sets of features includes a pathology severity prediction for each pathology of the plurality of pathologies at each respective anatomical location; and
each set of features of the plurality of sets of features is associated with a different sentence group of the plurality of sentence group and a different anatomical location of a plurality of anatomical locations in the corresponding radiological image;
each respective location tag is associated with a sentence included in the input text data and is indicative of an anatomical location in the corresponding radiological image.

### Claim 19 (depends on 16)

The method of claim 16, wherein the MTL-based machine learning network includes a plurality of jointly trained classifier heads, each jointly trained classifier head of the plurality of jointly trained classifier heads associated with a particular pathology of the plurality of pathologies.

### Claim 20 (depends on 19)

The method of claim 19, wherein the plurality of pathology severity predictions determined for each respective anatomical location are generated based on:
analyzing the particular sentence group associated with each respective anatomical location using the plurality of jointly trained classifier heads; and
determining, for each respective anatomical location, a pathology severity prediction for each pathology of the plurality of pathologies, wherein each jointly trained classifier head of the plurality of jointly trained classifier heads generates a pathology severity prediction associated with a different pathology.