# US 2021/0341504 A1 · Visualization analysis apparatus and visual learning methods

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0341504 A1
- Document ID: US20210341504A1
- Status: Published application
- Kind: A1
- Application number: 17/278,283
- Filed: 2019-09-19
- Published: 2021-11-04
- Earliest claimed: 2018-09-20
- Assignees: Siemens Healthcare Diagnostics Inc.
- Applicants: Siemens Healthcare Diagnostics Inc.
- Inventors: Rayal Raj Prasad Nalam Venkat; Benjamin S. Pollack; Yao-Jen Chang; Vivek Singh; Venkatesh NarasimhaMurthy
- Web page: https://patentgrep.com/patent/US20210341504A1

## CPC

- G01N 35/00732: {Identification of carriers, materials or components in automatic analysers}
- G01N 15/05: in blood
- G06F 18/2413: based on distances to training or reference patterns
- G06T 7/0012: {Biomedical image inspection}
- G06T 7/74: {involving reference images or patches}
- G06V 10/225: {based on a marking or identifier characterising the area}
- G06V 10/764: using classification, e.g. of video objects
- G01N 35/021: {having a flexible chain, e.g. "cartridge belt", conveyor for reaction cells or cuvettes}
- G01N 2015/045: {by optical analysis}
- G01N 2035/00752: {bar codes}
- G06T 7/11: Region-based segmentation
- G06T 2207/10004: Still image; Photographic image
- G06T 2207/10024: Color image
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2210/41: Medical
- G01N: INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES (measuring or testing apparatus or processes other than immunoassay, involving enzymes or microorganisms C12M, C12Q)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A method of characterizing a specimen in a specimen container includes capturing one or more images of the specimen container, wherein the one or more images include one or more objects of the specimen container, and wherein the capturing generates pixel data from a plurality of pixels. The method further includes identifying one or more selected objects from the one or more objects, displaying an image of the specimen container, and displaying, on the image of the specimen container, one or more locations of pixels used to identify the one or more selected objects. Other apparatus and methods are disclosed.

## Claims

### Claim 1

A method of identifying objects of a specimen container, comprising:
capturing one or more images of the specimen container, the one or more images including one or more objects of the specimen container, the capturing generating pixel data from a plurality of pixels;
identifying one or more selected objects from the one or more objects using one or more neural networks;
displaying an image of the specimen container; and
displaying, on the image of the specimen container, one or more locations of pixels used by the one or more neural networks to identify the one or more selected objects.

### Claim 2 (depends on 1)

The method of claim 1, wherein the one or more selected objects is at least one of a cap, an air gap, a serum or plasma portion, a settled blood portion, and a gel separator.

### Claim 3 (depends on 1)

The method of claim 1, wherein identifying one or more selected objects is performed, at least in part, by at least one of a segmentation network and a classification network.

### Claim 4 (depends on 1)

The method of claim 1, further comprising segmenting the one or more images into a plurality of pixel classes and wherein displaying comprises displaying an image showing one or more locations of one or more pixels constituting one or more of pixel classes relative to the specimen container.

### Claim 5 (depends on 1)

The method of claim 1, wherein displaying comprises displaying an image at least partially representing the specimen container.

### Claim 6 (depends on 5)

The method of claim 5, further comprising overlaying one or more images representing the one or more objects over an image at least partially representing the specimen container, wherein locations of the one or more objects are in locations of their respective pixels relative to the image at least partially representing the specimen container.

### Claim 7 (depends on 1)

The method of claim 1, wherein identifying one or more selected objects from the one or more objects comprises identifying a cap.

### Claim 8 (depends on 7)

The method of claim 7, further comprising identifying a color of the cap.

### Claim 9 (depends on 1)

The method of claim 1, wherein identifying one or more selected objects from the one or more objects comprises identifying a label.

### Claim 10 (depends on 9)

The method of claim 9, further comprising reading the label.

### Claim 11 (depends on 9)

The method of claim 9, further comprising identifying a physical condition of the label.

### Claim 12 (depends on 1)

The method of claim 1, wherein identifying one or more selected objects from the one or more objects comprises identifying a serum or plasma portion.

### Claim 13 (depends on 12)

The method of claim 12, further comprising identifying an interferent in the serum or plasma region.

### Claim 14 (depends on 12)

The method of claim 12, further comprising identifying at least one of hemolysis, icterus, or lipemia in the serum or plasma region.

### Claim 15 (depends on 1)

The method of claim 1, wherein identifying one or more selected objects comprises assigning a confidence gradient to one or more pixels used by the one or more neural networks to identify the one or more selected objects, and wherein displaying one or more locations of pixels used by the one or more neural networks to identify the one or more selected objects comprises displaying, on the image of the specimen container, one or more images indicating the confidence gradient of the one or more pixels used by the one or more neural networks to identify the one or more selected objects.

### Claim 16 (depends on 1)

The method of claim 1, comprising generating an activation map, and displaying, on an image of the specimen container, the activation map.

### Claim 17

A quality check module, comprising:
one or more image capture devices operative to capture one or more images from one or more viewpoints of a specimen container, wherein capturing one or more images generates pixel data from a plurality of pixels; and
a computer coupled to the one or more image capture devices, the computer configured and operative to:
identify one or more selected objects from one or more objects of the specimen container using one or more neural networks;
display an image of the specimen container; and
display, on the image of the specimen container, one or more locations of pixels used by the one or more neural networks to identify the one or more selected objects.

### Claim 18 (depends on 17)

The quality check module of claim 17, wherein the one or more selected objects is at least one of a cap, an air gap, a serum or plasma portion, a settled blood portion, and a gel separator.

### Claim 19 (depends on 17)

The quality check module of claim 17, wherein the computer is operative to generate an activation map.

### Claim 20 (depends on 19)

The quality check module of claim 19, wherein the computer is operative to overlay the activation map over the image of the specimen container.

### Claim 21

A specimen testing apparatus, comprising:
a track;
a carrier moveable on the track and configured to contain a specimen container containing a serum or plasma portion of a specimen therein;
a plurality of image capture devices arranged around the track and operative to capture one or more images from one or more viewpoints of the specimen container and the serum or plasma portion of the specimen, wherein capturing an image generates pixel data from a plurality of pixels; and
a computer coupled to the plurality of image capture devices, the computer configured and operative to:
identify one or more selected objects from one or more objects of the specimen container using one or more neural networks;
display an image of the specimen container; and
display, on the image of the specimen container, one or more locations of pixels used by the one or more neural networks to identify the one or more selected objects.
