# US 2020/0394430 A1 · Classifying compartments at security checkpoints by detecting a shape of an object

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0394430 A1
- Document ID: US20200394430A1
- Status: Published application
- Kind: A1
- Application number: 16/441,334
- Filed: 2019-06-14
- Published: 2020-12-17
- Earliest claimed: 2019-06-14
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Mohamed Nooman Ahmed; Samuel H. Woo
- Web page: https://patentgrep.com/patent/US20200394430A1

## CPC

- G01N 23/046: using tomography, e.g. computed tomography [CT]
- G01V 5/20: Detecting prohibited goods, e.g. weapons, explosives, hazardous substances, contraband or smuggled objects
- G06F 18/2115: by evaluating different subsets according to an optimisation criterion, e.g. class separability, forward selection or backward elimination
- G06F 18/2148: {characterised by the process organisation or structure, e.g. boosting cascade}
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06V 10/255: {Detecting or recognising potential candidate objects based on visual cues, e.g. shapes}
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/7747: {Organisation of the process, e.g. bagging or boosting}
- G06V 10/776: Validation; Performance evaluation
- G06V 10/82: using neural networks
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G06V 20/64: Three-dimensional [3D] objects
- G06V 2201/05: Recognition of patterns representing particular kinds of hidden objects, e.g. weapons, explosives, drugs
- G01N: INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES (measuring or testing apparatus or processes other than immunoassay, involving enzymes or microorganisms C12M, C12Q)
- G01V: GEOPHYSICS; GRAVITATIONAL MEASUREMENTS; DETECTING MASSES OR OBJECTS; TAGS (means for indicating the location of accidentally buried, e.g. snow-buried, persons A63B29/02)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A system and method for automatically detecting prohibited objects in a compartment at a security checkpoint includes receiving a three-dimensional representation of a compartment from an imaging device connected to the computing system, identifying, by the processor, a region within the compartment that has a highest likelihood of containing a prohibited object, based on a voxel classification of the three-dimensional representation using a first trained neural network, and classifying, by the processor, shapes of objects located within the identified region using a second trained neural network to determine whether any of the shapes correspond to a shape of a prohibited object known by the second neural network.

## Claims

### Claim 1

A method for automatically detecting prohibited objects in a compartment at a security checkpoint, comprising:
receiving, by a processor of a computing system, a three-dimensional representation of a compartment from an imaging device connected to the computing system;
identifying, by the processor, a region within the compartment that has a highest likelihood of containing a prohibited object, based on a voxel classification of the three-dimensional representation using a first trained neural network; and
classifying, by the processor, shapes of objects located within the identified region using a second trained neural network to determine whether any of the shapes correspond to a shape of a prohibited object known by the second neural network.

### Claim 2 (depends on 1)

The method of claim 1, wherein the three-dimensional representation is a set of images in sequence of the region of the compartment, output by a computed topography (CT) scanner, further wherein each image in the set of images has a width and a height, and when stacked together, form a depth.

### Claim 3 (depends on 1)

The method of claim 1, wherein the first trained neural network determines whether any voxel classifications of the three-dimensional representation correspond to a voxel classification of a prohibited material.

### Claim 4 (depends on 1)

The method of claim 1, wherein the first trained neural network is trained by:
isolating, by the processor, sub-volumes of different sizes around a voxel of a three-dimensional representation of a known prohibited material for each voxel of the three-dimensional representation of the known prohibited material;
applying, by the processor, three-dimensional convolutions on the sub-volumes of different sizes; and
feeding, by the processor, the three-dimensional convolutions into the trained neural network so that the trained neural network learns that the convoluted sub-volumes fed into the trained neural network corresponds to a voxel classification of the known prohibited material,

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
labeling, by the processor, each of the voxels determined to have a same voxel classification as a known prohibited material; and
suggesting, by the processor, the region within the compartment that has the highest likelihood of containing the prohibited object based on a location of the labeled voxels within the compartment.

### Claim 6 (depends on 5)

The method of claim 5, wherein the first trained neural network is a three-dimensional convolutional neural network (CNN) that detects a presence of the prohibited material in the compartment based on the voxel classification of the three-dimensional representation using an entire sub-volume isolated around each voxel, without requiring human inspection of an output of the imaging device.

### Claim 7 (depends on 1)

The method of claim 1, wherein the second trained neural network is trained by:
calculating, by the processor, a maximum intensity projection across each dimension of at least two orthogonal projections of a known three-dimensional representation of a prohibited object;
stacking, by the processor, each of the maximum intensity projections into a single image; and
feeding, by the processor, the single image into the second trained neural network so that the second trained neural network learns that the single image having the same maximum intensity projections corresponds to the prohibited object.

### Claim 8 (depends on 3)

The method of claim 3, wherein the second trained neural network classifies objects located within the region as a prohibited object in response to determining that a shape output from the first trained neural network fed into the second trained neural network compares to previously classified shapes of known prohibited objects known by the second trained neural network.

### Claim 9 (depends on 3)

The method of claim 3, wherein the second trained neural network classifies objects located within the region as a non-prohibited object in response to determining that a shape output from the first trained neural network fed into the second trained neural network does not compare to previously classified shapes of known prohibited objects known by the second trained neural network.

### Claim 10 (depends on 1)

The method of claim 1, wherein the second trained neural network is a two-dimensional convolutional neural network (CNN).

### Claim 11 (depends on 1)

The method of claim 1, wherein the second trained neural network is a three-dimensional convolutional neural network (CNN).

### Claim 12

A computing system, comprising:
a processor;
a memory device coupled to the processor; and
a computer readable storage device coupled to the processor, wherein the storage device contains program code executable by the processor via the memory device to implement a method for automatically detecting prohibited objects in a compartment at a security checkpoint comprising:
receiving, by a processor of a computing system, a three-dimensional representation of a compartment from an imaging device connected to the computing system;
identifying, by the processor, a region within the compartment that has a highest likelihood of containing a prohibited object, based on a voxel classification of the three-dimensional representation using a first trained neural network; and
classifying, by the processor, shapes of objects located within the identified region using a second trained neural network to determine whether any of the shapes correspond to a shape of a prohibited object known by the second neural network.

### Claim 13 (depends on 12)

The computing system of claim 12, wherein the first trained neural network determines whether any voxel classifications of the three-dimensional representation correspond to a voxel classification of a prohibited material.

### Claim 14 (depends on 12)

The computing system of claim 12, wherein the first trained neural network is trained by:
isolating, by the processor, sub-volumes of different sizes around a voxel of a three-dimensional representation of a known prohibited material, for each voxel of the three-dimensional representation of the known prohibited material;
applying, by the processor, three-dimensional convolutions on the sub-volumes of different sizes; and
feeding, by the processor, the three-dimensional convolutions into the trained neural network so that the trained neural network learns that the convoluted sub-volumes fed into the trained neural network corresponds to a voxel classification of the known prohibited material.

### Claim 15 (depends on 12)

The computing system of claim 12, further comprising:
labeling, by the processor, each of the voxels determined to have a same voxel classification as a known prohibited material; and
suggesting, by the processor, the region within the compartment that has the highest likelihood of containing the prohibited object based on a location of the labeled voxels within the compartment.

### Claim 16 (depends on 12)

The computing system of claim 12, wherein the first trained neural network is a three-dimensional convolutional neural network (CNN) that detects a presence of the prohibited material in the compartment based on the voxel classification of the three-dimensional representation using an entire sub-volume isolated around each voxel, without requiring human inspection of an output of the imaging device.

### Claim 17 (depends on 12)

The computing system of claim 12, wherein the second trained neural network is trained by:
calculating, by the processor, a maximum intensity projection across each dimension of at least two orthogonal projections of a known three-dimensional representation of a prohibited object;
stacking, by the processor, the maximum intensity projections into a single image; and
feeding, by the processor, the single image into the second trained neural network so that the second trained neural network learns that the single image having the same maximum intensity projections corresponds to the prohibited object.

### Claim 18 (depends on 12)

The computing system of claim 12, wherein:
the second trained neural network classifies objects located within the region as a prohibited object in response to determining that a shape output from the first trained neural network fed into the second trained neural network compares to previously classified shapes of known prohibited objects known by the second trained neural network;
wherein the second trained neural network classifies objects located within the region as a non-prohibited object in response to determining that a shape output from the first trained neural network fed into the second trained neural network does not compare to previously classified shapes of known prohibited objects known by the second trained neural network.

### Claim 19

A computer program product, comprising a computer readable hardware storage device storing a computer readable program code, the computer readable program code comprising an algorithm that when executed by a computer processor of a computing system implements a method for automatically detecting prohibited objects in a compartment at a security checkpoint comprising:
receiving, by a processor of a computing system, a three-dimensional representation of a compartment from an imaging device connected to the computing system;
identifying, by the processor, a region within the compartment that has a highest likelihood of containing a prohibited object, based on a voxel classification of the three-dimensional representation using a first trained neural network; and
classifying, by the processor, shapes of objects located within the identified region using a second trained neural network to determine whether any of the shapes correspond to a shape of a prohibited object known by the second neural network.

### Claim 20 (depends on 19)

The computer program product of claim 19, further comprising:
labeling, by the processor, each of the voxels determined to have a same voxel classification as a known prohibited material; and
suggesting, by the processor, the region within the compartment that has the highest likelihood of containing the prohibited object based on a location of the labeled voxels within the compartment.
