US 12,754,309 B2 · Human-perceptible and machine-readable shape generation and classification of hidden objects

Abstract

System and methodology are disclosed for approximating traditional SAR imaging on mobile mmWave devices. The presently disclosed technology enables human-perceptible and machine-readable shape generation and classification of hidden objects on mobile mmWave devices. The resulting system and corresponding methodology are capable of imaging through obstructions, like clothing, and under low visibility conditions. To this end, the presently disclosed technology incorporates a machine-learning model to recover the high-spatial frequencies in the object to reconstruct an accurate 2D shape and predict its 3D features and category. The technology is disclosed in particular for security applications, but the broader model disclosed is adaptable to different applications, even with limited training samples.

Claims46

Dependent claims

A method for approximating Synthetic Aperture Radar (SAR) imaging on mobile mmWave (mmWave) devices, to enable human-perceptible and machine-readable shape generation and classification of hidden objects on mobile mmWave devices, comprising:

obtaining from a mobile device 3D mmWave shape data for a target object;
and
using a machine-learning model to recover high-spatial frequencies in the object and reconstruct a 2D shape of the target object, wherein recovering the high-spatial frequencies comprises finding a plane that intersects with a 3D voxel of the 3D mmWave shape data having a highest energy, selecting a plurality of neighboring 2D slices parallel to the highest-energy plane towards and away from an aperture plane, and concatenating the selected 2D slices to a 2D deconvolution layer of the machine-learning model.

A system that approximates, on mobile mmWave devices, SAR imaging of full-sized systems, to enable human-perceptible and machine-readable shape generation and classification of hidden objects on mobile mmWave devices, comprising:

a conditional generative adversarial network (cGAN)-based machine-learning system, trained based on inputs of examples of 3D mmWave shapes and based on the corresponding ground truth shapes, to learn the association between 3D mmWave shapes and the corresponding 2D ground truth shapes;
an input to the cGAN-based machine-learning system from a mobile device of 3D mmWave shape data of target objects;
and
a display for producing corresponding human perceptible 2D shapes output from the cGAN-based machine-learning system based on the input thereto, wherein the cGAN-based machine-learning system is configured to process the 3D mmWave shape data by finding a plane that intersects with a 3D voxel of the 3D mmWave shape data having a highest energy, selecting a plurality of neighboring 2D slices parallel to the highest-energy plane towards and away from an aperture plane, and concatenating the selected 2D slices to a 2D deconvolution layer of the cGAN-based machine-learning system.

1 more element

A conditional generative adversarial network (cGAN)-based machine-learning system, comprising one or more processors programmed to use a machine-learning model to recover the high-spatial frequencies in imperceptible 3D mmWave shape data for a target object, and to reconstruct and display an accurate human-perceivable 2D shape for the target object,

wherein the one or more processors are further programmed:
to provide respective generator and discriminator network blocks of the machine-learning model, which collectively generate a full 2D image of the target object based on the 3D mmWave shape data for a target object;
to provide respective quantifier and classifier network blocks of the cGAN-based machine-learning system;
and
for the classifier network block to determine its loss function calculated as:
L
class
(
G
)
=
L
cGAN
+
λ
C
·
L
C
(
G
)
+
λ
B
·
L
B
(
G
)
,
where
L
C
(
G
)
=
-
∑
i
=
1
9
t
i
⁢
log
⁡
(
c
⁡
(
s
i
)
)
⁢
and
⁢
L
B
(
G
)
=
-
(
t
0
⁢
log
⁡
(
p
0
)
+
(
1
-
t
0
)
⁢
log
⁡
(
1
-
p
0
)
)
where c(si) and ti are the predicted and actual probabilities of ith class (categorical output), p0 and t0 are the predicted and actual probabilities of suspicious object (binary output), and the hyper-parameters (λL, λF, λC, λB) represent the networks' focus on shape reconstruction, features prediction, and classification.

93 more elements

Dependencies are inferred from the claim text.

Human-perceptible and machine-readable shape generation and classification of hidden objects

Life of this patent

Human-perceptible and machine-readable shape generation and classification of hidden objects

Life of this patent

Abstract

Claims46