# US 2018/0025213 A1 · Camera system for traffic enforcement > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2018/0025213 A1 - Document ID: US20180025213A1 - Status: Published application - Kind: A1 - Application number: 15/637,368 - Filed: 2017-06-29 - Published: 2018-01-25 - Earliest claimed: 2016-07-22 - Assignees: NEC Laboratories America, Inc.; NEC Hong Kong Limited - Applicants: NEC Laboratories America, Inc.; NEC Hong Kong Limited - Inventors: Manmohan Chandraker; Xiang Yu; Eric Lau; Elsa Wong - Web page: https://patentgrep.com/patent/US20180025213A1 ## CPC - G06F 21/32: using biometric data, e.g. fingerprints, iris scans or voiceprints - G06F 18/24143: {Distances to neighbourhood prototypes, e.g. restricted Coulomb energy networks [RCEN]} - G06F 21/6218: {to a system of files or objects, e.g. local or distributed file system or database} - G06N 3/045: Combinations of networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/084: Backpropagation, e.g. using gradient descent - G06N 3/09: Supervised learning - G06N 20/00: Machine learning - G06V 10/82: using neural networks - G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle - G06V 30/19173: {Classification techniques} - G06V 40/16: Human faces, e.g. facial parts, sketches or expressions - G06V 40/161: {Detection; Localisation; Normalisation} - G06V 40/166: {using acquisition arrangements} - G06V 40/171: {Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships} - G06V 40/172: {Classification, e.g. identification} - G06V 40/45: {Detection of the body part being alive} - G07C 9/37: using biometric data, e.g. fingerprints, iris scans or voice recognition - G07C 9/38: with central registration - H04L 63/0861: {using biometrical features, e.g. fingerprint, retina-scan (cryptographic mechanisms or cryptographic arrangements for entity authentication using biological data H04L9/3231)} - H04L 63/1483: {service impersonation, e.g. phishing, pharming or web spoofing (detection of rogue wireless access points H04W12/12)} - G06F 2221/2133: Verifying human interaction, e.g., Captcha - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G07C: TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract A traffic enforcement system and corresponding method are provided. The traffic enforcement system includes a camera configured to capture an input image of one or more subjects in a motor vehicle. The traffic enforcement system further includes a memory storing a deep learning model configured to perform multi-task learning for a pair of tasks including a liveness detection task and a face recognition task on one or more subjects in a motor vehicle depicted in the input image. The traffic enforcement system also includes a processor configured to apply the deep learning model to the input image to recognize an identity the one or more subjects in the motor vehicle and a liveness of the one or more subjects. The liveness detection task is configured to evaluate a plurality of different distractor modalities corresponding to different physical spoofing materials to prevent face spoofing for the face recognition task. ## Claims ### Claim 1 A traffic enforcement system, comprising: a camera configured to capture an input image of one or more subjects in a motor vehicle; a memory storing a deep learning model configured to perform multi-task learning for a pair of tasks including a liveness detection task and a face recognition task on one or more subjects in a motor vehicle depicted in the input image; and a processor configured to apply the deep learning model to the input image to recognize an identity the one or more subjects in the motor vehicle and a liveness of the one or more subjects, and wherein the liveness detection task is configured to evaluate a plurality of different distractor modalities corresponding to different physical spoofing materials to prevent face spoofing for the face recognition task. ### Claim 2 (depends on 1) The traffic enforcement system of claim 1, wherein the processor is further configured to forward the input image of the one or more subjects in the motor vehicle to a corresponding traffic enforcement authority, responsive to the motor vehicle committing a traffic violation. ### Claim 3 (depends on 1) The traffic enforcement system of claim 1, wherein the processor is further configured to forward the input image, that depicts a particular one of the one or more subjects that is located in an operating location in the motor vehicle, to a corresponding traffic enforcement authority, responsive to a liveness determination of another one of the one or more subjects in the motor vehicle being determined as not live and a location of the motor vehicle being in a high occupancy vehicle lane. ### Claim 4 (depends on 1) The traffic enforcement system of claim 1, wherein the processor is further configured to forward the input image, that depicts a particular one of the one or more subjects that is located in an operating location in the motor vehicle and another one of the one or more subjects located in a non-operating position in the motor vehicle, to a corresponding traffic enforcement authority, responsive to a liveness determination of the other one of the one or more subjects being determined as not live and a location of the motor vehicle being in a high occupancy vehicle lane. ### Claim 5 (depends on 1) The traffic enforcement system of claim 1, wherein the different physical spoofing materials comprise a spoofing materials selected from the group consisting of a 3D mask, a printed image, and an electronic display. ### Claim 6 (depends on 1) The traffic enforcement system of claim 1, wherein the processor is further configured to perform data preprocessing on the plurality of input images selected from the group consisting of image contrast enhancements, data augmentation, and cropping. ### Claim 7 (depends on 1) The traffic enforcement system of claim 1, wherein the processor is further configured to apply an image contrast enhancement to at least one of the plurality of input images by randomly adding one or more Gaussian distributed instances to an original RGB value of the at least one of the plurality of images. ### Claim 8 (depends on 1) The traffic enforcement system of claim 1, wherein the processor is further configured to apply an image contrast enhancement to at least one of the plurality of input images by adding one or more Gaussian distributed instances to a Hue, Saturation, and Value (HSV) domain and multiplying two or more coefficients in the HSV domain to further adjust the Saturation and the Value of the at least one of the plurality of input images. ### Claim 9 (depends on 1) The traffic enforcement system of claim 1, wherein the processor is further configured to perform random central cropping on at least some of the plurality of input images. ### Claim 10 (depends on 1) The traffic enforcement system of claim 1, wherein the processor is further configured to perform a data augmentation process on at least one of the plurality of input images to augment data in the images and increase a generality of the deep learning model to the different distractor modalities. ### Claim 11 (depends on 1) The traffic enforcement system of claim 1, wherein the deep learning model is formed to include a plurality of layers, and the processor is further configured to train the deep learning model by freezing each of the plurality of layers except a last fully connected layer therein, and performing multi-task training by combining the liveness detection task and the face recognition task, wherein during the multi-task training, all weights of the plurality of layers are unfrozen. ### Claim 12 (depends on 1) The traffic enforcement system of claim 1, wherein the deep learning model is formed using an inception structure configured to multiply an output of a convolution layer of the deep learning model with at least one of the plurality of input images to produce a new image having amplified extracted features relative to the at least one of the plurality of input images. ### Claim 13 (depends on 1) The traffic enforcement system of claim 1, wherein the deep learning model is formed using a loss structure that includes a face liveness penalty and a face recognition penalty. ### Claim 14 A computer-implemented method for traffic enforcement, the method comprising: capturing, by a camera, an input image of one or more subjects in a motor vehicle; storing, in a memory, a deep learning model configured to perform multi-task learning for a pair of tasks including a liveness detection task and a face recognition task on one or more subjects in a motor vehicle depicted in the input image; and applying, by a processor, the deep learning model to the input image to recognize an identity the one or more subjects in the motor vehicle and a liveness of the one or more subjects, and wherein the liveness detection task is configured to evaluate a plurality of different distractor modalities corresponding to different physical spoofing materials to prevent face spoofing for the face recognition task. ### Claim 15 (depends on 14) The computer-implemented method of claim 14, further comprising forwarding the input image of the one or more subjects in the motor vehicle to a corresponding traffic enforcement authority, responsive to the motor vehicle committing a traffic violation. ### Claim 16 (depends on 14) The computer-implemented method of claim 14, further comprising forwarding the input image, that depicts a particular one of the one or more subjects that is located in an operating location in the motor vehicle, to a corresponding traffic enforcement authority, responsive to a liveness determination of another one of the one or more subjects in the motor vehicle being determined as not live and a location of the motor vehicle being in a high occupancy vehicle lane. ### Claim 17 (depends on 14) The computer-implemented method of claim 14, further comprising forwarding the input image, that depicts a particular one of the one or more subjects that is located in an operating location in the motor vehicle and another one of the one or more subjects located in a non-operating position in the motor vehicle, to a corresponding traffic enforcement authority, responsive to a liveness determination of the other one of the one or more subjects being determined as not live and a location of the motor vehicle being in a high occupancy vehicle lane. ### Claim 18 (depends on 14) The computer-implemented method of claim 14, further comprising performing data preprocessing on the plurality of input images selected from the group consisting of image contrast enhancements, data augmentation, and cropping. ### Claim 19 (depends on 14) The computer-implemented method of claim 14, wherein the deep learning model is formed to include a plurality of layers, and the method further comprises training the deep learning model by: freezing each of the plurality of layers except a last fully connected layer therein; and performing multi-task training by combining the liveness detection task and the face recognition task, wherein during the multi-task training, all weights of the plurality of layers are unfrozen. ### Claim 20 A computer program product for traffic enforcement, the computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform a method comprising: capturing, by a camera, an input image of one or more subjects in a motor vehicle; storing, in a memory, a deep learning model configured to perform multi-task learning for a pair of tasks including a liveness detection task and a face recognition task on one or more subjects in a motor vehicle depicted in the input image; and applying, by a processor, the deep learning model to the input image to recognize an identity the one or more subjects in the motor vehicle and a liveness of the one or more subjects, and wherein the liveness detection task is configured to evaluate a plurality of different distractor modalities corresponding to different physical spoofing materials to prevent face spoofing for the face recognition task.