Back to results
US 12,684,095 B2Granted
Systems and methods for detecting traffic violations using mobile detection devices
AssigneeHayden AI Technologies, Inc.
InventorsVaibhav Ghadiok, Christopher Carson, Bo Shen
Life of this patent
Estimating the term…
Abstract
Disclosed are systems and methods for detecting traffic violations using one or more mobile detection devices. Videos captured by one or more mobile detection devices can be processed on the mobile detection devices to extract data and information concerning a potential traffic violation involving a vehicle and a restricted road area. The mobile detection devices can transmit such data and information to a server configured to make a determination as to whether a traffic violation has occurred by comparing the data and information received from the mobile detection devices.
Claims17
One or more non-transitory computer-readable media comprising instructions stored thereon, that when executed by one or more processors, perform steps comprising:
- segmenting frames of one or more videos at a pixel-level, the one or more videos captured by one or more video image sensors, the one or more videos acquired by the one or more video image sensors while the one or more video image sensors are in motion;
- associating a semantic label to each pixel of the frames of the one or more videos using a semantic segmentation function from a computer vision library;
- constructing a virtual three-dimensional (3D) environment representing a real-world environment captured by the one or more video image sensors;
- constructing a 3D semantic annotated map using the virtual 3D environment in combination with two-dimensional maps from public or government mapping databases along with the semantic labels associated with pixels of the frames of the one or more videos, the 3D semantic annotated map thereby being a fusion of mapping data and semantic labels obtained from multiple sources;
- identifying a vehicle and a restricted road area from frames of one or more videos by applying a plurality of functions from the computer vision library to the frames and passing the frames to one or more deep learning models;
- bounding the vehicle in the frames with a vehicular bounding polygon and bounding the restricted road area in the frames with a road bounding polygon;
- verifying the identified restricted road area includes a road area subject to prohibition against occupancy or entry based at least in part on the semantic annotated map;
- and
- responsive to verifying the restricted road area is subject to prohibition against occupancy or entry, detecting that a potential traffic violation has occurred based in part on overlap of the vehicular bounding polygon with the road bounding polygon.
6 more elements
A device for detecting a potential traffic violation, comprising:
- one or more video image sensors configured to capture one or more videos of a vehicle and a restricted road area;
- and
- one or more processors programmed to executed instructions to:
- segment frames of the one or more videos at a pixel-level, the one or more videos acquired by the one or more video image sensors while the one or more video image sensors are in motion;
- associate a semantic label to each pixel of the frames of the one or more videos using a semantic segmentation function from a computer vision library;
- construct a virtual three-dimensional (3D) environment representing a real-world environment captured by the one or more video image sensors;
- construct a 3D semantic annotated map using the virtual 3D environment in combination with two-dimensional maps from public or government mapping databases along with the semantic labels associated with pixels of the frames of the one or more videos, the 3D semantic annotated map thereby being a fusion of mapping data and semantic labels obtained from multiple sources;
- identify the vehicle and the restricted road area from frames of the one or more videos by applying a plurality of functions from the computer vision library to the frames and passing the frames to one or more deep learning models running on the device to identify the vehicle and the restricted road area;
- bound the vehicle in the frames with a plurality of vehicular bounding polygon and bound the restricted road area in the frames with a plurality of road bounding polygon;
- verifying the identified restricted road area includes a road area subject to prohibition against occupancy or entry based at least in part on semantic annotated map;
- and
- responsive to verifying the restricted road area is subject to prohibition against occupancy or entry, detect that a potential traffic violation has occurred based in part on overlap of the plurality of vehicular bounding polygon with the plurality of road bounding polygon.
9 more elements
A method of detecting a traffic violation, comprising:
- segmenting frames of one or more videos at a pixel-level, the one or more videos captured by one or more video image sensors, the one or more videos acquired by the one or more video image sensors while the one or more video image sensors are in motion;
- associating a semantic label to each pixel of the frames of the one or more videos using a semantic segmentation function from a computer vision library;
- constructing a virtual three-dimensional (3D) environment representing a real-world environment captured by the one or more video image sensors;
- constructing a 3D semantic annotated map using the virtual 3D environment in combination with two-dimensional maps from public or government mapping databases along with the semantic labels associated with pixels of the frames of the one or more videos, the 3D semantic annotated map thereby being a fusion of mapping data and semantic labels obtained from multiple sources;
- identifying a vehicle and a restricted road area from frames of one or more videos by applying a plurality of functions from the computer vision library to the frames and passing the frames to one or more deep learning models;
- bounding the vehicle in the frames with a vehicular bounding polygon and bounding the restricted road area in the frames with a road bounding polygon;
- verifying the identified restricted road area includes a road area subject to prohibition against occupancy or entry based at least in part on the semantic annotated map;
- and
- responsive to verifying the restricted road area is subject to prohibition against occupancy or entry, detecting that a potential traffic violation has occurred based in part on overlap of the vehicular bounding polygon with the road bounding polygon.
6 more elements
Dependencies are inferred from the claim text.