Back to results
US 11,445,169 B2Granted
Camera image and sensor fusion for mirror replacement system
AssigneeStoneridge Electronics AB
InventorsMilan Gavrilovic, Frans Kievith, Bárbara Martín Leung, Annette Lef, Parinaz Kasebzadeh
Life of this patent
Estimating the term…
Abstract
A measurement system for a vehicle including a first camera defining a field of view having a corresponding optical axis, and a motion detection sensor mechanically fixed to the first camera such that the motion detection sensor is configured to detect motion of the optical axis.
Claims18
A measurement system for a vehicle comprising:
- a first camera defining a field of view having a corresponding optical axis;
- a motion detection sensor mechanically connected to an application specific integrated circuit (ASIC) of the first camera such that motion of the ASIC corresponds to motion of the optical axis in a one to one fashion and the motion detection sensor is configured to detect motion of the optical axis, the ASIC being configured to output a fused data set comprising a combination of image data generated by the first camera and motion data generated by the motion detection sensor;
- a controller in communication with the ASIC, the controller including at least a blockage detection system and being configured to receive a motion sensor output and a camera output, and configured to fuse the motion sensor output and the camera output thereby creating a fused data set, the controller being configured to receive the fused data set, the controller being configured to analyze the fused data set using a machine learning based analysis, and the machine learning based analysis being configured to determine an orientation of the camera and alert an operator in response to the first camera facing an incorrect direction;
- and
- wherein the controller is configured to provide the fused data set to the blockage detection system, and wherein the blockage detection system is configured to compare image based motion detection and sensor based motion detection of the fused data to determine when the camera is at least partially blocked.
2 more elements
A method for detecting a blocked camera lens for a vehicle comprising:
- receiving image data and inertial motion unit (IMU) data from a camera system, the camera system including a motion detection sensor mechanically connected to an application specific integrated circuit (ASIC) of a first camera such that motion of the ASIC corresponds to motion of the optical axis in a one to one fashion and the motion detection sensor is configured to detect motion of the optical axis, the ASIC being configured to output a fused data set comprising a combination of image data generated by the first camera and motion data generated by the motion detection sensor;
- analyzing the IMU data to detect motion of the camera system using a sensor based motion detection system including a controller configured to receive the fused data set;
- analyzing the fused data set to detect motion of the camera using an image based machine learning motion detection analysis, the machine learning motion detection analysis being configured to determine at least an orientation of the camera and alert an operator in response to the first camera facing an incorrect direction;
- and
- determining a presence of an at least partially blocked lens in response to the image based motion detection analysis indicating no motion and the sensor based motion detection indicating motion.
2 more elements
A method for detecting a displaced camera system for a vehicle comprising:
- Receiving, at a controller, fused data including an image portion and an inertial motion unit (IMU) portion from a camera system, the camera system including a motion detection sensor mechanically connected to an application specific integrated circuit (ASIC) of a first camera such that motion of the ASIC corresponds to motion of the optical axis in a one to one fashion and the motion detection sensor is configured to detect motion of the optical axis, and the ASIC being configured to output a fused data set comprising a combination of image data generated by the first camera and motion data generated by the motion detection sensor;
- analyzing the image portion using an image based motion detection system of the controller and determining at least one parameter of motion in at least one object within the image;
- analyzing the IMU portion using a sensor based motion detection system of the controller and determining at least one parameter of motion of the IMU;
- comparing the at least one parameter of motion of the at least one object in the image and the at least one parameter of motion of the IMU, and determining a camera system displacement has occurred when the at least one parameter of motion of the at least one objection differs from the motion of the IMU by at least a predefined amount;
- and
- analyze the fused data set using a machine learning based analysis, the machine learning based analysis being configured to determine an orientation of the camera and alert an operator in response to the first camera facing an incorrect direction.
3 more elements
Dependencies are inferred from the claim text.