# US 10,870,427 B2 · Vehicular control system with remote processor > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,870,427 B2 - Document ID: US10870427B2 - Status: Granted - Kind: B2 - Application number: 16/199,487 - Filed: 2018-11-26 - Published: 2019-04-11 - Granted: 2020-12-22 - Earliest claimed: 2013-12-05 - Assignees: Magna Electronics Inc. - Applicants: Magna Electronics Inc. - Inventors: Krishna Koravadi - Web page: https://patentgrep.com/patent/US10870427B2 ## CPC - B60W 30/09: Taking automatic action to avoid collision, e.g. braking and steering - B60T 7/12: for automatic initiation; for initiation not subject to will of driver or passenger {(limiting speed of vehicles other than rail vehicles B60K31/00)} - B60T 7/18: operated by wayside apparatus - B62D 15/029: {Steering assistants using warnings or proposing actions to the driver without influencing the steering system} - G08G 1/164: {Centralised systems, e.g. external to vehicles} - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - B60T: VEHICLE BRAKE CONTROL SYSTEMS OR PARTS THEREOF; BRAKE CONTROL SYSTEMS OR PARTS THEREOF, IN GENERAL (electrodynamic brake systems for vehicle, in general B60L; brakes per se, i.e. devices where braking effect occurs, including ultimate brake actuators, F16D); ARRANGEMENT OF BRAKING ELEMENTS ON VEHICLES IN GENERAL; PORTABLE DEVICES FOR PREVENTING UNWANTED MOVEMENT OF VEHICLES; VEHICLE MODIFICATIONS TO FACILITATE COOLING OF BRAKES - B62D: MOTOR VEHICLES; TRAILERS (steering, or guiding on a desired track, of agricultural machines or implements A01B69/00; wheels, castors, axles, increasing wheel adhesion B60B; vehicle tyres, tyre inflation or tyre changing B60C; connections between vehicles of a train or the like B60D; vehicles for use on rail and road, amphibious or convertible vehicles B60F; suspension arrangements B60G; heating, cooling, ventilating or other air treating devices B60H; windows, windscreens, non-fixed roofs, doors or similar devices, protective coverings for vehicles not in use B60J; propulsion plant arrangements, auxiliary drives, transmissions, controls, instrumentation or dashboards B60K; electric equipment or propulsion of electrically-propelled vehicles B60L; power supply for electrically-propelled vehicles B60M; passenger accommodation not otherwise provided for B60N; adaptations for load transportation or to carry special loads or objects B60P; arrangement of signalling or lighting devices, the mounting or supporting thereof or circuits therefor, for vehicles in general B60Q; vehicles, vehicle fittings or vehicle parts not otherwise provided for B60R; servicing, cleaning, repairing, supporting, lifting, or manoeuvring, not otherwise provided for, B60S; brake arrangements, brake control systems or parts thereof B60T; air-cushion vehicles B60V; motor-cycles, accessories therefor B62J, B62K; testing of vehicles G01M) - G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93}) ## Abstract A vehicular control system includes a camera, a non-vision sensor and a control having at least one data processor. The camera is disposed at a vehicle and has a field of view at least forward of the vehicle. The non-vision sensor is disposed at the vehicle and has a field of sensing at least forward of the vehicle. The control, responsive at least in part to processing at the control of captured image data captured sensor data, determines a fault of the camera or of the non-vision sensor. Responsive to determination of the fault of the camera or of the non-vision sensor, the control wirelessly communicates an alert to a remote processor that is located remote from the vehicle and that is not part of the vehicle. Responsive to receipt of the communicated alert, the remote processor at least in part assumes control of the vehicle. ## Claims ### Claim 1 A vehicular control system, said vehicular control system comprising: a camera operable to capture image data; a non-vision sensor operable to capture sensor data; a control having at least one data processor; wherein image data captured by said camera and sensor data captured by said non-vision sensor are processed at said control; wherein said camera is disposed at a vehicle equipped with said control and wherein said camera has a field of view at least forward of the equipped vehicle; wherein said non-vision sensor is disposed at the equipped vehicle and wherein said non-vision sensor has a field of sensing at least forward of the equipped vehicle; wherein image data captured by said camera and sensor data captured by said non-vision sensor are provided to said control; wherein said control, responsive at least in part to processing at said control of image data captured by said camera and processing at said control of sensor data captured by said non-vision sensor, determines a fault of said camera or of said non-vision sensor; wherein, responsive to determination of the fault of said camera or of said non-vision sensor, said control wirelessly communicates an alert to a remote processor that is located remote from the equipped vehicle; wherein said remote processor is not part of the equipped vehicle; and wherein, responsive to receipt of the communicated alert, said remote processor at least in part assumes control of the equipped vehicle. ### Claim 2 (depends on 1) The vehicular control system of claim 1, wherein said camera comprises a forward viewing camera that is one of a plurality of cameras disposed at the equipped vehicle, and wherein said non-vision sensor comprises a forward sensing non-vision sensor that is one of a plurality of non-vision sensors disposed at the equipped vehicle. ### Claim 3 (depends on 2) The vehicular control system of claim 2, wherein said plurality of cameras comprises (i) at least one sideward viewing camera and (ii) at least one rearward viewing camera. ### Claim 4 (depends on 1) The vehicular control system of claim 1, wherein said non-vision sensor comprises a radar sensor. ### Claim 5 (depends on 4) The vehicular control system of claim 4, wherein said radar sensor comprises a forward sensing radar sensor that is one of a plurality of radar sensors disposed at the equipped vehicle. ### Claim 6 (depends on 1) The vehicular control system of claim 1, wherein said non-vision sensor comprises a LIDAR sensor. ### Claim 7 (depends on 6) The vehicular control system of claim 6, wherein said LIDAR sensor comprises a forward sensing LIDAR sensor that is one of a plurality of LIDAR sensors disposed at the equipped vehicle. ### Claim 8 (depends on 1) The vehicular control system of claim 1, wherein image data captured by said camera and provided to said control and sensor data captured by said non-vision sensor and provided to said control are processed at said control to detect and classify objects present external of the equipped vehicle. ### Claim 9 (depends on 1) The vehicular control system of claim 1, wherein the equipped vehicle is equipped with a GPS-enabled e-Horizon. ### Claim 10 (depends on 9) The vehicular control system of claim 9, wherein the GPS-enabled e-Horizon of the equipped vehicle provides to the equipped vehicle environment data related to one selected from the group consisting of (i) a speed limit, (ii) an exit ramp location, (iii) an entry ramp location, (iv) road curvature information and (v) traffic lanes. ### Claim 11 (depends on 1) The vehicular control system of claim 1, wherein said control wirelessly communicates to said remote processor vehicle data indicative of operation of the equipped vehicle, and wherein said control wirelessly communicates to said remote processor environment data indicative of the environment in which the equipped vehicle is traveling. ### Claim 12 (depends on 11) The vehicular control system of claim 11, wherein, responsive to vehicle data and environment data wirelessly received at said remote processor from the equipped vehicle, said remote processor determines if the equipped vehicle is at or approaching a hazardous condition. ### Claim 13 (depends on 12) The vehicular control system of claim 12, wherein, responsive to determination that the equipped vehicle is at or approaching a potentially hazardous condition, said remote processor controls a vehicle system of the equipped vehicle to mitigate the determined potentially hazardous condition. ### Claim 14 (depends on 13) The vehicular control system of claim 13, wherein the vehicle system comprises at least one system selected from the group consisting of (i) a brake system of the equipped vehicle, (ii) a steering system of the equipped vehicle, (iii) a torque control system of the equipped vehicle and (iv) a collision avoidance system of the equipped vehicle. ### Claim 15 (depends on 11) The vehicular control system of claim 11, wherein the communicated environment data comprises data pertaining to at least one selected from the group consisting of (i) map information, (ii) traffic information and (iii) weather condition information. ### Claim 16 (depends on 11) The vehicular control system of claim 11, wherein the communicated vehicle data comprises data pertaining to at least one selected from the group consisting of (i) longitudinal velocity of the equipped vehicle, (ii) lateral velocity of the equipped vehicle, (iii) longitudinal acceleration of the equipped vehicle, (iv) lateral acceleration of the equipped vehicle and (v) actuation of a turn signal indicator of the equipped vehicle. ### Claim 17 (depends on 1) The vehicular control system of claim 1, wherein said control, responsive at least in part to processing at said control of image data captured by said camera and processing at said control of sensor data captured by said non-vision sensor, autonomously controls the equipped vehicle. ### Claim 18 (depends on 1) The vehicular control system of claim 1, wherein said remote processor wirelessly receives communications from multiple other vehicles, and wherein said remote processor is not part of any vehicle of the multiple other vehicles. ### Claim 19 (depends on 18) The vehicular control system of claim 18, wherein said remote processor wirelessly receives, from the multiple other vehicles, respective vehicle data indicative of operation of the respective one of the multiple other vehicles and respective environment data indicative of the environment in which the respective one of the multiple other vehicles is operating. ### Claim 20 (depends on 19) The vehicular control system of claim 19, wherein each of the multiple other vehicles includes a vehicle-to-infrastructure communication system that is operable to communicate data to said remote processor. ### Claim 21 A vehicular control system, said vehicular control system comprising: a camera operable to capture image data; a radar sensor operable to capture sensor data; a control having at least one data processor; wherein image data captured by said camera and sensor data captured by said radar sensor are processed at said control; wherein said camera is disposed at a vehicle equipped with said control and wherein said camera has a field of view at least forward of the equipped vehicle; wherein said radar sensor is disposed at the equipped vehicle and wherein said radar sensor has a field of sensing at least forward of the equipped vehicle; wherein image data captured by said camera and sensor data captured by said radar sensor are provided to said control; wherein said control, responsive at least in part to processing at said control of image data captured by said camera and processing at said control of sensor data captured by said radar sensor, determines a fault of said camera or of said radar sensor; wherein, responsive to determination of the fault of said camera or of said radar sensor, said control wirelessly communicates an alert to a remote processor that is located remote from the equipped vehicle; wherein said remote processor is not part of the equipped vehicle; wherein, responsive to receipt of the communicated alert, said remote processor at least in part assumes control of the equipped vehicle; wherein said control wirelessly communicates to said remote processor vehicle data indicative of operation of the equipped vehicle, and wherein said control wirelessly communicates to said remote processor environment data indicative of the environment in which the equipped vehicle is traveling; wherein, responsive to vehicle data and environment data wirelessly received at said remote processor from the equipped vehicle, said remote processor determines if the equipped vehicle is at or approaching a hazardous condition; and wherein, responsive to determination that the equipped vehicle is at or approaching a potentially hazardous condition, said remote processor controls a vehicle system of the equipped vehicle to mitigate the determined potentially hazardous condition. ### Claim 22 (depends on 21) The vehicular control system of claim 21, wherein said camera comprises a forward viewing camera that is one of a plurality of cameras disposed at the equipped vehicle, and wherein said radar sensor comprises a forward sensing radar sensor that is one of a plurality of radar sensors disposed at the equipped vehicle. ### Claim 23 (depends on 21) The vehicular control system of claim 21, wherein the vehicle system comprises at least one system selected from the group consisting of (i) a brake system of the equipped vehicle, (ii) a steering system of the equipped vehicle, (iii) a torque control system of the equipped vehicle and (iv) a collision avoidance system of the equipped vehicle. ### Claim 24 (depends on 21) The vehicular control system of claim 21, wherein the communicated environment data comprises data pertaining to at least one selected from the group consisting of (i) map information, (ii) traffic information and (iii) weather condition information. ### Claim 25 (depends on 21) The vehicular control system of claim 21, wherein the communicated vehicle data comprises data pertaining to at least one selected from the group consisting of (i) longitudinal velocity of the equipped vehicle, (ii) lateral velocity of the equipped vehicle, (iii) longitudinal acceleration of the equipped vehicle, (iv) lateral acceleration of the equipped vehicle and (v) actuation of a turn signal indicator of the equipped vehicle. ### Claim 26 (depends on 21) The vehicular control system of claim 21, wherein said remote processor wirelessly receives communications from multiple other vehicles, and wherein said remote processor is not part of any vehicle of the multiple other vehicles. ### Claim 27 (depends on 26) The vehicular control system of claim 26, wherein said remote processor wirelessly receives, from the multiple other vehicles, respective vehicle data indicative of operation of the respective one of the multiple other vehicles and respective environment data indicative of the environment in which the respective one of the multiple other vehicles is operating. ### Claim 28 (depends on 27) The vehicular control system of claim 27, wherein each of the multiple other vehicles includes a vehicle-to-infrastructure communication system that is operable to communicate data to said remote processor. ### Claim 29 (depends on 21) The vehicular control system of claim 21, wherein said control, responsive at least in part to processing at said control of image data captured by said camera and processing at said control of sensor data captured by said radar sensor, autonomously controls the equipped vehicle. ### Claim 30 A vehicular control system, said vehicular control system comprising: a camera operable to capture image data; a LIDAR sensor operable to capture sensor data; a control having at least one data processor; wherein image data captured by said camera and sensor data captured by said LIDAR sensor are processed at said control; wherein said camera is disposed at a vehicle equipped with said control and wherein said camera has a field of view at least forward of the equipped vehicle; wherein said LIDAR sensor is disposed at the equipped vehicle and wherein said LIDAR sensor has a field of sensing at least forward of the equipped vehicle; wherein image data captured by said camera and sensor data captured by said LIDAR sensor are provided to said control; wherein said control, responsive at least in part to processing at said control of image data captured by said camera and processing at said control of sensor data captured by said LIDAR sensor, determines a fault of said camera or of said LIDAR sensor; wherein, responsive to determination of the fault of said camera or of said LIDAR sensor, said control wirelessly communicates an alert to a remote processor that is located remote from the equipped vehicle; wherein said remote processor is not part of the equipped vehicle; wherein, responsive to receipt of the communicated alert, said remote processor at least in part assumes control of the equipped vehicle; wherein said control wirelessly communicates to said remote processor vehicle data indicative of operation of the equipped vehicle, and wherein said control wirelessly communicates to said remote processor environment data indicative of the environment in which the equipped vehicle is traveling; wherein, responsive to vehicle data and environment data wirelessly received at said remote processor from the equipped vehicle, said remote processor determines if the equipped vehicle is at or approaching a hazardous condition; and wherein, responsive to determination that the equipped vehicle is at or approaching a potentially hazardous condition, said remote processor controls a vehicle system of the equipped vehicle to mitigate the determined potentially hazardous condition. ### Claim 31 (depends on 30) The vehicular control system of claim 30, wherein said camera comprises a forward viewing camera that is one of a plurality of cameras disposed at the equipped vehicle, and wherein said LIDAR sensor comprises a forward sensing LIDAR sensor that is one of a plurality of LIDAR sensors disposed at the equipped vehicle. ### Claim 32 (depends on 30) The vehicular control system of claim 30, wherein the vehicle system comprises at least one system selected from the group consisting of (i) a brake system of the equipped vehicle, (ii) a steering system of the equipped vehicle, (iii) a torque control system of the equipped vehicle and (iv) a collision avoidance system of the equipped vehicle. ### Claim 33 (depends on 30) The vehicular control system of claim 30, wherein the communicated environment data comprises data pertaining to at least one selected from the group consisting of (i) map information, (ii) traffic information and (iii) weather condition information. ### Claim 34 (depends on 30) The vehicular control system of claim 30, wherein the communicated vehicle data comprises data pertaining to at least one selected from the group consisting of (i) longitudinal velocity of the equipped vehicle, (ii) lateral velocity of the equipped vehicle, (iii) vehicle longitudinal acceleration of the equipped vehicle, (iv) lateral acceleration and (v) actuation of a turn signal indicator of the equipped vehicle. ### Claim 35 (depends on 30) The vehicular control system of claim 30, wherein said control, responsive at least in part to processing at said control of image data captured by said camera and processing at said control of sensor data captured by said LIDAR sensor, autonomously controls the equipped vehicle. ### Claim 36 (depends on 30) The vehicular control system of claim 30, wherein said remote processor wirelessly receives communications from multiple other vehicles, and wherein said remote processor is not part of any vehicle of the multiple other vehicles. ### Claim 37 (depends on 36) The vehicular control system of claim 36, wherein said remote processor wirelessly receives, from the multiple other vehicles, respective vehicle data indicative of operation of the respective one of the multiple other vehicles and respective environment data indicative of the environment in which the respective one of the multiple other vehicles is operating. ### Claim 38 (depends on 37) The vehicular control system of claim 37, wherein each of the multiple other vehicles includes a vehicle-to-infrastructure communication system that is operable to communicate data to said remote processor.