# US 2018/0025237 A1 · Vehicular control system with trailering assist function > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2018/0025237 A1 - Document ID: US20180025237A1 - Status: Published application - Kind: A1 - Application number: 15/722,150 - Filed: 2017-10-02 - Published: 2018-01-25 - Earliest claimed: 2012-09-26 - Assignees: Magna Electronics Inc. - Applicants: Magna Electronics Inc. - Inventors: Sebastian Pliefke; Paul Jarmola; Thomas Wierich; Steven V. Byrne; Yuesheng Lu - Web page: https://patentgrep.com/patent/US20180025237A1 ## CPC - G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads - B60K 35/22: Display screens - B60K 35/28: characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver - B60K 35/60: Instruments characterised by their location or relative disposition in or on vehicles (arrangements of lighting devices on dashboards B60Q3/10) - B60K 35/80: Arrangements for controlling instruments - B60R 1/23: with a predetermined field of view - B60R 11/04: Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle - B62D 13/06: for backing a normally drawn trailer - B62D 15/0295: {by overlaying a vehicle path based on present steering angle over an image without processing that image} - G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle - G06V 20/588: {Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road} - H04N 5/272: Means for inserting a foreground image in a background image, i.e. inlay, outlay - H04N 7/183: {for receiving images from a single remote source} - H04N 23/63: by using electronic viewfinders - B60K 35/211: {producing three-dimensional [3D] effects, e.g. stereoscopic images} - B60K 2360/173: Reversing assist - B60K 2360/21: using cameras - B60K 2360/27: using semi-transparent optical elements - B60K 2360/29: Holographic features - B60K 2360/777: on or in sun visors - B60R 1/12: Mirror assemblies combined with other articles, e.g. clocks - B60R 2001/1253: {with cameras, video cameras or video screens} - B60R 2300/105: using multiple cameras - B60R 2300/303: using joined images, e.g. multiple camera images - B60R 2300/307: virtually distinguishing relevant parts of a scene from the background of the scene - B60R 2300/607: from a bird's eye viewpoint - B60R 2300/80: characterised by the intended use of the viewing arrangement - B60R 2300/806: for aiding parking - B60R 2300/808: for facilitating docking to a trailer - B60R 2300/8086: for vehicle path indication - B60Y 2400/92: Driver displays - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - B60K: ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES - B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR - B62D: MOTOR VEHICLES; TRAILERS (steering, or guiding on a desired track, of agricultural machines or implements A01B69/00; wheels, castors, axles, increasing wheel adhesion B60B; vehicle tyres, tyre inflation or tyre changing B60C; connections between vehicles of a train or the like B60D; vehicles for use on rail and road, amphibious or convertible vehicles B60F; suspension arrangements B60G; heating, cooling, ventilating or other air treating devices B60H; windows, windscreens, non-fixed roofs, doors or similar devices, protective coverings for vehicles not in use B60J; propulsion plant arrangements, auxiliary drives, transmissions, controls, instrumentation or dashboards B60K; electric equipment or propulsion of electrically-propelled vehicles B60L; power supply for electrically-propelled vehicles B60M; passenger accommodation not otherwise provided for B60N; adaptations for load transportation or to carry special loads or objects B60P; arrangement of signalling or lighting devices, the mounting or supporting thereof or circuits therefor, for vehicles in general B60Q; vehicles, vehicle fittings or vehicle parts not otherwise provided for B60R; servicing, cleaning, repairing, supporting, lifting, or manoeuvring, not otherwise provided for, B60S; brake arrangements, brake control systems or parts thereof B60T; air-cushion vehicles B60V; motor-cycles, accessories therefor B62J, B62K; testing of vehicles G01M) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - B60Y: INDEXING SCHEME RELATING TO ASPECTS CROSS-CUTTING VEHICLE TECHNOLOGY ## Abstract A vehicular control system includes a camera having an exterior field of view at least rearward of the vehicle and operable to capture image data. A trailer is attached to the vehicle and image data captured by the camera includes image data captured when the vehicle is maneuvered with the trailer at an angle relative to the vehicle. The vehicular control system determines a trailer angle of the trailer and is operable to determine a path of the trailer responsive at least to a steering angle of the vehicle and the determined trailer angle of the trailer. The vehicular control system determines an object present exterior of the vehicle and the vehicular control system distinguishes a drivable surface from a prohibited space, and the vehicular control system plans a driving path for the vehicle that neither impacts the object nor violates the prohibited space. ## Claims ### Claim 1 A vehicular control system, said vehicular control system comprising: a camera disposed at a vehicle and having an exterior field of view at least rearward of the vehicle; wherein said camera is operable to capture image data; wherein a trailer is attached to the vehicle; an image processor operable to process captured image data; wherein image data captured by said camera during maneuvering of the vehicle and the trailer includes image data captured by said camera when the vehicle is maneuvered with the trailer at an angle relative to the vehicle; wherein said vehicular control system, responsive at least in part to image processing by said image processor of image data captured by said camera, determines a trailer angle of the trailer; wherein said vehicular control system is operable to determine a path of the trailer responsive at least to a steering angle of the vehicle and the determined trailer angle of the trailer; wherein said vehicular control system determines an object present exterior of the vehicle which ought not to be impacted during maneuvering of the vehicle and the trailer; wherein said vehicular control system distinguishes a drivable surface from a prohibited space; and wherein, based at least in part on image processing by said image processor of image data captured by said camera during maneuvering of the vehicle and the trailer, said vehicular control system plans a driving path for the vehicle that neither impacts the object nor violates the prohibited space. ### Claim 2 (depends on 1) The vehicular control system of claim 1, wherein, based at least in part on image processing by said image processor of image data captured by said camera during maneuvering of the vehicle and the trailer, said vehicular control system classifies the object. ### Claim 3 (depends on 1) The vehicular control system of claim 1, wherein said vehicular control system determines the object which ought not to be impacted during maneuvering of the vehicle and the trailer based at least in part on mapping information. ### Claim 4 (depends on 1) The vehicular control system of claim 1, wherein said vehicular control system determines the object which ought not be impacted during maneuvering of the vehicle and the trailer based at least in part on image processing by said image processor of image data captured by said camera. ### Claim 5 (depends on 1) The vehicular control system of claim 1, wherein said vehicular control system determines the object which ought not to be impacted during maneuvering of the vehicle and the trailer based at least in part on data wirelessly transmitted to the vehicle. ### Claim 6 (depends on 1) The vehicular control system of claim 1, wherein, based at least in part on image processing by said image processor of image data captured by said camera, said vehicular control system estimates height of the center of mass of the trailer. ### Claim 7 (depends on 6) The vehicular control system of claim 6, wherein said vehicular control system estimates the height of the center of mass of the trailer during maneuvering of the vehicle and the trailer around a curve. ### Claim 8 (depends on 1) The vehicular control system of claim 1, wherein, based at least in part on image processing by said image processor of image data captured by said camera, said vehicular control system estimates the trailer's nicking angle while the vehicle steers around a curve. ### Claim 9 (depends on 1) The vehicular control system of claim 1, wherein, based at least in part on image processing by said image processor of image data captured by said camera during maneuvering of the vehicle and the trailer, said vehicular control system classifies a road surface viewed by said camera. ### Claim 10 (depends on 1) The vehicular control system of claim 1, wherein said vehicular control system displays, at a display screen of the vehicle, images derived from image data captured by said camera, and wherein an overlay overlaid over the displayed images indicates a steering path for the vehicle that maneuvers the trailer around the obstacle. ### Claim 11 (depends on 10) The vehicular control system of claim 10, wherein the steering path for the vehicle is determined at least in part responsive to the steering angle of the vehicle. ### Claim 12 (depends on 1) The vehicular control system of claim 1, wherein said vehicular control system, based at least in part on image processing by said image processor of image data captured by said camera during maneuvering of the vehicle and the trailer, is operable to determine a distance between the trailer turning axis and a nicking point of the trailer by the trailer's nicking behavior while the vehicle and trailer are in motion. ### Claim 13 (depends on 1) The vehicular control system of claim 1, wherein said vehicular control system, based at least in part on image processing by said image processor of image data captured by said camera during maneuvering of the vehicle and the trailer, is operable to determine the trailer's width. ### Claim 14 (depends on 1) The vehicular control system of claim 1, wherein said vehicular control system gathers trailer data pertaining to physical characteristics of the trailer, and wherein said vehicular control system is operable to determine the path of the trailer at least in part responsive to the trailer data gathered. ### Claim 15 (depends on 1) The vehicular control system of claim 1, wherein said vehicular control system is operable to identify a certain trailer and to store properties of that certain trailer in a data file that is accessed when that certain trailer is connected at the vehicle and identified by said vehicular control system. ### Claim 16 (depends on 1) The vehicular control system of claim 1, wherein, when the vehicle and the trailer are maneuvering, and based at least in part on image processing by said image processor of image data captured by said camera during maneuvering of the vehicle and the trailer, said vehicular control system estimates a dimension of the trailer. ### Claim 17 (depends on 16) The vehicular control system of claim 16, wherein the dimension of the trailer comprises width of the trailer. ### Claim 18 (depends on 16) The vehicular control system of claim 16, wherein the dimension of the trailer comprises length of the trailer. ### Claim 19 A vehicular control system, said vehicular control system comprising: a camera disposed at a vehicle and having an exterior field of view at least rearward of the vehicle; wherein said camera is operable to capture image data; wherein a trailer is attached to the vehicle; an image processor operable to process captured image data; wherein image data captured by said camera during maneuvering of the vehicle and the trailer includes image data captured by said camera when the vehicle is maneuvered with the trailer at an angle relative to the vehicle; wherein said vehicular control system, responsive at least in part to image processing by said image processor of image data captured by said camera, determines a trailer angle of the trailer; wherein said vehicular control system is operable to determine a path of the trailer responsive at least to a steering angle of the vehicle and the determined trailer angle of the trailer; wherein said vehicular control system determines an object present exterior of the vehicle which ought not be impacted during maneuvering of the vehicle and the trailer based at least in part on image processing by said image processor of image data captured by said camera; wherein said vehicular control system distinguishes a drivable surface from a prohibited space; wherein, based at least in part on image processing by said image processor of image data captured by said camera during maneuvering of the vehicle and the trailer, said vehicular control system plans a driving path for the vehicle that neither impacts the object nor violates the prohibited space; and wherein said vehicular control system is operable to identify a certain trailer and to store properties of that certain trailer in a data file that is accessed when that certain trailer is connected at the vehicle and identified by said vehicular control system. ### Claim 20 (depends on 19) The vehicular control system of claim 19, wherein, based at least in part on image processing by said image processor of image data captured by said camera during maneuvering of the vehicle and the trailer, said vehicular control system classifies the object. ### Claim 21 (depends on 20) The vehicular control system of claim 20, wherein, when the vehicle and the trailer are maneuvering, and based at least in part on image processing by said image processor of image data captured by said camera during maneuvering of the vehicle and the trailer, said vehicular control system estimates a dimension of the trailer. ### Claim 22 A vehicular control system, said vehicular control system comprising: a camera disposed at a vehicle and having an exterior field of view at least rearward of the vehicle; wherein said camera is operable to capture image data; wherein a trailer is attached to the vehicle; an image processor operable to process image data captured by said camera; wherein image data captured by said camera during maneuvering of the vehicle and the trailer includes image data captured by said camera when the vehicle is maneuvered with the trailer at an angle relative to the vehicle; wherein said vehicular control system, responsive at least in part to image processing by said image processor of image data captured by said camera, determines a trailer angle of the trailer; wherein said vehicular control system is operable to determine a path of the trailer responsive at least to a steering angle of the vehicle and the determined trailer angle of the trailer; wherein said vehicular control system determines an object present exterior of the vehicle which ought not be impacted during maneuvering of the vehicle and the trailer based at least in part on image processing by said image processor of image data captured by said camera; wherein said vehicular control system distinguishes a drivable surface from a prohibited space; wherein, based at least in part on image processing by said image processor of image data captured by said camera during maneuvering of the vehicle and the trailer, said vehicular control system plans a driving path for the vehicle that neither impacts the object nor violates the prohibited space; wherein said vehicular control system is operable to identify a certain trailer and to store properties of that certain trailer in a data file that is accessed when that certain trailer is connected at the vehicle and identified by said vehicular control system; wherein, based at least in part on image processing by said image processor of image data captured by said camera during maneuvering of the vehicle and the trailer, said vehicular control system classifies at least one of (i) the object and (ii) a road surface viewed by said camera; and wherein, when the vehicle and the trailer are maneuvering, and based at least in part on image processing by said image processor of image data captured by said camera during maneuvering of the vehicle and the trailer, said vehicular control system estimates width of the trailer. ### Claim 23 (depends on 22) The vehicular control system of claim 22, wherein, based at least in part on image processing by said image processor of image data captured by said camera, said vehicular control system estimates height of the center of mass of the trailer during maneuvering of the vehicle and the trailer around a curve. ### Claim 24 (depends on 22) The vehicular control system of claim 22, wherein, based at least in part on image processing by said image processor of image data captured by said camera, said vehicular control system estimates the trailer's nicking angle while the vehicle steers around a curve. ### Claim 25 (depends on 22) The vehicular control system of claim 22, wherein, based at least in part on image processing by said image processor of image data captured by said camera during maneuvering of the vehicle and the trailer, said vehicular control system classifies a road surface viewed by said camera.