# US 12,001,213 B2 · Vehicle and trailer maneuver assist system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,001,213 B2 - Document ID: US12001213B2 - Status: Granted - Kind: B2 - Application number: 17/305,954 - Filed: 2021-07-19 - Published: 2021-11-04 - Granted: 2024-06-04 - Earliest claimed: 2017-08-25 - Assignees: Magna Electronics Inc. - Applicants: Magna Electronics Inc. - Inventors: Jyothi P. Gali; Richard C. Bozich; Nikhil Gupta; Daniel W. Hodgman; Timothy M. Phillips; Clarence P. McElroy - Web page: https://patentgrep.com/patent/US12001213B2 ## CPC - B60R 1/06: mounted on vehicle exterior {(B60R1/025, B60R1/08 take precedence)} - B60R 1/00: Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles - B60R 1/26: to the rear of the vehicle - B60W 10/18: including control of braking systems - B60W 10/20: including control of steering systems - B60W 30/18036: {Reversing} - B60W 50/14: Means for informing the driver, warning the driver or prompting a driver intervention - B62D 13/06: for backing a normally drawn trailer - B62D 15/025: {Active steering aids, e.g. helping the driver by actively influencing the steering system after environment evaluation (B62D1/28 takes precedence)} - B62D 15/0285: {Parking performed automatically} - G01C 21/3623: {using a camera or code reader, e.g. for optical or magnetic codes} - G01C 21/3664: {Details of the user input interface, e.g. buttons, knobs or sliders, including those provided on a touch screen; remote controllers; input using gestures} - G05D 1/00: Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots (drive control systems specially adapted for autonomous road vehicles B60W60/00) - G05D 1/0231: {using optical position detecting means (position-fixing by using electromagnetic waves other than radio waves, e.g. optical position detecting means G01S5/16)} - G05D 1/249: from positioning sensors located off-board the vehicle, e.g. from cameras - G05D 1/617: Safety or protection, e.g. defining protection zones around obstacles or avoiding hazards (arrangements for controlling the position or course of two or more vehicles for avoiding collisions therebetween G05D1/693; arrangements for reacting to or preventing system or operator failure G05D1/80) - B60R 2300/105: using multiple cameras - B60R 2300/20: characterised by the type of display used - B60R 2300/808: for facilitating docking to a trailer - B60W 2050/143: {Alarm means} - B60W 2050/146: {Display means} - B60W 2300/14: Tractor-trailers, i.e. combinations of a towing vehicle and one or more towed vehicles, e.g. caravans; Road trains - B60W 2420/403: Image sensing, e.g. optical camera - B60W 2554/00: Input parameters relating to objects - B60W 2554/4026: Cycles - B60W 2554/4029: Pedestrians - B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - B62D: MOTOR VEHICLES; TRAILERS (steering, or guiding on a desired track, of agricultural machines or implements A01B69/00; wheels, castors, axles, increasing wheel adhesion B60B; vehicle tyres, tyre inflation or tyre changing B60C; connections between vehicles of a train or the like B60D; vehicles for use on rail and road, amphibious or convertible vehicles B60F; suspension arrangements B60G; heating, cooling, ventilating or other air treating devices B60H; windows, windscreens, non-fixed roofs, doors or similar devices, protective coverings for vehicles not in use B60J; propulsion plant arrangements, auxiliary drives, transmissions, controls, instrumentation or dashboards B60K; electric equipment or propulsion of electrically-propelled vehicles B60L; power supply for electrically-propelled vehicles B60M; passenger accommodation not otherwise provided for B60N; adaptations for load transportation or to carry special loads or objects B60P; arrangement of signalling or lighting devices, the mounting or supporting thereof or circuits therefor, for vehicles in general B60Q; vehicles, vehicle fittings or vehicle parts not otherwise provided for B60R; servicing, cleaning, repairing, supporting, lifting, or manoeuvring, not otherwise provided for, B60S; brake arrangements, brake control systems or parts thereof B60T; air-cushion vehicles B60V; motor-cycles, accessories therefor B62J, B62K; testing of vehicles G01M) - G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S) - G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES ## Abstract A vehicular trailer assist system includes a plurality of vehicle cameras disposed at a vehicle, a trailer camera disposed at a rear portion of a trailer and viewing at least rearward of the trailer, and an electronic control unit (ECU) having a processor operable to process image data captured by the cameras. A display screen displays images of a region rearward of the trailer and includes a touch screen. Responsive to a touch input that is representative of a target location, the ECU, via processing of image data captured by the cameras, determines a path of travel for the vehicle and trailer to follow to maneuver the vehicle and trailer toward the target location. The ECU, responsive to an input indicative of an updated target location, updates the path of travel for the vehicle and trailer to follow to maneuver the vehicle and trailer toward the updated target location. ## Claims ### Claim 1 A vehicular trailer assist system, said vehicular trailer assist system comprising: a plurality of vehicle cameras disposed at a vehicle equipped with said vehicular trailer assist system, said plurality of vehicle cameras comprising a rear vehicle camera disposed at a rear portion of the vehicle and viewing at least rearward of the vehicle, a driver side vehicle camera disposed at a driver side of the vehicle and viewing at least sideward of the vehicle, and a passenger side vehicle camera disposed at a passenger side of the vehicle and viewing at least sideward of the vehicle; at least one trailer camera disposed at a rear portion of a trailer and viewing at least rearward of the trailer; an electronic control unit (ECU) comprising a processor operable to process image data captured by said plurality of vehicle cameras; wherein, with the trailer hitched to the vehicle, image data captured by said at least one trailer camera is processed at said ECU; a display screen disposed in the vehicle and viewable by a driver of the vehicle, wherein, with the trailer hitched to the vehicle, said display screen displays images derived from image data captured by at least one camera selected from the group consisting of (i) said at least one trailer camera and (ii) at least one vehicle camera of said plurality of vehicle cameras; wherein said display screen comprises a touch screen; wherein, responsive to a touch input at the touch screen that is representative of a target location rearward of the trailer, said ECU converts pixel coordinates of the touch input at the touch screen into real world coordinates of the target location, and wherein the target location indicates a destination for the trailer; wherein said ECU, with the trailer hitched to the vehicle, and via processing of image data captured by said plurality of vehicle cameras and said at least one trailer camera, and based at least in part on the real world coordinates representative of the touch input, determines a path of travel for the vehicle and trailer to follow to maneuver the vehicle and trailer toward the target location; wherein the determined path of travel is displayed on the display screen overlaying displayed video images derived from image data captured by at least one camera selected from the group consisting of (i) said at least one trailer camera and (ii) at least one vehicle camera of said plurality of vehicle cameras; wherein, responsive to an input indicative of an updated target location, said ECU updates the path of travel for the vehicle and trailer to follow to maneuver the vehicle and trailer toward the updated target location, and wherein the updated target location is at a different location from the target location; and wherein said ECU, responsive to determination of the path of travel for the vehicle and trailer to follow and responsive to the updated path of travel, controls at least steering of the vehicle to maneuver the vehicle and trailer along the updated path of travel toward the updated target location. ### Claim 2 (depends on 1) The vehicular trailer assist system of claim 1, wherein said ECU, with the trailer hitched to the vehicle, and via processing by said processor of image data captured by said plurality of vehicle cameras and said at least one trailer camera, detects at least one object present in the determined path of travel. ### Claim 3 (depends on 2) The vehicular trailer assist system of claim 2, wherein said ECU, responsive to detecting at least one object present in the determined path of travel, adjusts the path of travel to avoid the detected at least one object. ### Claim 4 (depends on 1) The vehicular trailer assist system of claim 1, wherein the input indicative of the updated target location is provided by the driver of the vehicle touching the touch screen. ### Claim 5 (depends on 1) The vehicular trailer assist system of claim 1, wherein the input indicative of the updated target location is provided by the driver of the vehicle moving a cursor at the display screen to the target location present in a displayed image at the display screen. ### Claim 6 (depends on 1) The vehicular trailer assist system of claim 1, wherein the input indicative of the updated target location is provided by a remote transmitter. ### Claim 7 (depends on 1) The vehicular trailer assist system of claim 1, wherein said ECU estimates control parameters at particular intervals along the determined path of travel. ### Claim 8 (depends on 7) The vehicular trailer assist system of claim 7, wherein said ECU updates the estimated control parameters while maneuvering the vehicle and trailer along the determined path of travel and adjusts the determined path of travel accordingly. ### Claim 9 (depends on 1) The vehicular trailer assist system of claim 1, wherein said at least one trailer camera comprises a plurality of trailer cameras disposed at the trailer. ### Claim 10 (depends on 1) The vehicular trailer assist system of claim 1, wherein said plurality of vehicle cameras comprises a forward viewing camera disposed at a front portion of the vehicle and viewing forward of the vehicle. ### Claim 11 (depends on 1) The vehicular trailer assist system of claim 1, wherein the rear vehicle camera comprises a rear backup camera of the vehicle. ### Claim 12 (depends on 1) The vehicular trailer assist system of claim 1, further comprising a remote sensor in wireless communication with said ECU, wherein said remote sensor is remote from the vehicle and the trailer, and wherein said ECU is operable to process data captured by said remote sensor while maneuvering the vehicle and trailer along the determined path of travel. ### Claim 13 (depends on 12) The vehicular trailer assist system of claim 12, wherein the remote sensor comprises a drone sensor disposed at a drone that is operable to maneuver rearward of the trailer. ### Claim 14 A vehicular trailer assist system, said vehicular trailer assist system comprising: a plurality of vehicle cameras disposed at a vehicle equipped with said vehicular trailer assist system, said plurality of vehicle cameras comprising a rear backup camera disposed at a rear portion of the vehicle and viewing at least rearward of the vehicle, a driver side vehicle camera disposed at a driver side of the vehicle and viewing at least sideward of the vehicle, and a passenger side vehicle camera disposed at a passenger side of the vehicle and viewing at least sideward of the vehicle; at least one trailer camera disposed at a rear portion of a trailer and viewing at least rearward of the trailer; an electronic control unit (ECU) comprising a processor operable to process image data captured by said plurality of vehicle cameras; wherein, with the trailer hitched to the vehicle, image data captured by said at least one trailer camera is processed at said ECU; a display screen disposed in the vehicle and viewable by a driver of the vehicle, wherein, with the trailer hitched to the vehicle, said display screen displays images derived from image data captured by at least one camera selected from the group consisting of (i) said at least one trailer camera and (ii) at least one vehicle camera of said plurality of vehicle cameras; wherein said display screen comprises a touch screen; wherein, responsive to a touch input at the touch screen that is representative of a target location rearward of the trailer, said ECU converts pixel coordinates of the touch input at the touch screen into real world coordinates of the target location, and wherein the target location indicates a destination for the trailer; wherein said ECU, with the trailer hitched to the vehicle, and via processing of image data captured by said plurality of vehicle cameras and said at least one trailer camera, and based at least in part on the real world coordinates representative of the touch input, determines a path of travel for the vehicle and trailer to follow to maneuver the vehicle and trailer toward the target location; wherein the determined path of travel is displayed on the display screen overlaying displayed video images derived from image data captured by at least one camera selected from the group consisting of (i) said at least one trailer camera and (ii) at least one vehicle camera of said plurality of vehicle cameras; wherein said ECU, with the trailer hitched to the vehicle, and via processing by said processor of image data captured by said plurality of vehicle cameras and said at least one trailer camera, detects at least one object present in the determined path of travel; wherein said ECU, responsive to detecting at least one object present in the determined path of travel, adjusts the path of travel to avoid the detected at least one object; wherein, responsive to an input indicative of an updated target location, said ECU updates the path of travel for the vehicle and trailer to follow to maneuver the vehicle and trailer toward the updated target location, and wherein the updated target location is at a different location from the target location; wherein said ECU, responsive to determination of the path of travel for the vehicle and trailer to follow and responsive to the updated path of travel, controls at least steering of the vehicle to maneuver the vehicle and trailer along the updated path of travel toward the updated target location; and wherein the input indicative of the updated target location is provided by the driver of the vehicle touching the touch screen. ### Claim 15 (depends on 14) The vehicular trailer assist system of claim 14, wherein the input indicative of the updated target location is provided by the driver of the vehicle moving a cursor at the display screen to the target location present in a displayed image at the display screen. ### Claim 16 (depends on 14) The vehicular trailer assist system of claim 14, wherein said ECU estimates control parameters at particular intervals along the determined path of travel. ### Claim 17 (depends on 16) The vehicular trailer assist system of claim 16, wherein said ECU updates the estimated control parameters while maneuvering the vehicle and trailer along the determined path of travel and adjusts the determined path of travel accordingly. ### Claim 18 A vehicular trailer assist system, said vehicular trailer assist system comprising: a plurality of vehicle cameras disposed at a vehicle equipped with said vehicular trailer assist system, said plurality of vehicle cameras comprising a rear backup camera disposed at a rear portion of the vehicle and viewing at least rearward of the vehicle, a driver side vehicle camera disposed at a driver side of the vehicle and viewing at least sideward of the vehicle, and a passenger side vehicle camera disposed at a passenger side of the vehicle and viewing at least sideward of the vehicle; at least one trailer camera disposed at a rear portion of a trailer and viewing at least rearward of the trailer; an electronic control unit (ECU) comprising a processor operable to process image data captured by said plurality of vehicle cameras; wherein, with the trailer hitched to the vehicle, image data captured by said at least one trailer camera is processed at said ECU; a display screen disposed in the vehicle and viewable by a driver of the vehicle, wherein, with the trailer hitched to the vehicle, said display screen displays images derived from image data captured by at least one camera selected from the group consisting of (i) said at least one trailer camera and (ii) at least one vehicle camera of said plurality of vehicle cameras; wherein said display screen comprises a touch screen; wherein, responsive to a touch input at the touch screen that is representative of a target location rearward of the trailer, said ECU converts pixel coordinates of the touch input at the touch screen into real world coordinates of the target location, and wherein the target location indicates a destination for the trailer; wherein said ECU, with the trailer hitched to the vehicle, and via processing of image data captured by said plurality of vehicle cameras and said at least one trailer camera, and based at least in part on the real world coordinates representative of the touch input, determines a path of travel for the vehicle and trailer to follow to maneuver the vehicle and trailer toward the target location; wherein the determined path of travel is displayed on the display screen overlaying displayed video images derived from image data captured by at least one camera selected from the group consisting of (i) said at least one trailer camera and (ii) at least one vehicle camera of said plurality of vehicle cameras; wherein said ECU estimates control parameters at particular intervals along the determined path of travel; wherein said ECU updates the estimated control parameters while maneuvering the vehicle and trailer along the determined path of travel and adjusts the determined path of travel accordingly; wherein, responsive to an input indicative of an updated target location, said ECU updates the path of travel for the vehicle and trailer to follow to maneuver the vehicle and trailer toward the updated target location, and wherein the updated target location is at a different location from the target location; wherein said ECU, responsive to determination of the path of travel for the vehicle and trailer to follow and responsive to the updated path of travel, controls at least steering of the vehicle to maneuver the vehicle and trailer along the updated path of travel toward the updated target location; and wherein the input indicative of the updated target location is provided by the driver of the vehicle moving a cursor at the display screen to the target location present in a displayed image at the display screen. ### Claim 19 (depends on 18) The vehicular trailer assist system of claim 18, wherein said ECU, with the trailer hitched to the vehicle, and via processing by said processor of image data captured by said plurality of vehicle cameras and said at least one trailer camera, detects at least one object present in the determined path of travel. ### Claim 20 (depends on 19) The vehicular trailer assist system of claim 19, wherein said ECU, responsive to detecting at least one object present in the determined path of travel, adjusts the path of travel to avoid the detected at least one object. ### Claim 21 (depends on 18) The vehicular trailer assist system of claim 18, wherein said at least one trailer camera comprises a plurality of trailer cameras disposed at the trailer.