# US 12,183,052 B2 · Vehicular trailering assist system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,183,052 B2 - Document ID: US12183052B2 - Status: Granted - Kind: B2 - Application number: 18/391,938 - Filed: 2023-12-21 - Published: 2024-05-09 - Granted: 2024-12-31 - Earliest claimed: 2020-07-24 - Assignees: Magna Electronics Inc. - Applicants: Magna Electronics Inc. - Inventors: Jyothi P. Gali; Mohammadmehdi Jalalmaab; Prasanna Ananthakrishnan; Harold E. Joseph; Alexander Velichko - Web page: https://patentgrep.com/patent/US12183052B2 ## CPC - G06V 10/44: Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components - B60D 1/06: Ball-and-socket hitches - B60D 1/245: {for facilitating push back or parking of trailers} - B60D 1/62: involving supply lines, electric circuits or the like - B60R 1/003: {for viewing trailer hitches} - B62D 13/00: Steering specially adapted for trailers (combined traction and steering hitches B60D) - G06T 7/74: {involving reference images or patches} - G06V 10/751: {Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching} - G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle - G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads - H04N 7/18: Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast - H04N 7/183: {for receiving images from a single remote source} - B62D 15/027: {Parking aids, e.g. instruction means} - G06T 2207/30252: Vehicle exterior; Vicinity of vehicle - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - B60D: VEHICLE CONNECTIONS - B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR - B62D: MOTOR VEHICLES; TRAILERS (steering, or guiding on a desired track, of agricultural machines or implements A01B69/00; wheels, castors, axles, increasing wheel adhesion B60B; vehicle tyres, tyre inflation or tyre changing B60C; connections between vehicles of a train or the like B60D; vehicles for use on rail and road, amphibious or convertible vehicles B60F; suspension arrangements B60G; heating, cooling, ventilating or other air treating devices B60H; windows, windscreens, non-fixed roofs, doors or similar devices, protective coverings for vehicles not in use B60J; propulsion plant arrangements, auxiliary drives, transmissions, controls, instrumentation or dashboards B60K; electric equipment or propulsion of electrically-propelled vehicles B60L; power supply for electrically-propelled vehicles B60M; passenger accommodation not otherwise provided for B60N; adaptations for load transportation or to carry special loads or objects B60P; arrangement of signalling or lighting devices, the mounting or supporting thereof or circuits therefor, for vehicles in general B60Q; vehicles, vehicle fittings or vehicle parts not otherwise provided for B60R; servicing, cleaning, repairing, supporting, lifting, or manoeuvring, not otherwise provided for, B60S; brake arrangements, brake control systems or parts thereof B60T; air-cushion vehicles B60V; motor-cycles, accessories therefor B62J, B62K; testing of vehicles G01M) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A vehicular trailer assist system includes a camera disposed at a vehicle and viewing at least a portion of a trailer hitched to the vehicle. The system, during a first turning maneuver of a calibration maneuver of the vehicle and trailer, stores a subset of the frames of image data captured by the camera. Each stored frame of the stored subset of the frames captures the trailer at a respective different trailer angle relative to a longitudinal axis of the vehicle. The system determines image data edges for each stored frame of image data. The system, after performing the first turning maneuver of the calibration maneuver and during a second turning maneuver of the calibration maneuver, and based at least in part on the determined image data edges for the stored frames of image data, determines a jack-knife angle of the trailer relative to the longitudinal axis of the vehicle. ## Claims ### Claim 1 A vehicular trailering assist system, the vehicular trailering assist system comprising: a camera disposed at a rear portion of a vehicle equipped with the vehicular trailering assist system, the camera viewing at least rearward of the vehicle, wherein, with a trailer hitched to the vehicle, the camera views at least a portion of the trailer hitched to the vehicle; wherein the camera is operable to capture frames of image data that include image data representative of at least a portion of the trailer hitched to the vehicle; an electronic control unit (ECU) comprising electronic circuitry and associated software; wherein the electronic circuitry comprises an image processor operable to process frames of image data captured by the camera as the vehicle and trailer travel along a road; wherein frames of image data captured by the camera are transferred to and are processed at the ECU; wherein the vehicular trailering assist system, during a first turning maneuver of a calibration maneuver of the vehicle and trailer, and at least in part via processing at the ECU of frames of image data captured by the camera during the first turning maneuver of the calibration maneuver, stores a subset of the frames of image data captured by the camera, and wherein each stored frame of image data of the stored subset of the frames of image data captured by the camera captures the trailer at a respective different trailer angle relative to a longitudinal axis of the vehicle; wherein the vehicular trailering assist system determines image data edges for each stored frame of image data; and wherein the vehicular trailering assist system, after performing the first turning maneuver of the calibration maneuver and during a second turning maneuver of the calibration maneuver, and based at least in part on the determined image data edges for the stored frames of image data, determines a jack-knife angle of the trailer relative to the longitudinal axis of the vehicle. ### Claim 2 (depends on 1) The vehicular trailering assist system of claim 1, wherein, during an angle sampling portion of the calibration maneuver, the vehicular trailering assist system determines a current trailer angle based on a kinematic model. ### Claim 3 (depends on 2) The vehicular trailering assist system of claim 2, wherein the angle sampling portion of the calibration maneuver completes after the vehicle completes a threshold portion of a turn of the calibration maneuver. ### Claim 4 (depends on 1) The vehicular trailering assist system of claim 1, wherein the stored subset of frames of image data comprises a plurality of frames of image data, and wherein the stored subset of frames of image data comprises less than ten frames of image data. ### Claim 5 (depends on 1) The vehicular trailering assist system of claim 1, wherein the vehicular trailering assist system, responsive to processing at the ECU of frames of image data captured by the camera during the calibration maneuver, determines an initial trailer template of the trailer hitched to the vehicle, and wherein the vehicular trailering assist system, responsive to determining image data edges for each stored frame of image data, matches each determined image data edge to the determined initial trailer template. ### Claim 6 (depends on 5) The vehicular trailering assist system of claim 5, wherein each stored frame of image data is cropped to encompass only a region of interest, and wherein the vehicular trailering assist system matches the region of interest to the determined initial trailer template. ### Claim 7 (depends on 5) The vehicular trailering assist system of claim 5, wherein the vehicular trailering assist system, responsive to matching each determined image data edge to the determined initial trailer template, determines a location of a hitch ball of the vehicle. ### Claim 8 (depends on 7) The vehicular trailering assist system of claim 7, wherein the vehicular trailering assist system, while determining the location of the hitch ball of the vehicle, provides an indication to an operator of the vehicle. ### Claim 9 (depends on 5) The vehicular trailering assist system of claim 5, wherein the vehicular trailering assist system, using the image data edges, generates an edge image for each stored frame of image data, and wherein the vehicular trailering assist system matches each generated edge image to the determined initial trailer template by determining a plurality of scores for each generated edge image and selecting a highest score for each generated edge image. ### Claim 10 (depends on 9) The vehicular trailering assist system of claim 9, wherein the vehicular trailering assist system determines the plurality of scores for each generated edge image by sliding each corresponding generated edge image along the determined initial trailer template. ### Claim 11 (depends on 9) The vehicular trailering assist system of claim 9, wherein the score for each generated edge image is associated with an amount of matching between the generated edge image and the determined initial trailer template. ### Claim 12 (depends on 9) The vehicular trailering assist system of claim 9, wherein each generated edge image comprises a warped region of interest. ### Claim 13 (depends on 1) The vehicular trailering assist system of claim 1, wherein, during the second turning maneuver of the calibration maneuver, the trailer angle of the trailer relative to the longitudinal axis of the vehicle remains constant. ### Claim 14 A vehicular trailering assist system, the vehicular trailering assist system comprising: a camera disposed at a rear portion of a vehicle equipped with the vehicular trailering assist system, the camera viewing at least rearward of the vehicle, wherein, with a trailer hitched to the vehicle, the camera views at least a portion of the trailer hitched to the vehicle; wherein the camera is operable to capture frames of image data that include image data representative of at least a portion of the trailer hitched to the vehicle; an electronic control unit (ECU) comprising electronic circuitry and associated software; wherein the electronic circuitry comprises an image processor operable to process frames of image data captured by the camera as the vehicle and trailer travel along a road; wherein frames of image data captured by the camera are transferred to and are processed at the ECU; wherein, during an angle sampling portion of a calibration maneuver of the vehicle and trailer, the vehicular trailering assist system determines a current trailer angle based on a kinematic model; wherein the vehicular trailering assist system, during a first turning maneuver of the calibration maneuver, and at least in part via processing at the ECU of frames of image data captured by the camera during the first turning maneuver of the calibration maneuver, stores a subset of the frames of image data captured by the camera, and wherein each stored frame of image data of the stored subset of the frames of image data captured by the camera captures the trailer at a respective different trailer angle relative to a longitudinal axis of the vehicle; wherein the stored subset of frames of image data comprises a plurality of frames of image data, and wherein the stored subset of frames of image data comprises less than ten frames of image data; wherein the vehicular trailering assist system determines image data edges for each stored frame of image data; and wherein the vehicular trailering assist system, after performing the first turning maneuver of the calibration maneuver and during a second turning maneuver of the calibration maneuver, and based at least in part on the determined image data edges for the stored frames of image data, determines a jack-knife angle of the trailer relative to the longitudinal axis of the vehicle. ### Claim 15 (depends on 14) The vehicular trailering assist system of claim 14, wherein the angle sampling portion of the calibration maneuver completes after the vehicle completes a threshold portion of a turn of the calibration maneuver. ### Claim 16 (depends on 14) The vehicular trailering assist system of claim 14, wherein the vehicular trailering assist system, responsive to processing at the ECU of frames of image data captured by the camera during the calibration maneuver, determines an initial trailer template of the trailer hitched to the vehicle, and wherein the vehicular trailering assist system, responsive to determining image data edges for each stored frame of image data, matches each determined image data edge to the determined initial trailer template. ### Claim 17 (depends on 16) The vehicular trailering assist system of claim 16, wherein each stored frame of image data is cropped to encompass only a region of interest, and wherein the vehicular trailering assist system matches the region of interest to the determined initial trailer template. ### Claim 18 (depends on 16) The vehicular trailering assist system of claim 16, wherein the vehicular trailering assist system, responsive to matching each determined image data edge to the determined initial trailer template, determines a location of a hitch ball of the vehicle. ### Claim 19 A vehicular trailering assist system, the vehicular trailering assist system comprising: a camera disposed at a rear portion of a vehicle equipped with the vehicular trailering assist system, the camera viewing at least rearward of the vehicle, wherein, with a trailer hitched to the vehicle, the camera views at least a portion of the trailer hitched to the vehicle; wherein the camera is operable to capture frames of image data that include image data representative of at least a portion of the trailer hitched to the vehicle; an electronic control unit (ECU) comprising electronic circuitry and associated software; wherein the electronic circuitry comprises an image processor operable to process frames of image data captured by the camera as the vehicle and trailer travel along a road; wherein frames of image data captured by the camera are transferred to and are processed at the ECU; wherein the vehicular trailering assist system, during a first turning maneuver of a calibration maneuver of the vehicle and trailer, and at least in part via processing at the ECU of frames of image data captured by the camera during the first turning maneuver of the calibration maneuver, stores a subset of the frames of image data captured by the camera, and wherein each stored frame of image data of the stored subset of the frames of image data captured by the camera captures the trailer at a respective different trailer angle relative to a longitudinal axis of the vehicle; wherein the stored subset of frames of image data comprises a plurality of frames of image data, and wherein the stored subset of frames of image data comprises less than ten frames of image data; wherein the vehicular trailering assist system determines image data edges for each stored frame of image data; wherein the vehicular trailering assist system, after performing the first turning maneuver of the calibration maneuver and during a second turning maneuver of the calibration maneuver, and based at least in part on the determined image data edges for the stored frames of image data, determines a jack-knife angle of the trailer relative to the longitudinal axis of the vehicle; and wherein, during the second turning maneuver of the calibration maneuver, the trailer angle of the trailer relative to the longitudinal axis of the vehicle remains constant. ### Claim 20 (depends on 19) The vehicular trailering assist system of claim 19, wherein, during an angle sampling portion of the calibration maneuver, the vehicular trailering assist system determines a current trailer angle based on a kinematic model. ### Claim 21 (depends on 20) The vehicular trailering assist system of claim 20, wherein the angle sampling portion of the calibration maneuver completes after the vehicle completes a threshold portion of a turn of the calibration maneuver. ### Claim 22 (depends on 19) The vehicular trailering assist system of claim 19, wherein the vehicular trailering assist system, responsive to processing at the ECU of frames of image data captured by the camera during the calibration maneuver, determines an initial trailer template of the trailer hitched to the vehicle, and wherein the vehicular trailering assist system, responsive to determining image data edges for each stored frame of image data, matches each determined image data edge to the determined initial trailer template.