# US 12,110,031 B2 · Systems and methods for estimating levelness of a trailer and providing levelling guidance > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,110,031 B2 - Document ID: US12110031B2 - Status: Granted - Kind: B2 - Application number: 17/662,339 - Filed: 2022-05-06 - Published: 2023-11-09 - Granted: 2024-10-08 - Earliest claimed: 2022-05-06 - Assignees: GM Global Technology Operations LLC - Applicants: GM Global Technology Operations LLC - Inventors: Mohamed A. Naiel; Yun Qian Miao; Cristinel Padurariu; Alaa M. Khamis - Web page: https://patentgrep.com/patent/US12110031B2 ## CPC - B60W 40/13: Load or weight - G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle - B60W 40/112: Roll movement - B60W 2300/14: Tractor-trailers, i.e. combinations of a towing vehicle and one or more towed vehicles, e.g. caravans; Road trains - B60W 2420/403: Image sensing, e.g. optical camera - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract Systems and methods for measuring how level a trailer is and providing trailer levelling guidance. The systems and methods include a vehicle comprising a vehicle levelness estimator, a trailer connectable to the vehicle and a rear facing camera mounted on the vehicle. The methods and systems include capturing an image of the trailer using the rear facing camera, receiving a vehicle levelness parameter from the vehicle levelness estimator of the vehicle, determining a rotation angle of the trailer relative to the vehicle based on the image, determining an estimated trailer levelness parameter based on the rotation angle and the vehicle levelness parameter, and outputting the estimated trailer levelness parameter. ## Claims ### Claim 1 A method of measuring how level a trailer is, wherein the trailer is connected to a vehicle, the method comprising: capturing an image of the trailer using a rear facing camera of the vehicle, the image encompassing a front face of the trailer; receiving, via at least one processor, a vehicle levelness parameter from a vehicle levelness estimator of the vehicle, the vehicle levelness parameter indicating how level the vehicle is, based on a digital elevation map and a current location of the vehicle; determining, via the at least one processor, a rotation angle of the trailer relative to the vehicle based on the image and a comparison of the image with a stored reference image that was captured at a known roll angle of the trailer; determining, via the at least one processor, a vision-based estimated trailer levelness parameter based on the rotation angle and the vehicle levelness parameter, the vision-based estimated trailer levelness parameter indicating how level the trailer is using the image from the rear facing camera of the vehicle, the vehicle level parameter from the digital elevation map and the current location of the vehicle, and the stored reference image; receiving, via the at least one processor, a sensor-based trailer levelness parameter based on a levelness measurement from a trailer levelness estimator disposed on the trailer, wherein the levelness estimator comprises an inertial measurement unit (IMU) that is disposed on a surface of the trailer; determining, via the at least one processor, a fused trailer levelness parameter based on a fusion of both (A) the vision-based estimated trailer levelness parameter with (B) the sensor-based trailer levelness parameter, utilizing different estimates at different respective times of both the vision-based estimated trailer levelness parameter and the sensor-based trailer levelness parameter, wherein the fusion of the estimated trailer levelness parameter with the sensor based trailer levelness parameter is weighted based on an inverse of variance that is tracked over time; outputting, via the at least one processor, the estimated trailer levelness parameter; and automatically adjusting a height under one or more wheels of the trailer, in accordance with instructions provided by the at least one processor. ### Claim 2 (depends on 1) The method of claim 1, wherein outputting the estimated trailer levelness parameter includes generating a user interface indicating the estimated trailer levelness parameter. ### Claim 3 (depends on 2) The method of claim 2, wherein the user interface is included on at least one of a native display device of the vehicle and a display device of a mobile computer. ### Claim 4 (depends on 1) The method of claim 1, wherein outputting the estimated trailer levelness parameter includes providing the estimated trailer levelness parameter to an automatic trailer levelling system. ### Claim 5 (depends on 1) The method of claim 1, wherein determining the sensor-based trailer levelness parameter includes regression from a model. ### Claim 6 (depends on 1) The method of claim 1, wherein determining the sensor-based trailer levelness parameter includes determining a height difference of the trailer at each of opposed sides. ### Claim 7 (depends on 6) The method claim 6, wherein determining the sensor-based trailer levelness parameter includes converting the height of the trailer at each of opposed sides to the roll angle. ### Claim 8 (depends on 1) The method of claim 1, wherein the levelness estimator comprises a multi-axis inertial measurement unit (IMU) that is included in a personal mobile computing device that is removably placed on a surface of the trailer. ### Claim 9 (depends on 1) The method of claim 1, further comprising: automatically placing a jack under one or the more wheels of the trailer, in accordance with instructions provided by the at least one processor, based on the fused trailer levelness parameter. ### Claim 10 (depends on 1) The method of claim 1, further comprising: automatically adjusting a height setting of a jack under the one or more wheels of the trailer, in accordance with instructions provided by the at least one processor, based on the fused trailer levelness parameter. ### Claim 11 (depends on 1) The method of claim 1, wherein the fusing is performed via the at least one processor in accordance with the following equation: L t f = median ( { w v w v + w i ⁢ l t _ v + w i w v + w i ⁢ l t _ i | t ¯ = t , t - 1 , … , t - N s } ) , and where: ltv represents an estimated trailer level at time t using the vision-based estimated trailer levelness parameter with variance σv; lti represents an estimated trailer level at time t using the sensor-based estimated trailer levelness parameter with variance σi; and wv and wi: are respective weights to aggregate the multiple estimates. ### Claim 12 (depends on 11) The method of claim 11, wherein the weights wv and wi are inversely proportional to their respective variances. ### Claim 13 (depends on 11) The method of claim 11, wherein the weight wv is inversely proportional to an image quality indicator (IQI) for the vision-based trailer levelness parameter. ### Claim 14 (depends on 11) The method of claim 11, wherein: the weights wv and wi are inversely related to their respective variances; and the weight wv is also inversely related to an IQI for the vision-based trailer levelness parameter. ### Claim 15 A system for measuring how level a trailer is, the system comprising: a vehicle comprising a vehicle levelness estimator; a trailer connectable to the vehicle; a rear facing camera mounted on the vehicle; at least one processor in operable communication with the rear facing camera, wherein the at least one processor is configured to execute program instructions, wherein the program instructions are configured to cause the at least one processor to: capture an image of the trailer using the rear facing camera, the image encompassing a front face of the trailer; receive a vehicle levelness parameter from the vehicle levelness estimator of the vehicle, the vehicle levelness parameter indicating how level the vehicle is, based on a digital elevation map and a current location of the vehicle; determine a rotation angle of the trailer relative to the vehicle based on the image and a comparison of the image with a stored reference image that was captured at a known roll angle of the trailer; determine a vision-based estimated trailer levelness parameter based on the rotation angle and the vehicle levelness parameter, the vision-based estimated trailer levelness parameter indicating how level the trailer is using the image from the rear facing camera of the vehicle, the vehicle level parameter from the digital elevation map and the current location of the vehicle, and the stored reference image; receive a sensor-based trailer levelness parameter based on a levelness measurement from a trailer levelness estimator disposed on the trailer, wherein the levelness estimator comprises an inertial measurement unit (IMU) that is disposed on a surface of the trailer; determine a fused trailer levelness parameter based on a fusion of both (A) the vision-based estimated trailer levelness parameter with (B) the sensor-based trailer levelness parameter, utilizing different estimates at different respective times of both the vision-based estimated trailer levelness parameter and the sensor-based trailer levelness parameter, wherein the fusion of the estimated trailer levelness parameter with the sensor based trailer levelness parameter is weighted based on an inverse of variance that is tracked over time; output the estimated trailer levelness parameter; and automatically adjusting a height under one or more wheels of the trailer, in accordance with instructions provided by the at least one processor. ### Claim 16 (depends on 15) The system of claim 15, wherein the levelness estimator comprises a multi-axis inertial measurement unit (IMU) that is included in a personal mobile computing device that is removably placed on a surface of the trailer. ### Claim 17 (depends on 15) The system of claim 15, wherein the fusing is performed via the at least one processor in accordance with the following equation: L t f = median ( { w v w v + w i ⁢ l t _ v + w i w v + w i ⁢ l t _ i | t ¯ = t , t - 1 , … , t - N s } ) , and wherein: ltv represents an estimated trailer level at time t using the vision-based estimated trailer levelness parameter with variance σv; lti: represents an estimated trailer level at time t using the sensor-based estimated trailer levelness parameter with variance σi; and wv and wi: are respective weights to aggregate the multiple estimates. ### Claim 18 (depends on 17) The system of claim 17, wherein the weights wv and wi are inversely proportional to their respective variances. ### Claim 19 (depends on 17) The system of claim 17, wherein the weight wv is inversely proportional to an image quality indicator (IQI) for the vision-based trailer levelness parameter. ### Claim 20 (depends on 17) The system of claim 17, wherein: the weights wv and wi are inversely related to their respective variances; and the weight wv is also inversely related to an IQI for the vision-based trailer levelness parameter.