# US 12,602,827 B2 · Camera monitoring system for vehicles including automatically calibrating camera > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,602,827 B2 - Document ID: US12602827B2 - Status: Granted - Kind: B2 - Application number: 18/079,987 - Filed: 2022-12-13 - Published: 2023-06-29 - Granted: 2026-04-14 - Earliest claimed: 2021-12-23 - Assignees: Stoneridge Electronics AB - Applicants: Stoneridge Electronics AB - Inventors: Utkarsh Sharma; Liang Ma; Nguyen Phan; Troy Otis Cooprider; Banuprakash Murthy - Web page: https://patentgrep.com/patent/US12602827B2 ## CPC - G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration - B60Q 9/00: Arrangement or adaptation of signal devices not provided for in one of main groups B60Q1/00 - B60Q7/00 {, e.g. haptic signalling} - B60R 1/26: to the rear of the vehicle - B60R 1/28: with an adjustable field of view - G06T 7/73: using feature-based methods - H04N 17/002: {for television cameras} - B60R 2300/105: using multiple cameras - B60R 2300/20: characterised by the type of display used - B60R 2300/8046: for replacing a rear-view mirror system - G06T 2207/30244: Camera pose - G06T 2207/30252: Vehicle exterior; Vicinity of vehicle - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - B60Q: ARRANGEMENT OF SIGNALLING OR LIGHTING DEVICES, THE MOUNTING OR SUPPORTING THEREOF OR CIRCUITS THEREFOR, FOR VEHICLES IN GENERAL - B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A camera monitoring system includes a first mirror replacement camera extending outward from a vehicle. The first mirror replacement camera defines a rearward facing field of view including at least one image feature during at least a first set of operating conditions. The at least one image feature has a fixed position within the field of view during the at least the first set of operating conditions. A camera monitoring system controller is configured to automatically calibrate an orientation of the first mirror replacement camera relative to the vehicle by comparing an expected position of the at least one image feature to an actual position of the at least one image feature while the vehicle is operating under the first set of operating conditions and identifying a shift of camera orientation based on the difference. ## Claims ### Claim 1 A camera monitoring system comprising: a first mirror replacement camera extending outward from a vehicle, the first mirror replacement camera defining a rearward facing field of view, wherein the rearward facing field of view includes at least one image feature during at least a first set of operating conditions, and wherein the at least one image feature has a fixed location within the field of view during the at least the first set of operating conditions, wherein the at least one image feature includes lane lines; and a camera monitoring system controller configured to automatically calibrate an orientation of the first mirror replacement camera relative to the vehicle by comparing an expected location of a first vanishing point of the lane lines to an actual location of a second vanishing point of the lane lines while the vehicle is operating under the first set of operating conditions and identifying a shift of camera orientation based on a difference in the first and second vanishing points, wherein the camera monitoring system controller is further configured to respond to the shift of camera orientation by adjusting a positioning of a displayed view when the shift is below a predefined magnitude. ### Claim 2 (depends on 1) The camera monitoring system of claim 1, wherein the first mirror replacement camera is disposed on a commercial vehicle cab, and the at least one image feature includes at least one feature of a trailer connected to the cab. ### Claim 3 (depends on 2) The camera monitoring system of claim 2, wherein the at least one image features includes at least one element from a set including at least a trailer wheel, a trailer end corner, a trailer rear edge, and a trailer top edge. ### Claim 4 (depends on 3) The Camera monitoring system of claim 3, wherein the at least one image feature includes at least two elements from the set. ### Claim 5 (depends on 1) The camera monitoring system of claim 1, wherein the first set of operating conditions includes a speed of the vehicle being above a first threshold and a steering angle of the vehicle being below a second threshold. ### Claim 6 (depends on 5) The camera monitoring system of claim 5, wherein the first set of operating conditions includes a road grade below a third threshold. ### Claim 7 (depends on 1) The camera monitoring system of claim 1, wherein the at least a first set of operating conditions includes a second set of operating conditions. ### Claim 8 (depends on 1) The camera monitoring system of claim 1, wherein adjusting the positioning of the displayed view when the shift is below the predefined magnitude includes adjusting a positioning of a class II view within a class IV view. ### Claim 9 (depends on 1) The camera monitoring system of claim 1, wherein a support structure for the mirror replacement camera includes at least one powered articulating element, the powered articulating element being controlled by the camera monitoring system controller, and wherein the camera monitoring system controller is configured to respond to the shift of camera orientation by articulating the powered articulating element such that the shift of camera orientation is reduced relative to a factory calibrated camera orientation. ### Claim 10 (depends on 1) The camera monitoring system of claim 1, wherein the camera monitoring system controller is configured to output at an alert to a vehicle operator when the shift or camera orientation exceeds a threshold magnitude. ### Claim 11 A method for automatically calibrating an orientation of a vehicle camera comprising: a camera monitoring system (CMS) controller receiving at least one image from a vehicle mounted camera that includes lane lines; comparing an expected location of a first vanishing point of the lane lines to an actual location of a second vanishing point of the lane lines while the vehicle is operating under at least a first set of operating conditions; identifying a shift of camera orientation based on a difference in the first and second vanishing points; and automatically providing an output based upon the difference, wherein the output includes adjusting a positioning of a displayed view when the shift is below a predefined magnitude. ### Claim 12 (depends on 11) The method of claim 11, wherein the vehicle mounted camera is disposed on a vehicle cab, and the at least one image feature includes at least one feature of a trailer connected to the vehicle cab. ### Claim 13 (depends on 12) The method of claim 12, wherein the at least one image feature includes at least one element from a set including at least a trailer wheel, a trailer end corner, a trailer rear edge, and a trailer top edge. ### Claim 14 (depends on 13) The method of claim 13, wherein the at least one image feature includes at least two elements from the set. ### Claim 15 (depends on 11) The method of claim 11, wherein the first set of operating conditions includes a speed of the vehicle being above a first threshold and a steering angle of the vehicle being below a second threshold. ### Claim 16 (depends on 15) The method of claim 15, wherein the first set of operating conditions includes a road grade below a third threshold. ### Claim 17 (depends on 11) The method of claim 11, wherein the at least the first set of operating conditions includes a second set of operating conditions. ### Claim 18 (depends on 11) The method of claim 11, wherein adjusting the positioning of the displayed view when the shift is below a predefined magnitude includes adjusting a positioning of a class II view within a class IV view. ### Claim 19 (depends on 11) The method of claim 11, wherein the output includes articulating a powered articulating element such that the shift of camera orientation is reduced relative to a factory calibrated camera orientation. ### Claim 20 (depends on 11) The method of claim 11, wherein the output includes providing an alert to a vehicle operator when the shift or camera orientation exceeds a threshold magnitude.