# US 11,999,299 B2 · Camera mirror system display camera calibration > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,999,299 B2 - Document ID: US11999299B2 - Status: Granted - Kind: B2 - Application number: 18/268,750 - Filed: 2021-12-16 - Published: 2024-02-08 - Granted: 2024-06-04 - Earliest claimed: 2020-12-23 - Assignees: Stoneridge Electronics AB - Applicants: Stoneridge Electronics AB - Inventors: John S. Germaine; Dominik Marx - Web page: https://patentgrep.com/patent/US11999299B2 ## CPC - B60R 1/22: for viewing an area outside the vehicle, e.g. the exterior of the vehicle - B60R 1/29: for viewing an area inside the vehicle, e.g. for viewing passengers or cargo - G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) - G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80) - G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration - B60R 2300/105: using multiple cameras - B60R 2300/402: Image calibration - B60R 2300/8006: for monitoring and displaying scenes of vehicle interior, e.g. for monitoring passengers or cargo - G06T 2207/10016: Video; Image sequence - G06T 2207/20084: Artificial neural networks [ANN] - G06T 2207/30208: Marker matrix - G06T 2207/30252: Vehicle exterior; Vicinity of vehicle - G06T 2207/30268: Vehicle interior - B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A camera mirror system for a vehicle includes, among other things, first and second exterior cameras that are configured to provide capture images outside of the vehicle that correspond to legally prescribed views. The system further includes first and second displays within a vehicle cabin that are configured to depict the captured images respectively from the first and second exterior cameras. The system further includes first and second interior cameras that are configured to capture driver images within the vehicle cabin. The first and second interior cameras respectively have first and second interior fields of view that overlap one another in a vehicle driver region within the vehicle cabin, the first display is in the second interior field of view, and the second display is in the first interior field of view. The system further includes a controller that is in communication with the first and second displays and the first and second interior cameras. The controller is configured to monitor a driver activity in the vehicle driver region using the first and second interior cameras. The controller is configured to initiate a calibration procedure in response to a calibration event. The calibration procedure includes first and second displays that respectively depict first and second patterns, and the first and second interior cameras detect respectively the second and first patterns to calibrate the first and second cameras in relation to the vehicle driver region to establish a distance between each of the first and second interior cameras and the second and first patterns respectively. ## Claims ### Claim 1 A camera mirror system for a vehicle, comprising: first and second exterior cameras configured to provide capture images outside of the vehicle respectively corresponding to first and second fields of views; first and second displays within a vehicle cabin configured to depict the captured images respectively from the first and second exterior cameras; first and second interior cameras configured to capture driver images within the vehicle cabin, the first and second interior cameras respectively having first and second interior fields of view that overlap one another in a vehicle driver region within the vehicle cabin, the first display is in the second interior field of view, and the second display is in the first interior field of view; and a controller in communication with the first and second displays and the first and second interior cameras, the controller is configured to monitor a driver activity in the vehicle driver region using the first and second interior cameras, the controller is configured to initiate a calibration procedure in response to a calibration event, the calibration procedure including first and second displays respectively depicting first and second patterns, and the first and second interior cameras detecting respectively the second and first patterns to calibrate the first and second interior cameras in relation to the vehicle driver region to establish a distance between each of the first and second interior cameras and the second and first patterns respectively. ### Claim 2 (depends on 1) The system of claim 1, wherein the first and second displays are arranged respectively at left and right side A-pillars within the vehicle cabin. ### Claim 3 (depends on 1) The system of claim 1, wherein the first and second interior cameras are provided respectively on the first and second displays and configured to be movable respectively therewith, the first and second interior cameras having operational positions that are adjustable relative to one another. ### Claim 4 (depends on 1) The system of claim 1, wherein the controller is configured to determine a distance from each of the first and second interior cameras to a driver feature to evaluate the driver activity. ### Claim 5 (depends on 4) The system of claim 4, wherein the driver activity includes at least one of driver gaze and drive pose. ### Claim 6 (depends on 1) The system of claim 1, wherein the calibration event corresponds to departure of at least one of the first and second interior cameras from a baseline operational position. ### Claim 7 (depends on 6) The system of claim 6, wherein the calibration event corresponds to departure of at least one of the first and second displays from the baseline operational position. ### Claim 8 (depends on 1) The system of claim 1, wherein the calibration event is configured to occur within a calibration window. ### Claim 9 (depends on 8) The system of claim 8, wherein the calibration window is during a vehicle startup procedure. ### Claim 10 (depends on 8) The system of claim 8, wherein the calibration window is below a threshold vehicle speed. ### Claim 11 (depends on 1) The system of claim 1, wherein at least one of the first and second patterns includes multiple lines. ### Claim 12 (depends on 11) The system of claim 11, wherein the at least one of the first and second patterns is black and white checkerboard pattern. ### Claim 13 A method of calibrating an interior camera used for driver observation within a vehicle cabin, wherein the method is configured to be used with a camera mirror system having first and second exterior cameras configured to capture images outside of the vehicle respectively corresponding to first and second fields of views, and first and second displays within the vehicle cabin are configured to depict the captured images respectively from the first and second exterior cameras, the method comprising the steps of: providing the interior camera movable between operational positions; detecting a calibration event that includes the interior camera being moved between the operational positions; displaying a pattern on at least one of the first and second displays subsequent to the calibration event detection; detecting the pattern with the interior camera; and calibrating the interior camera with the pattern to establish a distance between the interior camera and the pattern. ### Claim 14 (depends on 13) The method of claim 13, wherein the interior camera is one of a first interior camera and a second interior camera, and the first interior camera has the second display in its field of view, and the second interior camera has the first display in its field of view. ### Claim 15 (depends on 14) The method of claim 14, wherein the first and second interior cameras are provided respectively on the first and second displays and configured to be movable respectively therewith. ### Claim 16 (depends on 14) The method of claim 14, wherein another distance relative to a driver feature can be determined based upon the distance, the driver feature relates to a driver activity. ### Claim 17 (depends on 16) The method of claim 16, wherein the driver activity includes at least one of driver gaze and drive pose. ### Claim 18 (depends on 13) The method of claim 13, wherein the calibration event occurs during at least one of a vehicle startup procedure and below a threshold vehicle speed. ### Claim 19 (depends on 14) The method of claim 14, wherein the pattern is one of a first pattern and a second pattern, wherein the displaying step includes displaying the first pattern on the first display and the second pattern on the second display, the detecting step includes detecting the first and second patterns with the respective other of the first and second interior cameras, and the calibrating step includes calibrating each of the other of the first and second interior cameras with the respective other of the first and second patterns to establish the distance between each of the other of the first and second interior cameras and the respective other of the first and second patterns.