# US 10,455,226 B2 · Systems and methods for image capture device calibration for a materials handling vehicle > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,455,226 B2 - Document ID: US10455226B2 - Status: Granted - Kind: B2 - Application number: 15/158,893 - Filed: 2016-05-19 - Published: 2016-12-01 - Granted: 2019-10-22 - Earliest claimed: 2015-05-26 - Assignees: Crown Equipment Corporation - Applicants: Crown Equipment Corporation - Inventors: Jacob Jay Thomson; Timothy William Fanselow - Web page: https://patentgrep.com/patent/US10455226B2 ## CPC - G06T 7/74: {involving reference images or patches} - B60R 11/04: Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle - G01C 21/206: {specially adapted for indoor navigation} - G06V 30/422: Technical drawings; Geographical maps - H04N 17/002: {for television cameras} - G06T 2207/30244: Camera pose - G06T 2207/30252: Vehicle exterior; Vicinity of vehicle - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR - G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Systems and methods for calibrating an image capture device for a materials handling vehicle. One embodiment of a system includes the image capture device and a vehicle computing device, where the vehicle computing device stores logic that when executed by a processor, causes the materials handling vehicle to determine a current location of the materials handling vehicle in a warehouse and determine a seed value associated with the image capture device, the seed value representing initial calibration parameters of the image capture device. In some embodiments, the logic causes the materials handling vehicle to capture image data in the warehouse, compare the image data with a site map, and determine a calibrated value for the image capture device from the comparison and the seed value. ## Claims ### Claim 1 A system comprising a materials handling vehicle, the materials handling vehicle comprising an image capture device and a vehicle computing device, wherein: the vehicle computing device stores logic that when executed by a processor, causes the materials handling vehicle to perform at least the following: determine a current location of the materials handling vehicle in a warehouse; determine a seed value associated with the image capture device, the seed value representing initial calibration parameters of the image capture device; capture image data in the warehouse, the image data including a location identifier that is representative of a location in the warehouse; compare the image data with a site map, the site map representing imagery and location data of a ceiling in the warehouse; determine a calibrated value for the image capture device from the comparison and the seed value; update the seed value based on the calibrated value; and apply the updated seed value for determining a new location of the materials handling vehicle. ### Claim 2 (depends on 1) The system of claim 1, wherein the logic further causes the system to perform at least the following: compute a calibration confidence related to accuracy of the calibrated value for the image capture device, the calibration confidence being representative an accuracy of the calibrated value based on a comparison of the image data with the site map; determine whether the calibration confidence meets a predetermined threshold; provide a visual indication of the calibration confidence to a user; and in response to determining that the calibration confidence does not meet the predetermined threshold, provide an instruction to restart calibration. ### Claim 3 (depends on 2) The system of claim 2, wherein the logic further causes the system to report the calibration confidence to the user. ### Claim 4 (depends on 1) The system of claim 1, wherein the seed value includes data related to at least one of the following: pitch of the image capture device, roll of the image capture device, and yaw of the image capture device. ### Claim 5 (depends on 1) The system of claim 1, wherein the current location of the materials handling vehicle is received from at least one of the following: a user input and triggering of a sensor. ### Claim 6 (depends on 1) The system of claim 1, further comprising a remote computing device, wherein the logic further causes the vehicle computing device to communicate with the remote computing device to construct an optimization problem. ### Claim 7 (depends on 6) The system of claim 6, wherein the remote computing device utilizes a smoothing and mapping (SAM) library for constructing the optimization problem. ### Claim 8 (depends on 7) The system of claim 7, wherein the remote computing device utilizes simultaneous localization and mapping (SLAM) to construct the optimization problem. ### Claim 9 (depends on 7) The system of claim 7, wherein logic further causes the vehicle computing device to communicate at least one of the following to the remote computing device: the image data, data related to the site map, and data related to the image capture device. ### Claim 10 (depends on 7) The system of claim 7, wherein determining the calibrated value for the image capture device from the comparison includes receiving data from the remote computing device. ### Claim 11 (depends on 1) The system of claim 1, wherein the logic causes the materials handling vehicle to traverse a predetermined route to reach a destination. ### Claim 12 (depends on 1) The system of claim 1, wherein comparing the image data with the site map comprises identifying a segment of the site map that applies to the image data. ### Claim 13 (depends on 1) The system of claim 1, wherein the image data includes an image of a light fixture. ### Claim 14 A materials handling vehicle comprising an image capture device and a vehicle computing device, wherein: the vehicle computing device stores logic that when executed by a processor, causes the materials handling vehicle to perform at least the following: determine a current location of the materials handling vehicle; determine a seed value associated with the image capture device, the seed value representing initial calibration parameters of the image capture device; capture image data in a warehouse, the image data including a location identifier that is representative of a location in the warehouse; communicate with a remote computing device to compare the image data with a site map and create an optimization problem, the optimization problem being created via a communication with a mapping library, the site map representing imagery and location data of a ceiling in the warehouse; determine a calibrated value for the image capture device from an output of the optimization problem; update the seed value based on the calibrated value; and apply the updated seed value for determining a new location of the materials handling vehicle. ### Claim 15 (depends on 14) The materials handling vehicle of claim 14, wherein the logic further causes the materials handling vehicle to perform at least the following: compute a calibration confidence related to accuracy of the calibrated value for the image capture device, wherein the calibration confidence is representative an accuracy of the calibrated value based on a comparison of the image data with the site map; determine whether the calibration confidence meets a predetermined threshold; and in response to determining that the calibration confidence does not meet the predetermined threshold, provide an instruction to restart calibration. ### Claim 16 (depends on 14) The materials handling vehicle of claim 14, wherein the seed value includes data related to at least one of the following: pitch of the image capture device, roll of the image capture device, and yaw of the image capture device. ### Claim 17 (depends on 14) The materials handling vehicle of claim 14, wherein the remote computing device utilizes a smoothing and mapping (SAM) library for constructing the optimization problem. ### Claim 18 (depends on 14) The materials handling vehicle of claim 14, wherein logic further causes the vehicle computing device to communicate at least one of the following to the remote computing device: the image data, data related to the site map, and data related to the image capture device. ### Claim 19 A method for calibrating an image capture device for a materials handling vehicle comprising: determining a current location of the materials handling vehicle; estimating a seed value associated with the image capture device, wherein the seed value includes data related to at least one of the following: pitch of the image capture device, roll of the image capture device, and yaw of the image capture device; capturing image data in an environment, wherein the image data includes a location identifier that is representative of a location in a warehouse; communicating with a remote computing device to compare the image data with a site map to create an optimization problem, wherein the remote computing device utilizes a smoothing and mapping (SAM) library for creating the optimization problem; determining a calibrated value for the image capture device from the comparison and the seed value; computing a calibration confidence related to accuracy of the calibrated value for the image capture device, wherein the calibration confidence is representative an accuracy of the calibrated value based on a comparison of the image data with the site map; determining whether the calibration confidence meets a predetermined threshold; in response to determining that the calibration confidence does not meet the predetermined threshold, providing an instruction to restart calibration; and in response to determining that the calibration confidence does meet the predetermined threshold, applying the calibrated value for determining a new location of the materials handling vehicle.