# US 2019/0244377 A1 · Image processing method, electronic device, and non-transitory computer readable storage medium > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0244377 A1 - Document ID: US20190244377A1 - Status: Published application - Kind: A1 - Application number: 15/889,220 - Filed: 2018-02-06 - Published: 2019-08-08 - Earliest claimed: 2018-02-06 - Assignees: HTC Corporation - Applicants: HTC Corporation - Inventors: Wen-Hsiang YU - Web page: https://patentgrep.com/patent/US20190244377A1 ## CPC - G06T 7/55: from multiple images - H04N 1/32128: {attached to the image data, e.g. file header, transmitted message header, information on the same page or in the same computer file as the image (for information embedded in the image H04N1/32144)} - H04N 7/183: {for receiving images from a single remote source} - H04N 23/6812: {based on additional sensors, e.g. acceleration sensors} - H04N 23/687: {by shifting the lens or sensor position} - G06T 2207/10004: Still image; Photographic image - H04N 2201/3215: of a time or duration - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract An image processing method includes: obtaining, a first image by a camera, and a first lens position by a position sensor at a first timestamp; obtaining, by an inertial measurement unit sensor, an IMU signal indicating a movement of the camera between the first timestamp and a second timestamp; obtaining, a second image by the camera, and a second lens position by the position sensor at the second timestamp; calculating, by a processing circuit, a relative geometry value of the first image and the second image according to the first lens position, the second lens position, and the IMU signal; and performing, by the processing circuit, a depth calculation to the first image and the second image based on the relative geometry value on the condition that the relative geometry value meets a geometry threshold. ## Claims ### Claim 1 An image processing method comprising: obtaining, a first image by a camera, and a first lens position by a position sensor at a first timestamp; obtaining, by an inertial measurement unit sensor, an IMU signal indicating a movement of the camera between the first timestamp and a second timestamp; obtaining, a second image by the camera, and a second lens position by the position sensor at the second timestamp; calculating, by a processing circuit, a relative geometry value of the first image and the second image according to the first lens position, the second lens position, and the IMU signal; and performing, by the processing circuit, a depth calculation to the first image and the second image based on the relative geometry value on the condition that the relative geometry value meets a geometry threshold. ### Claim 2 (depends on 1) The image processing method of claim 1, wherein calculating the relative geometry value comprises: calculating a baseline distance between the camera at the first timestamp and at the second timestamp, wherein the geometry threshold is met on the condition that the baseline distance is greater than a predetermined value. ### Claim 3 (depends on 1) The image processing method of claim 1, further comprising: moving, by an actuator connected to a lens of the camera, the lens to obtain the first image and the second image with disparity on the condition that the IMU signal is lower than a predetermined threshold. ### Claim 4 (depends on 3) The image processing method of claim 3, wherein a speed of the actuator is lower than a shutter speed of the camera. ### Claim 5 (depends on 3) The image processing method of claim 3, wherein the lens is configured to move in a horizontal direction perpendicular to an optical axis of the lens. ### Claim 6 (depends on 1) The image processing method of claim 1, further comprising: enabling an optical image stabilization to the camera during the first timestamp to the second timestamp. ### Claim 7 (depends on 1) The image processing method of claim 1, further comprising: obtaining, a third image by a second camera, and a third lens position by the position sensor at the second timestamp; and performing, by the processing circuit, the depth calculation to the second image and the third image based on a fixed baseline between the camera and the second camera. ### Claim 8 An electronic device, comprising: a processing circuit; a camera electrically connected to the processing circuit; a position sensor electrically connected to the processing circuit; an inertial measurement unit sensor electrically connected to the processing circuit; a memory electrically connected to the processing circuit; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processing circuit, the one or more programs comprising instructions for: controlling the camera to obtain a first image and controlling the position sensor to obtain a first lens position at a first timestamp; controlling the inertial measurement unit sensor to obtain an IMU signal indicating a movement of the camera between the first timestamp and a second timestamp; controlling the camera to obtain a second image and controlling the position sensor to obtain a second lens position at the second timestamp; calculating a relative geometry value of the first image and the second image according to the first lens position, the second lens position, and the IMU signal; and performing a depth calculation to the first image and the second image based on the relative geometry value on the condition that the relative geometry value meets a geometry threshold. ### Claim 9 (depends on 8) The electronic device as claimed in claim 8, wherein the one or more programs further comprise instructions for: calculating a baseline distance between the camera at the first timestamp and at the second timestamp, wherein the geometry threshold is met on the condition that the baseline distance is greater than a predetermined value. ### Claim 10 (depends on 8) The electronic device as claimed in claim 8, further comprising: an actuator connected to a lens of the camera; and wherein the one or more programs further comprise instructions for: controlling the actuator to move the lens to obtain the first image and the second image with disparity on the condition that the IMU signal is lower than a predetermined threshold. ### Claim 11 (depends on 10) The electronic device as claimed in claim 10, wherein a speed of the actuator is lower than a shutter speed of the camera. ### Claim 12 (depends on 10) The electronic device as claimed in claim 10, wherein the lens is configured to move in a horizontal direction perpendicular to an optical axis of the lens. ### Claim 13 (depends on 8) The electronic device as claimed in claim 8, wherein the one or more programs further comprise instructions for: enabling an optical image stabilization to the camera during the first timestamp to the second timestamp. ### Claim 14 (depends on 8) The electronic device as claimed in claim 8, further comprising: a second camera electrically connected to the processing circuit; and wherein the one or more programs further comprise instructions for: controlling the second camera to obtain a third image and controlling the position sensor to obtain a third lens position at the second timestamp; and performing the depth calculation to the second image and the third image based on a fixed baseline between the camera and the second camera. ### Claim 15 A non-transitory computer readable storage medium storing one or more programs, comprising instructions, which when executed, causes a processing circuit to perform operations comprising: controlling a camera to obtain a first image and controlling a position sensor to obtain a first lens position at a first timestamp; controlling an inertial measurement unit sensor to obtain an IMU signal indicating a movement of the camera between the first timestamp and a second timestamp; controlling the camera to obtain a second image and controlling the position sensor to obtain a second lens position at the second timestamp; calculating a relative geometry value of the first image and the second image according to the first lens position, the second lens position, and the IMU signal; and performing a depth calculation to the first image and the second image based on the relative geometry value on the condition that the relative geometry value meets a geometry threshold. ### Claim 16 (depends on 15) The non-transitory computer readable storage medium as claimed in claim 15, further comprising instructions, which when executed, causes the processing circuit to further perform operations comprising: calculating a baseline distance between the camera at the first timestamp and at the second timestamp, wherein the geometry threshold is met on the condition that the baseline distance is greater than a predetermined value. ### Claim 17 (depends on 15) The non-transitory computer readable storage medium as claimed in claim 15, further comprising instructions, which when executed, causes the processing circuit to further perform operations comprising: controlling an actuator connected to a lens of the camera to move the lens to obtain the first image and the second image with disparity on the condition that the IMU signal is lower than a predetermined threshold. ### Claim 18 (depends on 17) The non-transitory computer readable storage medium as claimed in claim 17, further comprising instructions, which when executed, causes the processing circuit to further perform operations comprising: controlling the actuator to move the lens in a horizontal direction perpendicular to an optical axis of the lens with a speed lower than a shutter speed of the camera. ### Claim 19 (depends on 15) The non-transitory computer readable storage medium as claimed in claim 15, further comprising instructions, which when executed, causes the processing circuit to further perform operations comprising: enabling an optical image stabilization to the camera during the first timestamp to the second timestamp. ### Claim 20 (depends on 15) The non-transitory computer readable storage medium as claimed in claim 15, further comprising instructions, which when executed, causes the processing circuit to further perform operations comprising: controlling a second camera to obtain a third image and controlling the position sensor to obtain a third lens position at the second timestamp; and performing the depth calculation to the second image and the third image based on a fixed baseline between the camera and the second camera.