# US 2018/0322648 A1 · Methods and systems for binocular stereo vision > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2018/0322648 A1 - Document ID: US20180322648A1 - Status: Published application - Kind: A1 - Application number: 15/775,800 - Filed: 2016-11-09 - Published: 2018-11-08 - Earliest claimed: 2015-11-11 - Assignees: Zhejiang Dahua Technology Co., Ltd. - Applicants: Zhejiang Dahua Technology Co., Ltd. - Inventors: Wei LU; Qiankun LI; Shizhu PAN; Miao CHENG; Xiangming ZHOU - Web page: https://patentgrep.com/patent/US20180322648A1 ## CPC - G06T 7/593: from stereo images - G06F 18/2411: based on the proximity to a decision surface, e.g. support vector machines - G06T 7/85: {Stereo camera calibration} - G06T 2207/10021: Stereoscopic video; Stereoscopic image sequence - G06T 2207/20081: Training; Learning - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A system and method for binocular stereo vision is disclosed. The method includes acquiring a pair of images including a first image and a second image, the pair of images being captured by one or more cameras; obtaining a training model; determining a plurality of feature images according to the first image; determining one or more features of an object in the first image according to the training model and the plurality of feature images; determining a first area in the first image, the first area including at least one of the determined one or more features; obtaining depth information of the determined first area based on the first and second images; and determining a second area in the first image based on the determined first area and depth information, the second area including at least one of the determined first area. ## Claims ### Claim 1 A method for binocular stereo vision implemented on a computing device including one or more processors and one or more storage devices, the method comprising: acquiring a pair of images including a first image and a second image, the pair of images being captured by one or more cameras; determining a plurality of feature images according to the first image; determining one or more features of an object in the first image according to a training model and the plurality of feature images; determining a first area in the first image, the first area including at least one of the determined one or more features; obtaining depth information of the determined first area based on the first and second images; and determining a second area in the first image based on the determined first area and depth information, the second area including at least one of the determined first area. ### Claim 2 (depends on 1) The method of claim 1, wherein acquiring the pair of images including the first image and the second image comprises: identifying an object in view of a first camera; adjusting a parameter of a second camera so that the object appears in view of the second camera; and acquiring the pair of images captured by the first camera and the second camera when the object appears in view of both the first and second cameras. ### Claim 3 (depends on 1) The method of claim 1, further comprises obtaining the training model by: obtaining a plurality image samples; generating an initial root filter according to the samples; generating an updating root filter according to the initial root filter; generating a part filter according to the updating root filter; and generating the training model according to the samples and the part filter. ### Claim 4 (depends on 1) The method of claim 1, wherein the obtaining of the depth information of the determined first area based on the first and second images comprises: processing the pair of images to generate a pair of processed images, wherein vertical coordinates in an image coordinate system of two corresponding pixels in the pair of processed images are equal; generating a pair of processed images according to the pair of images; determining a target pixel in the first area in a first processed image; determining a matching pixel of the target pixel in a second processed image; determining depth information of the target pixel in a camera coordinate system according to the matching pixel; determining a height and a pitch angle of at least one of the one or more cameras; and transforming the depth information in the camera coordinate system into depth information in a world coordinate system according to the height and the pitch angle. ### Claim 5 (depends on 4) The method of claim 4, wherein an origin of the image coordinate system is a pixel in a highest position in a top left corner of the first image, an origin of the camera coordinate system is an optical center of the camera capturing the first image, and an origin of the world coordinate system is a point in the real world. ### Claim at position 6 6-7. (canceled) ### Claim 8 (depends on 4) The method of claim 4, wherein lenses of the one or more cameras are fixed lenses, the fixed lenses being non-rotatable along a first axis, a second axis, or a third axis, wherein the first axis, the second axis, and the third axis are perpendicular to each other. ### Claim 9 (depends on 4) The method of claim 4, wherein the determining of the height and the pitch angle of the at least one of the one or more cameras comprises: identifying a third area in the first processed image; determining a parallax between a first pixel in the third area and a second pixel corresponding to the first pixel in the second processed image; determining a relationship between a vertical coordinate of the first pixel in the image coordinate system and the parallax; determining a projection equation according to the relationship; and determining the height and the pitch angle of the at least one of the one or more cameras according to the projection equation. ### Claim 10 (depends on 2) The method of claim 2, wherein the obtaining of the depth information of the determined first area based on the first and second images comprises: obtaining a calibration parameter of the first camera and the second camera, the calibration parameter including a yaw angle, a pitch angle, a focal length, or a three dimensional coordinate of an origin of a camera coordinate system in a world coordinate system of the first camera and the second camera in an initial state; determining a pair of pixels in the pair of images; obtaining, when the object appears in view of both the first and second cameras, an adjustment parameter of at least one of the first camera and the second camera, the adjustment parameter including a yaw angle, a pitch angle, or a focal length of the first camera and the second camera; and determining depth information of the pair of pixels in the world coordinate system according to the calibration parameter and the adjustment parameter. ### Claim 11 (depends on 10) The method of claim 10, wherein the first camera and the second camera have at least one rotatable lens, the rotatable lens being a lens rotatable along a first axis, a second axis or a third axis, wherein the first axis, the second axis, and the third axis are perpendicular to each other. ### Claim 12 (depends on 11) The method of claim 11, wherein the initial state is a state that rotation angles along the first axis, the second axis, and the third axis of the at least one rotatable lens of the first camera and the second camera are 0. ### Claim 13 A system for binocular stereo vision comprising: one or more cameras; one or more storage devices comprising a set of instructions; and one or more processors configured to communicate with the one or more storage devices, wherein when executing the set of instructions, the one or more processors are directed to cause the system to: acquire a pair of images including a first image and a second image, the pair of images being captured by one or more cameras; determine a plurality of feature images according to the first image; determine one or more features of an object in the first image according to a training model and the plurality of feature images; determine a first area in the first image, the first area including at least one of the determined one or more features; obtain depth information of the determined first area based on the first and second images; and determine a second area in the first image based on the determined first area and depth information, the second area including at least one of the determined first area. ### Claim 14 (depends on 13) The system of claim 13, wherein to acquire the pair of images including the first image and the second image, the one or more processors are directed to cause the system to: identify an object in view of a first camera; adjust adjusting a parameter of a second camera so that the object appears in view of the second camera; and acquire the pair of images captured by the first camera and the second camera when the object appears in view of both the first and second cameras. ### Claim 15 (depends on 13) The system of claim 13, wherein the one or more processors are further directed to cause the system to obtain the training model by: obtaining a plurality image samples; generating an initial root filter according to the samples; generating an updating root filter according to the initial root filter; generating a part filter according to the updating root filter; and generating the training model according to the samples and the part filter. ### Claim 16 (depends on 13) The system of claim 13, wherein to obtain the depth information of the determined first area based on the first and second images, the one or more processors are directed to cause the system to: process the pair of images to generate a pair of processed images, wherein vertical coordinates in an image coordinate system of two corresponding pixels in the pair of processed images are equal; generate a pair of processed images according to the pair of images; determine a target pixel in the first area in a first processed image; determine a matching pixel of the target pixel in a second processed image; determine depth information of the target pixel in a camera coordinate system according to the matching pixel; determine a height and a pitch angle of at least one of the one or more cameras; and transform the depth information in the camera coordinate system into depth information in a world coordinate system according to the height and the pitch angle. ### Claim 17 (depends on 16) The system of claim 16, wherein an origin of the image coordinate system is a pixel in a highest position in a top left corner of the first image, an origin of the camera coordinate system is an optical center of the camera capturing the first image, and an origin of the world coordinate system is a point in the real world. ### Claim at position 17 18-19. (canceled) ### Claim 20 (depends on 16) The system of claim 16, wherein lenses of the one or more cameras are fixed lenses, the fixed lenses being non-rotatable along a first axis, a second axis, or a third axis, wherein the first axis, the second axis, and the third axis are perpendicular to each other. ### Claim 21 (depends on 16) The system of claim 16, wherein to determine the height and the pitch angle of the at least one of the one or more cameras, the one or more processors are directed to cause the system to: identify a third area in the first processed image; determine a parallax between a first pixel in the third area and a second pixel corresponding to the first pixel in the second processed image; determine a relationship between a vertical coordinate of the first pixel in the image coordinate system and the parallax; determine a projection equation according to the relationship; and determine the height and the pitch angle of the at least one of the one or more cameras according to the projection equation. ### Claim 22 (depends on 14) The system of claim 14, wherein to obtain the depth information of the determined first area based on the first and second images, the one or more processors are directed to cause the system to: obtain a calibration parameter of the first camera and the second camera, the calibration parameter including a yaw angle, a pitch angle, a focal length, or a three dimensional coordinate of an origin of a camera coordinate system in a world coordinate system of the first camera and the second camera in an initial state; determine a pair of pixels in the pair of images; obtain when the object appears in view of both the first and second cameras, an adjustment parameter of at least one of the first camera and the second camera, the adjustment parameter including a yaw angle, a pitch angle, or a focal length of the first camera and the second camera; and determine depth information of the pair of pixels in the world coordinate system according to the calibration parameter and the adjustment parameter. ### Claim 23 (depends on 22) The system of claim 22, wherein the first camera and the second camera have at least one rotatable lens, the rotatable lens being a lens rotatable along a first axis, a second axis or a third axis, wherein the first axis, the second axis, and the third axis are perpendicular to each other. ### Claim 24 (canceled) ### Claim 25 A non-transitory computer readable medium embodying a computer program product, the computer program product comprising instructions configured to cause a computing device to: acquire a pair of images including a first image and a second image, the pair of images being captured by one or more cameras; obtain a training model; determine a plurality of feature images according to the first image; determine one or more features of an object in the first image according to the training model and the plurality of feature images; determine a first area in the first image, the first area including at least one of the determined one or more features; obtain depth information of the determined first area based on the first and second images; and determine a second area in the first image based on the determined first area and depth information, the second area including at least one of the determined first area.