# US 10,769,951 B2 · Image processing apparatus, image processing method, and vehicle control system to determine the presence of an object from an image of a peripheral area of a moving body > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,769,951 B2 - Document ID: US10769951B2 - Status: Granted - Kind: B2 - Application number: 15/761,878 - Filed: 2016-08-30 - Published: 2018-10-18 - Granted: 2020-09-08 - Earliest claimed: 2015-09-30 - Assignees: Sony Corporation - Applicants: Sony Corporation - Inventors: Yasutaka Hirasawa; Hideki Oyaizu; Yuhi Kondo; Suguru Aoki; Taketo Akama - Web page: https://patentgrep.com/patent/US10769951B2 ## CPC - G08G 1/166: {for active traffic, e.g. moving vehicles, pedestrians, bikes} - G06T 7/11: Region-based segmentation - G06T 7/174: involving the use of two or more images - G06V 10/147: Details of sensors, e.g. sensor lenses (fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19) - G06V 20/588: {Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road} - G08G 1/16: Anti-collision systems (road vehicle drive control systems for predicting or avoiding probable or impending collision otherwise than by control of a particular sub-unit B60W30/08) - H04N 7/18: Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast - H04N 7/181: {for receiving images from a plurality of remote sources} - G06T 2207/30252: Vehicle exterior; Vicinity of vehicle - G06V 20/64: Three-dimensional [3D] objects - G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93}) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract An image acquisition unit 341-1 acquires a polarization image and a non-polarization image indicating a peripheral area of a moving body, such as the peripheral area of a vehicle. A discrimination information generation unit 342-1 uses the polarization image acquired by the image acquisition unit 341-1 and generates analysis object discrimination information indicating a road surface or the like. An image analysis unit 344-1 uses an image of an image analysis area set on the basis of the analysis object discrimination information generated by the discrimination information generation unit 342-1 with respect to the non-polarization image acquired by the image acquisition unit 341-1, and performs a discrimination of an object, such as an obstacle on the road surface. It is possible to efficiently perform a determination of the presence of the object from the non-polarization image of the peripheral area of the moving body. ## Claims ### Claim 1 An image processing apparatus comprising: a discrimination information generation unit configured to generate analysis object discrimination information by using a polarization image indicating a peripheral area of a moving body; and an image analysis unit configured to perform discrimination of an object by using an image of an image analysis area set on the basis of the analysis object discrimination information generated by the discrimination information generation unit to a non-polarization image indicating the peripheral area of the moving body, wherein the discrimination information generation unit generates, as the analysis object discrimination information, information related to a braking distance of the moving body in the peripheral area of the moving body by using the polarization image, and the image analysis unit sets an area within the braking distance as the image analysis area. ### Claim 2 (depends on 1) The image processing apparatus according to claim 1, wherein the discrimination information generation unit generates the analysis object discrimination information indicating a road surface in the peripheral area of the moving body on the basis of the polarization image, and the image analysis unit sets the image analysis area in accordance with an area of the road surface. ### Claim 3 (depends on 2) The image processing apparatus according to claim 2, wherein the discrimination information generation unit sets, to an area of the road surface, a continuous image area in which a direction of a normal line calculated from the polarization image is a vertical direction of the moving body. ### Claim 4 (depends on 1) The image processing apparatus according to claim 1, wherein, on the basis of a distance in the non-polarization image and the braking distance calculated from a roughness on a traveling surface and a moving speed, the discrimination information generation unit generates, as the analysis object discrimination information, information indicating an area within the braking distance in the non-polarization image. ### Claim 5 (depends on 1) The image processing apparatus according to claim 1, wherein the image analysis unit is provided in plurality, and the plurality of image analysis units share an area in which the non-polarization image is divided on the basis of the analysis object discrimination information and perform discrimination of the object. ### Claim 6 (depends on 5) The image processing apparatus according to claim 5, wherein the analysis object discrimination information is information related to the braking distance of the moving body in the peripheral area of the moving body and information indicating the road surface in the peripheral area of the moving body, and the plurality of image analysis units perform discrimination of the object in an area within the braking distance by using many image analysis units as compared with other image areas. ### Claim 7 (depends on 1) The image processing apparatus according to claim 1, wherein an image transfer unit that performs a transfer of the non-polarization image is provided, the image analysis unit is connected to the image transfer unit via a communication network, and the image transfer unit controls a transfer of the non-polarization image to the image analysis unit on the basis of the analysis object discrimination information generated by the discrimination information generation unit. ### Claim 8 (depends on 7) The image processing apparatus according to claim 7, wherein the image transfer unit transfers an image of the image analysis area. ### Claim 9 (depends on 7) The image processing apparatus according to claim 7, wherein the discrimination information generation unit generates the analysis object discrimination information by using the polarization image in each capturing direction, and the image transfer unit selects the non-polarization image transferred to the image analysis unit on the basis of the analysis object discrimination information generated by the discrimination information generation unit and transfers the selected non-polarization image along with the analysis object discrimination information. ### Claim 10 (depends on 7) The image processing apparatus according to claim 7, wherein the discrimination information generation unit generates the analysis object discrimination information by using the polarization image in each capturing direction, and the image transfer unit selects the non-polarization image transferred to the image analysis unit on the basis of the analysis object discrimination information generated by the discrimination information generation unit and transfers an image of the image analysis area in the selected non-polarization image along with the analysis object discrimination information. ### Claim 11 (depends on 7) The image processing apparatus according to claim 7, wherein an image analysis control unit and the plurality of image analysis units are provided, and the image analysis control unit generates image analysis control information to control that how discrimination of the object to the non-polarization image transferred to the image analysis unit from the image transfer unit is performed by the plurality of image analysis units on the basis of the analysis object discrimination information generated by the discrimination information generation unit and transmits the image analysis control information to the image analysis unit along with the non-polarization image. ### Claim 12 An image processing method comprising: generating analysis object discrimination information by a discrimination information generation unit by using a polarization image indicating a peripheral area of a moving body; and performing discrimination of an object by an image analysis unit by using an image of an image analysis area set on the basis of the analysis object discrimination information generated by the discrimination information generation unit to a non-polarization image indicating the peripheral area of the moving body, wherein generating analysis object discrimination information includes generating, as the analysis object discrimination information, information related to a braking distance of the moving body in the peripheral area of the moving body by using the polarization image, further including setting, by the image analysis unit, an area within the braking distance as the image analysis area. ### Claim 13 A vehicle control system comprising: a discrimination information generation unit configured to generate analysis object discrimination information by using a polarization image indicating a peripheral area of a vehicle; an image analysis unit configured to perform discrimination of an object by using an image of an image analysis area set on the basis of the analysis object discrimination information generated by the discrimination information generation unit to a non-polarization image indicating the peripheral area of the vehicle; and a driving system control unit configured to perform control of the vehicle on the basis of a discrimination result of the object in the image analysis unit, wherein the discrimination information generation unit generates, as the analysis object discrimination information, information related to a braking distance of the vehicle in the peripheral area of the vehicle by using the polarization image, and the image analysis unit sets an area within the braking distance as the image analysis area.