# US 11,443,520 B2 · Image processing apparatus, image processing method, and image processing system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,443,520 B2 - Document ID: US11443520B2 - Status: Granted - Kind: B2 - Application number: 16/328,381 - Filed: 2017-09-01 - Published: 2021-09-09 - Granted: 2022-09-13 - Earliest claimed: 2016-09-16 - Assignees: Sony Corporation - Applicants: Sony Corporation - Inventors: Nobuhiro Tsunashima - Web page: https://patentgrep.com/patent/US11443520B2 ## CPC - G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle - G08G 1/16: Anti-collision systems (road vehicle drive control systems for predicting or avoiding probable or impending collision otherwise than by control of a particular sub-unit B60W30/08) - B60R 1/25: to the sides of the vehicle - B60R 1/26: to the rear of the vehicle - B60R 1/28: with an adjustable field of view - G06T 1/00: General purpose image data processing - G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads - H04N 23/635: {Region indicators; Field of view indicators} - H04N 23/698: for achieving an enlarged field of view, e.g. panoramic image capture - H04N 23/81: for suppressing or minimising disturbance in the image signal generation - B60R 2300/30: characterised by the type of image processing - B60R 2300/302: combining image information with GPS information or vehicle data, e.g. vehicle speed, gyro, steering angle data - B60R 2300/602: with an adjustable viewpoint - B60R 2300/605: the adjustment being automatic - B60R 2300/802: for monitoring and displaying vehicle exterior blind spot views - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93}) - B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract The present technology relates to an image processing apparatus, an image processing method, and an image processing system enabling the presentation of more useful information. An image in a viewpoint direction corresponding to the state of an object is generated from a captured image. Alternatively, an image of the periphery of the object is captured at a wide angle, and then an image obtained by projecting a part of the wide-angle captured image thus obtained onto a plane in a viewpoint direction corresponding to the state of the object is displayed. For example, an image of the periphery of an object is captured at a wide angle, an image in a viewpoint direction corresponding to the state of the object is generated from the obtained captured image, and the generated image is displayed. ## Claims ### Claim 1 An image processing apparatus, comprising: circuitry configured to: receive a wide-angle view image of a periphery of a vehicle; receive vehicle information of the vehicle from at least one sensor; determine a state of the vehicle based on the received vehicle information, wherein the state of the vehicle is one of an ordinary traveling state or a slow traveling state, the ordinary traveling state indicates that a speed of the vehicle is equal to or greater than a determined threshold, and the slow traveling state indicates that the speed of the vehicle is less than the determined threshold; and generate an image in a prescribed viewpoint direction by projection of a part of the wide-angle view image onto a plane in the prescribed viewpoint direction, wherein the generation of the image in the prescribed viewpoint direction is based on the determined state of the vehicle, and the generation of the image in the prescribed viewpoint direction is toward a rear side of the vehicle based on the determination that the state of the vehicle is the ordinary traveling state. ### Claim 2 (depends on 1) The image processing apparatus according to claim 1, wherein the circuitry is further configured to execute a distortion correction on the part of the wide-angle view image prior to the generation of the image in the prescribed viewpoint direction. ### Claim 3 (depends on 1) The image processing apparatus according to claim 1, wherein the state of the vehicle is related to a movement of the vehicle. ### Claim 4 (depends on 1) The image processing apparatus according to claim 1, wherein the state of the vehicle is related to an operation performed with respect to the vehicle. ### Claim 5 (depends on 1) The image processing apparatus according to claim 1, wherein the circuitry is further configured to: determine the state of the vehicle is one of a right turning state or a left turning state based on the vehicle information, wherein the vehicle information indicates a winker switch is set in one of a right direction or a left direction; and generate the image in the prescribed viewpoint direction toward an oblique rear side of the vehicle based on the determination that the vehicle is in one of the right turning state or the left turning state. ### Claim 6 (depends on 1) The image processing apparatus according to claim 1, wherein the circuitry is further configured to: determine the state of the vehicle is the slow traveling state based on the received vehicle information which indicates that the speed of the vehicle is less than the determined threshold; and generate the image in the prescribed viewpoint direction toward an oblique-lower front side of a lateral surface of the vehicle based on the determination that the state of the vehicle is the slow traveling state. ### Claim 7 (depends on 1) The image processing apparatus according to claim 1, wherein the circuitry is further configured to: determine the state of the vehicle is a backward movement state based on the received vehicle information; and generate the image in the prescribed viewpoint direction toward a lower side of a lateral surface of the vehicle based on the determination that the state of the vehicle is the backward movement state. ### Claim 8 (depends on 1) The image processing apparatus according to claim 1, further comprising a display screen configured to display the generated image. ### Claim 9 (depends on 8) The image processing apparatus according to claim 8, wherein the display screen is further configured to display the generated image in a layout corresponding to the state of the vehicle. ### Claim 10 (depends on 8) The image processing apparatus according to claim 8, wherein the display screen is further configured to display the generated image in at least one position of a plurality of positions corresponding to the state of the vehicle. ### Claim 11 An image processing method, comprising: receiving a wide-angle view image of a periphery of a vehicle; receiving vehicle information of the vehicle from at least one sensor; determining a state of the vehicle based on the received vehicle information, wherein the state of the vehicle is one of an ordinary traveling state or a slow traveling state, the ordinary traveling state indicates that a speed of the vehicle is equal to or greater than a determined threshold, and the slow traveling state indicates that the speed of the vehicle is less than the determined threshold; and generating an image in a prescribed viewpoint direction by projection of a part of the wide-angle view image onto a plane in the prescribed viewpoint direction, wherein the generation of the image in the prescribed viewpoint direction is based on the determined state of the vehicle, and the generation of the image in the prescribed viewpoint direction is toward a rear side of the vehicle based on the determination that the state of the vehicle is the ordinary traveling state. ### Claim 12 An image processing apparatus, comprising: an image sensor configured to capture a wide-angle view image of a periphery of a vehicle; circuitry configured to: receive vehicle information of the vehicle from at least one sensor; determine a state of the vehicle based on the received vehicle information, wherein the state of the vehicle is one of an ordinary traveling state or a slow traveling state, the ordinary traveling state indicates that a speed of the vehicle is equal to or greater than a determined threshold, and the slow traveling state indicates that the speed of the vehicle is less than the determined threshold; and generate an image in a prescribed viewpoint direction by projection of a part of the wide-angle view image onto a plane in the prescribed viewpoint direction, wherein the generation of the image in the prescribed viewpoint direction is based on the determined state of the vehicle, and the generation of the image in the prescribed viewpoint direction is toward a rear side of the vehicle based on the determination that the state of the vehicle is the ordinary traveling state; and a display screen configured to display the generated image in the prescribed viewpoint direction. ### Claim 13 An image processing method, comprising: capturing a wide-angle view image of a periphery of a vehicle; receiving vehicle information of the vehicle from at least one sensor; determining a state of the vehicle based on the received vehicle information, wherein the state of the vehicle is one of an ordinary traveling state or a slow traveling state, the ordinary traveling state indicates that a speed of the vehicle is equal to or greater than a determined threshold, and the slow traveling state indicates that the speed of the vehicle is less than the determined threshold; generating an image in a prescribed viewpoint direction by projection of a part of the wide-angle view image onto a plane in the prescribed viewpoint direction, wherein the generation of the image in the prescribed viewpoint direction is based on the determined state of the vehicle, and the generation of the image in the prescribed viewpoint direction is toward a rear side of the vehicle based on the determination that the state of the vehicle is the ordinary traveling state; and displaying the generated image in the prescribed viewpoint direction. ### Claim 14 An image processing system, comprising: an imaging apparatus configured to capture a wide-angle view image of a periphery of a vehicle; an image processing apparatus comprising circuitry, wherein the circuitry is configured to: receive vehicle information of the vehicle from at least one sensor; determine a state of the vehicle based on the received vehicle information, wherein the state of the vehicle is one of an ordinary traveling state or a slow traveling state, the ordinary traveling state indicates that a speed of the vehicle is equal to or greater than a determined threshold, and the slow traveling state indicates that the speed of the vehicle is less than the determined threshold; and generate an image in a prescribed viewpoint direction by projection of a part of the wide-angle view image onto a plane in the prescribed viewpoint direction, wherein the generation of the image in the prescribed viewpoint direction is based on the determined state of the vehicle, and the generation of the image in the prescribed viewpoint direction is toward a rear side of the vehicle based on the determination that the state of the vehicle is the ordinary traveling state; and a display apparatus configured to display the generated image in the prescribed viewpoint direction. ### Claim 15 (depends on 1) The image processing apparatus according to claim 1, wherein the circuitry is further configured to: determine that the state of the vehicle is one of a right turning state or a left turning state based on the vehicle information, wherein the vehicle information indicates that a winker switch is set in one of a right direction or a left direction; and generate the image in the prescribed viewpoint direction which is toward an oblique front side of the vehicle based on the determination that the vehicle is in one of the right turning state or the left turning state. ### Claim 16 (depends on 1) The image processing apparatus according to claim 1, wherein the circuitry is further configured to: determine that the state of the vehicle is a backward movement state based on the vehicle information; and generate the image in the prescribed viewpoint direction which is toward an oblique-lower rear side of a lateral surface of the vehicle based on the determination that the state of the vehicle is the backward movement state. ### Claim 17 (depends on 1) The image processing apparatus according to claim 1, wherein the circuitry is further configured to: recognize an object that indicates an obstacle in the periphery of the vehicle based on the wide-angle view image; determine a possibility of collision of the vehicle with the object based on the determined state of the vehicle; and generate notification information based on a result of the determination of the possibility of the collision of the vehicle with the object. ### Claim 18 (depends on 17) The image processing apparatus according to claim 17, wherein the circuitry is further configured to one of emphasize or enlarge the object in the generated image. ### Claim 19 (depends on 17) The image processing apparatus according to claim 17, wherein the circuitry is further configured to change the notification information based on a distance between the object and the vehicle. ### Claim 20 (depends on 17) The image processing apparatus according to claim 17, wherein the notification information includes voice, and the circuitry is further configured to change one of voice content, a sound volume, or a sound quality of the voice based on a distance between the object and the vehicle.