# US 2014/0153776 A1 · Image processing device, image processing method, and program > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2014/0153776 A1 - Document ID: US20140153776A1 - Status: Published application - Kind: A1 - Application number: 14/086,720 - Filed: 2013-11-21 - Published: 2014-06-05 - Earliest claimed: 2012-12-04 - Assignees: Sony Corporation - Applicants: Sony Corporation - Inventors: Hideto Takeuchi - Web page: https://patentgrep.com/patent/US20140153776A1 ## CPC - G06V 20/54: of traffic, e.g. cars on the road, trains or boats - G06V 10/255: {Detecting or recognising potential candidate objects based on visual cues, e.g. shapes} - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract There is provided an image processing device including an image input unit configured to input captured image data of a portion of parking stalls that are compartmented at least by a first side line extending in a first direction and a second side line extending in the first direction, an image processing unit configured to generate edge image data by performing an edge extraction process on the captured image data, and a parking determination unit configured to obtain an integrated value of edge pixels of the first direction portion corresponding to the parking stalls in each position of a second direction that is orthogonal to the first direction based on the edge image data, and then to determine whether or not vehicles are parked in the parking stalls based on the obtained integrated value of each position of the second direction. ## Claims ### Claim 1 An image processing device comprising: an image input unit configured to input captured image data of a portion of parking stalls that are compartmented at least by a first side line extending in a first direction and a second side line extending in the first direction; an image processing unit configured to generate edge image data by performing an edge extraction process on the captured image data; and a parking determination unit configured to obtain an integrated value of edge pixels of the first direction portion corresponding to the parking stalls in each position of a second direction that is orthogonal to the first direction based on the edge image data, and then to determine whether or not vehicles are parked in the parking stalls based on the obtained integrated value of each position of the second direction. ### Claim 2 (depends on 1) The image processing device according to claim 1, further comprising: a projective transformation processing unit configured to perform a projective transformation process on the edge image data to transform an image captured with a perspective into an image viewed from straight above, wherein the parking determination unit obtains the integrated value of each position of the second direction based on the edge image data that has undergone the projective transformation process. ### Claim 3 (depends on 1) The image processing device according to claim 1, wherein each of the parking stalls is compartmented by the first side line and the second side line and by a base line that is set to be on the opposite side of the entrance side and extends in the second direction, and wherein the first direction portion corresponding to each of the parking stalls is set to be in a range from the entrance of the parking stall to the front of the base line. ### Claim 4 (depends on 1) The image processing device according to claim 1, wherein the image processing unit has an edge extraction unit configured to generate edge intensity image data by performing an edge extraction process on the captured image data, a quantization unit configured to generate edge image data by performing a quantization process on the edge intensity image data, an edge labeling unit configured to regard edge pixel regions having connectivity with the edge image data as a same region, to perform a process of giving the same label number to edge pixels of a same region, and to generate labeled image data, a first rectangle objectification unit configured to perform a process of giving a circumscribed rectangle to the edge pixel regions to which the same label number is given based on the labeled image data, and to generate a rectangular object, and a first fine object removal unit configured to perform a process on the edge image data to remove an edge pixel region having a small number of connected pixels and an edge pixel region in a rectangular object having short vertical and horizontal widths as noise based on the labeled image data and the rectangular object, and to generate output edge image data. ### Claim 5 (depends on 4) The image processing device according to claim 4, wherein the first fine object removal unit further performs a process on the labeled image data to remove the edge pixel region having a small number of connected pixels and the edge pixel region in the rectangular object having short vertical and horizontal widths as noise based on the labeled image data and the rectangular object, and to generate output labeled image data, wherein a projective transformation unit configured to perform a projective transformation process on the edge image data and the labeled image data that are output from the first fine object removal unit to transform an image captured with a perspective into an image viewed from straight above is further included, wherein the parking determination unit has a second rectangle objectification unit configured to generate a rectangular object by giving a circumscribed rectangle to the edge pixel regions to which the same label number is given based on the labeled image data output from the projective transformation unit, and a second fine object removal unit configured to perform a process on the edge image data output from the projective transformation unit to remove an edge pixel region that does not include an edge intensity equal to or greater than a predetermined value and an edge pixel region in a rectangular object having a width in the second direction smaller than a predetermined value as noise based on the labeled image data and the rectangular object, and to generate output edge image data, and wherein the integrated value of each position of the second direction is obtained based on the edge image data output from the second fine object removal unit. ### Claim 6 (depends on 1) The image processing device according to claim 1, for a parking lane that is constituted by consecutively providing a plurality of parking stalls in the second direction, the device further comprising: a parking lane congestion degree computation unit configured to compute the degree of congestion of vehicles in the parking lane based on a determination result made by the parking determination unit with regard to the plurality of parking stalls. ### Claim 7 (depends on 6) The information processing device according to claim 6, for the parking lane that has a dual lane structure of a front lane and a rear lane, wherein, with regard to each of the parking stalls of the front lane, the parking determination unit obtains the integrated value of edge pixels of the first direction portion corresponding to the parking stalls in each position of the second direction that is orthogonal to the first direction based on the edge image data, and then determines whether or not vehicles are parked in the parking stalls based on the obtained integrated value of each position, and wherein, with regard to each of the parking stalls of the rear lane, the parking determination unit obtains, as a first integrated value, the integrated value of the edge pixels of the first direction portion corresponding to the parking stalls in each position of the second direction that is orthogonal to the first direction based on the edge image data, obtains, as a second integrated value, the integrated value of edge pixels of a third direction portion that is orthogonal to both of the first direction and the second direction corresponding to the entrances of the parking stalls in each position of the second direction based on the edge image data, and determines whether or not vehicles are parked in the parking stalls based on the obtained first and second integrated values of each position. ### Claim 8 (depends on 7) The image processing device according to claim 7, further comprising: a projective transformation unit configured to perform a projective transformation process on the edge image data to transform an image captured with a perspective into an image viewed from straight above, wherein the parking determination unit obtains the integrated value based on the edge image data that has undergone the projective transformation process except for the second integrated value. ### Claim 9 An image processing method comprising: inputting captured image data of a portion of parking stalls that are compartmented at least by a first side line extending in a first direction and a second side line extending in the first direction; generating edge image data by performing an edge extraction process on the captured image data; and obtaining an integrated value of edge pixels of the first direction portion corresponding to the parking stalls in each position of a second direction that is orthogonal to the first direction based on the edge image data, and then determining whether or not vehicles are parked in the parking stalls based on the obtained integrated value of each position. ### Claim 10 A program for causing a computer to function as: a device configured to input captured image data of a portion of parking stalls that are compartmented at least by a first side line extending in a first direction and a second side line extending in the first direction; a device configured to generate edge image data by performing an edge extraction process on the captured image data; and a device configured to obtain an integrated value of edge pixels of the first direction portion corresponding to the parking stalls in each position of a second direction that is orthogonal to the first direction based on the edge image data, and then to determine whether or not vehicles are parked in the parking stalls based on the obtained integrated value of each position.