# US 12,214,510 B2 · Display apparatus > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,214,510 B2 - Document ID: US12214510B2 - Status: Granted - Kind: B2 - Application number: 18/300,012 - Filed: 2023-04-13 - Published: 2023-08-10 - Granted: 2025-02-04 - Earliest claimed: 2011-08-26 - Assignees: Panasonic Automotive Systems Co., Ltd. - Applicants: Panasonic Intellectual Property Management Co., Ltd. - Inventors: Masayoshi Michiguchi; Tatsuto Ryugo; Kenji Okano; Yukiko Kanno - Web page: https://patentgrep.com/patent/US12214510B2 ## CPC - G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle - B25J 9/161: {Hardware, e.g. neural networks, fuzzy logic, interfaces, processor} - B25J 9/1666: {Avoiding collision or forbidden zones} - B25J 9/1679: {characterised by the tasks executed} - B25J 9/1694: {characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion} - B25J 11/008: {Manipulators for service tasks} - B25J 19/023: {including video camera means} - B60R 1/00: Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles - B60R 1/27: providing all-round vision, e.g. using omnidirectional cameras - G01S 15/931: of land vehicles - H04N 5/2624: {for obtaining an image which is composed of whole input images, e.g. splitscreen} - H04N 7/181: {for receiving images from a plurality of remote sources} - H04N 7/185: {from a mobile camera, e.g. for remote control} - H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums - B25J 9/0003: {Home robots, i.e. small robots for domestic use} - B60R 2300/105: using multiple cameras - B60R 2300/20: characterised by the type of display used - B60R 2300/301: combining image information with other obstacle sensor information, e.g. using RADAR/LIDAR/SONAR sensors for estimating risk of collision - B60R 2300/607: from a bird's eye viewpoint - B60R 2300/802: for monitoring and displaying vehicle exterior blind spot views - G01S 17/88: Lidar systems specially adapted for specific applications - G01S 17/894: Three-dimensional [3D] imaging with simultaneous measurement of time-of-flight at a two-dimensional [2D] array of receiver pixels, e.g. time-of-flight cameras or flash lidar - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50}) - B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR - G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A driving assistance apparatus is provided in which the detection range of a left-front-corner sonar (12a) located at the vehicle's left front corner is included in the field of view of a second imaging means (14) located at the vehicle's left front corner. When the left-front-corner sonar (12a) detects a three-dimensional object at the vehicle's left front corner, an image processing means (3) synthesizes an image of the image created using a second imaging means (14) and the images created with four cameras (7a-7d) for imaging the complete periphery of the vehicle, and creates a bird's-eye-view image (40b). The detection range of the left-front-corner sonar (12a) is included within a region of the bird's-eye image (40b) on the basis of the image created with the second imaging means (14). ## Claims ### Claim 1 A control method for a control device mounted in a vehicle, the control device connected to a first camera, a second camera, a third camera, a fourth camera, a sensor, and a display, the first camera being configured to capture first images of first ambient view at a first out portion of the vehicle, the second camera being configured to capture second images of second ambient view at a second out portion of the vehicle, the third camera being configured to capture third images of third ambient view at a third out portion of the vehicle, the fourth camera being configured to capture fourth images of fourth ambient view at a fourth out portion of the vehicle, the first out portion of the vehicle being distinct from the second out portion, the third out portion, or the fourth out portion of the vehicle, the second out portion of the vehicle being distinct from the third out portion or the fourth out portion of the vehicle, the third out portion of the vehicle being distinct from the fourth out portion of the vehicle, the sensor having a detection area at the second out portion of the vehicle, and configured to detect a three-dimensional object within the detection area, the control method comprising: displaying a first one image having at least a first region, a second region, and a third region on a screen of the display, the first region corresponding to one of the first images, the second region corresponding to one of the third images, the third region corresponding to one of the fourth images, when the sensor does not detect a three-dimensional object at the second out portion of the vehicle; and displaying a second one image having at least a fourth region, a fifth region, a sixth region, and a seventh region on the screen of the display, the fourth region corresponding to one of the first images, the fifth region corresponding to one of the second images, the sixth region corresponding to one of the third images, the seventh region corresponding to one of the fourth images, the fifth region of the second one image including—at least a part of the detection area of the sensor, the fifth region of the second one image including at least a part of the three-dimensional object, when the sensor detects a three-dimensional object at the second out portion of the vehicle. ### Claim 2 (depends on 1) The control method according to claim 1, wherein the sensor and the second camera respectively include optical axes that substantially match each other. ### Claim 3 (depends on 1) The control method according to claim 1, wherein the first one image comprises a first bird's-eye view image combined with at least one of the first images, one of the third images, and one of the fourth images. ### Claim 4 (depends on 3) The control method according to claim 3, wherein the first one image includes an image of the vehicle on a center of the first bird's-eye view image. ### Claim 5 (depends on 4) The control method according to claim 4, wherein the first one image does not include one of the second images in the first bird's-eye view image. ### Claim 6 (depends on 1) The control method according to claim 1, wherein the second one image comprises a second bird's-eye view image combined with at least one of the first images, one of the second images, one of the third images, and one of the fourth images. ### Claim 7 (depends on 6) The control method according to claim 6, wherein the second one image includes an image of the vehicle on a center of the second bird's-eye view image. ### Claim 8 (depends on 1) The control method according to claim 1, wherein the control device comprises a processor. ### Claim 9 (depends on 1) The control method according to claim 1, wherein an angle of view of the second camera is set to approximately 180 degrees in a horizontal plane. ### Claim 10 (depends on 1) The control method according to claim 1, wherein the detection area of the sensor is set to be equal to or less than 180 degrees. ### Claim 11 A control device configured to be mounted in a vehicle, the control device comprising: an input configured to be connected to a first camera, a second camera, a third camera, a fourth camera, and a sensor, the first camera being configured to capture first images of first ambient view at a first out portion of the vehicle, the second camera being configured to capture second images of second ambient view at a second out portion of the vehicle, the third camera being configured to capture third images of third ambient view at a third out portion of the vehicle, the fourth camera being configured to capture fourth images of fourth ambient view at a fourth out portion of the vehicle, the first out portion of the vehicle being distinct from the second out portion, the third out portion, or the fourth out portion of the vehicle, the second out portion of the vehicle being distinct from the third out portion or the fourth out portion of the vehicle, the third out portion of the vehicle being distinct from the fourth out portion of the vehicle, the sensor having a detection area at the second out portion of the vehicle, and being configured to detect a three-dimensional object within the detection area; an output configured to be connected with a display, wherein: when the sensor does not detect a three-dimensional object at the second out portion of the vehicle, the control device displays a first one image having at least a first region, a second region and a third region on a screen of the display, the first region corresponding to one of the first images, the second region corresponding to one of the third images, the third region corresponding to one of the fourth images; and when the sensor detects a three-dimensional object at the second out portion of the vehicle, the control device displays a second one image having at least a fourth region, a fifth region, a sixth region, and a seventh region on the screen of the display, the fourth region corresponding to one of the first images, the fifth region corresponding to one of the second images, the sixth region corresponding to one of the third images, the seventh region corresponding to one of the fourth images, the fifth region of the second one image including at least a part of the detection area of the sensor, the fifth region of the second one image including at least a part of the three-dimensional object. ### Claim 12 (depends on 11) The control device according to claim 11, wherein the sensor and the second camera respectively include optical axes that substantially match each other. ### Claim 13 (depends on 11) The control device according to claim 11, wherein the first one image comprises a first bird's-eye view image combined with at least one of the first images, one of the third images, and one of the fourth images. ### Claim 14 (depends on 13) The control device according to claim 13, wherein the first one image includes an image of the vehicle on a center of the first bird's-eye view image. ### Claim 15 (depends on 14) The control device according to claim 14, wherein the first one image does not include one of the second images in the first bird's-eye view image. ### Claim 16 (depends on 11) The control device according to claim 11, wherein the second one image comprises a second bird's-eye view image combined with at least one of the first images, one of the second images, one of the third images, and one of the fourth images. ### Claim 17 (depends on 16) The control device according to claim 16, wherein the second one image includes an image of the vehicle on a center of the second bird's-eye view image. ### Claim 18 (depends on 11) The control device according to claim 11, further comprising a processor. ### Claim 19 (depends on 11) The control device according to claim 11, wherein an angle of view of the second camera is set to approximately 180 degrees in a horizontal plane. ### Claim 20 (depends on 11) The control device according to claim 11, wherein the detection area of the sensor is set to be equal to or less than 180 degrees.