# US 10,373,378 B2 · Augmented reality system for vehicle blind spot prevention > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,373,378 B2 - Document ID: US10373378B2 - Status: Granted - Kind: B2 - Application number: 14/751,891 - Filed: 2015-06-26 - Published: 2016-12-29 - Granted: 2019-08-06 - Earliest claimed: 2015-06-26 - Assignees: PACCAR Inc - Applicants: Andrew Paul Harbach; Bruce Lee - Inventors: Andrew Paul Harbach; Bruce Lee - Web page: https://patentgrep.com/patent/US10373378B2 ## CPC - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - B60R 1/081: {avoiding blind spots, e.g. by using a side-by-side association of mirrors} - B60R 1/12: Mirror assemblies combined with other articles, e.g. clocks - B60R 1/26: to the rear of the vehicle - B60R 11/0235: {of flat type, e.g. LCD} - G06F 3/1423: {controlling a plurality of local displays, e.g. CRT and flat panel display} - G06F 3/147: using display panels - G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle - H04N 23/63: by using electronic viewfinders - H04N 23/698: for achieving an enlarged field of view, e.g. panoramic image capture - B60R 2001/1215: {with information displays} - B60R 2001/1253: {with cameras, video cameras or video screens} - B60R 2011/0017: {Head-rests} - B60R 2300/307: virtually distinguishing relevant parts of a scene from the background of the scene - B60R 2300/50: characterised by the display information being shared, e.g. external display, data transfer to other traffic participants or centralised traffic controller - B60R 2300/8026: in addition to a rear-view mirror system - G06T 2207/30252: Vehicle exterior; Vicinity of vehicle - G09G 2380/10: Automotive applications - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00) ## Abstract The present disclosure relates to systems and methods for providing various types of information to a vehicle driver. Such information can be used by the vehicle driver singularly or in conjunction with other information available to the vehicle driver in order to allow the driver to operate the vehicle in an increasingly safe manner and/or to reduce the likelihood of property damage and/or possible bodily injuries to the driver, etc. In some instances, such information is presented to the driver as an augmented reality environment such that the driver can “see through” objects that may be occluding the driver's vision. ## Claims ### Claim 1 In a vehicle platoon including a host vehicle controlled by a first driver and a trailing vehicle that is physically separate from and behind the host vehicle and is controlled by a second driver, the host vehicle having one or more displays, and the trailing vehicle having an information source mounted thereon, a method implemented in computer-executable instructions for displaying information about vehicle surroundings to the first driver, the method comprising: obtaining vehicle environment data from one or more information sources, the vehicle environment data indicative of at least a part of a scene occluded from view of the first driver when operating the host vehicle, wherein the one or more information sources include the information source mounted to the trailing vehicle, wherein the vehicle environment data includes a real image from the information source mounted to the trailing vehicle, and wherein the real image depicts an area behind the trailing vehicle that is occluded from view of the first driver by the trailing vehicle; and presenting to the first driver, with the aid of the one or more displays, an augmented reality environment based on the vehicle environment data and representative of an area surrounding the host vehicle but obstructed from view of the first driver; wherein said presenting includes causing the one or more displays to render at least a portion of the real image of the area occluded from view of the first driver by the trailing vehicle. ### Claim 2 (depends on 1) The method of claim 1, wherein said presenting includes causing the one or more displays to render one or more virtual design elements representative of one or more objects located in the scene occluded from view of the first driver. ### Claim 3 (depends on 1) The method of claim 1, wherein said presenting includes generating one or more virtual design elements representative of one or more objects located in the scene occluded from view of the first driver; obtaining a real image of the scene, the real image based on the point of view of the first driver; and causing the virtual design elements and the real image to be displayed together by the one or more displays. ### Claim 4 (depends on 3) The method of claim 3, wherein said causing the virtual design elements and the real image to be displayed together by the one or more displays includes arranging the generated one or more virtual design elements over a preselected area of the real image. ### Claim 5 (depends on 1) The method of claim 1, wherein said obtaining vehicle environment data includes obtaining additional data from a source discrete from the host vehicle, the external source including one of a beacon and a traffic camera. ### Claim 6 (depends on 1) The method of claim 1, wherein the vehicle environment data is associated with one or more sensing zones that correspond to one or more external areas of the host vehicle, wherein the one or more external areas of the host vehicle represent at least in part areas occluded from view of the first driver when operating the host vehicle. ### Claim 7 A non-transitory computer-readable medium having modules for conveying information to a first driver of a host vehicle in a vehicle platoon regarding vehicle surroundings, wherein the vehicle platoon includes the host vehicle and a trailing vehicle that is physically separate from and behind the host vehicle and controlled by a second driver, the modules comprising: an information gathering module configured to collect data from one or more information sources associated with one or more sensing zones, wherein the one or more information sources include an information source mounted to the trailing vehicle, wherein the data includes a real image from the information source mounted to the trailing vehicle, wherein the real image depicts an area behind the trailing vehicle that is occluded from view of the first driver by the trailing vehicle; an augmented reality image rendering module configured to generate from the collected data one or more virtual design elements representative of objects occluded from view of the first driver; and a display module configured to: cause the virtual design elements to be presented to a display; and cause the real image of the area occluded from view of the first driver by the trailing vehicle to be presented to a display. ### Claim 8 (depends on 7) The computer-readable medium of claim 7, wherein the display module is further configured to cause the real image to be presented with the virtual design elements, wherein the real image and the virtual design elements together form an augmented reality environment. ### Claim 9 (depends on 8) The computer-readable medium of claim 8, wherein the augmented reality image rendering module is further configured to arrange and align the generated one or more virtual design elements over a preselected area of the real image. ### Claim 10 (depends on 7) The computer-readable medium of claim 7, wherein the one or more information sources include information sources selected from the group consisting of a digital camera, a thermal imaging device, radar, and lidar. ### Claim 11 (depends on 7) The computer-readable medium of claim 7, wherein the one or more sensing zones correspond to one or more external areas of the host vehicle. ### Claim 12 (depends on 11) The computer-readable medium of claim 11, wherein the one or more external areas of the host vehicle represent at least in part areas occluded from view of the first driver when operating the host vehicle. ### Claim 13 A system for providing information to a first driver of a host vehicle of a vehicle platoon, wherein the vehicle platoon includes the host vehicle and a trailing vehicle that is physically separate from and behind the host vehicle and is controlled by a second driver, the system comprising: one or more displays; and a display generator communicatively coupled to one or more information sources configured to generate data associated with one or more sensing zones located at one or more areas surrounding the host vehicle, wherein the one or more information sources include an information source mounted to the trailing vehicle, wherein the data includes a real image from the information source mounted to the trailing vehicle, and wherein the real image depicts an area behind the trailing vehicle that is occluded from view of the first driver by the trailing vehicle; wherein the display generator is configured to: collect data from the one or more information sources associated with the one or more sensing zones; generate from the collected data one or more virtual design elements representative of objects occluded from view of the first driver; cause the virtual design elements to be presented to a display; and cause at least a portion of the real image of the area occluded from view of the first driver by the trailing vehicle to be presented to a display. ### Claim 14 (depends on 13) The system of claim 13, wherein the one or more displays include a transparent display configured to overlay one of a transparent layer and a reflective layer. ### Claim 15 (depends on 14) The system of claim 14, wherein the transparent layer includes a window and the reflective layer includes a mirror. ### Claim 16 (depends on 15) The system of claim 15, wherein the mirror includes one of a side mirror and a rear view mirror. ### Claim 17 (depends on 13) The system of claim 13, wherein the display generator includes a memory storing one or more modules having program instructions for conveying vehicle information on the display; and a processor configured to execute the program instructions of the one or more modules. ### Claim 18 (depends on 13) The system of claim 13, wherein the one or more information sources include information sources selected from the group consisting of a digital camera, a thermal imaging device, radar, and lidar. ### Claim 19 (depends on 13) The system of claim 13, wherein one of the one or more information sources includes a receiver configured to receive data associated with the one or more sensing zones from a source discrete from the host vehicle, wherein the source discrete from the host vehicle is one of a beacon and a traffic camera.