# US 11,440,467 B2 · Autonomous vehicle visual based communication > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,440,467 B2 - Document ID: US11440467B2 - Status: Granted - Kind: B2 - Application number: 17/187,829 - Filed: 2021-02-28 - Published: 2021-06-17 - Granted: 2022-09-13 - Earliest claimed: 2019-03-31 - Assignees: GM Cruise Holdings LLC - Applicants: GM Cruise Holdings LLC - Inventors: Christopher Shih - Web page: https://patentgrep.com/patent/US11440467B2 ## CPC - G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)} - B60Q 1/50: for indicating other intentions or conditions, e.g. request for waiting or overtaking - B60Q 1/507: {specific to autonomous vehicles} - G05D 1/0088: {characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours (using knowledge based models G06N5/00)} - G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle - G06V 40/103: {Static body considered as a whole, e.g. static pedestrian or occupant recognition} - B60Q 2400/50: Projected symbol or information, e.g. onto the road or car body - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - B60Q: ARRANGEMENT OF SIGNALLING OR LIGHTING DEVICES, THE MOUNTING OR SUPPORTING THEREOF OR CIRCUITS THEREFOR, FOR VEHICLES IN GENERAL - G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract An autonomous vehicle including a projector and a sensor system configured to output data representative of an object in an environment exterior of the autonomous vehicle. The autonomous vehicle can further include a data store that stores height map data. A computing system of the autonomous vehicle can classify an object as a person and can detect an initial position of the person. Responsive to the detection, the computing system can calibrate a notification to be projected on a ground of the environment exterior of the autonomous vehicle. The notification is based on output of a transform from a position of the projector to the ground and the transform is applied based on the predefined height map of the ground. The notification can be projected adjacent the person. The notification may be configured to inform the person that the autonomous vehicle detects a position of the person. ## Claims ### Claim 1 An autonomous vehicle, comprising: a sensor system configured to output data representative of an object in an environment external to the autonomous vehicle; a projector; and a computing system in communication with the sensor system and the projector, wherein the computing system comprises: a processor; and memory that stores computer-executable instructions that, when executed by the processor, cause the processor to perform acts comprising: classifying the object as a person based on the data outputted by the sensor system; detecting a location of the person in the environment over time based on the data outputted by the sensor system; detecting a gaze of the person in the environment being directed at the autonomous vehicle based on the data outputted by the sensor system; and causing the projector to project a notification on a ground in the environment external to the autonomous vehicle, wherein the projector is caused to project the notification on the ground adjacent to the location of the person during a period of time while one or more of the person moves within the environment during at least a portion of the period of time or the autonomous vehicle moves within the environment during at least a portion of the period of time, and wherein the projector is further caused to project the notification on the ground adjacent to the location of the person during the period of time responsive to the gaze of the person being detected as directed at the autonomous vehicle. ### Claim 2 (depends on 1) The autonomous vehicle of claim 1, the acts further comprising: calibrating the notification to be projected on the ground in the environment external to the autonomous vehicle adjacent to the location of the person during the period of time based on a distance from the projector to the location of the person in the environment over time and an elevation of the projector relative to the ground adjacent to the location of the person over time. ### Claim 3 (depends on 1) The autonomous vehicle of claim 1, wherein the projector is caused to continue to project the notification on the ground adjacent to the location of the person during the period of time after the gaze of the person is detected as moving away from the autonomous vehicle. ### Claim 4 (depends on 1) The autonomous vehicle of claim 1, wherein the period of time during which the projector is caused to project the notification on the ground adjacent to the location of the person is a time period during which the person is within a threshold distance from the autonomous vehicle. ### Claim 5 (depends on 1) The autonomous vehicle of claim 1, wherein the period of time during which the projector is caused to project the notification on the ground adjacent to the location of the person is a time period during which the person performs a predefined maneuver. ### Claim 6 (depends on 1) The autonomous vehicle of claim 1, wherein the period of time during which the projector is caused to project the notification on the ground adjacent to the location of the person is a predefined projection time period. ### Claim 7 (depends on 1) The autonomous vehicle of claim 1, wherein the projector is caused to project the notification on the ground adjacent to the location of the person during the period of time such that the notification is an image on the ground surrounding a footprint of the person during the period of time. ### Claim 8 (depends on 1) The autonomous vehicle of claim 1, wherein the projector is caused to project the notification on the ground adjacent to the location of the person during the period of time such that the notification is an image on the ground in front of the person in a direction of travel of the person. ### Claim 9 (depends on 1) The autonomous vehicle of claim 1, wherein the notification has a characteristic, and wherein the acts further comprise: selecting the characteristic of the notification based on a classification of the person. ### Claim 10 (depends on 1) The autonomous vehicle of claim 1, wherein the notification projected on the ground adjacent to the location of the person during the period of time comprises an indicator specifying an identity of the autonomous vehicle. ### Claim 11 A method performed by an autonomous vehicle in an environment, comprising: classifying an object in the environment external to the autonomous vehicle as a person based on data outputted by a sensor system of the autonomous vehicle; detecting a location of the person in the environment over time based on the data outputted by the sensor system; detecting a gaze of the person in the environment being directed at the autonomous vehicle based on the data outputted by the sensor system; and causing a projector of the autonomous vehicle to project a notification on a ground in the environment external to the autonomous vehicle, wherein the projector is caused to project the notification on the ground adjacent to the location of the person during a period of time while one or more of the person moves within the environment during at least a portion of the period of time or the autonomous vehicle moves within the environment during at least a portion of the period of time, and wherein the projector is further caused to project the notification on the ground adjacent to the location of the person during the period of time responsive to the gaze of the person being detected as directed at the autonomous vehicle. ### Claim 12 (depends on 11) The method of claim 11, further comprising: calibrating the notification to be projected on the ground in the environment external to the autonomous vehicle adjacent to the location of the person during the period of time based on a distance from the projector to the location of the person in the environment over time and an elevation of the projector relative to the ground adjacent to the location of the person over time. ### Claim 13 (depends on 11) The method of claim 11, wherein the period of time during which the projector is caused to project the notification on the ground adjacent to the location of the person is at least one of a time period during which the person is within a threshold distance from the autonomous vehicle, a time period during which the person performs a predefined maneuver, or a predefined projection time period. ### Claim 14 (depends on 11) The method of claim 11, wherein the projector is caused to project the notification on the ground adjacent to the location of the person during the period of time such that the notification is an image on the ground surrounding a footprint of the person during the period of time. ### Claim 15 (depends on 11) The method of claim 11, wherein the projector is caused to project the notification on the ground adjacent to the location of the person during the period of time such that the notification is an image on the ground in front of the person in a direction of travel of the person. ### Claim 16 (depends on 11) The method of claim 11, wherein the notification projected on the ground adjacent to the location of the person during the period of time comprises an indicator specifying an identity of the autonomous vehicle. ### Claim 17 (depends on 11) The method of claim 11, wherein the projector is caused to continue to project the notification on the ground adjacent to the location of the person during the period of time after the gaze of the person is detected as moving away from the autonomous vehicle. ### Claim 18 An autonomous vehicle, comprising: a sensor system configured to output data representative of an object in an environment external to the autonomous vehicle; a projector; and a computing system in communication with the sensor system and the projector, wherein the computing system comprises: a processor; and memory that stores computer-executable instructions that, when executed by the processor, cause the processor to perform acts comprising: classifying the object as a person based on the data outputted by the sensor system; detecting a location of the person in the environment based on the data outputted by the sensor system; detecting a gaze of the person in the environment being directed at the autonomous vehicle based on the data outputted by the sensor system; and responsive to the gaze of the person being detected as directed at the autonomous vehicle, causing the projector to project a notification on a ground in the environment external to the autonomous vehicle, wherein the projector is caused to project the notification on the ground adjacent to the location of the person during a period of time. ### Claim 19 (depends on 18) The autonomous vehicle of claim 18, wherein the projector is caused to continue to project the notification on the ground adjacent to the location of the person during the period of time after the gaze of the person is detected as moving away from the autonomous vehicle. ### Claim 20 (depends on 18) The autonomous vehicle of claim 18, wherein the location of the person in the environment is detected over time, and wherein the projector is caused to project the notification on the ground adjacent to the location of the person during the period of time while one or more of the person moves within the environment during at least a portion of the period of time or the autonomous vehicle moves within the environment during at least a portion of the period of time.