# US 10,769,452 B2 · Evaluating and presenting pick-up and drop-off locations in a situational-awareness view of an autonomous vehicle > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,769,452 B2 - Document ID: US10769452B2 - Status: Granted - Kind: B2 - Application number: 15/812,749 - Filed: 2017-11-14 - Published: 2018-05-17 - Granted: 2020-09-08 - Earliest claimed: 2016-11-14 - Assignees: Lyft, Inc. - Applicants: Lyft, Inc. - Inventors: Robert Earl Rasmusson, Jr.; Taggart Matthiesen; Craig Dehner; Linda Dong; Frank Taehyun Yoo; Karina van Schaardenburg; John Tighe; Matt Vitelli; Jisi Guo; Eli Guerron - Web page: https://patentgrep.com/patent/US10769452B2 ## CPC - G01C 21/3638: {including 3D objects and buildings (three dimensional [3D] modelling, e.g. data description of 3D objects G06T17/00; geographic models G06T17/05)} - B60W 60/00253: {Taxi operations} - B60W 60/0027: {using trajectory prediction for other traffic participants} - B62D 15/0285: {Parking performed automatically} - G01C 21/365: {Guidance using head up displays or projectors, e.g. virtual vehicles or arrows projected on the windscreen or on the road itself} - G05D 1/00: Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots (drive control systems specially adapted for autonomous road vehicles B60W60/00) - G05D 1/0044: {by providing the operator with a computer generated representation of the environment of the vehicle, e.g. virtual reality, maps (maps used for automatic navigation G05D1/0274; flight directors G01C23/005)} - G05D 1/0088: {characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours (using knowledge based models G06N5/00)} - G05D 1/0212: {with means for defining a desired trajectory (involving a plurality of land vehicles G05D1/0287)} - G05D 1/0246: {using a video camera in combination with image processing means} - G05D 1/0274: {using mapping information stored in a memory device (navigation using map-matching G01C21/30)} - G05D 1/221: Remote-control arrangements - G05D 1/2247: {providing the operator with simple or augmented images from one or more cameras} - G05D 1/225: operated by off-board computers - G05D 1/227: Handing over between remote control and on-board control; Handing over between remote control arrangements - G05D 1/246: using environment maps, e.g. simultaneous localisation and mapping [SLAM] - G05D 1/249: from positioning sensors located off-board the vehicle, e.g. from cameras - G05D 1/646: Following a predefined trajectory, e.g. a line marked on the floor or a flight path - G06F 18/24: Classification techniques - G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) - G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle - B60W 2420/403: Image sensing, e.g. optical camera - B60W 2420/408: Radar; Laser, e.g. lidar - B60W 2554/00: Input parameters relating to objects - B60W 2555/20: Ambient conditions, e.g. wind or rain - B60W 2556/10: Historical data - G05D 2101/10: using artificial intelligence [AI] techniques - G06T 2207/30252: Vehicle exterior; Vicinity of vehicle - G06T 2207/30256: Lane; Road marking - G06T 2207/30261: Obstacle - G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S) - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - B62D: MOTOR VEHICLES; TRAILERS (steering, or guiding on a desired track, of agricultural machines or implements A01B69/00; wheels, castors, axles, increasing wheel adhesion B60B; vehicle tyres, tyre inflation or tyre changing B60C; connections between vehicles of a train or the like B60D; vehicles for use on rail and road, amphibious or convertible vehicles B60F; suspension arrangements B60G; heating, cooling, ventilating or other air treating devices B60H; windows, windscreens, non-fixed roofs, doors or similar devices, protective coverings for vehicles not in use B60J; propulsion plant arrangements, auxiliary drives, transmissions, controls, instrumentation or dashboards B60K; electric equipment or propulsion of electrically-propelled vehicles B60L; power supply for electrically-propelled vehicles B60M; passenger accommodation not otherwise provided for B60N; adaptations for load transportation or to carry special loads or objects B60P; arrangement of signalling or lighting devices, the mounting or supporting thereof or circuits therefor, for vehicles in general B60Q; vehicles, vehicle fittings or vehicle parts not otherwise provided for B60R; servicing, cleaning, repairing, supporting, lifting, or manoeuvring, not otherwise provided for, B60S; brake arrangements, brake control systems or parts thereof B60T; air-cushion vehicles B60V; motor-cycles, accessories therefor B62J, B62K; testing of vehicles G01M) - G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract In one embodiment, a method includes identifying, based on map data, an area for pick-up or drop-off of a user by an autonomous vehicle. The method also includes determining, based on autonomous-vehicle sensor data that represents an external environment of the autonomous vehicle, one or more potential pick-up or drop-off locations within the area. The method also includes calculating, based at least in part on the autonomous-vehicle sensor data and historical data a viability score for each of the potential pick-up or drop-off locations. The historical data includes past pick-up or drop-off locations for one or more past users. The method also includes providing for display a visual representation of at least a portion of the area for pick-up or drop-off that indicates at least one of the one or more potential pick-up or drop-off locations. ## Claims ### Claim 1 A method comprising, by one or more computing devices: identifying, based on map data, an area for pick-up or drop-off of a user by a vehicle; determining, based on the map data and a location of the vehicle, that the vehicle is within a threshold distance of the area for pick-up or drop-off of the user; in response to the determination that the vehicle is within the threshold distance of the area, determining, based on vehicle sensor data that represents an external environment of the vehicle, one or more potential pick-up or drop-off locations within the area; calculating, based at least in part on the vehicle sensor data and historical data including past pick-up or drop-off events involving one or more past users for each potential pick-up or drop-off location, a viability score for each of the potential pick-up or drop-off locations; and providing for display a visual representation of at least a portion of the area for pick-up or drop-off that indicates at least one of the one or more potential pick-up or drop-off locations. ### Claim 2 (depends on 1) The method of claim 1, further comprising: selecting, based on the viability score or a user selection, one of the one or more potential pick-up or drop-off locations for pick-up or drop-off of the user by the vehicle; and providing an indication of the selected one of the one or more potential pick-up or drop-off locations to the vehicle for navigation. ### Claim 3 (depends on 2) The method of claim 2, wherein the indication comprises instructions for navigation to the selected one of the one or more potential pick-up or drop-off locations. ### Claim 4 (depends on 1) The method of claim 1, wherein the historical data includes past pick-up or drop-off events involving a current user of the vehicle. ### Claim 5 (depends on 1) The method of claim 1, further comprising: providing for display in the visual representation a visual indication of the viability score for each of one or more of the one or more potential pick-up or drop-off locations. ### Claim 6 (depends on 1) The method of claim 1, further comprising: providing a visual indication that a first one of the one or more potential pick-up or drop-off locations has a higher viability score than a second one of the one or more potential pick-up or drop-off locations. ### Claim 7 (depends on 6) The method of claim 6, wherein the visual indication comprises rendering the first one of the one or more potential pick-up or drop-off locations in a color that is different from the second one of the one or more potential pick-up or drop-off locations. ### Claim 8 (depends on 1) The method of claim 1, wherein the historical data comprises a number, a frequency, or a proportion of passengers that were previously picked-up or dropped-off at the one or more potential pick-up or drop-off locations during a particular time window. ### Claim 9 (depends on 1) The method of claim 1, wherein the viability score is further based on a characterization score that represents how suitable each of the one or more potential pick-up or drop-off locations is based on one or more physical characteristics of each of the one or more potential pick-up or drop-off locations. ### Claim 10 (depends on 9) The method of claim 9, wherein the characterization score is based at least in part on a size of the potential pick-up or drop-off location, a lighting measurement of the potential pick-up or drop-off location, a distance between the potential pick-up location and an origin associated with a current ride, or a distance between the potential drop-off location and a destination associated with a current ride. ### Claim 11 (depends on 9) The method of claim 9, wherein the viability score is based on a weighted composite of the characterization score and a historical score based on the historical data, the viability score being represented as VS=(CS)w+HS(v), wherein VS is the viability score, CS is the characterization score, HS is the historical score, w is a first weighting value assigned to CS, and v is a second weighting value assigned to HS. ### Claim 12 (depends on 1) The method of claim 1, wherein the display provides a situational-awareness view for a passenger of the vehicle. ### Claim 13 (depends on 1) The method of claim 1, wherein determining one or more potential pick-up or drop-off locations within the area comprises: determining a location of an object within a threshold distance of the vehicle; or determining at least one of a location or a size of one or more negative spaces within a space parameter defined for the vehicle, wherein the space parameter comprises one or more rules for determining whether a negative space within a threshold distance is acceptable for use as the one or more potential pick-up or drop-off locations. ### Claim 14 (depends on 13) The method of claim 13, wherein the one or more rules comprise restrictions based on: passenger safety; vehicle safety; pedestrian safety; or vehicle operation restrictions. ### Claim 15 (depends on 1) The method of claim 1, wherein calculating the viability score for each of the potential pick-up or drop-off locations comprises: determining a pick-up or drop-off preference of one or more of the past users for the area for pick-up or drop off of the user. ### Claim 16 A system comprising: one or more processors and one or more computer-readable non-transitory storage media coupled to one or more of the processors, the one or more computer-readable non-transitory storage media comprising instructions operable when executed by one or more of the processors to cause the system to perform operations comprising: identifying, based on map data, an area for pick-up or drop-off of a user by a vehicle; determining, based on the map data and a location of the vehicle, that the vehicle is within a threshold distance of the area for pick-up or drop-off of the user; in response to the determination that the vehicle is within the threshold distance of the area, determining, based on vehicle sensor data that represents an external environment of the vehicle, one or more potential pick-up or drop-off locations within the area; calculating, based at least in part on the vehicle sensor data and historical data including past pick-up or drop-off events involving one or more past users for each potential pick-up or drop-off location, a viability score for each of the potential pick-up or drop-off locations; and providing for display a visual representation of at least a portion of the area for pick-up or drop-off that indicates at least one of the one or more potential pick-up or drop-off locations. ### Claim 17 (depends on 16) The system of claim 16, wherein the processors are further operable when executing the instructions to perform operations comprising: selecting, based on the viability score or a user selection, one of the one or more potential pick-up or drop-off locations for pick-up or drop-off of the user by the vehicle; and providing an indication of the selected one of the one or more potential pick-up or drop-off locations to the vehicle for navigation. ### Claim 18 (depends on 16) The system of claim 16, wherein the processors are further operable when executing the instructions to perform operations comprising: providing for display in the visual representation a visual indication of the viability score for each of one or more of the one or more potential pick-up or drop-off locations. ### Claim 19 (depends on 16) The system of claim 16, wherein the processors are further operable when executing the instructions to perform operations comprising: providing a visual indication that a first one of the one or more potential pick-up or drop-off locations has a higher viability score than a second one of the one or more potential pick-up or drop-off locations. ### Claim 20 One or more computer-readable non-transitory storage media embodying software that is operable when executed to cause one or more processors to perform operations comprising: identifying, based on map data, an area for pick-up or drop-off of a user by a vehicle; determining, based on the map data and a location of the vehicle, that the vehicle is within a threshold distance of the area for pick-up or drop-off of the user; in response to the determination that the vehicle is within the threshold distance of the area, determining, based on vehicle sensor data that represents an external environment of the vehicle, one or more potential pick-up or drop-off locations within the area; calculating, based at least in part on the vehicle sensor data and historical data including past pick-up or drop-off events involving one or more past users for each potential pick-up or drop-off location, a viability score for each of the potential pick-up or drop-off locations; and providing for display a visual representation of at least a portion of the area for pick-up or drop-off that indicates at least one of the one or more potential pick-up or drop-off locations.