# US 10,372,129 B1 · System and method of providing recommendations to users of vehicles

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,372,129 B1
- Document ID: US10372129B1
- Status: Granted
- Kind: B1
- Application number: 15/488,836
- Filed: 2017-04-17
- Granted: 2019-08-06
- Earliest claimed: 2010-10-05
- Assignees: Waymo LLC
- Applicants: Waymo LLC
- Inventors: Christopher Paul Urmson; Dmitri A. Dolgov; Andrew Hughes Chatham; Philip Nemec
- Web page: https://patentgrep.com/patent/US10372129B1

## CPC

- B60R 1/23: with a predetermined field of view
- B60R 1/00: Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
- B60T 7/22: initiated by contact of vehicle, e.g. bumper, with an external object, e.g. another vehicle {, or by means of contactless obstacle detectors mounted on the vehicle}
- B60T 8/00: Arrangements for adjusting wheel-braking force to meet varying vehicular or ground-surface conditions, e.g. limiting or varying distribution of braking force (by changing number of effective brake cylinders in power brake systems B60T17/10)
- B60T 8/17: Using electrical or electronic regulation means to control braking {(detecting or indicating faulty operation B60T8/885)}
- B60T 8/885: {using electrical circuitry}
- B60T 17/18: Safety devices; Monitoring
- B60T 17/221: {Procedure or apparatus for checking or keeping in a correct functioning condition of brake systems (hydraulic pressure systems in general F15B19/00, F15B21/04; testing structures or apparatus G01M)}
- B60W 30/08: {Active safety systems} predicting or avoiding probable or impending collision {or attempting to minimise its consequences}
- B60W 40/06: Road conditions
- B60W 50/14: Means for informing the driver, warning the driver or prompting a driver intervention
- B60W 60/0013: {specially adapted for occupant comfort}
- G01C 21/3617: {using user history, behaviour, conditions or preferences, e.g. predicted or inferred from previous use or current movement}
- G01S 17/86: Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
- G01S 17/931: of land vehicles
- G05D 1/00: Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots (drive control systems specially adapted for autonomous road vehicles B60W60/00)
- G05D 1/0055: {with safety arrangements}
- G05D 1/0088: {characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours (using knowledge based models G06N5/00)}
- G05D 1/021: {specially adapted to land vehicles}
- G05D 1/0212: {with means for defining a desired trajectory (involving a plurality of land vehicles G05D1/0287)}
- G05D 1/0214: {in accordance with safety or protection criteria, e.g. avoiding hazardous areas (monitoring the location of vehicles within a certain area, e.g. forbidden or allowed areas, in traffic control systems for road vehicles G08G1/13)}
- G05D 1/0276: {using signals provided by a source external to the vehicle (involving a plurality of vehicles G05D1/0287; automatically controlling vehicle speed responsive to externally generated signals B60K31/0058)}
- G05D 1/81: Handing over between on-board automatic and on-board manual control
- G06Q 10/02: Reservations, e.g. for tickets, services or events
- G06Q 30/02: Marketing; Price estimation or determination; Fundraising
- G06Q 30/0207: Discounts or incentives, e.g. coupons or rebates
- G06Q 40/08: Insurance
- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- G06T 7/223: using block-matching
- G06T 7/231: using full search
- G06T 7/521: from laser ranging, e.g. using interferometry; from the projection of structured light
- G06T 7/74: {involving reference images or patches}
- G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G06V 20/64: Three-dimensional [3D] objects
- G07C 9/00563: {using personal physical data of the operator, e.g. finger prints, retinal images, voicepatterns}
- B60R 2300/30: characterised by the type of image processing
- B60T 2201/022: Collision avoidance systems
- B60T 2210/32: Vehicle surroundings
- B60T 2270/406: Test-mode; Self-diagnosis
- B60W 30/186: excessive wear or burn out of friction elements, e.g. clutches
- B60W 2050/0292: {Fail-safe or redundant systems, e.g. limp-home or backup systems}
- B60W 2420/403: Image sensing, e.g. optical camera
- B60W 2420/408: Radar; Laser, e.g. lidar
- B60W 2540/043: Identity of occupants
- B60W 2540/30: Driving style
- B60W 2552/05: Type of road, e.g. motorways, local streets, paved or unpaved roads
- B60W 2555/60: Traffic rules, e.g. speed limits or right of way
- B60W 2556/10: Historical data
- B60W 2556/50: of positioning data, e.g. GPS [Global Positioning System] data
- B62D 6/00: Arrangements for automatically controlling steering depending on driving conditions sensed and responded to, e.g. control circuits (means for initiating a change in direction B62D1/00; steering valves B62D5/06; combined with means for inclining the vehicle body or wheels on bends B62D9/00)
- G01S 13/865: {Combination of radar systems with lidar systems}
- G01S 13/867: {Combination of radar systems with cameras}
- G05B 2219/2637: Vehicle, car, auto, wheelchair
- G05D 1/024: {in combination with a laser (lasers per se H01S)}
- G05D 1/0246: {using a video camera in combination with image processing means}
- G05D 1/0257: {using a radar (radar systems designed for anti-collision purposes between land vehicles or between land vehicle and fixed obstacles G01S13/931)}
- G05D 1/0274: {using mapping information stored in a memory device (navigation using map-matching G01C21/30)}
- G05D 1/0278: {using satellite positioning signals, e.g. GPS}
- G06T 2207/10004: Still image; Photographic image
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/30236: Traffic on road, railway or crossing
- G06T 2207/30252: Vehicle exterior; Vicinity of vehicle
- G06T 2207/30261: Obstacle
- G06V 10/255: {Detecting or recognising potential candidate objects based on visual cues, e.g. shapes}
- G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle
- G06V 20/588: {Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road}
- B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60T: VEHICLE BRAKE CONTROL SYSTEMS OR PARTS THEREOF; BRAKE CONTROL SYSTEMS OR PARTS THEREOF, IN GENERAL (electrodynamic brake systems for vehicle, in general B60L; brakes per se, i.e. devices where braking effect occurs, including ultimate brake actuators, F16D); ARRANGEMENT OF BRAKING ELEMENTS ON VEHICLES IN GENERAL; PORTABLE DEVICES FOR PREVENTING UNWANTED MOVEMENT OF VEHICLES; VEHICLE MODIFICATIONS TO FACILITATE COOLING OF BRAKES
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G07C: TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
- B62D: MOTOR VEHICLES; TRAILERS (steering, or guiding on a desired track, of agricultural machines or implements A01B69/00; wheels, castors, axles, increasing wheel adhesion B60B; vehicle tyres, tyre inflation or tyre changing B60C; connections between vehicles of a train or the like B60D; vehicles for use on rail and road, amphibious or convertible vehicles B60F; suspension arrangements B60G; heating, cooling, ventilating or other air treating devices B60H; windows, windscreens, non-fixed roofs, doors or similar devices, protective coverings for vehicles not in use B60J; propulsion plant arrangements, auxiliary drives, transmissions, controls, instrumentation or dashboards B60K; electric equipment or propulsion of electrically-propelled vehicles B60L; power supply for electrically-propelled vehicles B60M; passenger accommodation not otherwise provided for B60N; adaptations for load transportation or to carry special loads or objects B60P; arrangement of signalling or lighting devices, the mounting or supporting thereof or circuits therefor, for vehicles in general B60Q; vehicles, vehicle fittings or vehicle parts not otherwise provided for B60R; servicing, cleaning, repairing, supporting, lifting, or manoeuvring, not otherwise provided for, B60S; brake arrangements, brake control systems or parts thereof B60T; air-cushion vehicles B60V; motor-cycles, accessories therefor B62J, B62K; testing of vehicles G01M)
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)

## Abstract

A system and method are arranged to provide recommendations to a user of a vehicle. In one aspect, the vehicle navigates in an autonomous mode and the sensors provide information that is based on the location of the vehicle and output from sensors directed to the environment surrounding the vehicle. In further aspects, both current and previous sensor data is used to make the recommendations, as well as data based on the sensors of other vehicles.

## Claims

### Claim 1

A method of generating recommendations for destination locations, the method comprising:
maneuvering, by one or more processors, a vehicle in an autonomous driving mode at a particular location at a first point in time;
identifying, by the one or more processors, a particular point of interest is proximate to the particular location;
identifying, by the one or more processors, a period of time during which a passenger stayed at the particular point of interest at a second point in time earlier than the first point in time;
determining, by the one or more processors, whether the period of time meets a threshold period of time; and
when the period of time meets the threshold period of time, generating, by the one or more processors, a recommendation suggesting that the vehicle transport the passenger to the particular point of interest; and
providing, by the one or more processors, the recommendation to the passenger.

### Claim 2 (depends on 1)

The method of claim 1, wherein the particular point of interest is a restaurant.

### Claim 3 (depends on 1)

The method of claim 1, further comprising, after providing the recommendation, automatically maneuvering the vehicle in the autonomous driving mode to the particular point of interest.

### Claim 4 (depends on 1)

The method of claim 1, wherein the threshold period of time is at least 30 minutes.

### Claim 5 (depends on 1)

The method of claim 1, further comprising, prior to generating the recommendation:
receiving instructions to maneuver the vehicle in order to transport the passenger to a particular destination location; and
maneuvering the vehicle in the autonomous driving mode along a first route towards the particular destination location based on the instructions, wherein maneuvering the vehicle at the particular location at the first point in time corresponds to maneuvering the vehicle along the first route.

### Claim 6 (depends on 5)

The method of claim 5, wherein identifying the particular point of interest is further based on a second route traveled by the vehicle at a third point in time that is at least one day prior to the first point in time, the second route being different from the first route.

### Claim 7 (depends on 5)

The method of claim 5, wherein identifying the particular point of interest is further based on a second route traveled by the vehicle at a third point in time that is at least one day prior to the first point in time, the second route being a same route as the first route.

### Claim 8 (depends on 5)

The method of claim 5, wherein identifying the particular point of interest is further based on a second route traveled by the vehicle at a third point in time that is at least one day prior to the first point in time, and the second route at least partially overlaps with the first route at the first point in time.

### Claim 9 (depends on 5)

The method of claim 5, wherein generating the recommendation is further based on the particular destination location.

### Claim 10 (depends on 1)

The method of claim 1, wherein identifying the particular point of interest of is further based on pre-stored information identifying that the vehicle has visited the particular point of interest in the past.

### Claim 11 (depends on 1)

The method of claim 1, wherein the period of time corresponds to a period of time during which the vehicle stayed at the particular point of interest.

### Claim 12

A system for generating recommendations for destination locations, the system comprising one or more processors configured to:
maneuver a vehicle in an autonomous driving mode at a particular location at a first point in time;
identify a particular point of interest is proximate to the particular location;
identify a period of time during which a passenger stayed at the particular point of interest at a second point in time earlier than the first point in time;
determine whether the period of time meets a threshold period of time; and
when the period of time meets the threshold period of time, generate a recommendation suggesting that the vehicle transport the passenger to the particular point of interest; and
providing, by the one or more processors, the recommendation to the passenger.

### Claim 13 (depends on 12)

The system of claim 12, wherein the one or more processors are further configured to, after providing the recommendation, automatically maneuver the vehicle in the autonomous driving mode to the particular point of interest.

### Claim 14 (depends on 12)

The system of claim 12, wherein the threshold period of time is at least 30 minutes.

### Claim 15 (depends on 12)

The system of claim 12, wherein the one or more processors are further configured to, prior to generating the recommendation:
receive instructions to maneuver the vehicle in order to transport the passenger to a particular destination location; and
maneuver the vehicle in the autonomous driving mode along a first route towards the particular destination location based on the instructions, wherein maneuvering the vehicle at the particular location at the first point in time corresponds to maneuvering the vehicle along the first route.

### Claim 16 (depends on 15)

The system of claim 15, wherein identifying the particular point of interest is further based on a second route traveled by the vehicle at a third point in time that is at least one day prior to the first point in time, the second route being different from the first route.

### Claim 17 (depends on 15)

The system of claim 15, wherein identifying the particular point of interest is further based on a second route traveled by the vehicle at a third point in time that is at least one day prior to the first point in time, the second route being a same route as the first route.

### Claim 18 (depends on 15)

The system of claim 15, wherein identifying the particular point of interest is further based on a second route traveled by the vehicle at a third point in time that is at least one day prior to the first point in time, and the second route at least partially overlaps with the first route at the first point in time.

### Claim 19 (depends on 15)

The system of claim 15, wherein generating the recommendation is further based on the particular destination location.

### Claim 20 (depends on 12)

The system of claim 12, further comprising the vehicle.