# US 2019/0316913 A1 · Determining Autonomous Vehicle Status Based on Mapping of Crowdsourced Object Data

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0316913 A1
- Document ID: US20190316913A1
- Status: Published application
- Kind: A1
- Application number: 15/951,087
- Filed: 2018-04-11
- Published: 2019-10-17
- Earliest claimed: 2018-04-11
- Assignees: Micron Technology, Inc.
- Applicants: Micron Technology, Inc.
- Inventors: Gil Golov
- Web page: https://patentgrep.com/patent/US20190316913A1

## CPC

- G06F 16/29: Geographical information databases
- B60W 60/0053: {from vehicle to occupant}
- G01C 21/30: Map- or contour-matching
- G01C 21/3602: {Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera}
- G01C 21/3614: {through interaction with a road map, e.g. selecting a POI icon on a road map}
- G05D 1/246: using environment maps, e.g. simultaneous localisation and mapping [SLAM]
- G05D 1/2462: {using feature-based mapping}
- G05D 1/81: Handing over between on-board automatic and on-board manual control
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections
- G06N 3/09: Supervised learning
- G06N 3/092: Reinforcement learning
- G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle
- G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G06V 20/582: {of traffic signs}
- G06V 20/584: {of vehicle lights or traffic lights}
- G06V 20/588: {Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road}
- G07C 5/008: {communicating information to a remotely located station (transmission systems for measured values G08C)}
- G06N 3/02: Neural networks
- G06N 3/048: Activation functions
- G06N 3/08: Learning methods
- G06N 20/00: Machine learning
- G08G 1/166: {for active traffic, e.g. moving vehicles, pedestrians, bikes}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G07C: TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
- G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93})

## Abstract

A map in a cloud service stores physical objects previously detected by other vehicles that have previously traveled over the same road that a current vehicle is presently traveling on. New data received by the cloud service from the current vehicle regarding new objects that are being encountered by the current vehicle can be compared to the previous object data stored in the map. Based on this comparison, an operating status of the current vehicle is determined. In response to determining the status, an action such as terminating an autonomous navigation mode of the current vehicle is performed.

## Claims

### Claim 1

A method comprising:
receiving, by at least one processor, data regarding objects detected by a plurality of vehicles, the detected objects including a first object;
storing, by the at least one processor, a map comprising the detected objects, each object having an object type and a location;
subsequent to receiving the data regarding objects detected by the plurality of vehicles, receiving first data regarding objects detected by a first vehicle;
determining, based on comparing the received first data to the map, that the first vehicle has failed to detect the first object; and
in response to determining that the first vehicle has failed to detect the first object, performing an action.

### Claim 2 (depends on 1)

The method of claim 1, wherein the first object is a traffic sign, a traffic light, a road lane, or a physical structure.

### Claim 3 (depends on 1)

The method of claim 1, further comprising determining a location of the first vehicle, wherein determining that the first vehicle has failed to detect the first object includes comparing the location of the first vehicle to the location of the first object stored in the map.

### Claim 4 (depends on 1)

The method of claim 1, wherein the first vehicle is a vehicle other than the plurality of vehicles.

### Claim 5 (depends on 1)

The method of claim 1, further comprising analyzing the first data, wherein performing the action comprises configuring, based on analyzing the first data, at least one action performed by the first vehicle.

### Claim 6 (depends on 1)

The method of claim 1, wherein the first vehicle is an autonomous vehicle comprising a controller and a storage device, wherein the action comprises updating firmware of the controller, and wherein the updated firmware is stored in the storage device.

### Claim 7 (depends on 1)

The method of claim 1, further comprising training a computer model using at least one of supervised or unsupervised learning, wherein the training is done using data collected from the plurality of vehicles, and wherein determining that the first vehicle has failed to detect the first object is based at least in part on an output from the computer model.

### Claim 8 (depends on 1)

The method of claim 1, wherein the first data comprises image data obtained from at least one sensor of the first vehicle.

### Claim 9 (depends on 1)

The method of claim 1, further comprising analyzing the first data, wherein the first data comprises image data, and analyzing the first data comprises performing pattern recognition using the image data to determine a type of object detected by the first vehicle.

### Claim 10 (depends on 1)

The method of claim 1, further comprising providing the first data as an input to an artificial neural network model, wherein the action performed is based on an output from the artificial neural network model.

### Claim 11

A system comprising:
at least one processor; and
memory storing instructions configured to instruct the at least one processor to:
receive data regarding objects, each object detected by at least one of a plurality of vehicles, and the detected objects including a first object;
store, based on the received data, a map including the detected objects, each of the detected objects associated with a respective location;
receive first data regarding at least one object detected by a first vehicle;
determine, based on comparing the received first data to the map, that the first vehicle has failed to detect the first object; and
in response to determining that the first vehicle has failed to detect the first object, perform at least one action.

### Claim 12 (depends on 11)

The system of claim 11, wherein performing the at least one action comprises sending a communication to the first vehicle, the communication causing the first vehicle to perform at least one of deactivating an autonomous driving mode of the first vehicle or activating a backup navigation device of the first vehicle.

### Claim 13 (depends on 11)

The system of claim 11, wherein performing the at least one action comprises sending at least one communication to a computing device other than the first vehicle.

### Claim 14 (depends on 13)

The system of claim 13, wherein the computing device is a server that monitors a respective operating status for each of a plurality of vehicles.

### Claim 15 (depends on 13)

The system of claim 13, wherein the instructions are further configured to instruct the at least one processor to determine that an accident involving the first vehicle has occurred, and wherein the at least one communication to the computing device comprises data associated with operation of the first vehicle prior to the accident.

### Claim 16 (depends on 11)

The system of claim 11, wherein the instructions are further configured to instruct the at least one processor to compare a location of an object detected by the first vehicle to a location of the first object, wherein determining that the first vehicle has failed to detect the first object is based at least in part on comparing the location of the object detected by the first vehicle to the location of the first object.

### Claim 17 (depends on 11)

The system of claim 11, wherein the received data regarding objects detected by the plurality of vehicles includes data collected by a plurality of sensors for each of the vehicles.

### Claim 18 (depends on 11)

The system of claim 11, wherein performing the at least one action is based on an output from a machine learning model, and wherein the machine learning model is trained using training data, the training data comprising data collected by sensors of the plurality of vehicles.

### Claim 19

A non-transitory computer storage medium storing instructions which, when executed on a computing device, cause the computing device to perform a method comprising:
receiving data regarding objects detected by a plurality of vehicles;
storing a map including respective locations for each of the detected objects;
subsequent to receiving the data regarding objects detected by the plurality of vehicles, receiving first data regarding at least one object detected by a first vehicle, the first data comprising location data for the at least one object;
determining, based on comparing the received first data to the map, that the first data fails to match data for at least one object stored in the map; and
in response to determining that the first data fails to match data for at least one object stored in the map, performing an action.

### Claim 20 (depends on 19)

The non-transitory computer storage medium of claim 19, wherein the first data comprises data obtained from an artificial neural network model of the first vehicle.
