# US 11,897,501 B2 · ADS perception development

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,897,501 B2
- Document ID: US11897501B2
- Status: Granted
- Kind: B2
- Application number: 17/741,599
- Filed: 2022-05-11
- Published: 2022-11-24
- Granted: 2024-02-13
- Earliest claimed: 2021-05-24
- Assignees: Zenseact AB
- Applicants: Zenseact AB
- Inventors: Magnus Gyllenhammar; Carl Zandén; Majid Khorsand Vakilzadeh
- Web page: https://patentgrep.com/patent/US11897501B2

## CPC

- B60W 60/001: {Planning or execution of driving tasks}
- G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle
- B60W 40/12: related to parameters of the vehicle itself {, e.g. tyre models}
- G06F 18/217: {Validation; Performance evaluation; Active pattern learning techniques}
- G07C 5/008: {communicating information to a remotely located station (transmission systems for measured values G08C)}
- B60W 2420/403: Image sensing, e.g. optical camera
- B60W 2554/4049: Relationship among other objects, e.g. converging dynamic objects
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G07C: TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE

## Abstract

An assessment system for performance evaluation and updating of a PMUD of an ego-vehicle. The assessment system obtains world view data from a perception module configured to generate the world view data based on sensor data obtained from vehicle-mounted sensors; obtains other world view data generated by another perception module; forms a joint world view by matching the world view data the other world view data; and obtains perception data based on a perception model and sensor data obtained from one or more vehicle-mounted sensors. The assessment system further matches the perception data to the formed joint world view; evaluates the obtained perception data in reference to the joint world view to determine an estimation deviation in an identified match between the perceptive parameter of the perception data and a corresponding perceptive parameter in the joint world view; and updates parameters of the perception model based on the estimation deviation.

## Claims

### Claim 1

A method performed by an assessment system for performance evaluation and updating of a Perception Module Under Development, PMUD, of an ego-vehicle equipped with an Automated Driving System, ADS, the method comprising:
obtaining world view data from an ego-vehicle perception module configured to generate the world view data based on sensor data obtained from one or more vehicle-mounted sensors;
obtaining other world view data generated by another perception module of at least a first other vehicle in a surrounding environment of the ego-vehicle;
forming a joint world view by matching the world view data and the other world view data in a common reference system;
obtaining perception data from the PMUD, the PMUD being configured to generate the perception data based on a perception model and the sensor data obtained from the one or more vehicle-mounted sensors, the perception data being indicative of a perceptive parameter of the surrounding environment of the ego-vehicle;
matching the perception data to the formed joint world view;
evaluating the obtained perception data in reference to the joint world view in order to determine an estimation deviation in an identified match between the perceptive parameter of the perception data and a corresponding perceptive parameter in the joint world view;
updating one or more parameters of the perception model based on the estimation deviation; and
transmitting the one or more updated parameters of the perception model of the PMUD to a remote entity.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the world view data and the other world view data comprise respective data adequacy attributes indicating respective adequacy of contributing to the joint world view, the forming of the joint world view comprising forming the joint world view with the extent of contribution from the world view data and the other world view data respectively being based on the respective data adequacy attributes.

### Claim 3 (depends on 2)

The method according to claim 2, wherein the obtaining other world view data comprises obtaining the other world view data from the at least first other vehicle based on the at least first other vehicle fulfilling vehicle adequacy criteria comprising conditions of under what circumstances another vehicle is deemed adequate to obtain other world view data from.

### Claim 4 (depends on 3)

The method according to claim 3, further comprising:
receiving a set of globally updated parameters of the perception model of the PMUD from a remote entity, wherein the set of globally updated parameters are based on information obtained from a plurality of vehicles comprising the PMUD; and
updating the perception model of the PMUD based on the received set of globally updated parameters.

### Claim 5 (depends on 2)

The method according to claim 2, further comprising:
receiving a set of globally updated parameters of the perception model of the PMUD from a remote entity, wherein the set of globally updated parameters are based on information obtained from a plurality of vehicles comprising the PMUD; and
updating the perception model of the PMUD based on the received set of globally updated parameters.

### Claim 6 (depends on 2)

The method according to claim 2, wherein the perceptive parameter of the surrounding environment of the ego-vehicle comprises at least one of:
an object detection estimation;
an object classification estimation;
an object state estimation;
a road reference estimation;
a free-space estimation;
an object trajectory estimation;
a drivable-area estimation;
an object property estimation;
an ego vehicle property estimation;
an Operating Design Domain, ODD, parameter estimation;
a road segment class estimation; and
an electronic horizon estimation.

### Claim 7 (depends on 1)

The method according to claim 1, wherein the obtaining other world view data comprises obtaining the other world view data from one or more other vehicles fulfilling vehicle adequacy criteria comprising conditions of under what circumstances another vehicle is deemed adequate to obtain other world view data from.

### Claim 8 (depends on 1)

The method according to claim 1, wherein the perceptive parameter of the surrounding environment of the ego-vehicle comprises at least one of:
an object detection estimation;
an object classification estimation;
an object state estimation;
a road reference estimation;
a free-space estimation;
an object trajectory estimation;
a drivable-area estimation;
an object property estimation;
an ego vehicle property estimation;
an Operating Design Domain, ODD, parameter estimation;
a road segment class estimation; and
an electronic horizon estimation.

### Claim 9 (depends on 1)

The method according to claim 1, further comprising:
receiving a set of globally updated parameters of the perception model of the PMUD from the remote entity, wherein the set of globally updated parameters are based on information obtained from a plurality of vehicles comprising the PMUD; and
updating the perception model of the PMUD based on the received set of globally updated parameters.

### Claim 10 (depends on 1)

The method according to claim 1, further comprising:
receiving a set of globally updated parameters of the perception model of the PMUD from a remote entity, wherein the set of globally updated parameters are based on information obtained from a plurality of vehicles comprising the PMUD; and
updating the perception model of the PMUD based on the received set of globally updated parameters.

### Claim 11

An assessment system for performance evaluation and updating of a Perception Module Under Development, PMUD, of an ego-vehicle equipped with an Automated Driving System, ADS, the assessment system comprising:
a world view obtaining unit configured to obtain world view data from an ego-vehicle perception module configured to generate the world view data based on sensor data obtained from one or more vehicle-mounted sensors;
an other world view obtaining unit configured to obtain other world view data generated by another perception module of at least a first other vehicle in a surrounding environment of the ego-vehicle;
a joint world view forming unit configured to form a joint world view by matching the world view data and the other world view data in a common reference system;
a PMUD data obtaining unit configured to obtain perception data from the PMUD, the PMUD being configured to generate the perception data based on a perception model and sensor data obtained from one or more vehicle-mounted sensors, the perception data being indicative of a perceptive parameter of the surrounding environment of the ego-vehicle;
a matching unit configured to match the perception data to the formed joint world view;
an estimation deviation determining unit configured to evaluate the obtained perception data in reference to the joint world view in order to determine an estimation deviation in an identified match between the perceptive parameter of the perception data and a corresponding perceptive parameter in the joint world view;
a parameter updating unit configured to update one or more parameters of the perception model based on the estimation deviation; and
a parameter transmitting unit configured to transmit the one or more updated parameters of the perception model of the PMUD to a remote entity.

### Claim 12 (depends on 11)

The assessment system according to claim 11, wherein the world view data and the other world view data comprise respective data adequacy attributes indicating respective adequacy of contributing to the joint world view, the joint world view forming unit being configured to form the joint world view with extent of contribution from the world view data and the other world view data respectively being based on the respective data adequacy attributes.

### Claim 13 (depends on 12)

The assessment system according to claim 12, wherein the other world view obtaining unit is configured to obtain the other world view data from the at least first other vehicle based on the at least first other vehicle fulfilling vehicle adequacy criteria comprising conditions of under what circumstances an other vehicle is deemed adequate to obtain the other world view data.

### Claim 14 (depends on 11)

The assessment system according to claim 11, wherein the other world view obtaining unit is configured to obtain the other world view data from the at least first other vehicle based on the at least first other vehicle fulfilling vehicle adequacy criteria comprising conditions of under what circumstances an other vehicle is deemed adequate to obtain the other world view data.

### Claim 15 (depends on 11)

The assessment system according to claim 11, wherein the perceptive parameter of the surrounding environment of the ego-vehicle comprises at least one of:
an object detection estimation;
an object classification estimation;
an object state estimation;
a road reference estimation;
a free-space estimation;
an object trajectory estimation;
a drivable-area estimation;
an object property estimation;
an ego vehicle property estimation;
an Operating Design Domain, ODD, parameter estimation;
a road segment class estimation; and
an electronic horizon estimation.

### Claim 16 (depends on 11)

The assessment system according to claim 11, further comprising:
a parameter receiving unit configured to receive a set of globally updated parameters of the perception model of the PMUD from a remote entity, wherein the set of globally updated parameters are based on information obtained from a plurality of vehicles comprising the PMUD; and
a model updating unit configured to update the perception model of the PMUD based on the received set of globally updated parameters.

### Claim 17 (depends on 11)

The assessment system according to claim 11, wherein the assessment system is comprised in the ego-vehicle.

### Claim 18

A computer storage medium storing a computer program containing computer program code that when executed causes one of a computer and a processor to perform a method for performance evaluation and updating of a Perception Module Under Development, PMUD, of an ego-vehicle equipped with an Automated Driving System, ADS, the method comprising:
obtaining world view data from an ego-vehicle perception module configured to generate the world view data based on sensor data obtained from one or more vehicle-mounted sensors;
obtaining other world view data generated by another perception module of at least a first other vehicle in a surrounding environment of the ego-vehicle;
forming a joint world view by matching the world view data and the other world view data in a common reference system;
obtaining perception data from the PMUD, the PMUD being configured to generate the perception data based on a perception model and sensor data obtained from one or more vehicle-mounted sensors, the perception data being indicative of a perceptive parameter of the surrounding environment of the ego-vehicle;
matching the perception data to the formed joint world view;
evaluating the obtained perception data in reference to the joint world view in order to determine an estimation deviation in an identified match between the perceptive parameter of the perception data and a corresponding perceptive parameter in the joint world view;
updating one or more parameters of the perception model based on the estimation deviation; and
transmitting the one or more updated parameters of the perception model of the PMUD to a remote entity.