# US 2025/0095424 A1 · Method Of Operating A Data Recording Device And System

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0095424 A1
- Document ID: US20250095424A1
- Status: Published application
- Kind: A1
- Application number: 18/814,680
- Filed: 2024-08-26
- Published: 2025-03-20
- Earliest claimed: 2023-09-14
- Assignees: Aptiv Technologies AG
- Applicants: Aptiv Technologies AG
- Inventors: Dennis MÜLLER; Ori MAOZ; Michael ARNOLD; Lutz ROESE-KOERNER
- Web page: https://patentgrep.com/patent/US20250095424A1

## CPC

- G07C 5/085: {using electronic data carriers}
- G06N 3/02: Neural networks
- G07C 5/008: {communicating information to a remotely located station (transmission systems for measured values G08C)}
- G07C 5/0825: {using optical means}
- G06N 3/08: Learning methods
- G07C: TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A method of operating a data recording device for Autonomous Driving, or Advanced Driver Assistant Systems applications for vehicles is provided. The data recording device comprises a data logging unit, a computing unit, a storage unit and a relevancy determination unit including at least one relevancy determination module. The storage unit includes a circular buffer and a persistent storage region. The method comprises retrieving sensor data acquired by sensors, storing the sensor data in the circular buffer, retrieving and storing perception results or neural network embeddings from the sensor data and deciding whether the retrieved perception results or neural network embeddings are considered relevant or irrelevant, and triggering a transfer of the sensor data from the circular buffer to the persistent storage region in case the perception results or neural network embeddings corresponding to the sensor data are considered relevant.

## Claims

### Claim 1

A method of operating a data recording device for Autonomous Driving or Advanced Driver Assistant Systems applications for vehicles, the data recording device comprising:
a data logging unit, a computing unit, a storage unit and a relevancy determination unit including at least one relevancy determination module, the storage unit including a circular buffer and a persistent storage region,
wherein the method comprises the steps of:
retrieving, with the data logging unit and the computing unit, sensor data acquired by in-vehicle sensors,
storing, with the data logging unit, the sensor data in the circular buffer,
analyzing, with the computing unit, analyzing the sensor data and using the sensor data to derive at least one of perception results or neural network embeddings from the retrieved sensor data,
retrieving and storing, with the relevancy determination unit, the at least one of the perception results or the neural network embeddings,
deciding, with the relevancy determination unit, whether the retrieved at least one of the perception results or the neural network embeddings are considered relevant or irrelevant,
triggering, with the relevancy determination unit, a transfer of the sensor data from the circular buffer to the persistent storage region for persistent storage of the sensor data in case the at least one of the perception results or the neural network embeddings corresponding to the sensor data are considered relevant by the relevancy determination unit.

### Claim 2 (depends on 1)

The method of claim 1, wherein the deciding, with the at least one relevancy determination module of the relevancy determination unit, whether the retrieved at least one of the perception results or the neural network embeddings are considered relevant or irrelevant includes applying a predefined rule set to the retrieved perception results.

### Claim 3 (depends on 1)

The method of claim 1, further comprising reducing, with the computing unit, the dimensionality of the neural network embeddings by Principal Component Analysis, transmitting, with the computing unit, the dimensionality reduced neural network embeddings to the relevancy determination unit, and deciding, with the relevancy determination unit, on the relevancy based on the dimensionality reduced neural network embeddings.

### Claim 4 (depends on 1)

The method of claim 1, wherein the data recording device further comprises a display unit provided with a display and a touch panel, and wherein the method further comprises transmitting, with the computing unit, the perception results to the display unit, displaying, with the display unit, the perception results on the display, recognizing, with the display unit, a manual user interaction on the touch panel responsive to the perception results being displayed on the display, and, upon recognizing the manual user interaction, transmitting, with the display unit, the information to the relevancy determination unit that the perception results are relevant.

### Claim 5 (depends on 1)

The method of claim 1, further comprising: receiving, with the computing unit, text-based user-queries, encoding, with the computing unit, the retrieved text-based user queries into neural network embeddings, transmitting, with the computing unit, the neural network embeddings to the relevancy determination unit, and deciding, with the at least one relevancy determination module, whether neural network embeddings derived from sensor data are considered relevant or irrelevant based on a similarity measure, using cosine-similarity, between the neural network embeddings derived from sensor data and the encoded user query neural network embeddings.

### Claim 6 (depends on 1)

The method of claim 1, wherein the at least one relevancy determination module bases its decision whether the at least one of the retrieved perception results or the neural network embeddings are considered relevant or irrelevant on a comparison of the neural network embeddings with neural network embeddings previously stored in the relevancy determination unit.

### Claim 7 (depends on 6)

The method of claim 6, wherein the at least one relevancy determination module performs the following:
a) retrieving a predetermined number of neural network embeddings derived from sensor data taken at consecutive times,
b) determining, for each of the neural network embeddings, nearest neighbors within the whole dataset of neural network embeddings retrieved by the at least one relevancy determination module,
c) at least one of determining, for each of the neural network embeddings, a distance to the nearest neighbors or evaluating a density of the neural network embedding within the whole dataset of neural network embeddings retrieved by the relevancy determination module by applying a “k-nearest-neighbors” density estimation method,
d) calculating at least one of average distances or average densities for temporary coherent subsets of the predetermined number of neural network embeddings, the at least one of the average distances the average densities being calculated as a running mean of at least one of the determined distance or the density measurements for the smaller subsets of the predetermined number of neural network embeddings,
e) evaluating the subset having at least one of an average distance larger than a distance threshold or an average density smaller than a density threshold, and considering the respective subset as relevant,
f) continue with step a).

### Claim 8 (depends on 1)

A method of operating a system comprising a data center and a plurality of data recording devices configured to perform the method of claim 1, the data center comprising a global data base and a global relevancy determination unit, wherein the method comprises the steps of:
exchancing, with the data center, data with each of the plurality of data recording devices having over-the-air transmission capabilities,
at least one of (i) querying and retrieving, with the data center, at least one of the sensor data, the perception results, or the neural network embeddings from each of the plurality of data recording devices in regular time intervals, in a continuous manner, or (ii) querying and retrieving, with the data center, an updated predefined rule set for deciding whether at least one of the perception results or the neural network embeddings are considered relevant or irrelevant in regular time intervals, in a continuous manner,
storing, with the data center the data in the global data base as at least one of global sensor data, global perception results, or global neural network embeddings,
analyzing, with the global relevancy determination unit, the global data base and providing, with the global relevancy determination unit, estimates for at least one of the global perception densities or the global embedding densities,
deciding, with the system, whether at least one of the retrieved perception results or the neural network embeddings of a particular data recording device are considered relevant or irrelevant based on at least one of the perception results, the neural network embeddings of the particular data recording device, the global perception results, global neural network embeddings, or the updated predefined rule set.

### Claim 9 (depends on 8)

The method of claim 8, further comprising synchronizing, with the relevancy determination unit, the particular data recording device at least one of the predetermined rule set, the global perception densities, or the global embedding densities with the global relevancy determination unit, and deciding, with the relevancy determination unit of the particular data recording device, whether at least one of the perception results or the neural network embeddings of the particular data recording device are considered relevant or irrelevant by using at least one of the updated predefined rule set, the global perception densities, or the global embedding densities together with at least one relevancy determination module.

### Claim 10 (depends on 8)

The method of claim 8, further comprising deciding, with the global relevancy determination unit, whether at least one of the perception results or the neural network embeddings of the particular data recording device are considered relevant or irrelevant by using at least one of the updated predefined rule set, the global perception densities, or the global embedding densities together with at least one relevancy determination module and transmitting, with the global relevancy determination unit, the relevancy decision to the relevancy determination unit of the particular data recording device.

### Claim 11 (depends on 8)

The method of claim 8, wherein the data center further comprises an upload unit which queries and receives sensor data considered relevant and stored in the persistent storage region of each of the plurality of data recording devices and transmits said the data to the global data base, wherein the sensor data is queried after predetermined time intervals of one of 12, 24, 48 or 72 hours.

### Claim 12 (depends on 1)

A data recording device configured to perform the method of claim 1.

### Claim 13 (depends on 8)

A system configured to perform the method of claim 8.

### Claim 14

(canceled)

### Claim 15

(canceled)
