# US 2021/0279480 A1 · Data collection apparatus

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0279480 A1
- Document ID: US20210279480A1
- Status: Published application
- Kind: A1
- Application number: 17/150,431
- Filed: 2021-01-15
- Published: 2021-09-09
- Earliest claimed: 2020-03-09
- Assignees: Subaru Corporation
- Applicants: Subaru Corporation
- Inventors: Ikuo GOTO; Terunori KOBAYASHI; Kotaro UKI; Hideyuki TAKAO
- Web page: https://patentgrep.com/patent/US20210279480A1

## CPC

- G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle
- G06F 16/245: Query processing
- G06V 40/174: {Facial expression recognition}
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A data collection apparatus includes a processor and a storage. The processor is configured to: estimate an emotion of a person watching an own vehicle on the basis of an image data of a surrounding environment of the own vehicle; generate emotion data including an estimate of the emotion of the person watching the own vehicle; specify a state of the own vehicle viewed by the person watching the own vehicle; and correlate the emotion data with the specified state of the own vehicle. The storage is configured to be caused by the processor to store the emotion data correlated with the specified state of the own vehicle.

## Claims

### Claim 1

A data collection apparatus comprising:
a processor configured to
estimate an emotion of a person watching an own vehicle on a basis of an image data of a surrounding environment of the own vehicle,
generate emotion data including an estimate of the emotion of the person watching the own vehicle,
specify a state of the own vehicle viewed by the person watching the own vehicle, and
correlate the emotion data with the specified state of the own vehicle; and
a storage configured to be caused by the processor to store the emotion data correlated with the specified state of the own vehicle.

### Claim 2 (depends on 1)

The data collection apparatus according to claim 1, wherein
the processor is configured to
specify data on the surrounding environment of the own vehicle viewed by the person watching the own vehicle, and
correlate the emotion data with the data on the surrounding environment, and
cause the storage to store the emotion data correlated with the data on the surrounding environment.

### Claim 3 (depends on 2)

The data collection apparatus according to claim 2, wherein
the processor is configured to specify data on at least one of a building, a natural object, a road, another vehicle, or weather as the data on the surrounding environment of the own vehicle viewed by the person watching the own vehicle.

### Claim 4 (depends on 1)

The data collection apparatus according to claim 1, wherein
the processor is further configured to
specify a portion of the own vehicle that the person is watching on a basis of the image data,
correlate the emotion data with data on the portion of the own vehicle that the person is watching, and
cause the storage to store the emotion data correlated with the data on the portion of the own vehicle that the person is watching.

### Claim 5 (depends on 2)

The data collection apparatus according to claim 2, wherein
the processor is further configured to
specify a portion of the own vehicle that the person is watching on a basis of the image data,
correlate the emotion data with data on the portion of the own vehicle that the person is watching, and
cause the storage to store the emotion data correlated with the data on the portion of the own vehicle that the person is watching.

### Claim 6 (depends on 1)

The data collection apparatus according to claim 1, wherein
the processor is further configured to
specify an attribute of the person watching the own vehicle on a basis of the image data,
correlate the emotion data with data on the attribute of the person, and
cause the storage to store the emotion data correlated with the data on the attribute of the person.

### Claim 7 (depends on 2)

The data collection apparatus according to claim 2, wherein
the processor is further configured to
specify an attribute of the person watching the own vehicle on a basis of the image data,
correlate the emotion data with data on the attribute of the person, and
cause the storage to store the emotion data correlated with the data on the attribute of the person.

### Claim 8 (depends on 6)

The data collection apparatus according to claim 6, wherein,
in a case where the person watching the own vehicle is an occupant in another vehicle, the data on the attribute of the person includes data on a speed of the occupant relative to the own vehicle.

### Claim 9 (depends on 7)

The data collection apparatus according to claim 7, wherein,
in a case where the person watching the own vehicle is an occupant in another vehicle, the data on the attribute of the person includes data on a speed of the occupant relative to the own vehicle.

### Claim 10 (depends on 1)

The data collection apparatus according to claim 1, wherein
the processor is further configured to
correlate the emotion data with data on at least one of season, calendar, time, or a position on map data, and
cause the storage to store the emotion data correlated with the data on at least one of the season, the calendar, the time, or the position on map data.

### Claim 11 (depends on 2)

The data collection apparatus according to claim 2, wherein
the processor is further configured to
correlate the emotion data with data on at least one of season, calendar, time, or a position on map data, and
cause the storage to store the emotion data correlated with the data on at least one of the season, the calendar, the time, or the position on map data.

### Claim 12 (depends on 1)

The data collection apparatus according to claim 1, wherein
the processor is configured to cause the storage to store the emotion data of the person as time-series data.

### Claim 13 (depends on 2)

The data collection apparatus according to claim 2, wherein
the processor is configured to cause the storage to store the emotion data of the person as time-series data.

### Claim 14 (depends on 1)

The data collection apparatus according to claim 1, wherein
the processor is configured to
calculate a degree of interest in the own vehicle of the person watching the own vehicle on a basis of the image data, and
estimate the emotion of the person in a case where the degree of interest is equal to or greater than a predetermined threshold.

### Claim 15 (depends on 2)

The data collection apparatus according to claim 2, wherein
the processor is configured to
calculate a degree of interest in the own vehicle of the person watching the own vehicle on a basis of the image data, and
estimate the emotion of the person in a case where the degree of interest is equal to or greater than a predetermined threshold.

### Claim 16 (depends on 14)

The data collection apparatus according to claim 14, wherein
the degree of interest in the own vehicle comprises a rate of duration time in which a line of sight of the person is directed to the own vehicle relative to a predetermined time.

### Claim 17 (depends on 15)

The data collection apparatus according to claim 15, wherein
the degree of interest in the own vehicle comprises a rate of duration time in which a line of sight of the person is directed to the own vehicle relative to a predetermined time.
