# US 2024/0199032 A1 · Electronic device, method for controlling electronic device, and program

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0199032 A1
- Document ID: US20240199032A1
- Status: Published application
- Kind: A1
- Application number: 18/555,949
- Filed: 2022-04-07
- Published: 2024-06-20
- Earliest claimed: 2021-04-27
- Assignees: KYOCERA Corporation
- Applicants: KYOCERA Corporation
- Inventors: Yusuke NISHII; Jungo MIYAZAKI
- Web page: https://patentgrep.com/patent/US20240199032A1

## CPC

- B60W 40/08: related to drivers or passengers
- G06T 7/73: using feature-based methods
- A61B 5/163: {by tracking eye movement, gaze, or pupil change}
- A61B 5/18: for vehicle drivers {or machine operators}
- A61B 3/113: for determining or recording eye movement
- A61B 2503/22: Motor vehicles operators, e.g. drivers, pilots, captains
- A61B 2560/0252: using ambient temperature
- B60W 2540/225: Direction of gaze
- B60W 2540/229: Attention level, e.g. attentive to driving, reading or sleeping
- G06T 2207/10016: Video; Image sequence
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)

## Abstract

An electronic device includes an encoder and a decoder. The encoder is configured to estimate an unknown value on the basis of first biological information including a line of sight of a subject extracted from an image of the subject, subject's environmental information representing an environment of the subject, and subject's internal state information representing an internal state of the subject. The decoder is configured to estimate second biological information including the line of sight of the subject on the basis of the unknown value, the subject's environmental information, and the subject's internal state information. The electronic device adjusts parameters of the encoder and the decoder on the basis of reproducibility of the first biological information from the second biological information.

## Claims

### Claim at position 1

1-2. (canceled)

### Claim 3

An electronic device comprising:
an encoder configured to estimate an unknown value, based on first biological information including a line of sight of a subject extracted from an image of the subject, subject's environmental information representing an environment of the subject, and a value assumed to be subject's internal state information representing an internal state of the subject;
a decoder configured to estimate second biological information including the line of sight of the subject, based on the unknown value, the subject's environmental information, and the value assumed to be the subject's internal state information; and
an estimator configured to assume a plurality of values to be the subject's internal state information, and estimate a value of the plurality of values to be the subject's internal state information, the value corresponding to the highest reproducibility of the first biological information from the second biological information.

### Claim 4 (depends on 3)

The electronic device according to claim 3, wherein the electronic device outputs a predetermined alarm when a value of the plurality of values satisfies a predetermined condition, the value corresponding to the highest reproducibility of the first biological information from the second biological information.

### Claim 5 (depends on 3)

The electronic device according to claim 3, wherein the estimator performs estimation, based on a probability or logarithmic probability representing a likelihood that the unknown value estimated by the encoder is generated from a predetermined probability distribution.

### Claim 6 (depends on 3)

The electronic device according to claim 3, wherein the subject's internal state information includes information representing a concentration level of the subject.

### Claim 7 (depends on 6)

The electronic device according to claim 6, wherein the subject's internal state information includes information representing a concentration level of the subject during driving of a vehicle.

### Claim 8 (depends on 3)

The electronic device according to claim 3, wherein at least one of the first biological information or the second biological information includes coordinates of the line of sight of the subject.

### Claim 9 (depends on 3)

The electronic device according to claim 3, wherein the subject's environmental information includes information of a scene image in front of the subject.

### Claim 10 (depends on 3)

The electronic device according to claim 3, wherein the subject's environmental information includes information of an image captured to include a direction in which the line of sight of the subject is directed.

### Claim 11 (depends on 9)

The electronic device according to claim 9, wherein the subject's environmental information includes information extracted from the image.

### Claim 12 (depends on 11)

The electronic device according to claim 11, wherein the subject's environmental information includes information predicting the line of sight of the subject in the image.

### Claim 13 (depends on 3)

The electronic device according to claim 3, wherein at least one of the first biological information or the second biological information of the subject includes information representing a pupil radius of the subject.

### Claim 14 (depends on 13)

The electronic device according to claim 13, wherein the subject's environmental information includes brightness in the environment of the subject.

### Claim 15 (depends on 3)

The electronic device according to claim 3, wherein at least one of the first biological information or the second biological information of the subject includes information representing an amount of sweat of the subject.

### Claim 16 (depends on 15)

The electronic device according to claim 15, wherein the subject's environmental information includes at least one of a temperature or a humidity in the environment of the subject.

### Claim 17 (depends on 3)

The electronic device according to claim 3, wherein at least one selected from a group consisting of the subject's environmental information, the first biological information, and the second biological information is time series information acquired in a predetermined period.

### Claim 18

(canceled)

### Claim 19

A method for controlling an electronic device, the method comprising:
estimating an unknown value, based on first biological information including a line of sight of a subject extracted from an image of the subject, subject's environmental information representing an environment of the subject, and a value assumed to be subject's internal state information representing an internal state of the subject;
estimating second biological information including the line of sight of the subject, based on the unknown value, the subject's environmental information, and the value assumed to be the subject's internal state information; and
assuming a plurality of values to be the subject's internal state information, and estimating a value of the plurality of values to be the subject's internal state information, the value corresponding to the highest reproducibility of the first biological information from the second biological information.

### Claim 20

(canceled)

### Claim 21

A non-transitory computer-readable recording medium storing computer program instructions, which when executed by an electronic device, cause the electronic device to:
estimate an unknown value, based on first biological information including a line of sight of a subject extracted from an image of the subject, subject's environmental information representing an environment of the subject, and a value assumed to be subject's internal state information representing an internal state of the subject;
estimate second biological information including the line of sight of the subject, based on the unknown value, the subject's environmental information, and the value assumed to be the subject's internal state information; and
assume a plurality of values to be the subject's internal state information, and estimating a value of the plurality of values to be the subject's internal state information, the value corresponding to the highest reproducibility of the first biological information from the second biological information.
