# US 2023/0267752 A1 · Electronic device, information processing apparatus, method for inference, and program for inference > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2023/0267752 A1 - Document ID: US20230267752A1 - Status: Published application - Kind: A1 - Application number: 18/004,175 - Filed: 2021-07-01 - Published: 2023-08-24 - Earliest claimed: 2020-07-17 - Assignees: KYOCERA Corporation - Applicants: KYOCERA Corporation - Inventors: Takayuki SHINOZAKI - Web page: https://patentgrep.com/patent/US20230267752A1 ## CPC - A61B 5/163: {by tracking eye movement, gaze, or pupil change} - G06V 20/597: {Recognising the driver's state or behaviour, e.g. attention or drowsiness} - A61B 3/113: for determining or recording eye movement - A61B 5/004: {adapted for image acquisition of a particular organ or body part (A61B5/0082 takes precedence; arrangements for optical scanning A61B5/0062)} - A61B 5/0077: {Devices for viewing the surface of the body, e.g. camera, magnifying lens} - A61B 5/168: {Evaluating attention deficit, hyperactivity} - A61B 5/18: for vehicle drivers {or machine operators} - A61B 5/7267: {involving training the classification device} - G06N 3/08: Learning methods - G06V 10/70: using pattern recognition or machine learning (optical pattern recognition or electronic computations therefor G06V10/88) - G06V 40/18: Eye characteristics, e.g. of the iris - G08G 1/04: using optical or ultrasonic detectors - A61B 2576/02: specially adapted for a particular organ or body part - G06N 3/04: Architecture, e.g. interconnection topology - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93}) ## Abstract An electronic device 10 includes an imaging unit 11, a gaze-line detector 12, and a controller 14. The imaging unit 11 is configured to create at least one image corresponding to a view by image capturing. The gaze-line detector is configured to detect a gaze line of a subject, the gaze line being directed to the view. The controller 14 is configured to serve as a first inference unit 15 and a second inference unit 16. The first inference unit 15 is capable of inferring a gaze-line prediction map based on the at least one image. The second inference unit 16 is capable of inferring biological information regarding an alertness level of the subject based on the gaze-line prediction map and the gaze line of the subject. ## Claims ### Claim 1 An electronic device comprising: an imaging unit configured to create at least one image corresponding to a view by image capturing; a gaze-line detector configured to detect a gaze line of a subject, the gaze line being directed to the view; and a controller configured to estimate an alertness level of the subject based on the at least one image and the gaze line, wherein the controller is configured to serve as a first inference unit capable of inferring a gaze-line prediction map that is based on the at least one image and that represents a probability of finding the gaze line at each position in the at least one image, the first inference unit being constructed based on learning data obtained by machine learning regarding a relationship between a training image and a gaze line of a subject for training, the gaze line being directed to the training image, and a second inference unit capable of inferring biological information regarding the alertness level of the subject based on the gaze-line prediction map and the gaze line of the subject, the second inference unit being constructed based on learning data obtained by machine learning regarding a relationship among a gaze-line prediction map for training inferred by the first inference unit based on the training image, the gaze line of the subject for training, the gaze line being directed to the training image, and biological information regarding an alertness level of the subject for training. ### Claim 2 (depends on 1) The electronic device according to claim 1, wherein the at least one image comprises a plurality of continuously captured images and the controller is configured to estimate the alertness level of the subject based on the plurality of continuously captured images and the gaze line for each of the plurality of images. ### Claim 3 (depends on 2) The electronic device according to claim 2, wherein the controller is configured to exclude a combination of a gaze line whose moving speed exceeds a threshold and a corresponding image from the plurality of continuously captured images and the gaze line for each of the plurality of continuously captured images when estimating the alertness level of the subject. ### Claim 4 An information processing apparatus comprising: an acquiring unit configured to acquire at least one image corresponding to a view and a gaze line of a subject, the gaze line being directed to the view; a controller configured to estimate an alertness level of the subject based on the at least one image and the gaze line; and an output unit configured to output the alertness level, wherein the controller is configured to serve as a first inference unit capable of creating a gaze-line prediction map that is based on the at least one image and that represents a frequency of finding the gaze line in the at least one image, the first inference unit being constructed based on learning data obtained by machine learning regarding a relationship between a training image and a gaze line of a subject for training, the gaze line being directed to the training image, and as a second inference unit capable of inferring biological information regarding the alertness level of the subject based on the gaze-line prediction map and the gaze line of the subject, the second inference unit being constructed based on learning data obtained by machine learning regarding a relationship among a gaze-line prediction map for training inferred by the first inference unit based on the training image, the gaze line of the subject for training, the gaze line being directed to the training image, and biological information regarding an alertness level of the subject for training. ### Claim 5 A method for inference, comprising the steps of: creating at least one image corresponding to a view by image capturing; detecting a gaze line of a subject, the gaze line being directed to the view; and estimating an alertness level of the subject based on the at least one image and the gaze line, wherein the step of estimating includes a first inference step in which a gaze-line prediction map can be inferred based on the at least one image, the gaze-line prediction map representing a probability of finding the gaze line at each position in the at least one image, the first inference step being constructed based on learning data obtained by machine learning regarding a relationship between a training image and a gaze line of a subject for training, the gaze line being directed to the training image, and a second inference step in which biological information regarding the alertness level of the subject can be inferred based on the gaze-line prediction map and the gaze line of the subject, the second inference step being constructed based on learning data obtained by machine learning regarding a relationship among a gaze-line prediction map for training inferred in the first inference step based on the training image, the gaze line of the subject for training, the gaze line being directed to the training image, and biological information regarding an alertness level of the subject for training. ### Claim 6 A non-transitory computer-readable recording medium including program for inference causing a computer to serve as an imaging unit configured to create at least one image corresponding to a view by image capturing; a gaze-line detector configured to detect a gaze line of a subject, the gaze line being directed to the view; and a controller configured to estimate an alertness level of the subject based on the at least one image and the gaze line, wherein the controller is configured to serve as a first inference unit capable of inferring a gaze-line prediction map that is based on the at least one image and that represents a probability of finding the gaze line at each position in the at least one image, the first inference unit being constructed based on learning data obtained by machine learning regarding a relationship between a training image and a gaze line of a subject for training, the gaze line being directed to the training image, and a second inference unit capable of inferring biological information regarding the alertness level of the subject based on the gaze-line prediction map and the gaze line of the subject, the second inference unit being constructed based on learning data obtained by machine learning regarding a relationship among a gaze-line prediction map for training inferred by the first inference unit based on the training image, the gaze line of the subject for training, the gaze line being directed to the training image, and biological information regarding an alertness level of the subject for training.