# US 2019/0318180 A1 · Methods and systems for processing driver attention data > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0318180 A1 - Document ID: US20190318180A1 - Status: Published application - Kind: A1 - Application number: 15/954,855 - Filed: 2018-04-17 - Published: 2019-10-17 - Earliest claimed: 2018-04-17 - Assignees: GM Global Technology Operations LLC - Applicants: GM Global Technology Operations LLC - Inventors: Upali P. Mudalige; Donald K. Grimm; Wende Zhang - Web page: https://patentgrep.com/patent/US20190318180A1 ## CPC - G06V 20/597: {Recognising the driver's state or behaviour, e.g. attention or drowsiness} - G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)} - G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads - G06V 30/226: of cursive writing - G06V 40/18: Eye characteristics, e.g. of the iris - G06T 2207/30201: Face - G06T 2207/30261: Obstacle - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract Methods and systems are provided for processing attention data. In one embodiment, a method includes: receiving, by a processor, object data associated with at least one object of an exterior environment of the vehicle; receiving upcoming behavior data determined from a planned route of the vehicle; receiving gaze data sensed from an occupant of the vehicle; processing, by the processor, the object data, the upcoming behavior data, and the gaze data to determine an attention score associated with an attention of the occupant of the vehicle; and selectively generating, by the processor, signals to at least one of notify the occupant and control the vehicle based on the attention score. ## Claims ### Claim 1 A method of processing attention data, comprising: receiving, by a processor, object data associated with at least one object of an exterior environment of a vehicle; receiving upcoming behavior data determined from a planned route of the vehicle; receiving gaze data sensed from an occupant of the vehicle; processing, by the processor, the object data, the upcoming behavior data, and the gaze data to determine an attention score associated with an attention of the occupant of the vehicle; and selectively generating, by the processor, signals to at least one of notify the occupant and control the vehicle based on the attention score. ### Claim 2 (depends on 1) The method of claim 1, wherein the processing comprises computing an object location from the object data, computing a gaze location from the gaze data, computing an upcoming behavior location from the upcoming behavior data, and computing the attention score based on the gaze location, the object location, and the upcoming behavior location. ### Claim 3 (depends on 2) The method of claim 2, wherein the computing the attention score comprises computing a matching level between the object location and the gaze location. ### Claim 4 (depends on 3) The method of claim 3, wherein the computing the attention score further comprises applying a weighting factor to the matching level. ### Claim 5 (depends on 3) The method of claim 3, wherein the computing the matching level is based on a duration of matching. ### Claim 6 (depends on 3) The method of claim 3, wherein the computing the matching level is based on a breadth of matching. ### Claim 7 (depends on 4) The method of claim 4, wherein the weighting factor is dynamically determined based on environmental conditions. ### Claim 8 (depends on 4) The method of claim 4, wherein the weighting factor is dynamically determined based on a driving scenario. ### Claim 9 (depends on 2) The method of claim 2, wherein the computing the attention score comprises computing a matching level between the upcoming behavior location and the gaze location. ### Claim 10 (depends on 9) The method of claim 9, wherein the computing the attention score further comprises applying a weighting factor to the matching level. ### Claim 11 (depends on 9) The method of claim 9, wherein the computing the matching level is based on a duration of matching. ### Claim 12 (depends on 9) The method of claim 9, wherein the computing the matching level is based on a breadth of matching. ### Claim 13 (depends on 1) The method of claim 1, further comprising determining a distraction level based on a matching between the gaze location and a distraction region, and wherein the attention score is further determined based on the distraction level. ### Claim 14 (depends on 13) The method of claim 13, further comprising applying a weighting factor to the distraction level and wherein the attention score is further based on the weighted distraction level. ### Claim 15 (depends on 14) The method of claim 14, wherein the weighting factor is dynamically determined based on environmental conditions. ### Claim 16 (depends on 14) The method of claim 14, wherein the weighting factor is dynamically determined based on a driving scenario. ### Claim 17 (depends on 1) The method of claim 1, further comprising receiving gaze data from occupants of other vehicles at a same location; determining a general gaze direction based on the gaze data from the occupants of other vehicles; and wherein the processing the object data, the upcoming behavior data, and the gaze data to determine the attention score further comprises processing the object data, the upcoming behavior data, the gaze data from the occupant, and the general gaze direction to determine the attention score associated with an attention of the occupant of the vehicle. ### Claim 18 A system for processing attention data, comprising: a first non-transitory module that processes, by a processor, object data associated with at least one object of an exterior environment of a vehicle to determine an object location; a second non-transitory module that processes, by a processor, upcoming behavior data determined from a planned route of the vehicle to determine an upcoming behavior location; a third non-transitory module that processes, by a processor, gaze data sensed from an occupant of the vehicle to determine a gaze location; a fourth non-transitory module that processes, by a processor, the object location, the upcoming behavior location, and the gaze location to determine an attention score associated with an attention of the occupant of the vehicle; and a fifth non-transitory module that, by a processor, selectively generates signals to at least one of notify the occupant and control the vehicle based on the attention score. ### Claim 19 (depends on 18) The system of claim 18, wherein the fourth non-transitory module computes the attention score based on a matching level between the object location and the gaze location and a matching level between the upcoming behavior location and the gaze location. ### Claim 20 (depends on 19) The system of claim 19, wherein the fourth non-transitory module computes the attention score based on a weighting factor applied to the matching level between the object location and the gaze location and a weighting factor applied to the matching level between the upcoming behavior location and the gaze location.