# US 10,769,459 B2 · Method and system for monitoring driving behaviors > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,769,459 B2 - Document ID: US10769459B2 - Status: Granted - Kind: B2 - Application number: 15/751,339 - Filed: 2016-08-30 - Published: 2018-08-23 - Granted: 2020-09-08 - Earliest claimed: 2015-08-31 - Assignees: SRI International - Applicants: SRI International - Inventors: Amir Tamrakar; Gregory Ho; David Salter; Jihua Huang - Web page: https://patentgrep.com/patent/US10769459B2 ## CPC - G06V 20/597: {Recognising the driver's state or behaviour, e.g. attention or drowsiness} - G06V 40/167: {using comparisons between temporally consecutive images} - G06V 40/171: {Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships} - G08B 21/06: indicating a condition of sleep, e.g. anti-dozing alarms - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G08B: SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS ## Abstract A method and a system are provided for monitoring driving conditions. The method includes receiving video data comprising video frames from one or more sensors where the video frames may represent an interior or exterior of a vehicle, detecting and recognizing one or more features from the video data where each feature is associated with at least one driving condition, extracting the one or more features from the video data, developing intermediate features by associating and aggregating the extracted features among the extracted features, and developing a semantic meaning for the at least one driving condition by utilizing the intermediate features and the extracted one or more features. ## Claims ### Claim 1 A method of monitoring driving conditions, comprising: receiving video data comprising video frames from one or more sensors; identifying a face of a person within the video frames; identifying a plurality of landmarks on the face of the person and an orientation of the face; tracking motion of the landmarks and the orientation of the face within the video frames; overlaying a facial image over the face of the person in the video frames; transferring the tracked motion of the landmarks and the orientation to the facial image overlaying the face of the person in the video frames; extracting one or more features from the video frames, wherein each feature is associated with at least one driving condition; developing intermediate features by associating and aggregating the extracted features; and developing a semantic meaning for the at least one driving condition by utilizing at least the intermediate features. ### Claim 2 (depends on 1) The method of claim 1, wherein the facial image comprises a set of image landmarks, and transferring the tracked motion comprises transferring the tracked motion of the plurality of landmarks of the face of the person to motion of the set of image landmarks of the facial image. ### Claim 3 (depends on 1) The method of claim 1, further comprising correlating at least two extracted features to develop the semantic meaning by running two independent regressions on the at least two extracted features and running a joint regression on results of the two independent regressions. ### Claim 4 A method of monitoring driving conditions via a monitoring module comprising program instructions stored in memory and causing one or more processors to execute the steps of: receiving video data comprising video frames from one or more sensors, wherein the video frames represent an interior or exterior of a vehicle; detecting and recognizing one or more features from the video data, wherein each feature is associated with at least one driving condition; extracting the one or more features from the video data; developing intermediate features by running regressions on the one or more extracted features and correlating at least two of the extracted features; and developing a semantic meaning for the at least one driving condition by utilizing at least the intermediate features. ### Claim 5 (depends on 4) The method of claim 4, further comprising: receiving safety data; and integrating the intermediate features and the safety data to develop the semantic meaning for driving conditions. ### Claim 6 (depends on 4) The method of claim 4, wherein detecting and recognizing the one or more features comprises training a detector by utilizing historical video data, and using the trained detector for extracting the one or more features from the video data. ### Claim 7 (depends on 6) The method of claim 6, wherein training the detector comprises running a regression on the historical video data utilizing a machine learning methodology. ### Claim 8 (depends on 4) The method of claim 4, wherein detecting and recognizing the one or more features comprises training a customized detector by using the received video data to generalize an identity for a driver of the vehicle, and using the customized detector for extracting the one or more features from the video data. ### Claim 9 (depends on 4) The method of claim 4, wherein detecting and recognizing the one or more features comprises developing a model by averaging distances between identifiable points for the one or more features in historical video data, and using the model for extracting the one or more features from the video data. ### Claim 10 (depends on 9) The method of claim 9, further comprising enhancing the model by utilizing the extracted one or more features from the received video data. ### Claim 11 (depends on 4) The method of claim 4, further comprising correlating at least two extracted features to develop the semantic meaning. ### Claim 12 (depends on 4) The method of claim 4, wherein correlating at least two extracted features comprises running at least two independent regressions for the at least two extracted features, and the semantic meaning is developed by running a joint regression on results of the at least two independent regressions. ### Claim 13 (depends on 4) The method of claim 4, further comprising displaying the extracted one or more features in a user interface.