# US 11,308,325 B2 · Systems and methods for predicting real-time behavioral risks using everyday images > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,308,325 B2 - Document ID: US11308325B2 - Status: Granted - Kind: B2 - Application number: 16/654,868 - Filed: 2019-10-16 - Published: 2020-04-16 - Granted: 2022-04-19 - Earliest claimed: 2018-10-16 - Assignees: Duke University - Applicants: Duke University - Inventors: Francis McClernon; Matthew Engelhard; Lawrence Carin; Jason Oliver - Web page: https://patentgrep.com/patent/US11308325B2 ## CPC - G08B 31/00: Predictive alarm systems characterised by extrapolation or other computation using updated historic data - G06F 18/24323: {Tree-organised classifiers} - G06F 18/2451: linear, e.g. hyperplane - G06N 3/04: Architecture, e.g. interconnection topology - G06N 3/045: Combinations of networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/08: Learning methods - G06N 3/09: Supervised learning - G06V 10/764: using classification, e.g. of video objects - G06V 20/20: in augmented reality scenes - G08B 7/06: using electric transmission {, e.g. involving audible and visible signalling through the use of sound and light sources} - G09B 19/00: Teaching not covered by other main groups of this subclass (teaching or practice apparatus for gun-aiming or gun-laying F41G3/26) - G09B 19/0092: {Nutrition} - G09B 23/28: for medicine - G06N 20/20: Ensemble learning - G08B: SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B) ## Abstract A system includes a camera configured to generate image data and a computing device in electronic communication with the camera. The computing device includes at least one processor and is configured to receive, from the camera, one or more images representative of a location. The computing device is further configured to apply a trained classifier to the one or more images to classify the location into one of at least two risk categories, wherein the classification is based on a likelihood of a subject performing a target behavior based on presence of the subject in the location. The computing device is additionally configured to issue a risk alert responsive to the trained classifier classifying the location into a high-risk category. ## Claims ### Claim 1 A system comprising: a camera configured to generate image data; and a computing device in electronic communication with the camera, the computing device comprising at least one processor, wherein the computing device is configured to: receive, from the camera, one or more images representative of a location; apply a trained classifier to the one or more images to classify the location into one of at least two risk categories, wherein the classification is based on a likelihood of a subject performing a target behavior based on presence of the subject in the location, wherein the target behavior comprises smoking and wherein the trained classifier comprises a convolutional neural network trained to identify one or more objects in the one or more images, and wherein the trained classifier differentiates between a first risk category indicative of a smoking environment and a second risk category indicative of a nonsmoking environment with an accuracy greater than 70%; and issue a risk alert responsive to the trained classifier classifying the location into a high-risk category, wherein the risk alert comprises one or more of an audible alert, a visual alert, or a tactile alert. ### Claim 2 (depends on 1) The system according to claim 1, wherein the target behavior comprises one or more of anxiety, eating disorders, falling, insomnia, ADHD, obsessive compulsive disorder, sensory integration disorder, eating behavior, physical activity, or alcoholism. ### Claim 3 (depends on 1) The system according to claim 1, wherein the trained classifier further comprises an interpretable linear classifier trained to generate a risk index based on the one or more identified objects in the one or more images, wherein the location is classified into a risk category based on the risk index. ### Claim 4 (depends on 1) The system according to claim 1, wherein the system comprises a display interface configured to display an image of the location with at least one identified object highlighted to indicate an association between the at least one identified object and the target behavior. ### Claim 5 (depends on 1) The system according to claim 1, wherein the trained classifier comprises a decision tree. ### Claim 6 (depends on 1) The system according to claim 1, wherein the trained classifier is further trained to distinguish between a first category of high-risk locations in which the subject is at risk of performing the target behavior while present in one of the first category of high-risk locations and a second group of high-risk locations in which the subject is at risk of performing the target behavior subsequent to being present in one of the second category of high-risk locations. ### Claim 7 (depends on 1) The system according to claim 1, wherein the computing device is a wearable computing device and wherein the camera is physically coupled to the wearable computing device. ### Claim 8 (depends on 1) The system according to claim 1, wherein the at least one processor is configured to issue the risk alert responsive to the trained classifier classifying the location into the high-risk category based on the trained classifier classifying the location into the high-risk category for each image of a predetermined threshold number of images. ### Claim 9 (depends on 1) The system according to claim 1, wherein the computing device is further configured to: determine whether the subject performed the target behavior in the location; and further train the classifier based on the determination whether the subject performed the target behavior in the location. ### Claim 10 (depends on 9) The system according to claim 9, further comprising one or more sensors in electronic communication with the computing device, wherein the computing device is configured to determine whether the subject performed the target behavior in the location based on sensor data from the one or more sensors. ### Claim 11 (depends on 1) The system according to claim 1, wherein the computing device is further configured to provide at least one behavioral recommendation to reduce risk of the subject performing the target behavior. ### Claim 12 (depends on 11) The system according to claim 11 in which the at least one behavioral recommendation to reduce the risk of the subject performing the target behavior comprises one or more of: utilizing a behavioral coping strategy, engaging in an alternate behavior, reconfiguring the location to reduce risk, leaving the location, utilizing a therapy aid, or contacting a support individual. ### Claim 13 (depends on 1) The system according to claim 1, wherein the trained classifier is further trained to classify the location based on a likelihood of a subject having a physiological event based on presence of the subject in the location. ### Claim 14 (depends on 13) The system according to claim 13, wherein the physiological event comprises one or more of an asthma attack, seizures, an allergic reaction, high blood pressure, or a stroke. ### Claim 15 A method comprising: receiving, from a camera, one or more images representative of a location; applying a trained classifier to the one or more images to classify the location into one of at least two risk categories, wherein the classification is based on a likelihood of a subject performing a target behavior based on presence of the subject in the location, wherein the target behavior comprises smoking and wherein the trained classifier comprises a convolutional neural network trained to identify one or more objects in the one or more images, and wherein the trained classifier differentiates between a first risk category indicative of a smoking environment and a second risk category indicative of a nonsmoking environment with an accuracy greater than 70%; and issuing a risk alert responsive to the trained classifier classifying the location into a high-risk category, wherein the risk alert comprises one or more of an audible alert, a visual alert, or a tactile alert. ### Claim 16 A mobile computing device comprising: a camera configured to generate one or more images representative of a location of the mobile computing device; and a non-transitory computer-readable medium comprising program instructions executable by at least one processor to cause the at least one processor to: apply a trained classifier to the one or more images to classify the location into one of at least two risk categories, wherein the classification is based on a likelihood of a subject performing a target behavior based on presence of the subject in the location, wherein the target behavior comprises smoking and wherein the trained classifier comprises a convolutional neural network trained to identify one or more objects in the one or more images, and wherein the trained classifier differentiates between a first risk category indicative of a smoking environment and a second risk category indicative of a nonsmoking environment with an accuracy greater than 70%; and issue a risk alert responsive to the trained classifier classifying the location into a high-risk category, wherein the risk alert comprises one or more of an audible alert, a visual alert, or a tactile alert.