US 10,356,303 B1Granted

Systems and methods for controlling smart devices based upon image data from image sensors

AssigneeState Farm Mutual Automobile Insurance Company

InventorsJackie O. Jordan, II, John Donovan, David Turrentine, Torri Wollenschlager, Bryan R. Nussbaum, Deanna Stockweather, Jeffrey W. Stoiber, Kerstin Markwardt, Gail L. Carlson, Kyle C. Schiebel, Troy Winslow, Joseph P. Harr, Ellakate Wagner, Michael Harris, Jr., Jennylind Sun

Life of this patent

Estimating the term…

patentgrep

Loading patent…

Abstract

The present embodiments relate to managing the operation of devices within a home or other property based upon received image data. According to certain aspects, a controller within the home may receive, with customer permission or consent, image data from image sensors disposed throughout the home. The controller may analyze the image data in combination with profile data for individuals associated with the home to determine that a certain individual is indicated in the image data. The controller may further determine an action to facilitate based upon the individual being detected in a certain location of the property, and may direct smart devices to perform that action. As a result, the present embodiments may facilitate providing occupant-location based (or occupant-presence or preference based) functionality, functions, or services, and/or directing operations of smart devices located about a property based upon occupant location, presence, preferences, and/or activities at the property.

Claims16

Dependent claims

A computer-implemented method of controlling a plurality of devices associated with a property, the property populated with a hardware controller in communication with the plurality of devices, the method comprising:

receiving, by the hardware controller via a communication network, image data from a set of image sensors disposed at a location of the property;
accessing, from memory, at least one occupant profile associated with at least one occupant of the property, the at least one occupant profile containing a set of images depicting the at least one occupant and a set of physical attributes of the at least one occupant;
analyzing the image data to detect an individual and a set of characteristics of the individual;
determining that the at least one occupant is depicted in the image data, including:
comparing the set of characteristics of the individual to the set of physical attributes of the at least one occupant, and
performing a facial recognition technique on the image data and the set of images contained in the at least one occupant profile;
determining, by the processor, a set of additional auxiliary items depicted in the image data and associated with the at least one occupant;
determining, by the processor from the image data, how the at least one occupant is interacting with the set of additional auxiliary items;
based upon the at least one occupant profile:
determining, further based upon how the at least one occupant is interacting with the set of additional auxiliary items, an action to be performed, and
determining, based upon the location of the set of image sensors, at least one of the plurality of devices to perform the action;
and
directing the at least one of the plurality of devices to perform the action.

10 more elements

A hardware controller for controlling a plurality of devices associated with a property, the property populated with a hardware controller in communication with the plurality of devices, comprising:

at least one transceiver configured to receive image data from a set of image sensors disposed at a location of the property;
a memory configured to store non-transitory computer executable instructions and a set of occupant profiles;
and
a processor interfacing with the at least one transceiver and the memory, wherein the processor is configured to execute the non-transitory computer executable instructions to cause the processor to:
access, from the memory, at least one occupant profile of the set of occupant profiles, the at least one occupant profile associated with at least one occupant of the property, the at least one occupant profile containing a set of images depicting the at least one occupant and a set of physical attributes of the at least one occupant,
determine that the at least one occupant is depicted in the image data, including:
compare the set of characteristics of the individual to the set of physical attributes of the at least one occupant, and
perform a facial recognition technique on the image data and the set of images contained in the at least one occupant profile,
determine a set of additional auxiliary items depicted in the image data and associated with the at least one occupant,
determine, from the image data, how the at least one occupant is interacting with the set of additional auxiliary items;
based upon the data contained in the at least one occupant profile:
determine, further based upon how the at least one occupant is interacting with the set of additional auxiliary items, an action to be performed, and
determine, based upon the location of the set of image sensors, at least one of the plurality of devices to perform the action, and
direct the at least one of the plurality of devices to perform the action.

11 more elements

Dependencies are inferred from the claim text.

Facts

Application: 14/873,783Filed 2 Oct 2015 · us provisional application
Priority: 7 Oct 2014
Granted: 16 Jul 2019
Term ends: Not estimatedThe source records don’t support an estimate.

Classification38 codes

G05B 15/02
Systems controlled by a computer: electric
G01C 21/20
Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00: instruments for performing navigational calculations
G06Q 10/20
Administration; Management: administration of product repair or maintenance
G06Q 30/01
Commerce: customer relationship services

Family

Application62/060,962Us provisional application · filed 7 Oct 2014
Application62/105,407Us provisional application · filed 20 Jan 2015
Application62/187,651Us provisional application · filed 1 Jul 2015
Application62/187,624Us provisional application · filed 1 Jul 2015
Application62/187,642Us provisional application · filed 1 Jul 2015
Application62/187,666Us provisional application · filed 1 Jul 2015
Application62/187,645Us provisional application · filed 1 Jul 2015
Application62/189,329Us provisional application · filed 7 Jul 2015
Application62/193,317Us provisional application · filed 16 Jul 2015
Application62/197,343Us provisional application · filed 27 Jul 2015
Application62/198,813Us provisional application · filed 30 Jul 2015
Application62/200,375Us provisional application · filed 3 Aug 2015
Application62/201,671Us provisional application · filed 16 Aug 2015
Application62/220,383Us provisional application · filed 18 Sep 2015
US 10,356,303 B1This patent

From the US filing history. Foreign family members aren’t listed.