# US 2019/0244102 A1 · Devices and methods using machine learning to reduce resource usage in surveillance > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0244102 A1 - Document ID: US20190244102A1 - Status: Published application - Kind: A1 - Application number: 16/390,049 - Filed: 2019-04-22 - Published: 2019-08-08 - Earliest claimed: 2018-01-31 - Inventors: Thomas Danaher Harvey - Web page: https://patentgrep.com/patent/US20190244102A1 ## CPC - G06V 10/82: using neural networks - G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting - G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] - G06N 3/09: Supervised learning - G06N 3/096: Transfer learning - G06N 20/20: Ensemble learning - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract A method and system where a first subsystem makes observations and performs surveillance using sensors in a mode that conserves a resource such as power, data transmission band width or processing cycles. This is accomplished by reducing illumination, pixel count, sampling rate or other functions that result in a limited granularity or data collection rate. A machine model is applied to the limited data and, when it evaluates to a suitable result or a prediction of an interesting condition, another subsystem or the same subsystem in a different mode collects data at a finer granularity with a higher data collection size or rate and evaluates that data to determine the nature of the first evaluation. The machine model may be trained in stages on a large scale server and on a small field processor. Data from the sensor may be used for training to improve the second step. ## Claims ### Claim 1 A system to acquire data comprising: (a) a first subsystem adapted to make observations with a sensor operating at a course level of granularity which consumes resources at a limited level, (b) a processor to evaluate a machine learning model trained to determine existence of a requirement for data of finer granularity from the observations of the first subsystem, and (c) a second subsystem adapted to make observations with a sensor at a level of granularity finer than the first subsystem which is not limited to the level of resource use of the first subsystem wherein the second subsystem is used when the requirement for observations of finer granularity has been determined with the use of the machine learning model and the observations of the sensor of the first subsystem by the processor to exist and is not used when the requirement has not been determined by the processor to exist. ### Claim 2 (depends on 1) The system of claim 1 wherein: (a) the processor is a first processor; (b) the machine learning model has been trained on a second processor; (c) the machine learning model has been downloaded to the first processor subsequent to training on the second processor; (d) the machine learning model has been further trained subsequent to the downloading with data from observations made by the first subsystem and subsequently the determination of the requirement is made with machine learning model. ### Claim 3 (depends on 1) The system of claim 1 wherein: (a) the processor is a first processor; (b) the machine learning model has been trained on a second processor; (c) the machine learning model has been downloaded to the first processor subsequent to training on the second processor; (d) the machine learning model has been further trained subsequent to the downloading with data from observations made by the second subsystem and subsequently the determination of the requirement is made with machine learning model. ### Claim 4 (depends on 2) The system of claim 2 wherein: (a) the sensor of one subsystem is optical, and (b) the sensor of the another subsystem is acoustic. ### Claim 5 (depends on 2) The system of claim 2 wherein: (a) the sensor of one subsystem is optical, and (b) the sensor of the another subsystem is a radio frequency receiver. ### Claim 6 (depends on 2) The system of claim 2 wherein: (a) the sensor of one subsystem is acoustic, and (b) the sensor of the another subsystem is a radio frequency receiver. ### Claim 7 A method of making observations comprising: a) making a first observation of an object using a sensor at a course level of granularity chosen to conserve a resource; b) using a processor with a machine learning program with a machine learning model trained to use the first observation to determine the need to make a second observation at a finer level of granularity wherein the second observation requires a greater use of the resource than the first observation, c) making an observation of the object using a sensor at the finer level of granularity. ### Claim 8 (depends on 7) The method of claim 7 wherein: (a) the processor is a first processor; (b) the machine learning model has been trained on a second processor; (c) the machine learning model has been downloaded to the first processor subsequent to training on the second processor; (d) the machine learning model has been further trained subsequent to the downloading with data from observations made at the coarse level of granularity, and (e) the determination of the requirement is made subsequent to the further training with machine learning model. ### Claim 9 (depends on 7) The method of claim 7 wherein: (a) the processor is a first processor; (b) the machine learning model has been trained on a second processor; (c) the machine learning model has been downloaded to the first processor subsequent to training on the second processor; (d) the machine learning model has been further trained subsequent to the downloading with data from observations made by the sensor at the fine level of granularity, and (e) the determination of the requirement is made subsequent to the further training with machine learning model. ### Claim 10 (depends on 7) The method of claim 7 wherein: the sensor comprises an optical array of pixels with the fine level of granularity using a greater number of pixels than the course level of granularity. ### Claim 11 (depends on 7) The method of claim 7 wherein: the sensor comprises an optical array of pixels with the fine level of granularity using a pixel to observe a narrower angle of view than the pixel when observing at the coarse level of granularity. ### Claim 12 (depends on 7) The method of claim 7 wherein: (a) the first observation is made with an illumination system turned off, (b) the second observation is made with the illumination system turned on, and (c) the illumination system emits at least one of visible light, infrared radiation, ultraviolet radiation, x-rays, radio frequency waves, and acoustic radiation. ### Claim 13 A method of gathering data with comprising: a) operating a sensor with at least two modes of operation comprising a first mode of coarse granularity which requires a lower level of consumption of a resource and a second mode of fine granularity which requires a greater level of consumption of the resource, (b) generating a first set of data with the sensor in the first mode; b) operating a processor with a memory containing a machine learning model trained to identify objects of a class of objects from input of data from the sensor; c) using the first set of sensor data as input to the machine learning model to determine if an object of the class is present; d) using the sensor in the second mode with a greater consumption of the resource to produce a second set of data concerning the object which cannot be produced in the first mode. ### Claim 14 (depends on 13) The method of claim 13 wherein: (a) the processor is a first processor; (b) the machine learning model has been trained on a second processor; (c) the machine learning model has been downloaded to the first processor subsequent to training on the second processor; (d) the machine learning model has been further trained subsequent to the downloading with data from observations made by sensor prior to the generation of the first set of data. ### Claim 15 (depends on 14) The method of claim 14 wherein: the training subsequent to the downloading is done from data made by the sensor in the second mode. ### Claim 16 (depends on 13) The method of claim 13 wherein: the sensor comprises an optical array of pixels with the fine level of granularity using a greater number of pixels than the course level of granularity. ### Claim 17 (depends on 13) The method of claim 13 wherein: the sensor comprises an optical array of pixels with the fine level of granularity using a pixel to observe a narrower angle of view than the pixel when observing at the coarse level of granularity. ### Claim 18 (depends on 13) The method of claim 13 wherein: the sensor is acoustic. ### Claim 19 (depends on 13) The method of claim 13 wherein: the sensor is an antenna to receive radio frequency electromagnetic signals. ### Claim 20 (depends on 13) The method of claim 13 wherein: (a) the first set of data is produced with an illumination system which is turned off, (b) the second set of data is produced with the illumination system turned on, and (c) the illumination system emits at least one of visible light, infrared radiation, ultraviolet radiation, x-rays, radio frequency waves, and acoustic radiation.