# US 2019/0171897 A1 · System and method for automatically improving gathering of data using a data gathering device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0171897 A1
- Document ID: US20190171897A1
- Status: Published application
- Kind: A1
- Application number: 16/212,309
- Filed: 2018-12-06
- Published: 2019-06-06
- Earliest claimed: 2017-12-06
- Assignees: La SociéTé Dataperformers Inc.
- Inventors: Mehdi Merai; Ali Elawad
- Web page: https://patentgrep.com/patent/US20190171897A1

## CPC

- G06N 3/08: Learning methods
- G06V 10/94: Hardware or software architectures specially adapted for image or video understanding
- G06F 18/217: {Validation; Performance evaluation; Active pattern learning techniques}
- G06F 18/24: Classification techniques
- G06N 3/008: based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/09: Supervised learning
- G06N 3/092: Reinforcement learning
- G06N 5/048: Fuzzy inferencing
- G06N 20/00: Machine learning
- G06V 10/776: Validation; Performance evaluation
- G06V 10/993: {Evaluation of the quality of the acquired pattern}
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- H04N 23/62: Control of parameters via user interfaces
- H04N 23/64: {Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image}
- H04N 23/69: Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
- H04N 23/695: Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
- H04N 23/72: Combination of two or more compensation controls
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A computer-implemented system and method for automatically improving gathering of data includes processing first input data acquired by a data gathering device to generate a first prediction and a corresponding confidence score. If the confidence score is below a threshold, then generating and applying a set of at least one action to the data gathering device, such as changing acquisition settings of the device. Second input data acquired using the data gathering device having the action applied thereto is further received and processed to generate a second prediction and corresponding confidence score. The set of action to-be-applied to the device is further modified based on the difference. The generation of the at least one action, such as changing the acquisition settings, can be based on an acquisition settings adjustment model and the modification of the action can include updating the model by machine learning (ex: reinforcement learning). The updated model can be applied to subsequent iterations of data gathering.

## Claims

### Claim 1

A computer-implemented method for automatically improving gathering of data, the method comprising:
receiving first input data acquired using a data gathering device;
processing the first input data to generate a first prediction and a corresponding confidence score of the first prediction; and
if the confidence score of the first prediction is below a predetermined threshold:
generating a set of at least one action to apply to the data gathering device based on the first input data;
applying the set of at least one action to the data gathering device;
receiving second input data acquired using the data gathering device;
processing the second input data to generate a second prediction and a corresponding confidence score of the second prediction;
calculating a difference between the confidence of the first and second predictions; and
modifying the set of at least one action to apply to the data gathering device based on the difference.

### Claim 2 (depends on 1)

The method of claim 1, wherein the first input data is acquired in accordance with a first set of acquisition settings of the data gathering device;
wherein generating the set of at least one action to apply to the data gathering device based on the first input data comprises generating a second set of acquisition settings of the data gathering device based on a computer-implemented acquisition settings adjustment model;
wherein applying the set of at least one action to the data gathering device comprises configuring the data gathering device in accordance with the second set of acquisition settings;
wherein acquiring the second input data using the data gathering device comprises acquiring the second input data while the data gathering device is configured to the second set of acquisition settings; and
wherein modifying the sets of at least one action to apply to the data gathering device based on the difference comprises updating the acquisition settings adjustment model based on the difference.

### Claim 3 (depends on 2)

The method of claim 2, wherein the acquisition settings adjustment model is further updated based on the first set of acquisition settings and the second set of acquisition settings.

### Claim 4 (depends on 2)

The method of claim 2, wherein the computer-implemented acquisition settings adjustment model is pre-trained using machine learning; and
wherein updating the acquisition settings adjustment model based on the difference is carried out using reinforcement learning.

### Claim 5 (depends on 2)

The method of claim 2, further comprising:
acquiring a third input data using the data gathering device;
processing the third input data to generate a third prediction and a corresponding confidence of the third prediction; and
if the third prediction is below the predetermined threshold:
generating an additional set of at least one action to apply to the data gathering device based on the third input data by generating a third set of acquisition settings of the data gathering device based on the acquisition settings adjustment model as updated;
applying the additional set of at least one action to the data gathering device;
acquiring fourth input data using the gathering device having the additional set at least one action applied thereto.

### Claim 6 (depends on 1)

The method of claim 1, wherein the data gathering device is an image capture device;
wherein the first input data is a first digital image captured by the image capture device.

### Claim 7 (depends on 6)

The method of claim 6, wherein processing the first input data to generate a first prediction comprises applying object classification on the captured digital image;
wherein the first prediction is a classification of an object within the scene of the first captured digital image;
wherein the confidence of the first prediction is a classification score for the classified object of the first captured digital image;
wherein the second input data is a second digital image captured by the image capture device;
wherein the second prediction is a classification of the object within the scene of the second captured digital image; and
wherein the confidence of the second prediction is a classification score for the classified object of the second captured digital image.

### Claim 8 (depends on 7)

The method of claim 7, wherein the first set of acquisition settings of the data gathering device is a first set of image capture settings of the image capture device;
wherein generating the set of at least one action to apply to the data gathering device based on the first input data comprises generating a second set of image capture settings of the image capture device based on an image capture settings adjustment model;
wherein applying the set of actions to the data gathering device comprises configuring the image capture device in accordance with the second set of image capture settings;
wherein acquiring second input data using the data gathering device comprises acquiring the second digital image while the image capture device is configured to the second set of image capture settings; and
wherein modifying the set of at least one action to apply to the data gathering device based on the difference comprises updating the image capture settings adjustment model based on the difference between the classification score for the first captured digital image and the classification score for the second captured digital image.

### Claim 9 (depends on 8)

The method of claim 8, wherein the image capture settings adjustment model is further updated based on the first set of image capture settings and the second set of image capture settings.

### Claim 10 (depends on 8)

The method of claim 8, wherein the first or second image capture settings comprises one or more of zoom, pan, tilt, white balance, exposure, ISO settings, aperture, shutter speed, depth sensor, frame rate, position of a vehicle having the image capture device mounted thereto and lighting subsystem.

### Claim 11 (depends on 8)

The method of claim 8, wherein the image capture settings adjustment model is pre-trained using machine learning; and
wherein updating the image capture settings adjustment model based on the difference is carried out using reinforcement learning.

### Claim 12 (depends on 8)

The method of claim 8, further comprising:
receiving a third digital image acquired using the image capture device;
processing the third digital image to apply a classification of an object within the scene of the third digital image and to generate a classification score for the classified object of the third captured digital image;
if the classification score for the classified object of the third captured digital image is below the predetermined threshold:
generating an additional set of image capture settings to apply to the image capture device based on the third digital image and the image capture settings adjustment model as updated;
applying the additional set of image capture settings to the image capture device;
receiving a fourth digital image acquired using the image capture device having the additional set of image capture settings applied thereto.

### Claim 13

A computer-implemented system for automatically improving gathering of data, the system comprising:
at least one data storage device; and
at least one processor operably coupled to the at least one storage device, the at least one processor being configured for:
receiving first input data acquired using a data gathering device;
processing the first input data to generate a first prediction and a corresponding confidence score of the first prediction; and
if the confidence score of the first prediction is below a predetermined threshold:
generating a set of at least one action to apply to the data gathering device based on the first input data;
applying the set of at least one action to the data gathering device;
receiving second input data acquired using the data gathering device;
processing the second input data to generate a second prediction and a corresponding confidence score of the second prediction;
calculating a difference between the confidence of the first and second predictions; and
modifying the set of at least one action to apply to the data gathering device based on the difference.

### Claim 14 (depends on 13)

The system of claim 13, wherein the first input data is acquired in accordance with a first set of acquisition settings of the data gathering device;
wherein generating the set of at least one action to apply to the data gathering device based on the first input data comprises generating a second set of acquisition settings of the data gathering device based on a computer-implemented acquisition settings adjustment model;
wherein applying the set of at least one action to the data gathering device comprises configuring the data gathering device in accordance with the second set of acquisition settings;
wherein acquiring the second input data using the data gathering device comprises acquiring the second input data while the data gathering device is configured to the second set of acquisition settings; and
wherein modifying the sets of at least one action to apply to the data gathering device based on the difference comprises updating the acquisition settings adjustment model based on the difference.

### Claim 15 (depends on 14)

The system of claim 14, wherein the acquisition settings adjustment model is further updated based on the first set of acquisition settings and the second set of acquisition settings.

### Claim 16 (depends on 14)

The system of claim 14, wherein the computer-implemented acquisition settings adjustment model is pre-trained using machine learning; and
wherein updating the acquisition settings adjustment model based on the difference is carried out using reinforcement learning.

### Claim 17 (depends on 14)

The system of claim 14, wherein the processor is further configured for:
acquiring a third input data using the data gathering device;
processing the third input data to generate a third prediction and a corresponding confidence of the third prediction; and
if the third prediction is below the predetermined threshold:
generating an additional set of at least one action to apply to the data gathering device based on the third input data by generating a third set of acquisition settings of the data gathering device based on the acquisition settings adjustment model as updated;
applying the additional set of at least one action to the data gathering device;
acquiring fourth input data using the gathering device having the additional set at least one action applied thereto.

### Claim 18 (depends on 13)

The system of claim 13, wherein the data gathering device is an image capture device;
wherein the first input data is a first digital image captured by the image capture device.

### Claim 19 (depends on 18)

The system of claim 18, wherein processing the first input data to generate a first prediction comprises applying object classification on the captured digital image;
wherein the first prediction is a classification of an object within the scene of the first captured digital image;
wherein the confidence of the first prediction is a classification score for the classified object of the first captured digital image;
wherein the second input data is a second digital image captured by the image capture device;
wherein the second prediction is a classification of the object within the scene of the second captured digital image; and
wherein the confidence of the second prediction is a classification score for the classified object of the second captured digital image.

### Claim 20 (depends on 19)

The system of claim 19, wherein the first set of acquisition settings of the data gathering device is a first set of image capture settings of the image capture device;
wherein generating the set of at least one action to apply to the data gathering device based on the first input data comprises generating a second set of image capture settings of the image capture device based on an image capture settings adjustment model;
wherein applying the set of actions to the data gathering device comprises configuring the image capture device in accordance with the second set of image capture settings;
wherein acquiring second input data using the data gathering device comprises acquiring the second digital image while the image capture device is configured to the second set of image capture settings; and
wherein modifying the set of at least one action to apply to the data gathering device based on the difference comprises updating the image capture settings adjustment model based on the difference between the classification score for the first captured digital image and the classification score for the second captured digital image.

### Claim 21 (depends on 20)

The system of claim 20, wherein the image capture settings adjustment model is further updated based on the first set of image capture settings and the second set of image capture settings.

### Claim 22 (depends on 20)

The system of claim 20, wherein the first or second image capture settings comprises one or more of zoom, pan, tilt, white balance, exposure, ISO settings, aperture, shutter speed, depth sensor, frame rate, position of a vehicle having the image capture device mounted thereto and lighting subsystem.

### Claim 23 (depends on 20)

The system of claim 20, wherein the image capture settings adjustment model is pre-trained using machine learning; and
wherein updating the image capture settings adjustment model based on the difference is carried out using reinforcement learning.

### Claim 24 (depends on 20)

The system of claim 20, wherein the processor is further configured for:
receiving a third digital image acquired using the image capture device;
processing the third digital image to apply a classification of an object within the scene of the third digital image and to generate a classification score for the classified object of the third captured digital image;
if the classification score for the classified object of the third captured digital image is below the predetermined threshold:
generating an additional set of image capture settings to apply to the image capture device based on the third digital image and the image capture settings adjustment model as updated;
applying the additional set of image capture settings to the image capture device;
receiving a fourth digital image acquired using the image capture device having the additional set of image capture settings applied thereto.

### Claim 25 (depends on 1)

A non-transitory computer-readable medium having instructions stored thereon for execution by a processor, the instructions when executed by the processor performing the method of claim 1.