# US 10,769,442 B1 · Scene change detection in image data

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,769,442 B1
- Document ID: US10769442B1
- Status: Granted
- Kind: B1
- Application number: 15/700,536
- Filed: 2017-09-11
- Granted: 2020-09-08
- Earliest claimed: 2017-09-11
- Assignees: Amazon Technologies, Inc.
- Applicants: Amazon Technologies, Inc.
- Inventors: Siyi Ding; Xiaoran Wang; Manika Puri; Mashhour Solh; Philip James Taylor
- Web page: https://patentgrep.com/patent/US10769442B1

## CPC

- G06T 7/254: involving subtraction of images
- G06F 18/22: Matching criteria, e.g. proximity measures
- G06T 7/223: using block-matching
- G06T 7/248: {involving reference images or patches}
- G06V 10/761: {Proximity, similarity or dissimilarity measures}
- G06V 20/41: {Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49)}
- G06V 20/46: {Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames}
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G06V 40/20: Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16)
- G06T 2207/10016: Video; Image sequence
- G06T 2207/10024: Color image
- G06T 2207/10048: Infrared image
- G06T 2207/30232: Surveillance
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Techniques are generally described for scene change detection. A first and second histogram representing frames of a video may be received. A Euclidean distance between the first and second histogram may be determined. A third histogram of a third frame may be received. Values of the third histogram may be compared to corresponding values of a background model of the environment. A fourth frame and a fifth frame of image data of the video may be received. A flow value between a first block of pixels of the fourth frame and a corresponding second block of pixels of the fifth frame may be determined. The flow value may be determined based on a motion vector and a sum of absolute differences between the first and second block. A portion of the video may be streamed to a remote computing device if the flow value exceeds a threshold value.

## Claims

### Claim 1

A method comprising:
determining a Euclidean distance between a first histogram of a first frame of video data representing an environment and a second histogram of a second frame of the video data;
determining that the Euclidean distance exceeds a threshold value;
determining, in response to the Euclidean distance exceeding the threshold value, a flow value representing changes in the environment represented in a third frame of the video data and a fourth frame of the video data, wherein the flow value is determined based at least in part on a motion vector between a first block of pixels of the third frame of the video data and a second block of pixels of the fourth frame of the video data, and a sum of absolute differences (SAD) between the first block of pixels and the second block of pixels;
determining that the flow value exceeds a flow threshold value; and
transmitting to a remote computing device at least a first portion of the video data subsequent to the first frame based at least in part on the flow value exceeding the flow threshold value.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
receiving a plurality of other histograms, wherein each of the plurality of other histograms is associated with a respective one of a plurality of sequential frames of the video data following the first frame and the second frame, and wherein each of the plurality of other histograms has a plurality of bins with each of the plurality of bins comprising a first bin value; and
generating a model of the environment, wherein the model comprises a plurality of distributions, wherein each of the plurality of distributions is associated with a corresponding one of the plurality of bins, wherein each distribution has a corresponding standard deviation band of +/−3σ.

### Claim 3 (depends on 2)

The method of claim 2, further comprising:
receiving a third histogram of a fifth frame of the video data, wherein the third histogram comprises a plurality of histogram values; and
for each of a first number of bins of the third histogram, determining that a corresponding first bin value of the first number of bins lies outside a standard deviation band of the model of the environment.

### Claim 4 (depends on 1)

The method of claim 1, further comprising determining an average luma value of each block of pixels of the third frame, wherein the determining the flow value is further based at least in part on the average luma value of the first block of pixels.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
determining a second flow value representing changes in the environment represented in a fifth frame of the video data and a sixth frame of the video data, wherein the second flow value is determined based at least in part on a second motion vector between a third block of pixels of the fifth frame of the video data and a fourth block of pixels of the sixth frame of the video data, and a second sum of absolute differences (SAD) between the third block of pixels and the fourth block of pixels;
determining that the second flow value exceeds the flow threshold value; and
sending a second number of frames of the video data to the remote computing device, wherein the second number of frames represents a second portion of the video that is consecutive to the first portion of the video.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
storing the flow value in a memory;
receiving an indication of an exposure change of a camera device capturing the video data; and
resetting the flow value stored in the memory to an initial value.

### Claim 7 (depends on 1)

The method of claim 1, wherein the flow value is stored in a memory, the method further comprising:
resetting the flow value to an initial value;
determining a second flow value representing changes in the environment represented in a fifth frame of the video data and a sixth frame of the video data;
determining, after a first period of time, that the second flow value does not exceed the flow threshold value;
determining a second Euclidean distance between a third histogram of a seventh frame and of the video data and a fourth histogram of an eighth frame of the video data;
determining that the second Euclidean distance does not exceed the threshold value; and
ceasing streaming of the video data following the first portion of the video data.

### Claim 8 (depends on 1)

The method of claim 1, further comprising:
receiving, from a camera device, an indication that infrared lighting conditions of the environment have changed from a first level to a second level;
receiving a fifth frame of the video data, wherein the fifth frame of the video data is captured by the camera device within a first amount of time after receiving the indication;
determining that the fifth frame comprises unstable lighting conditions based on the first amount of time being less than a threshold amount of time; and
ceasing to transmit at least a second portion of the video data to the remote computing device.

### Claim 9

A computing device comprising:
a camera device;
at least one processor configured in communication with the camera device; and
a non-transitory, computer-readable memory configured in communication with the at least one processor, the memory storing instructions that, when executed by the at least one processor, are effective to cause the at least one processor to perform a method comprising:
determining a Euclidean distance between a first histogram of a first frame of video data representing an environment and a second histogram of a second frame of the video data;
determining that the Euclidean distance exceeds a threshold value;
determining, in response to the Euclidean distance exceeding the threshold value, a flow value representing changes in the environment represented in a third frame of the video data and a fourth frame of the video data, wherein the flow value is determined based at least in part on a motion vector between a first block of pixels of the third frame of the video data and a second block of pixels of the fourth frame of the video data, and a sum of absolute differences (SAD) between the first block of pixels and the second block of pixels;
determining that the flow value exceeds a flow threshold value; and
transmitting to a remote computing device at least a first portion of the video data subsequent to the first frame based at least in part on the flow value exceeding the flow threshold value.

### Claim 10 (depends on 9)

The computing device of claim 9, wherein the instructions, when executed by the at least one processor, are effective to perform the method, further comprising:
receiving a plurality of other histograms, wherein each of the plurality of other histograms is associated with a respective one of a plurality of sequential frames of the video data following the first frame and the second frame, and wherein each of the plurality of other histograms has a plurality of bins with each of the plurality of bins comprising a first bin value; and
generating a model of the environment, wherein the model comprises a plurality of distributions, wherein each of the plurality of distributions is associated with a corresponding one of the plurality of bins, wherein each distribution has a corresponding standard deviation band of +/−3σ.

### Claim 11 (depends on 10)

The computing device of claim 10, wherein the instructions, when executed by the at least one processor, are effective to perform the method, further comprising:
receiving a third histogram of a fifth frame of the video data, wherein the third histogram comprises a plurality of histogram values; and
for each of a first number of bins of the third histogram, determining that a corresponding first bin value of the first number of bins lies outside a standard deviation band of the model of the environment.

### Claim 12 (depends on 9)

The computing device of claim 9, wherein the instructions, when executed by the at least one processor, are effective to perform the method, further comprising:
determining an average luma value of each block of pixels of the third frame, wherein the determining the flow value is further based at least in part on the average luma value of the first block of pixels.

### Claim 13 (depends on 9)

The computing device of claim 9, wherein the instructions, when executed by the at least one processor, are effective to perform the method, further comprising:
determining a second flow value representing changes in the environment represented in a fifth frame of the video data and a sixth frame of the video data, wherein the second flow value is determined based at least in part on a second motion vector between a third block of pixels of the fifth frame of the video data and a fourth block of pixels of the sixth frame of the video data, and a second sum of absolute differences (SAD) between the third block of pixels and the fourth block of pixels;
determining that the second flow value exceeds the flow threshold value; and
sending a second number of frames of the video data to the remote computing device, wherein the second number of frames represents a second portion of the video data that is consecutive to the first portion of the video data.

### Claim 14 (depends on 9)

The computing device of claim 9, wherein the instructions, when executed by the at least one processor, are effective to perform the method, further comprising:
storing the flow value in the memory;
receiving an indication of an exposure change of the camera device; and
resetting the flow value stored in the memory to an initial value.

### Claim 15 (depends on 9)

The computing device of claim 9, wherein the instructions, when executed by the at least one processor, are effective to perform the method, further comprising:
resetting the flow value to an initial value;
determining a second flow value representing changes in the environment represented in a fifth frame of the video data and a sixth frame of the video data;
determining, after a first period of time, that the second flow value does not exceed the flow threshold value;
determining a second Euclidean distance between a third histogram of a seventh frame and of the video data and a fourth histogram of an eighth frame of the video data;
determining that the second Euclidean distance does not exceed the threshold value; and
ceasing streaming of the video data following the first portion of the video data.

### Claim 16 (depends on 9)

The computing device of claim 9, wherein the flow value is stored in the memory, and the instructions, when executed by the at least one processor, are effective to perform the method, further comprising:
receiving, from the camera device, an indication that infrared lighting conditions of the environment have changed from a first level to a second level;
receiving a fifth frame of the video data, wherein the fifth frame of the video data is captured by the camera device within a first amount of time after receiving the indication;
determining that the fifth frame comprises unstable lighting conditions based on the first amount of time being less than a threshold amount of time; and
ceasing to transmit at least a second portion of the video data to the remote computing device.

### Claim 17

A method comprising:
determining a Euclidean distance between a first histogram of a first frame of video data representing an environment and a second histogram of a second frame of the video data;
determining that the Euclidean distance exceeds a threshold value;
determining a first flow value representing changes in the environment represented in a third frame of the video data and a fourth frame of the video data, wherein the first flow value is determined based at least in part on a first motion vector between a first block of pixels of the third frame of the video data and a second block of pixels of the fourth frame of the video data, and a first sum of absolute differences (SAD) between the first block of pixels and the second block of pixels;
determining that the first flow value exceeds a flow threshold value;
sending a first number of frames of the video data representing a first portion of the video data to a remote computing device;
determining a second flow value representing changes in the environment represented in a fifth frame of the video data and a sixth frame of the video data, wherein the second flow value is determined based at least in part on a second motion vector between a third block of pixels of the fifth frame of the video data and a fourth block of pixels of the sixth frame of the video data, and a second sum of absolute differences (SAD) between the third block of pixels and the fourth block of pixels;
determining that the second flow value exceeds the flow threshold value; and
sending a second number of frames of the video data to the remote computing device, wherein the second number of frames represents a second portion of the video that is consecutive to the first portion of the video data.

### Claim 18 (depends on 17)

The method of claim 17, further comprising:
receiving a plurality of other histograms, wherein each of the plurality of other histograms is associated with a respective one of a plurality of sequential frames of the video data following the first frame and the second frame, and wherein each of the plurality of other histograms has a plurality of bins with each of the plurality of bins comprising a first bin value; and
generating a model of the environment, wherein the model comprises a plurality of distributions, wherein each of the plurality of distributions is associated with a corresponding one of the plurality of bins, wherein each distribution has a corresponding standard deviation band of +/−3σ.

### Claim 19 (depends on 18)

The method of claim 18, further comprising:
receiving a third histogram of a fifth frame of the video data, wherein the third histogram comprises a plurality of histogram values; and
for each of a first number of bins of the third histogram, determining that a corresponding first bin value of the first number of bins lies outside a standard deviation band of the model of the environment.

### Claim 20 (depends on 17)

The method of claim 17, further comprising determining an average luma value of each block of pixels of the third frame, wherein the determining the flow value is further based at least in part on the average luma value of the first block of pixels.