# US 2016/0353715 A1 · Agricultural monitoring system using image analysis

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2016/0353715 A1
- Document ID: US20160353715A1
- Status: Published application
- Kind: A1
- Application number: 15/173,614
- Filed: 2016-06-03
- Published: 2016-12-08
- Earliest claimed: 2015-06-03
- Assignees: Keltronix, Inc.
- Applicants: Keltronix, Inc.
- Inventors: Kelton Temby; Jonathan Simpson
- Web page: https://patentgrep.com/patent/US20160353715A1

## CPC

- A01K 47/06: Other details of beehives, e.g. ventilating devices, entrances to hives, guards, partitions or bee escapes
- G06V 20/53: {Recognition of crowd images, e.g. recognition of crowd congestion}
- G06V 10/46: Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features (colour feature extraction G06V10/56)
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- A01K: ANIMAL HUSBANDRY; AVICULTURE; APICULTURE; PISCICULTURE; FISHING; REARING OR BREEDING ANIMALS, NOT OTHERWISE PROVIDED FOR; NEW BREEDS OF ANIMALS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Aspects of this disclosure include a system for providing non-contact, computer-vision based monitoring of the health and pollination activity of a beehive. The system may include camera positioned proximate to a beehive. The camera may include an onboard processor that analyzes video of the beehive captured by the camera and calculates an activity value that estimates a number of bees moving about the beehive. The video calculated activity values may be uploaded to a server where they can be accessed via a user device. The user device may allow the user to display interactive plots of the activity values over a variety of time bases. The disclosed beehive monitoring system relies on relatively lost-cost hardware and requires neither modification to the hive nor special constraints on the placement of the camera.

## Claims

### Claim 1

An apparatus for monitoring a beehive, the apparatus comprising:
a camera that captures a sequence of video frames of the exterior of the beehive at a predetermined frame rate, the sequence of video frames including at least a first video frame, a second video frame, and a third video frame; and,
a processor in communication with the camera system, the processor performs image analysis on the sequence of video frames to calculate an activity value that represents an estimate of the number of bees in the vicinity of the beehive, wherein the image analysis performed by the processor includes:
generating a first difference frame by subtracting the second video frame from the first video frame;
generating a second difference frame by subtracting the third video frame from the second video frame;
generating a differential image frame by combining the first difference frame and the second difference frame using a logical AND operation;
generating a modified differential image frame by performing a thresholding operation on the differential image frame;
identifying a set of contours in the modified differential image frame by performing blob detection on the modified differential image frame;
generating a count of the number of contours in the set of contours having an area that falls within a predefined range;
calculating the activity value by multiplying the count by the predetermined frame rate; and,
storing the activity value on a storage device for display on a user device.

### Claim 2 (depends on 1)

The apparatus of claim 1 further comprising a local storage device for storing the activity value in association with the sequence of video frames.

### Claim 3 (depends on 1)

The apparatus of claim 1 further comprising a communication system in communication with the processor, wherein the communication system transmits the activity value to a server for retrieval by the user device.

### Claim 4 (depends on 3)

The apparatus of claim 3, wherein the communication system transmits the sequence of video frames to the server.

### Claim 5 (depends on 3)

The apparatus of claim 3, wherein the communication system transmits data received from one or more environmental sensors to the server.

### Claim 6 (depends on 1)

The apparatus of claim 1 further comprising a mobile platform coupled to the camera system.

### Claim 7 (depends on 1)

The apparatus of claim 1 further comprising solar-powered power supply and a battery.

### Claim 8

An apparatus for monitoring a beehive, the apparatus comprising:
a camera that captures a sequence of video frames of the exterior of a beehive; and,
a processor in communication with the camera system, the processor performs image analysis on the sequence of video frames to calculate an activity value that represents an estimate of the number of bees in the vicinity of the beehive, wherein the image analysis performed by the processor includes:
generating a series of contour count values representing a number of contours identified in each frame, wherein generating the series includes:
generating an average frame by combining a current frame and at least one previous frame;
generating a differential frame by subtracting the average frame from the current frame;
generating a first modified differential frame by performing an erosion operation on the differential frame;
generating a second modified differential frame by performing a dilation operation on the first modified differential frame;
identifying a set of contours in the second modified differential frame; and,
generating a count of the number of contours in the set of contours;
generating a filtered series of contour count values by applying a low-pass filter to the series of contour count values;
incrementing the activity value for each increase in the filtered series of contour values; and,
storing the activity value on a storage device for display on a user device.

### Claim 9 (depends on 8)

The apparatus of claim 8 further comprising a local storage device for storing the activity value in association with the sequence of video frames.

### Claim 10 (depends on 8)

The apparatus of claim 8 further comprising a communication system in communication with the processor, wherein the communication system transmits the activity value to a server for retrieval by the user device.

### Claim 11 (depends on 9)

The apparatus of claim 9, wherein the communication system transmits the sequence of video frames to the server.

### Claim 12 (depends on 9)

The apparatus of claim 9, wherein the communication system transmits data received from one or more environmental sensors to the server.

### Claim 13 (depends on 7)

The apparatus of claim 7 further comprising a mobile platform coupled to the camera system.

### Claim 14 (depends on 7)

The apparatus of claim 7 further comprising a solar-powered power supply and a battery.

### Claim 15

A system for monitoring the health of a beehive, the system comprising:
a camera that captures a sequence of video frames of the exterior of the beehive;
a processor in communication with the camera, the processor analyzes the sequence of video frames to calculate one or more activity values that represents an estimate of the number of bees in the vicinity of the beehive in the sequence of video frames;
a server in communication with the processor, the server stores the sequence of video frames in association with the one or more activity values; and,
a user device in communication with the server, the user device displays a plot of activity values over a time base.

### Claim 16 (depends on 15)

The system of claim 15, wherein the plot displayed on the user device is interactive and allows the user to modify the time base of the plot.

### Claim 17 (depends on 15)

The system of claim 15, wherein the plot is interactive and allows the user to select a time index and play back a video sequence associated with the selected time index.

### Claim 18 (depends on 15)

The system of claim 15, wherein the plot of activity values include peak activity values.

### Claim 19 (depends on 15)

The system of claim 15, wherein the plot of activity values include average activity values.

### Claim 20 (depends on 15)

The system of claim 15, wherein the server obtains first and second activity value series for a predetermined time period from first and second beehives, respectively, determines a correlation value based on the first and second activity value series, and transmits a notification to the user device if the correlation value falls below a predetermined threshold.
