# US 2018/0025480 A1 · Apparatus and method for analysis of growing items

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0025480 A1
- Document ID: US20180025480A1
- Status: Published application
- Kind: A1
- Application number: 15/549,046
- Filed: 2016-02-05
- Published: 2018-01-25
- Earliest claimed: 2015-02-05
- Assignees: The Technology Research Centre Ltd.
- Applicants: The Technology Research Centre Ltd.
- Inventors: Laurence Dingle; Chris Roberts; Michael Kalyn; David Samworth
- Web page: https://patentgrep.com/patent/US20180025480A1

## CPC

- A01G 7/00: Botany in general
- G01N 21/84: Systems specially adapted for particular applications
- G06F 16/51: Indexing; Data structures therefor; Storage structures
- G06F 18/2113: by ranking or filtering the set of features, e.g. using a measure of variance or of feature cross-correlation
- G06T 7/001: {using an image reference approach}
- G06T 7/10: Segmentation; Edge detection (motion-based segmentation G06T7/215)
- G06T 7/74: {involving reference images or patches}
- G06T 7/85: {Stereo camera calibration}
- G06V 20/188: {Vegetation}
- G06V 20/38: {Outdoor scenes}
- G01N 33/025: {Fruits or vegetables}
- G01N 2021/8466: {Investigation of vegetal material, e.g. leaves, plants, fruits}
- G06Q 30/0206: {Price or cost determination based on market factors}
- G06T 2207/10021: Stereoscopic video; Stereoscopic image sequence
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/30188: Vegetation; Agriculture
- G06T 2207/30242: Counting objects in image
- G06T 2207/30252: Vehicle exterior; Vicinity of vehicle
- A01G: HORTICULTURE; CULTIVATION OF VEGETABLES, FLOWERS, RICE, FRUIT, VINES, HOPS OR SEAWEED; FORESTRY; WATERING (picking of fruits, vegetables, hops or the like A01D46/00; propagating unicellular algae C12N1/12)
- G01N: INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES (measuring or testing apparatus or processes other than immunoassay, involving enzymes or microorganisms C12M, C12Q)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

A vehicle-mounted apparatus for analysis of growing items having a camera configured to capture multiple sequential images of growing items, a memory comprising an image buffer for storing the images, a processor, software, which when executed on the processor is configured to repeatedly call an image from the image buffer, process the image to identify individual growing items to create item data relating to individual growing items, and update a growing item database by adding the item data wherein the software is configured to run whilst the camera captures further images, and the images are discarded once processed.

## Claims

### Claim 1

An apparatus for analysis of growing items, the apparatus suitable for installation on a vehicle and comprising:
at least one camera configured to capture multiple sequential images of growing items;
a memory comprising an image buffer for storing the images;
a processor;
software, which when executed on the processor is configured to repeatedly:
call an image from the image buffer;
process the image to identify individual growing items to create item data relating to individual growing items;
update a growing item database by adding the item data to the growing item database;
wherein during vehicle motion:
the software is configured to run whilst the camera captures further images; and,
at least some of the images are discarded once they have been processed by the software.

### Claim 2 (depends on 1)

An apparatus according to claim 1, comprising a GPS receiver for generating location data, in which the software is configured to combine the item data with location data and update the growing item database with the combined item data and location data.

### Claim 3

(canceled)

### Claim 4 (depends on 3)

An apparatus according to claim 3, in which the software is configured to identify and disregard growing items which were previously added to the growing item database by carrying out the following steps:
identifying a growing item in a first image;
predicting the position of the growing item in a second, subsequent image; and,
disregarding the growing item in the subsequent image if the position matches the predicted position within a predetermined margin of error.

### Claim 5 (depends on 4)

An apparatus according to claim 4, in which the step of predicting the position of the growing item is carried out using a velocity vector based on the speed and direction of a vehicle on which the apparatus is mounted.

### Claim 6 (depends on 5)

An apparatus according to claim 5, in which the velocity vector is refined by calculating the speed and direction of the growing object based on changes in its position between at least two images.

### Claim 7 (depends on 1)

An apparatus according to claim 1, in which images are removed from the buffer when they are called by the software.

### Claim at position 8

-13. (canceled)

### Claim 14 (depends on 1)

An apparatus according to claim 1, in which the software is configured to identify seed pixels in the image using a co-occurrence matrix, and to carry out a segmentation step to identify further candidate pixels proximate the seed pixels.

### Claim at position 10

-17. (canceled)

### Claim 18 (depends on 14)

An apparatus according to claim 14, in which in the segmentation step, an algorithm is selected from the list comprising a flood fill algorithm and a spiral algorithm based on a set of segmentation parameters less stringent than the seed parameters.

### Claim 19

(canceled)

### Claim 20 (depends on 18)

An apparatus according to claim 18, in which the software is configured to filter individual regions of pixels are based on size, and regions below a certain size are disregarded.

### Claim 21 (depends on 20)

An apparatus according to claim 20, in which the software is configured to produce item data by analysis of at least one of the size, shape, colour and aspect ratio of the regions of pixels.

### Claim 22 (depends on 21)

An apparatus according to claim 21, in which the software is configured to identify partially obscured growing items and to extrapolate at least one of their size and shape.

### Claim at position 16

-27. (canceled)

### Claim 28 (depends on 1)

An apparatus according to claim 1, in which the at least one camera comprises a first camera and a second camera having an overlapping field of vision with the first camera, in which the software is configured to calculate a depth of field between the cameras and the growing items using a stereoscopic measuring technique based on an overlap between simultaneously captured images.

### Claim 29 (depends on 28)

An apparatus according to claim 28, in which the software is configured to process the image to size individual growing items based on the calculated depth of field.

### Claim 30 (depends on 29)

An apparatus according to claim 29, in which individual growing items outside the overlapping field of vision are sized based on the calculated depth of field.

### Claim 31 (depends on 1)

An apparatus according to claim 1, configured to:
determine a status of the buffer;
output a speed feedback signal based on the status of the buffer.

### Claim 32 (depends on 31)

An apparatus according to claim 31, in which in determining the status of the buffer the apparatus determines the amount of data in the buffer.

### Claim 33 (depends on 31)

An apparatus according to claim 31, comprising a user display, in which the feedback signal is fed to the user display.

### Claim at position 23

-35. (canceled)

### Claim 36 (depends on 1)

An apparatus according to claim 1, configured to:
count the number of growing items on a given plant, plurality of plants or part of a plant;
compare the counted items to an expected count; and,
if the counted number of items differs from the expected count by a first predetermined margin, prompt a user to perform a calibration process.

### Claim 37

(canceled)

### Claim 38 (depends on 36)

An apparatus according to claim 36, in which the calibration process is configured to receive a manual count and compare the manual count to the counted number of items, in which if the manual count differs by a second predetermined margin from the counted number of items, to apply or update a factor to the counted number of items to bring the counted number of items within the second predetermined margin.

### Claim at position 27

-41. (canceled)

### Claim 42

A method of analysis of growing items comprising the steps of:
providing a vehicle;
providing an apparatus mounted on the vehicle, the apparatus comprising:
at least one camera configured to capture multiple sequential images of growing items;
a memory comprising an image buffer for storing the images;
a processor;
software, which when executed on the processor is configured to repeatedly:
call an image from the image buffer;
process the image to identify individual growing items to create item data relating to individual growing items; and
update a growing item database by adding the item data;
causing the vehicle to move, and whilst the vehicle is in motion;
running the software whilst the at least one camera captures images; and
discarding at least some of the images once processed by the software.

### Claim at position 29

-109. (canceled)