# US 9,516,854 B2 · Vision system for facilitating the automated application of disinfectant to the teats of dairy livestock

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 9,516,854 B2
- Document ID: US9516854B2
- Status: Granted
- Kind: B2
- Application number: 14/169,677
- Filed: 2014-01-31
- Published: 2014-05-29
- Granted: 2016-12-13
- Earliest claimed: 2010-08-31
- Assignees: Technologies Holdings Corp.
- Applicants: Technologies Holdings Corp.
- Inventors: Henk Hofman; Peter Willem van der Sluis; Ype Groensma
- Web page: https://patentgrep.com/patent/US9516854B2

## CPC

- A01J 7/04: for treatment of udders or teats, e.g. for cleaning
- A01J 5/003: Movable milking machines
- A01J 5/007: Monitoring milking processes; Control or regulation of milking machines
- A01J 5/0175: {Attaching of clusters}
- A01J 7/00: Accessories for milking machines or devices
- A01K 1/126: {Carousels}
- A01K 13/001: {Washing, cleaning, or drying devices (for veterinary purposes A61D7/00, A61D11/00)}
- A01K 29/00: Other apparatus for animal husbandry
- A01K 29/004: {for influencing biological activity}
- B25J 9/0027: {Means for extending the operation range}
- B25J 9/0093: {co-operating with conveyor means}
- B25J 9/0096: {co-operating with a working support, e.g. work-table}
- B25J 9/1679: {characterised by the tasks executed}
- B25J 9/1684: {Tracking a line or surface by means of sensors}
- B25J 9/1697: {Vision controlled systems}
- B25J 11/0045: {Manipulators used in the food industry}
- G05B 19/042: using digital processors (G05B19/05 takes precedence)
- G06T 1/0014: {Image feed-back for automatic industrial control, e.g. robot with camera (robots B25J19/023)}
- G06T 7/0004: {Industrial image inspection}
- G06T 7/13: Edge detection
- G06T 7/73: using feature-based methods
- H04N 7/183: {for receiving images from a single remote source}
- G05B 2219/2661: Milking robot
- G05B 2219/45113: Animal handling, milking robot
- G06T 7/74: {involving reference images or patches}
- G06T 2200/04: involving 3D image data
- G06T 2207/10004: Still image; Photographic image
- G06T 2207/10021: Stereoscopic video; Stereoscopic image sequence
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- Y10S 901/02: Arm motion controller
- Y10S 901/06: Communication with another machine
- Y10S 901/07: Conveyor
- Y10S 901/09: Closed loop, sensor feedback controls arm movement
- Y10S 901/41: Tool
- Y10S 901/43: Spray painting or coating
- Y10S 901/47: Optical
- A01J: MANUFACTURE OF DAIRY PRODUCTS
- A01K: ANIMAL HUSBANDRY; AVICULTURE; APICULTURE; PISCICULTURE; FISHING; REARING OR BREEDING ANIMALS, NOT OTHERWISE PROVIDED FOR; NEW BREEDS OF ANIMALS
- B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50})
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- Y10S: TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS

## Abstract

A method for determining a spray position for a spray tool includes accessing an image signal generated by a camera, the image signal corresponding to at least an udder of a dairy livestock. The method further includes processing the accessed image signal to determine a tangent to the rear and a tangent to the bottom of the udder of the dairy livestock. The method concludes by determining a spray position from which a spray tool may apply disinfectant to the teats of the dairy livestock, wherein the spray position is a position relative to the intersection of the two tangents.

## Claims

### Claim 1

A method, comprising:
accessing an image signal generated by a camera, the image signal corresponding to one or more features of a dairy livestock;
determining, based on the image signal, positions of each of the hind legs of the dairy livestock;
determining, based on the image signal and the determined positions of the hind legs of the dairy livestock, a position of an udder of the dairy livestock;
determining, based on the image signal and the determined position of the udder of the dairy livestock, a spray position from which a spray tool may apply disinfectant to the teats of the dairy livestock, wherein determining the spray position comprises processing the accessed image signal to determine a tangent at the rear of the udder and a tangent at the bottom of the udder, the spray position being a position relative to the intersection of the two tangents.

### Claim 2 (depends on 1)

The method of claim 1, wherein the spray position is located beneath one or more teats of the dairy livestock.

### Claim 3 (depends on 1)

The method of claim 1, wherein the image signal comprises a three-dimensional video image signal.

### Claim 4 (depends on 1)

The method of claim 1, wherein determining positions of each of the hind legs of the dairy livestock comprises processing the accessed image signal to locate edges in depth, the edges in depth corresponding to the edges of the hind legs of the dairy livestock.

### Claim 5 (depends on 4)

The method of claim 4, wherein determining the position of the udder of the dairy livestock comprises processing the accessed image signal to trace one or more of the located edges in depth upwardly until they intersect with the udder.

### Claim 6 (depends on 1)

The method of claim 1, further comprising communicating, in response to determining positions of each of the hind legs of the dairy livestock, one or more signals to one or more actuators of a robotic arm on which the camera is located, the one or more signals causing the one or more actuators to move the robotic arm between the hind legs of the dairy livestock.

### Claim 7 (depends on 1)

The method of claim 1, further comprising communicating, in response to determining the spray position, one or more signals to one or more actuators of a robotic arm, the one or more signals causing the one or more actuators to move a spray tool located on the robotic arm to the spray position such that spray tool may apply disinfectant to the teats of the dairy livestock.

### Claim 8

A method, comprising:
accessing an image signal generated by a camera, the image signal corresponding to at least an udder of a dairy livestock;
processing the accessed image signal to determine a tangent at the rear of the udder of the dairy livestock;
processing the accessed image signal to determine a tangent at the bottom of the udder of the dairy livestock; and
determining a spray position from which a spray tool may apply disinfectant to a plurality of teats of the dairy livestock, wherein the spray position is a position relative to the intersection of the two tangents.

### Claim 9 (depends on 8)

The method of claim 8, wherein the spray position is located beneath one or more teats of the dairy livestock.

### Claim 10 (depends on 8)

The method of claim 8, wherein the image signal comprises a three-dimensional video image signal.

### Claim 11 (depends on 8)

The method of claim 8, further comprising moving a robotic arm carrying the camera between hind legs of the dairy livestock.

### Claim 12 (depends on 8)

The method of claim 8, further comprising moving the spray tool located on a robotic arm to the spray position such that spray tool may apply disinfectant to the teats of the dairy livestock.

### Claim 13

A method, comprising:
accessing an image signal generated by a camera, the image signal corresponding to at least an udder of a dairy livestock;
using the image signal to determine a tangent at the rear of the udder and a tangent at the bottom of the udder;
determining a spray position from which a spray tool may apply disinfectant to a plurality of teats of the dairy livestock, wherein the spray position is a position relative to the intersection of the two tangents.

### Claim 14 (depends on 13)

The method of claim 13, wherein the spray position is located beneath one or more teats of the dairy livestock.

### Claim 15 (depends on 13)

The method of claim 13, wherein the image signal comprises a three-dimensional video image signal.

### Claim 16 (depends on 13)

The method of claim 13, further comprising moving a robotic arm carrying the camera between hind legs of the dairy livestock.

### Claim 17 (depends on 13)

The method of claim 13, further comprising moving the spray tool located on a robotic arm to the spray position such that spray tool may apply disinfectant to the teats of the dairy livestock.