# US 2018/0322353 A1 · Systems and methods for electronically identifying plant species

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0322353 A1
- Document ID: US20180322353A1
- Status: Published application
- Kind: A1
- Application number: 15/973,660
- Filed: 2018-05-08
- Published: 2018-11-08
- Earliest claimed: 2017-05-08
- Applicants: PlantSnap, Inc.
- Inventors: Eric Ralls
- Web page: https://patentgrep.com/patent/US20180322353A1

## CPC

- G06F 16/5838: {using colour}
- G06F 16/51: Indexing; Data structures therefor; Storage structures
- G06F 16/5854: {using shape and object relationship}
- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F 18/2415: based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
- G06F 18/41: {Interactive pattern learning with a human teacher}
- G06V 10/17: {using hand-held instruments}
- G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V 10/7788: {the supervisor being a human, e.g. interactive learning with a human teacher}
- G06V 10/95: {structured as a network, e.g. client-server architectures}
- G06V 20/20: in augmented reality scenes
- G06N 20/00: Machine learning
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A system is described herein comprising an application running on a processor of a mobile device, the application receiving a query image, wherein the application is communicatively coupled with one or more applications running on at least one processor of at least one remote server. The system includes the application providing the query image to the one or more applications, the one or more applications processing the query image to identify a query type corresponding to the query image, wherein the query type comprises a plurality of species. The system includes using a query type recognition engine corresponding to the query type to process the query image, the processing the query image including identifying at least one species corresponding to the query image. The system includes the one or more applications providing information of the at least one species to the application, wherein the application displays the information.

## Claims

### Claim 1

A system comprising,
an application running on a processor of a mobile device, the application receiving a query image, wherein the application is communicatively coupled with one or more applications running on at least one processor of at least one remote server;
the application providing the query image to the one or more applications, the one or more applications processing the query image to identify a query type corresponding to the query image, wherein the query type comprises a plurality of species;
using a query type recognition engine corresponding to the query type to process the query image, wherein the one or more applications include the query type recognition engine, the processing the query image including identifying at least one species corresponding to the query image;
the one or more applications providing information of the at least one species to the application, wherein the application displays the information of the at least one species.

### Claim 2 (depends on 1)

The system of claim 1, wherein the providing the information includes providing a level of confidence for each species of the at least one species.

### Claim 3 (depends on 1)

The system of claim 1, wherein the query type comprises one or more of a leaf, a flower, a whole plant, and grass.

### Claim 4 (depends on 1)

The system of claim 1, comprising providing training images to the one or more applications for each combination of query type and species of the plurality of species.

### Claim 5 (depends on 4)

The system of claim 4, the one or more applications using the training images to train the query type recognition engine, the training the query type recognition engine comprising defining attributes for each combination of query type and species of the plurality of species.

### Claim 6 (depends on 5)

The system of claim 5, the query type recognition engine using information of the attributes to identify the at least one species.

### Claim 7 (depends on 4)

The system of claim 4, the providing the training images comprising curating the training images from at least one database.

### Claim 8 (depends on 7)

The system of claim 7, wherein the at least one database includes a United States Department of Agriculture (USDA) database and an Encyclopedia of Life™ database.

### Claim 9 (depends on 4)

The system of claim 4, the providing the training images comprising curating the training images through image searching using one or more image search engines.

### Claim 10 (depends on 9)

The system of claim 9, wherein the one or more image search engines include Google™ and flickr™.

### Claim 11 (depends on 4)

The system of claim 4, the providing the training images comprising augmenting the training images, the augmenting comprising producing additional images derived from at least one image of the training images.

### Claim 12 (depends on 11)

The system of claim 11, the producing the additional images including one or more of rotating the at least one image, cropping the at least one image, manipulating the at least one image to simulate variable camera angles, segmenting the at least one image, and superimposing the at least one image on at least one different background.

### Claim 13 (depends on 1)

The system of claim 1, the receiving the query image comprising receiving the query image through operation of a camera of the mobile device.

### Claim 14 (depends on 13)

The system of claim 13, the receiving the query image including receiving a GPS location of the mobile device at the moment the mobile device camera captures the query image.

### Claim 15 (depends on 1)

The system of claim 1, the application receiving a request for additional information regarding the at least one species.

### Claim 16 (depends on 15)

The method of claim 15, the application requesting the additional information from api.earth.com.

### Claim 17 (depends on 16)

The system of claim 16, wherein the additional information comprises at least one of species, common name, kingdom, order, family, genus, title, and description.

### Claim 18 (depends on 1)

The system of claim 1, the application receiving a refusal to accept an identification of the at least one species, the application receiving a suggested identification of the at least one species.

### Claim 19

A system comprising,
an application running on a processor of a mobile device, the application receiving a query image, wherein the application is communicatively coupled with one or more applications running on at least one processor of at least one remote server;
the application providing the query image to the one or more applications, the one or more applications processing the query image to identify a query type corresponding to the query image, wherein the query type comprises a plurality of species;
using a query type recognition engine corresponding to the query type to process the query image, wherein the one or more applications include the query type recognition engine, the processing the query image including identifying at least one species corresponding to the query image;
providing training images to the one or more applications for each combination of query type and species of the plurality of species to train the query type recognition engine in identifying the at least one species;
the one or more applications providing information of the at least one species to the application, wherein the application displays the information of the at least one species.
