# US 8,787,682 B2 · Fast image classification by vocabulary tree based image retrieval

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 8,787,682 B2
- Document ID: US8787682B2
- Status: Granted
- Kind: B2
- Application number: 13/339,240
- Filed: 2011-12-28
- Published: 2012-09-27
- Granted: 2014-07-22
- Earliest claimed: 2011-03-22
- Assignees: NEC Laboratories America, Inc.
- Applicants: Ming Yang; Xiaoyu Wang; Yuanqing Lin; Kai Yu
- Inventors: Ming Yang; Xiaoyu Wang; Yuanqing Lin; Kai Yu
- Web page: https://patentgrep.com/patent/US8787682B2

## CPC

- G06V 10/464: {using a plurality of salient features, e.g. bag-of-words [BoW] representations}
- G06V 10/513: {Sparse representations}
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Systems and methods are disclosed to categorize images by detecting local features for each image; applying a tree structure to index local features in the images; and extracting a rank list of candidate images with category tags based on a tree indexing structure to estimate a label of a query image.

## Claims

### Claim 1

A method to categorize images, comprising:
detecting local features for each image;
applying a tree structure to index local features in the images;
extracting a rank list of candidate images with category tags based on a tree indexing structure to estimate a label of a query image, and
determining a category of the query image C(q) by
C
⁡
(
q
)
=
arg
⁢
⁢
max
N
c
=
1
⁢
∑
i
=
1
K
⁢
⁢
1
⁢
(
C
⁡
(
I
i
)
=
c
)
i
,
where retrieved top K candidate images Ii are sorted according to matching score si in descending order {Ii,si}1K, si>sj, ∀i<j and where C(Ii) is an image category index of Iiι.
wherein the above steps are computer implemented.

### Claim 2 (depends on 1)

The method of claim 1, comprising applying a weighted voting according to a rank.

### Claim 3 (depends on 1)

The method of claim 1, comprising performing vocabulary tree based image retrieval in a large-scale image category classifier.

### Claim 4 (depends on 3)

The method of claim 3, wherein the vocabulary tree based image retrieval comprises determining image content based nearest neighbor searching.

### Claim 5 (depends on 3)

The method of claim 3, wherein the vocabulary tree based image retrieval comprises classifying with a linear support vector machine (SVM).

### Claim 6 (depends on 1)

The method of claim 1, comprising generating inverted indices representing local invariant feature of an image.

### Claim 7 (depends on 1)

The method of claim 1, comprising performing a sematic category classification to generate a semantic label for the query image.

### Claim 8

A system to categorize images, comprising:
means for detecting local features for each image;
means for applying a tree structure to index local features in the images;
means for extracting a rank list of candidate images with category tags based on a tree indexing structure to estimate a label of a query image, and
means for determining a category of the query image C(q)
C
⁡
(
q
)
=
arg
⁢
⁢
max
N
c
=
1
⁢
∑
i
=
1
K
⁢
⁢
1
⁢
(
C
⁡
(
I
i
)
=
c
)
i
,
where retrieved top K candidate images Ii are sorted according to matching score si in descending order {Ii,si}1K, si>sj, ∀i<j and where C(Ii) is an image category index of Ii,
wherein the above means are computer implemented.

### Claim 9 (depends on 1)

The method of claim 1, comprising ranking scores of all categories to train a multi-class linear SVM.

### Claim 10 (depends on 1)

The method of claim 1, comprising concatenating ranking scores build a feature xεRN for an image q:
x
=
{
∑
i
=
1
K
⁢
⁢
1
⁢
(
C
⁡
(
I
i
)
=
c
)
i
}
c
=
1
N
.

### Claim 11 (depends on 8)

The system of claim 8, comprising means for concatenating ranking scores build a feature xεRN for an image q:
x
=
{
∑
i
=
1
K
⁢
⁢
1
⁢
(
C
⁡
(
I
i
)
=
c
)
i
}
c
=
1
N
.

### Claim 12 (depends on 8)

The system of claim 8, comprising means for applying a weighted voting according to a rank.

### Claim 13 (depends on 8)

The system of claim 8, comprising means for performing vocabulary tree based image retrieval in a large-scale image category classifier.

### Claim 14 (depends on 8)

The system of claim 8, wherein the vocabulary tree based image retrieval means comprises means for determining image content based nearest neighbor searching.

### Claim 15 (depends on 13)

The system of claim 13, wherein the vocabulary tree based image retrieval means comprises means for classifying with a linear support vector machine (SVM).

### Claim 16 (depends on 8)

The system of claim 8, comprising means for generating inverted indices representing local invariant feature of an image.

### Claim 17 (depends on 8)

The system of claim 8, comprising means for performing a sematic category classification to generate a semantic label for the query image.

### Claim 18 (depends on 8)

The system of claim 8, comprising means for ranking scores of all categories to train a multi-class linear SVM.