# US 2019/0318195 A1 · Robust feature identification for image-based object recognition

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0318195 A1
- Document ID: US20190318195A1
- Status: Published application
- Kind: A1
- Application number: 16/450,876
- Filed: 2019-06-24
- Published: 2019-10-17
- Earliest claimed: 2014-04-24
- Assignees: Nant Holdings IP, LLC
- Applicants: Nant Holdings IP, LLC
- Inventors: Bing Song; Liwen Lin; Mustafa Jaber
- Web page: https://patentgrep.com/patent/US20190318195A1

## CPC

- G06V 10/50: by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
- G06F 18/22: Matching criteria, e.g. proximity measures
- G06T 3/00: Geometric image transformations in the plane of the image
- G06T 3/40: Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06V 10/42: Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
- G06V 10/75: Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Techniques are provided that include identifying robust features within a training image. Training features are generated by applying a feature detection algorithm to the training image, each training feature having a training feature location within the training image. At least a portion of the training image is transformed into a transformed image in accordance with a predefined image transformation. Transform features are generated by applying the feature detection algorithm to the transformed image, each transform feature having a transform feature location within the transformed image. The training feature locations of the training features are mapped to corresponding training feature transformed locations within the transformed image in accordance with the predefined image transformation, and a robust feature set is compiled by selecting robust features, wherein each robust feature represents a training feature having a training feature transformed location proximal to a transform feature location of one of the transform features.

## Claims

### Claim at position 1

1-31. (canceled)

### Claim 32

A product image feature detection device comprising:
a tangible, non-transitory, computer-readable memory configured to store robust feature detection software instructions including at least one implementation of a feature detection algorithm, and at least one product training image; and
at least one processor coupled with the memory that, upon execution of the robust feature detection software instructions:
generates training features from the at least one product training image according to the at least one implementation of the feature detection algorithm, wherein each training feature has a corresponding training feature location within the at least one product training image;
transforms at least a portion of the at least one product training image according to an image transformation, thereby forming a transformed image portion;
generates transform features from the transformed image portion according to the at least one implementation of the feature detection algorithm, wherein each transform feature has a corresponding transform feature location within the transformed image portion; and
stores in the memory a set of robust features, wherein each robust feature in the set represents a training feature, based on the image transform, having a training feature transformed location proximal to a transform feature location.

### Claim 33 (depends on 32)

The device of claim 32, wherein the image transformation is user-selectable.

### Claim 34 (depends on 32)

The device of claim 32, wherein the image transformation comprises an image perturbation.

### Claim 35 (depends on 32)

The device of claim 32, wherein the image transformation comprises an image processing transform.

### Claim 36 (depends on 35)

The device of claim 35, wherein the image processing transform includes at least one the following: a Gaussian filter, a color transform, an edge enhancing transform, and a lossy compression.

### Claim 37 (depends on 32)

The device of claim 32, wherein the image transformation comprises a geometric transform.

### Claim 38 (depends on 37)

The device of claim 37, wherein the geometric transform comprises at least one of the following: a skewing, a rotation, and a shearing.

### Claim 39 (depends on 32)

The device of claim 32, wherein the image transformation includes a scale transform.

### Claim 40 (depends on 39)

The device of claim 39, wherein the scale transform comprises an up-scaling.

### Claim 41 (depends on 39)

The device of claim 39, wherein the scale transform comprises a down-scaling.

### Claim 42 (depends on 39)

The device of claim 39, wherein the scale transform comprises a linear scaling.

### Claim 43 (depends on 39)

The device of claim 39, wherein the scale transform comprises a scaling factor of at least two.

### Claim 44 (depends on 39)

The device of claim 39, wherein the scale transform comprises a scaling factor based on subject matter of the at least one product training image.

### Claim 45 (depends on 32)

The device of claim 32, wherein the at least one product training image comprises a near-sequence image.

### Claim 46 (depends on 32)

The device of claim 32, wherein the at least the portion of the at least one product training image comprises a patch.

### Claim 47 (depends on 32)

The device of claim 32, wherein the at least one product training image includes a consumer product.

### Claim 48 (depends on 32)

The device of claim 32, wherein the at least one product training image includes a vehicle.

### Claim 49 (depends on 32)

The device of claim 32, wherein the at least one product training image includes clothing.

### Claim 50 (depends on 32)

The device of claim 32, wherein the at least one product training image includes a game.

### Claim 51 (depends on 32)

The device of claim 32, wherein the at least one product training image includes a package.

### Claim 52 (depends on 32)

The device of claim 32, wherein the at least one product training image includes a device.

### Claim 53 (depends on 32)

The device of claim 32, wherein the at least one product training image comprises at least one video frame.

### Claim 54 (depends on 32)

The device of claim 32, wherein each robust feature comprises a robustness measure with respect to the image transformation.

### Claim 55 (depends on 54)

The device of claim 54, wherein the robustness measure reflects subject matter characteristics of the at least one product training image.

### Claim 56 (depends on 32)

The device of claim 32, wherein the at least one product training image is a member of a test library of product images.

### Claim 57 (depends on 32)

The device of claim 32, wherein the at least one implementation of the feature detection algorithm includes an implementation of one of the following: a scale-invariant feature transform (SIFT), Fast Retina Keypoint (FREAK), Histograms of Oriented Gradient (HOG), Speeded Up Robust Features (SURF), DAISY, Binary Robust Invariant Scalable Keypoints (BRISK), FAST, Binary Robust Independent Elementary Features (BRIEF), Harris Corners, Edges, Gradient Location and Orientation Histogram (GLOH), Energy of image Gradient (EOG), an edge detection algorithm, and Transform Invariant Low-rank Textures (TILT) feature detection algorithm.

### Claim 58 (depends on 32)

The device of claim 32, wherein the device is a mobile device.

### Claim 59

A product image feature detection method comprising:
generating training features from at least one product training image according to at least one implementation of a feature detection algorithm, wherein each training feature has a corresponding training feature location within the at least one product training image;
transforming at least a portion of the at least one product training image according to an image transformation, thereby forming a transformed image portion;
generating transform features from the transformed image portion according to the at least one implementation of the feature detection algorithm, wherein each transform feature has a corresponding transform feature location within the transformed image portion; and
storing in memory a set of robust features, wherein each robust feature in the set represents a training feature, based on the image transform, having a training feature transformed location proximal to a transform feature location.

### Claim 60

A computer program product embedded in a non-transitory computer readable medium comprising product image feature detection instructions executable by a computer processor to perform operations comprising:
generating training features from at least one product training image according to at least one implementation of a feature detection algorithm, wherein each training feature has a corresponding training feature location within the at least one product training image;
transforming at least a portion of the at least one product training image according to an image transformation, thereby forming a transformed image portion;
generating transform features from the transformed image portion according to the at least one implementation of the feature detection algorithm, wherein each transform feature has a corresponding transform feature location within the transformed image portion; and
storing in memory a set of robust features, wherein each robust feature in the set represents a training feature, based on the image transform, having a training feature transformed location proximal to a transform feature location.
