# US 12,542,000 B2 · Method and apparatus for distinguishing different configuration states of an object based on an image representation of the object

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,542,000 B2
- Document ID: US12542000B2
- Status: Granted
- Kind: B2
- Application number: 18/035,629
- Filed: 2021-10-27
- Published: 2023-12-21
- Granted: 2026-02-03
- Earliest claimed: 2020-11-05
- Assignees: Gestigon GmbH
- Applicants: Gestigon GmbH
- Inventors: Johanna Awada; Florian Hartmann
- Web page: https://patentgrep.com/patent/US12542000B2

## CPC

- G06V 10/44: Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V 40/28: {Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113)}
- G06V 10/242: {by image rotation, e.g. by 90 degrees}
- G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI]
- G06V 10/26: Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V 10/28: Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns
- G06V 10/50: by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/7715: {Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods}
- G06V 20/64: Three-dimensional [3D] objects
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A method for distinguishing different configuration states of an object based on an image representation of the object involves acquiring original image data representing an original image comprising a picture of the object; detecting the object within the original image; determining an orientation of the detected object; determining a region-of-interest within the original image representing the picture of the object at least in parts; generating normalized image data representing a two-dimensional normalized image by transforming the ROI of the original image based on the determined orientation of the object to a reference frame in which the ROI is represented by the normalized image with a predefined standard orientation and scaling; extracting characteristic features of the object as represented by the normalized image data; and classifying the extracted characteristic features to determine a specific configuration state of the object associated with the classification.

## Claims

### Claim 1

A method of distinguishing different configuration states of an object based on an image representation of the object, the method comprising:
acquiring original image data representing an original image comprising a picture of the object;
detecting the object within the original image;
determining an orientation of the detected object within the original image;
determining within the original image a region-of-interest, ROI, representing the picture of the object at least in parts;
generating normalized image data representing a two-dimensional normalized image by transforming the ROI of the original image based on the determined orientation of the object to a reference frame in which the ROI is represented by the normalized image with a predefined standard orientation and scaling;
extracting one or more characteristic features of the object as represented by the normalized image data; and
classifying the extracted one or more characteristic features according to a related classification scheme to determine among a set of two or more possible configuration states of the object a specific configuration state that is associated with the result of the classification;
wherein the normalized image is a digital image and at least one of the characteristic features of the object is extracted based on histogram data, wherein the histogram data represents a histogram that associates each row of at least a subset of the rows of image points of the normalized image along a selected one of the normalized image's dimensions with a respective rate or number of object-related image points in the respective row, wherein an object-related image point is an image point that represents a respective portion of the object within the normalized image.

### Claim 2 (depends on 1)

The method of claim 1, wherein the at least one characteristic feature of the object is extracted based on a ratio of the number or ratio of object-related image points in a first subset of the rows along said dimension of the normalized image and a number or ratio of object-related image points in a second, different subset of the rows of the same dimension of the normalized image.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
detecting within the normalized image at least a subset of those rows along said dimension of the normalized image that comprise multiple discrete row segments, each row segment consisting of object-related image points and each row segment being separate from its one or more neighboring row segments of the same row by a gap consisting of at least one image point that is not an object-related image point,
wherein the extracting the at least one characteristic feature of the object is further based on the presence of one or more of such gaps in at least one of said rows of the normalized image.

### Claim 4 (depends on 3)

The method of claim 3, wherein the extracting of the at least one characteristic feature of the object is based on one or more of the number, the sizes, the locations of the one or more gaps, and for at least one selected gap the number of contiguously neighboring rows having a corresponding gap being interconnected with the selected gap either directly or via one or more other gaps.

### Claim 5 (depends on 4)

The method of claim 4, wherein the extracting of the at least one characteristic feature of the object based on the presence of one or more of gaps in at least one of said rows of the normalized image is predominantly or solely based on that set of gaps within the normalized image that comprises the greatest number of contiguously neighboring rows having a corresponding gap being interconnected with the selected gap either directly or via one or more other gaps.

### Claim 6 (depends on 3)

The method of claim 3, wherein at least one of the gaps is identified by scanning a respective row of the normalized image for transitions from an object-related image point to a non-object-related image point and vice versa.

### Claim 7 (depends on 1)

The method of claim 1, further comprising preprocessing the original image data before detecting the object therein to improve the image quality of the image represented by the original image data.

### Claim 8 (depends on 1)

The method of claim 1, wherein determining the spatial orientation of the object within the original image is based on one or more of:
an orientation of a bounding box enclosing the object, of a contour of the object, or of an arrangement of two or more extreme points on the contour of the object within the original image;
a distribution of object-related image points within a bounding box enclosing the object;
an orientation defined by the relative position of a specific extreme point of the object and the geometric center or the center of gravity of the object or another extreme point of the object;
amplitude image data representing an amplitude image of the object and being provided as part of or in addition to the original image data; and
depth image data representing a depth image of the object and being provided as part of or in addition to the original image data.

### Claim 9 (depends on 1)

The method of claim 1, wherein generating the normalized image data comprises transforming the ROI to the reference frame by means of an affine transformation.

### Claim 10 (depends on 1)

The method of claim 1, wherein generating the normalized image data comprises transforming the ROI to the reference frame in such a way that the thus-transformed image of the ROI completely coincides with the normalized image.

### Claim 11 (depends on 1)

The method of claim 1, wherein the original image data comprises a sequence of contiguous image frames, and the method further comprises:
maintaining history data representing for each image frame of at least a subset of the sequence the respective classification result that was previously determined for the corresponding image frame, wherein the subset is a contiguous series of image frames that each picture the object, at least in parts,
wherein the determining a current configuration state of the object is based on the result of the classification of a current image frame and the history data.

### Claim 12 (depends on 11)

The method of claim 11, wherein the current configuration state of the object is determined based on a majority decision or averaging with respect to the result of the classification of the current image frame and the classification results represented by the history data.

### Claim 13 (depends on 1)

The method of claim 1, wherein the object is a human hand and the method is configured to distinguish, when a gesture is performed by the hand, a pointing gesture from a swiping gesture based on classifying one or more characteristic features of the hand being extracted from the normalized image data.

### Claim 14 (depends on 1)

A non-transitory computer-readable storage medium comprising instructions that, when executed on a computer or multi-computer platform, cause the computer or multi-computer platform, respectively, to perform the method of claim 1.

### Claim 15 (depends on 1)

An image processing apparatus for distinguishing different configuration states of an object based on an image representation of the object, the image processing apparatus being configured to perform the method of claim 1.