# US 2018/0322338 A1 · Computer implemented method for sign language characterization

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0322338 A1
- Document ID: US20180322338A1
- Status: Published application
- Kind: A1
- Application number: 16/024,176
- Filed: 2018-06-29
- Published: 2018-11-08
- Earliest claimed: 2017-05-02
- Assignees: King Fahd University of Petroleum and Minerals
- Applicants: King Fahd University of Petroleum and Minerals
- Inventors: Sabri A. MAHMOUD; Ala Addin SIDIG
- Web page: https://patentgrep.com/patent/US20180322338A1

## CPC

- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06V 40/113: {Recognition of static hand signs}
- G06V 40/28: {Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113)}
- G06V 10/464: {using a plurality of salient features, e.g. bag-of-words [BoW] representations}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A sign language recognizer is configured to detect interest points in an extracted sign language feature, wherein the interest points are localized in space and time in each image acquired from a plurality of frames of a sign language video; apply a filter to determine one or more extrema of a central region of the interest points; associate features with each interest point using a neighboring pixel function; cluster a group of extracted sign language features from the images based on a similarity between the extracted sign language features; represent each image by a histogram of visual words corresponding to the respective image to generate a code book; train a classifier to classify each extracted sign language feature using the code book; detect a posture in each frame of the sign language video using the trained classifier; and construct a sign gesture based on the detected postures.

## Claims

### Claim 1

A computer-implemented method of recognizing sign language, the method comprising:
detecting, via circuitry, one or more interest points in an extracted sign language feature, wherein the one or more interest points are localized in space and time in each of ,a plurality of images acquired from a plurality of frames of a sign language video including the extracted sign language feature, wherein the detecting is carried out using a Scale Invariant Features Transform (SIFT) descriptor and the interest points represent corners in each image;
applying a digital filter to determine one or more extrema of a central region of the one or more interest points;
associating one or more features with each interest point of the one or more interest points using a neighboring pixel function;
clustering, via the circuitry, a group of extracted sign language features from the plurality of images based on a similarity between the extracted sign language features according to the associating;
representing each image of the plurality of images by a histogram of visual words corresponding to the respective image to generate a code book;
training, via the circuitry, a classifier based on labels assigned to the plurality of images to classify each extracted sign language feature using the code book;
detecting, via the circuitry, a posture in each frame of the plurality of frames of the sign language video using the trained classifier;
constructing, via the circuitry, a sign gesture based on the detected postures, and
identifying text words that correspond with the sign gesture and presenting the text on a display so as to ease communication between deaf people and non-deaf people.

### Claim 2 (depends on 1)

The method of claim 1, wherein the digital filter includes a Difference of Gaussian (DoG) filter to detect a blob-like shape in each image as the one or more interest points.

### Claim 3

(canceled)

### Claim 4 (depends on 1)

The method of claim 1, wherein the digital filter includes a Harris Laplace.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
training a Support Vector Machine to classify each frame of the sign language video.

### Claim 6 (depends on 1)

The method of claim 1. further comprising:
classifying each sign gesture using a K-Nearest Neighbors (K-NN) classifier.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
splitting a sequence of postures, and generating a split histogram of the postures for the respective split sequence of postures: and
concatenating the split sequence of postures to preserve an order of the sequence of postures in the respective sign gesture.

### Claim 8 (depends on 7)

The method of claim 7, further comprising:
noimalizing the split histogram of the sequence of postures of the sign language video to account for a difference in signing speed.

### Claim at position 9

9-20. (canceled)