Method for segmenting sign language into morphemes, method for predicting morpheme positions, and method for augmenting data

A sign language analysis method comprising:

• recognizing key points of a speaker from a sign language video, including extracting key-point data including body, hand, and facial features from each frame of the sign-language video so that each feature of the key-point data is recognized in a corresponding frame temporal order and frame groups, from among frames of the sign-language video, are identified as temporally contiguous frame segments each corresponding to a candidate morpheme interval according to temporal continuity of the frames in the sign language video;
• inputting the recognized key points into an artificial intelligence (AI) model, and segmenting sign language into morphemes and predicting position information of the segmented morphemes using the AI model,
• wherein the AI model is trained to minimize a loss in the predicting position information of the segmented morphemes and the AI model is used in performing frame-based morpheme recognition of the sign language video on a frame basis, thereby improving accuracy of sign language translation during the frame-based morpheme recognition;
• generating additional training data by combining or omitting morpheme segments to enhance robustness of the AI model;
• and
• training the AI model using training data including recognized key-point data, morpheme information of the sign language video, and the generated additional training data, thereby enhancing the robustness of the AI model.

A sign language analysis system comprising:

• one or more processors configured to:
• recognize key points of a speaker from a sign language video for training, including extracting key-point data including body, hand, and facial features from each frame of the sign-language video so that each feature of the key-point data is recognized in a corresponding frame temporal order and frame groups, from among frames of the sign-language video, are identified as temporally contiguous frame segments each corresponding to a candidate morpheme interval according to temporal continuity of the frames in the sign language video;
• recognize key points of a speaker from a sign language video;
• input the recognized key points into the AI model which is trained, segment sign language into morphemes, and predict position information of the segmented morphemes, wherein the AI model is trained to minimize a loss in the predicting position information of the segmented morphemes and the AI model is used in performing frame-based morpheme recognition of the sign language video on a frame basis, thereby improving accuracy of sign language translation during the frame-based morpheme recognition;
• generate additional training data by combining or omitting morpheme segments to enhance robustness of the AI model;
• and
• train the AI model using training data including recognized key-point data, morpheme information of the sign language video, and the generated additional training data, thereby enhancing the robustness of the AI model.

A sign language analysis method comprising:

• recognizing key points of a speaker from a sign language video for training, including extracting key-point data including body, hand, and facial features from each frame of the sign-language video so that each feature of the key-point data is recognized in a corresponding frame temporal order and frame groups, from among frames of the sign-language video, are identified as temporally contiguous frame segments each corresponding to a candidate morpheme interval according to temporal continuity of the frames in the sign language video;
• inputting the recognized key points into the AI model, segmenting sign language into morphemes, and predicting position information of the segmented morphemes,
• wherein the AI model is trained to minimize a loss in the predicting position information of the segmented morphemes and the AI model is used in performing frame-based morpheme recognition of the sign language video on a frame basis, thereby improving accuracy of sign language translation during the frame-based morpheme recognition;
• generating additional training data by combining or omitting morpheme segments to enhance robustness of the AI model;
• and
• training the AI model using training data including recognized key-point data, morpheme information of the sign language video, and the generated additional training data, thereby enhancing the robustness of the AI model.