# US 11,906,658 B2 · Systems and methods for shapelet decomposition based gesture recognition using radar > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,906,658 B2 - Document ID: US11906658B2 - Status: Granted - Kind: B2 - Application number: 17/037,335 - Filed: 2020-09-29 - Published: 2021-07-01 - Granted: 2024-02-20 - Earliest claimed: 2019-12-18 - Assignees: Tata Consultancy Services Limited - Applicants: Tata Consultancy Services Limited - Inventors: Arijit Chowdhury; Smriti Rani; Tapas Chakravarty; Arpan Pal - Web page: https://patentgrep.com/patent/US11906658B2 ## CPC - G01S 7/415: {Identification of targets based on measurements of movement associated with the target} - G06F 18/00: Pattern recognition - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06N 5/04: Inference or reasoning models - G06N 20/00: Machine learning - G06F 2218/16: by matching signal segments - G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract This disclosure relates to systems and methods for shapelet decomposition based recognition using radar. State-of-the-art solutions involve use of standard machine learning classification techniques for gesture recognition which suffer with problem of dependency on collected data. The present disclosure overcome the limitations faced by the state-of-the-art solutions by obtaining a plurality of time domain signal using a radar system comprising three vertically arranged radars and one or more sensors, identifying one or more gesture windows to obtain one or more shapelets corresponding to one or gestures which are further decomposed into a plurality of sub shapelets. Further, at least one of (i) a positive or (i) a negative time delay is applied to each of the plurality of sub shapelets to obtain a plurality of composite shapelets which are further mapped with a plurality of trained shapelets to recognize gestures comprised in one or more activities performed by a subject. ## Claims ### Claim 1 A processor implemented method, comprising: obtaining, using a radar system, a plurality of time domain signals reflected by a subject performing one or more activities, wherein the subject refers to a living being including a human or an animal or a bird; identifying one or more gesture windows in the plurality of obtained time domain signals, by applying one or more signal properties on the plurality of obtained time domain signals to detect a no motion region and a motion region in the plurality of obtained time domain signals, wherein width of each gesture window is indicative of a time duration in which a gesture is performed, wherein applying the one or more signal properties comprises determining an average energy for the plurality of obtained time domain signals to identify a region where the average energy is minimum as the no motion region; obtaining, using a spectrogram, one or more shapelets of one or more gestures captured from the one or more gesture windows; creating a dictionary of basic shapelets of one or more motion patterns of a plurality of gestures comprised in the one or more activities performed by the subject; decomposing, using the created dictionary of basic shapelets, the one or more shapelets to obtain a plurality of sub-shapelets of the one or more gestures captured from the one or more gesture windows, wherein each sub-shapelet is representative of a specific motion pattern of each of the one or more gestures captured from the one or more gesture windows; applying, at least one of (i) a positive or (ii) negative time delay to each of the plurality of sub-shapelets of the one or more gestures to obtain a plurality of composite shapelets for the one or more gestures, wherein the plurality of composite shapelets for the one or more gestures are obtained by superimposing the positive and the negative time delays of each of the plurality of sub-shapelets, and wherein the positive and the negative time delay is provided to each of the plurality of sub-shapelets based on a displacement and a time pattern of the one or more gestures; and performing, a mapping of each of the plurality of composite shapelets with a plurality of trained shapelets stored in a system database to recognize the plurality of gestures comprised in the one or more activities being performed by the subject, wherein each of the plurality of composite shapelet is converted into a 2-dimensional or 3-dimensional vector and mapping is performed to recognize a new gesture performed by the subject. ### Claim 2 (depends on 1) The processor implemented method of claim 1, wherein the one or more gesture windows are identified by eliminating a portion of the plurality of obtained time domain signals with no motion. ### Claim 3 (depends on 1) The processor implemented method of claim 1, wherein the spectrogram of the one or more gestures is obtained by transforming the plurality of obtained time domain signals into time-frequency domain using a short time Fourier transform (STFT) technique. ### Claim 4 (depends on 1) The processor implemented method of claim 1, wherein the one or more shapelets of the one or more gestures are characterized by a set of features obtained from a spectrogram contour of the one or more gestures. ### Claim 5 A system, comprising: one or more data storage devices operatively coupled to one or more hardware processors and configured to store instructions configured for execution via the one or more hardware processors to: obtain, using a radar system, a plurality of time domain signals reflected by a subject performing one or more activities, wherein the subject refers to a living being including a human or an animal or a bird; identify, one or more gesture windows in the plurality of obtained time domain signals, by applying one or more signal properties on the plurality of obtained time domain signals to detect a no motion region and a motion region in the plurality of obtained time domain signals, wherein width of each gesture window is indicative of a time duration in which a gesture is performed, wherein applying the one or more signal properties comprises determining an average energy for the plurality of obtained time domain signals to identify a region where the average energy is minimum as the no motion region; obtain, using a spectrogram, one or more shapelets of one or more gestures captured from the one or more gesture windows; create a dictionary of basic shapelets of one or more motion patterns of a plurality of gestures comprised in the one or more activities performed by the subject; decompose, using the created dictionary of basic shapelets, the one or more shapelets to obtain a plurality of sub-shapelets of the one or more gestures captured from the one or more gesture windows, wherein each sub-shapelet is representative of a specific motion pattern of each of the one or more gestures captured from the one or more gesture windows; apply, at least one of (i) a positive or (ii) negative time delay to each of the plurality of sub-shapelets of the one or more gestures to obtain a plurality of composite shapelets for the one or more gestures, wherein the plurality of composite shapelets for the one or more gestures are obtained by superimposing the positive and the negative time delays of each of the plurality of sub-shapelets, and wherein the positive and the negative time delay is provided to each of the plurality of sub-shapelets based on a displacement and a time pattern of the one or more gestures; and perform, a mapping of each of the plurality of composite shapelets with a plurality of trained shapelets stored in a system database to recognize the plurality of gestures comprised in the one or more activities being performed by the subject, wherein each of the plurality of composite shapelet is converted into a 2-dimensional or 3-dimensional vector and mapping is performed to recognize a new gesture performed by the subject. ### Claim 6 (depends on 5) The system of claim 5, wherein the one or more gesture windows are identified by eliminating a portion of the plurality of obtained time domain signals with no motion. ### Claim 7 (depends on 5) The system of claim 5, wherein the spectrogram of the one or more gestures is obtained by transforming the plurality of obtained time domain signals into time-frequency domain using a short time Fourier transform (STFT) technique. ### Claim 8 (depends on 5) The system of claim 5, wherein the one or more shapelets of the one or more gestures are characterized by a set of features obtained from a spectrogram contour of the one or more gestures. ### Claim 9 A computer program product comprising a non-transitory computer readable medium having a computer readable program embodied therein, wherein the computer readable program, when executed on a computing device, causes the computing device to: obtain, using a radar system, a plurality of time domain signals reflected by a subject performing one or more activities, wherein the subject refers to a living being including a human or an animal or a bird; identify, one or more gesture windows in the plurality of obtained time domain signals, by applying one or more signal properties on the plurality of obtained time domain signals to detect a no motion region and a motion region in the plurality of obtained time domain signals, wherein width of each gesture window is indicative of a time duration in which a gesture is performed, wherein applying the one or more signal properties comprises determining an average energy for the plurality of obtained time domain signals to identify a region where the average energy is minimum as the no motion region; obtain, using a spectrogram, one or more shapelets of one or more gestures captured from the one or more gesture windows; create a dictionary of basic shapelets of one or more motion patterns of a plurality of gestures comprised in the one or more activities performed by the subject; decompose, using the created dictionary of basic shapelets, the one or more shapelets to obtain a plurality of sub-shapelets of the one or more gestures captured from the one or more gesture windows, wherein each sub-shapelet is representative of a specific motion pattern of each of the one or more gestures captured from the one or more gesture windows; apply, at least one of (i) a positive or (ii) negative time delay to each of the plurality of sub-shapelets of the one or more gestures to obtain a plurality of composite shapelets for the one or more gestures, wherein the plurality of composite shapelets for the one or more gestures are obtained by superimposing positive and the negative time delays of each of the plurality of sub-shapelets, and wherein the positive and the negative time delay is provided to each of the plurality of sub-shapelets based on a displacement and a time pattern of the one or more gestures; and perform, a mapping of each of the plurality of composite shapelets with a plurality of trained shapelets stored in a system database to recognize the plurality of gestures comprised in the one or more activities being performed by the subject, wherein each of the plurality of composite shapelet is converted into a 2-dimensional or 3-dimensional vector and mapping is performed to recognize a new gesture performed by the subject. ### Claim 10 (depends on 9) The computer program product of claim 9, wherein the one or more gesture windows are identified by eliminating a portion of the plurality of obtained time domain signals with no motion. ### Claim 11 (depends on 9) The computer program product of claim 9, wherein the spectrogram of the one or more gestures is obtained by transforming the plurality of obtained time domain signals into time-frequency domain using a short time Fourier transform (STFT) technique. ### Claim 12 (depends on 9) The computer program product of claim 9, wherein the one or more shapelets of the one or more gestures are characterized by a set of features obtained from a spectrogram contour of the one or more gestures.