# US 12,183,015 B2 · Interacting multi-model tracking algorithm using rest state model > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,183,015 B2 - Document ID: US12183015B2 - Status: Granted - Kind: B2 - Application number: 17/582,823 - Filed: 2022-01-24 - Published: 2022-07-28 - Granted: 2024-12-31 - Earliest claimed: 2021-01-27 - Assignees: Infineon Technologies AG - Applicants: Infineon Technologies AG - Inventors: Prachi Vaishnav; Avik Santra - Web page: https://patentgrep.com/patent/US12183015B2 ## CPC - G01S 13/50: Systems of measurement based on relative movement of target - G06T 7/251: {involving models} - G01S 13/42: Simultaneous measurement of distance and other co-ordinates (indirect measurement G01S13/46) - G01S 13/58: Velocity or trajectory determination systems; Sense-of-movement determination systems - G01S 17/50: Systems of measurement based on relative movement of target - G01S 17/58: Velocity or trajectory determination systems; Sense-of-movement determination systems - G06T 7/521: from laser ranging, e.g. using interferometry; from the projection of structured light - G06T 7/593: from stereo images - G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract In an embodiment, a method includes: obtaining depth data indicative of a time-resolved measurement of a 3-D position of a moveable target in a field-of-view of a depth sensor; and processing the depth data using an interacting multi-model (IMM) tracking algorithm, the IMM tracking algorithm providing, for each one of multiple iterations, tracking data comprising a respective estimate of the 3-D position of the moveable target, wherein the IMM tracking algorithm comprises a first model providing a first output, a second model providing a second output, and a fusion module fusing the first output and the second output to provide the tracking data, wherein the first model models a movement state of the moveable target, and wherein the second model models a rest state of the moveable target. ## Claims ### Claim 1 A method, comprising: obtaining depth data indicative of a time-resolved measurement of a 3-D position of a moveable target in a field-of-view of a depth sensor; and processing the depth data using an interacting multi-model (IMM) tracking algorithm, the IMM tracking algorithm providing, for each one of multiple iterations, tracking data comprising a respective estimate of the 3-D position of the moveable target, wherein: the IMM tracking algorithm comprises a first model providing a first output, a second model providing a second output, and a fusion module fusing the first output and the second output to provide the tracking data, the first model models a movement state of the moveable target, the second model models a rest state of the moveable target, the second model determines the second output based on the estimate of the 3-D position of the moveable target with respect to an input element of a user interface predefined within the field-of-view, the user interface comprises a plurality of input elements, the second output comprises an indication of a given input element of the plurality of input elements being addressed by the moveable target, and the tracking data comprises, at least for some of the multiple iterations, the indication of the given input element. ### Claim 2 (depends on 1) The method of claim 1, wherein the second model determines the second output under an assumption of zero velocity of the moveable target and using an estimation covariance determined based on a predefined probability distribution around the 3-D position. ### Claim 3 (depends on 1) The method of claim 1, wherein: the first output comprises a first state vector comprising the 3-D position and a velocity measure of the moveable target; the second output comprises a second state vector comprising the 3-D position of the moveable target; and the second state vector is padded using zero values for the velocity measure prior to the fusing. ### Claim 4 (depends on 1) The method of claim 1, wherein: the fusion module determines a first likelihood of the moveable target being in the movement state and a second likelihood of the moveable target being in the rest state; the fusion module determines the tracking data based on a weighted combination using weighting factors set in accordance with the first likelihood and the second likelihood; and the fusion module determines at least one of the first likelihood or the second likelihood based on an evolution of an accuracy of the respective one of the first model or the second model over two or more previous iterations of the multiple iterations. ### Claim 5 (depends on 1) The method of claim 1, wherein the moveable target is selected from a group comprising of: hand, part of a hand, and handheld pointing device. ### Claim 6 (depends on 1) The method of claim 1, wherein the tracking data selectively comprises the indication of the given input element depending on a likelihood of the moveable target being in the rest state. ### Claim 7 (depends on 1) The method of claim 1, further comprising determining whether one of the plurality of input elements is actuated by a user based on the indication of the given input element being addressed by the moveable target. ### Claim 8 (depends on 1) The method of claim 1, further comprising, depending on a likelihood that the moveable target is in the movement state, selectively applying a gesture classification based on a dynamic trajectory defined by the estimates of the 3-D position of the moveable target provided in the multiple iterations by the IMM tracking algorithm. ### Claim 9 (depends on 1) The method of claim 1, wherein the depth sensor is selected from a group comprising: radar sensor, time-of-flight sensor, stereo camera, and LIDAR sensor. ### Claim 10 (depends on 1) The method of claim 1, wherein the IMM tracking algorithm updates, for each iteration of the multiple iterations, an a-priori estimate of the 3-D position of the moveable target based on a Markovian decision process, the first output, the second output, a first likelihood of the moveable target being in the movement state in a preceding iteration, and a second likelihood of the moveable target being in the rest state in the preceding iteration. ### Claim 11 (depends on 1) The method of claim 1, wherein the first model determines the first output based on a prediction of a position and a velocity of the moveable target using an unscented Kalman filter and a coordinated turn model. ### Claim 12 (depends on 1) The method of claim 1, wherein the tracking data is indicative of a likelihood that the moveable target is in the movement state or in the rest state. ### Claim 13 (depends on 12) The method of claim 12, further comprising selecting a post-processing algorithm for post-processing the tracking data depending on the likelihood that the moveable target is in the movement state or the rest state. ### Claim 14 (depends on 1) A device comprising a processor and a memory with instructions stored thereon, wherein the instructions, when executed by the processor, enable the device to perform the method of claim 1. ### Claim 15 (depends on 14) The device of claim 14, wherein the device comprises a time-of-flight sensor, a radar sensor, a stereo camera, or a LIDAR sensor. ### Claim 16 A device comprising: a radar sensor comprising a transmitter configured to transmit radar signals towards a field-of-view of the radar sensor, a receiver configured to receive reflected radar signals from the field-of-view, wherein the radar sensor is configured to provide raw measurement data based on the reflected radar signals; and a processor configured to: generate depth data indicative of a time-resolved measurement of a 3-D position of a moveable target in the field-of-view based on the raw measurement data; and process the depth data using an interacting multi-model (IMM) tracking algorithm, the IMM tracking algorithm providing, for each one of multiple iterations, tracking data comprising a respective estimate of the 3-D position of the moveable target, wherein: the IMM tracking algorithm comprises a first model configured to provide a first output, a second model configured to provide a second output, and a fusion module configured to fuse the first output and the second output to provide the tracking data, the first model is configured to model a movement state of the moveable target, the second model is configured to model a rest state of the moveable target, the second model determines the second output based on the estimate of the 3-D position of the moveable target with respect to an input element of a user interface predefined within the field-of-view, the user interface comprises a plurality of input elements, the second output comprises an indication of a given input element of the plurality of input elements being addressed by the moveable target, and the tracking data comprises, at least for some of the multiple iterations, the indication of the given input element. ### Claim 17 (depends on 16) The device of claim 16, wherein the processor is configured to selectively apply a gesture classification based on a dynamic trajectory defined by the estimates of the 3-D position of the moveable target provided in the multiple iterations by the IMM tracking algorithm depending on a likelihood that the moveable target is in the movement state. ### Claim 18 A method, comprising: obtaining depth data indicative of a time-resolved measurement of a 3-D position of a moveable target in a field-of-view of a depth sensor; and processing the depth data using an interacting multi-model (IMM) tracking algorithm, the IMM tracking algorithm providing, for each one of multiple iterations, tracking data comprising a respective estimate of the 3-D position of the moveable target, wherein the IMM tracking algorithm comprises a first model providing a first output, a second model providing a second output, and a fusion module fusing the first output and the second output to provide the tracking data, wherein the first model models a movement state of the moveable target, wherein the second model models a rest state of the moveable target, and wherein processing the depth data further comprises determining the second output based on the estimate of the 3-D position of the moveable target with respect to an input element of a user interface predefined within the field-of-view using the second model. ### Claim 19 (depends on 18) The method of claim 18, wherein the second model determines the second output under an assumption of zero velocity of the moveable target and using an estimation covariance determined based on a predefined probability distribution around the 3-D position. ### Claim 20 (depends on 18) The method of claim 18, wherein the moveable target is selected from a group comprising of: hand, part of a hand, and handheld pointing device.