Interactive system for identification and tracking of objects and gestures

A method, comprising:

• generate a time-series of 3-D heat data frames, each of the 3-D heat data frame including heat data related to a 2-D array of pixels on the grid on the surface of the PCAP sensors and at discrete “z” levels above the PCAP sensors;
• pre-processing the time-series of 3-D heat data frames to remove noise from the time series of 3-D heat data frames to generate noiseless PCAP heat data grid;
• detecting 2-D contours of one or more objects at each of the discrete “z” levels and to connect points of the 2-D contours at each of the discrete “z” levels to points on the 2-D contours of other of the one or more of the other “z” levels to generate 3-D contours of the one or more objects;
• extracting one or more features from the 3-D contours of one or more objects and to generate an input vector for a machine learning model;
• and
• generating in input vector for a machine learning model using the features extracted from the 3-D contours.

One or more non-transitory computer-readable storage media encoding computer-executable instructions for executing on a computer system a computer process, the computer process comprising:

• receiving a time-series of 3-D heat data frames, each of the 3-D heat data frame including heat data related to a 2-D array of pixels on the grid on the surface of the PCAP sensors and at discrete “z” levels above a grid of plurality of projected capacitance (PCAP) sensors;
• pre-processing the time-series of 3-D heat data frames to remove noise from the time series of 3-D heat data frames to generate noiseless PCAP heat data grid;
• detecting 2-D contours of one or more objects at each of the discrete “z” levels and to connect points of the 2-D contours at each of the discrete “z” levels to points on the 2-D contours of other of the one or more of the other “z” levels to generate 3-D contours of the one or more objects;
• extracting one or more features from the 3-D contours of one or more objects and to generate an input vector for a machine learning model;
• and
• generating in input vector for a machine learning model using the features extracted from the 3-D contours.