Back to results
US 12,656,477 B2Granted
Systems and methods for motion estimation
AssigneeThruwave Inc.
InventorsElina B. Richards, Daniel Arnitz, Andreas Pedross-Engel, Matthew S. Reynolds
Life of this patent
Estimating the term…
Abstract
The present application is directed to a video-based motion extraction system that uses mmWave imaging. In one example, an object velocity is extracted in real time from at least one motion vector obtained from a compressed video. The velocity information is combined with distance information obtained from a distance sensor. The combination of distance and velocity is used to calculate a position of the object at each time instance. A velocity profile may be generated. The velocity profile may then be used to reconstruct an image of an object passing in front of an mmWave antenna array.
Claims20
A system for producing millimeter wave images, comprising:
- a video camera operative to capture video frames of a moving object,
- wherein the video camera is offset from a radar array that provides radar data;
- a millimeter wave antenna array proximate to the video camera and operative to capture millimeter wave data frames;
- a motion vector processor configured to extract one or more motion vectors from the video frames,
- wherein the one or more motion vectors represent relative motion between the video camera and the object;
- at least one of a position processor and a velocity processor,
- wherein the position processor is configured to determine one or more position estimates of the object relative to the millimeter wave antenna array using the one or more motion vectors,
- wherein the velocity processor is configured to determine one or more velocity estimates of the object relative to the millimeter wave antenna array using the one or more motion vectors, and
- wherein the one or more velocity estimates are determined using a quantile average determined from the one or more motion vectors and a correction factor associated with the radar data, the correction factor obtained from a training dataset comprising randomly generated velocity profiles;
- and
- an image reconstruction processor configured to reconstruct a millimeter wave image of the object using the millimeter wave data frames and at least one of the corresponding position and velocity estimates.
8 more elements
A method for producing a millimeter wave image of a moving object, the method comprising:
- capturing two or more video frames of the moving object using a video camera proximate to a millimeter wave antenna array, the object transported by a conveyor belt or robot, the video camera offset from a radar array that provides radar data;
- capturing two or more millimeter wave data frames using the millimeter wave antenna array;
- extracting, from the two or more video frames, one or more motion vectors representing the relative motion between the video camera and the object;
- determining one or more position estimates and one or more velocity estimates of the object relative to the millimeter wave antenna array using the one or more motion vectors,
- wherein the one or more velocity estimates are determined using a quantile average determined from the one or more motion vectors and a correction factor associated with the radar data, the correction factor obtained from a training dataset comprising randomly generated velocity profiles;
- reconstructing the millimeter wave image using the millimeter wave data frames and the corresponding at least one of the position estimates and the velocity estimates using an image reconstruction algorithm;
- and
- controlling a velocity of the conveyor belt or robot based on the reconstructed millimeter wave image to increase an image focus.
5 more elements
A method for estimating motion of an object in a video, comprising:
- receiving at least one recording of an object passing in front of a millimeter wave antenna array, the object transported by a conveyor belt or robot;
- receiving radar data from a radar array offset from the millimeter wave antenna array;
- determining at least one velocity associated with the object,
- wherein the at least one velocity is determined using a quantile average determined from at least one motion vector and a correction factor associated with the radar data, the correction factor obtained from a training dataset comprising randomly generated velocity profiles;
- receiving, from a distance sensor, at least one measurement of distance from a distance sensor associated with the object;
- combining the at least one velocity and the at least one measurement of distance;
- based on the combination of the at least one velocity and the at least one measurement of distance, generating a velocity profile of the object;
- reconstructing at least one millimeter wave image based on the velocity profile of the object;
- and
- controlling a velocity of the conveyor belt or robot based on the reconstructed millimeter wave image to increase an image focus.
7 more elements
Dependencies are inferred from the claim text.