Back to results
US 9,069,075 B2Granted
Coupled range and intensity imaging for motion estimation
AssigneeGM Global Technology Operations LLC
InventorShuqing Zeng
Life of this patent
Estimating the term…
Abstract
A method and system may operate a coupled range imager and intensity imager to obtain a sequence of at least two automatically registered images, each automatically registered image including concurrently acquired range data and intensity data with regard to an imaged scene. The registered range and intensity data is analyzed to yield an estimated motion of an element of an imaged object of the imaged scene.
Claims4
A method comprising:
- directing modulated light at an object in motion over at least one set time interval;
- using a beam splitter to direct a first portion of the modulated light after reflection from the object onto a range imager and a second portion of the modulated light onto an intensity imager so as to generate time-dependent, pixel-specific, intensity data and range data of the object;
- and
- using a processor to:
- perform pixel-level fusion of the intensity data and the range data, and
- calculate three-dimensional, pixel-level motion flow from the pixel-level fusion of the intensity data and the range data,
- wherein the pixel-level fusion is implemented by minimizing
- ∑
- u
-
- [
- I
-
- (
- R
-
- (
- u
- ,
- p
- +
- Δ
-
-
- p
- )
- )
- -
- T
-
- (
- u
- )
- ]
- 2
- such that pixel motion p is constrained within a planar patch as defined by
- [
- ∂
- u
-
- Z
- ∂
- u
-
- X
- ∂
- v
-
- Z
- ∂
- v
-
- Y
-
-
- 1
- ]
-
- p
- T
- =
- Δ
- t
-
- Z
- ,
- wherein:
- T(u) is previously acquired frame,
- I(u) is an input image,
- R(u,p) is a warp function describing a motion of image elements between T and I,
- u refers to image pixel coordinates in terms of row and column, (u,v),
- pT is motion of image elements between T and I, and
- Δp is a change in p from a previously calculated value,
- ∂uZ and ∂uX are range image gradients along pixel row, u,
- ∂vZ and ∂vY are range image gradients along pixel column, v,
- ΔtZ is partial derivative with respect to time, t,
- X, Y, Z refers to three-dimensional coordinates of range image expressed in pixel coordinates u, v, and t.
74 more elements
A method comprising:
- directing modulated light at an object in motion over at least one set time interval;
- using a beam splitter to direct a first portion of the modulated light after reflection from the object onto a range imager and a second portion of the modulated light onto an intensity imager so as to generate time-dependent, pixel-specific, intensity data and range data of the object;
- and
- using a processor to:
- perform pixel-level fusion of the intensity data and the range data,
- calculate three-dimensional, pixel-level motion flow from the pixel-level fusion of the intensity data and the range data,
- wherein the pixel-level fusion is implemented by minimizing
- ∑
- u
-
- [
- I
-
- (
- R
-
- (
- u
- ,
- p
- +
- Δ
-
-
- p
- )
- )
- -
- T
-
- (
- u
- )
- ]
- 2
- such that pixel motion p is constrained within a planar patch as defined by
- [
- ∂
- u
-
- Z
- ∂
- u
-
- X
- ∂
- v
-
- Z
- ∂
- v
-
- Y
-
-
- 1
- ]
-
- p
- T
- =
- Δ
- t
-
- Z
- ,
- wherein:
- T(u) is previously acquired frame,
- I(u) is an input image,
- R(u,p) is a warp function describing a motion of image elements between T and I,
- u refers to image pixel coordinates in terms of row and column, (u,v),
- pT is motion of image elements between T and I, and
- Δp is a change in p from a previously calculated value,
- ∂uZ and ∂uX are range image gradients along pixel row, u,
- ∂vZ and ∂vY are range image gradients along pixel column, v,
- ΔtZ is partial derivative with respect to time, t,
- X, Y, Z refers to three-dimensional coordinates of range image expressed in pixel coordinates u, v, and t, and
- wherein the minimizing
- ∑
- u
-
- [
- I
-
- (
- R
-
- (
- u
- ,
- p
- +
- Δ
-
-
- p
- )
- )
- -
- T
-
- (
- u
- )
- ]
- 2
- is implemented by applying Lucas-Kanade algorithm.
104 more elements
A system comprising:
- a modulated light source configured to direct modulated light at an object in motion over at least one set time interval;
- a beam splitter configured to direct a first portion of the modulated light after reflection from the object onto a range imager detector and a second portion of the modulated light onto an intensity imager detector so as to generate time-dependent, pixel-specific, intensity data and range data of the object;
- and
- a processor configured to:
- perform pixel-level fusion of the intensity data and the range data, and
- calculate three-dimensional, pixel-level motion flow from the pixel-level fusion of the intensity data and the range data,
- the pixel-level fusion is implemented by minimizing
- ∑
- u
-
- [
- I
-
- (
- R
-
- (
- u
- ,
- p
- +
- Δ
-
-
- p
- )
- )
- -
- T
-
- (
- u
- )
- ]
- 2
- such that pixel motion p is constrained within a planar patch as defined by
- [
- ∂
- u
-
- Z
- ∂
- u
-
- X
- ∂
- v
-
- Z
- ∂
- v
-
- Y
-
-
- 1
- ]
-
- p
- T
- =
- Δ
- t
-
- Z
- ,
- wherein:
- T(u) is previously acquired frame,
- I(u) is an input image,
- R(u,p) is a warp function describing a motion of image elements between T and I,
- u refers to image pixel coordinates in terms of row and column, (u,v),
- pT is motion of image elements between T and I, and
- Δp is a change in p from a previously calculated value,
- ∂uZ and ∂uX are range image gradients along pixel row, u,
- ∂vZ and ∂vY are range image gradients along pixel column, v,
- ΔtZ is partial derivative with respect to time, t,
- X, Y, Z refers to three-dimensional coordinates of range image expressed in pixel coordinates u, v, and t.
74 more elements
A system comprising:
- a modulated light source configured to direct modulated light at an object in motion over at least one set time interval;
- a beam splitter configured to direct a first portion of the modulated light after reflection from the object onto a range imager detector and a second portion of the modulated light onto an intensity imager detector so as to generate time-dependent, pixel-specific, intensity data and range data of the object;
- and
- a processor configured to:
- perform pixel-level fusion of the intensity data and the range data, and
- calculate three-dimensional, pixel-level motion flow from the pixel-level fusion of the intensity data and the range data, pixel-level fusion is implemented by minimizing
- ∑
- u
-
- [
- I
-
- (
- R
-
- (
- u
- ,
- p
- +
- Δ
-
-
- p
- )
- )
- -
- T
-
- (
- u
- )
- ]
- 2
- such that pixel motion p is constrained within a planar patch as defined by
- [
- ∂
- u
-
- Z
- ∂
- u
-
- X
- ∂
- v
-
- Z
- ∂
- v
-
- Y
-
-
- 1
- ]
-
- p
- T
- =
- Δ
- t
-
- Z
- ,
- wherein:
- T(u) is previously acquired frame,
- I(u) is an input image,
- R(u,p) is a warp function describing a motion of image elements between T and I,
- u refers to image pixel coordinates in terms of row and column, (u,v),
- pT is motion of image elements between T and I, and
- Δp is a change in p from a previously calculated value,
- ∂uZ and ∂uX are range image gradients along pixel row, u,
- ∂vZ and ∂vY are range image gradients along pixel column, v,
- ΔtZ is partial derivative with respect to time, t,
- X, Y, Z refers to three-dimensional coordinates of range image expressed in pixel coordinates u, v, and t,
- wherein the minimizing
- ∑
- u
-
- [
- I
-
- (
- R
-
- (
- u
- ,
- p
- +
- Δ
-
-
- p
- )
- )
- -
- T
-
- (
- u
- )
- ]
- 2
- is implemented by applying Lucas-Kanade algorithm.
103 more elements
Dependencies are inferred from the claim text.