US 8,805,023 B2 · Object motion estimating device, object motion estimating method, program, and recording medium

Abstract

An embodiment of the present invention provides an object motion estimating device and the like, which solve a problem of assuming translational movement (that is, a spatial gradient is 0) of an optical flow and are suitable to estimation of an optical flow by an image analysis of captured images. The object motion estimating device performs the image analysis of the captured images of the object to estimate motion of the object. The object motion estimating device includes an image analyzing unit 15. The image analyzing unit 15 assumes time invariance of the optical flow while not assuming the translational movement, and the image analyzing unit 15 estimates the optical flow with intensity at each point of the captured images as measurement variable while simultaneously estimating a spatial gradient of the optical flow. The assumption about the translational movement of the optical flow is an assumption about the optical flow that is actually unknown. The proposed object motion estimating device can relax this assumption of the translational movement of the optical flow as much as possible.

An object motion estimating device that performs image analysis of captured images to estimate motion of an object, the captured images being the images of the object captured by an image capturing unit,

the object motion estimating device comprising an image analyzing unit, the image analyzing unit including a state estimating unit that performs sequential estimation of a state variable in an estimation model or an equation computing unit that solves simultaneous equations, and
the image analyzing unit comprises:
an intensity gradient computing unit that computes an intensity gradient at a particular point of the captured images;
and
a temporal gradient computing unit that computes a temporal gradient from the intensity gradient,
wherein, in the estimation model, the state variable includes not only an optical flow by assuming that a time invariance that a first-order total differential of intensity of the object in the captured images with respect to time is zero holds as for the optical flow that is a velocity distribution of relative motion between the image capturing unit and the object, but also a spatial gradient of the optical flow by not assuming that translational movement that the object moves on a plane perpendicular to an image capturing axis of the image capturing unit holds,
wherein the estimation model includes a measurement equation in which intensity of the spatial neighborhood of a particular point on the captured images is set to a measurement variable,
wherein the state estimating unit performs the sequential estimation of the state variable to simultaneously estimate the optical flow and a spatial gradient of the estimated optical flow,
wherein the simultaneous equations comprise:
the estimation model and a temporal differentiation of the intensity included in the estimation model, and
the optical flow, wherein a spatial gradient of the optical flow is set to a variable, and a temporal differentiation of the intensity and intensity gradient and second-order differential coefficients of the intensity and of the intensity gradient with respect to time are set to coefficients,
wherein the equation computing unit solves the simultaneous equations to simultaneously estimate the optical flow and a spatial gradient of the estimated optical flow, and
wherein the captured images are two-dimensional images having size of at least 3 pixels by 3 pixels, at least the three captured images whose image capturing times differ from one another are stored in a captured image storage unit,
the intensity gradient computing unit computes intensity gradients Ix(x0,y0,t), Iy(x0,y0,t), Ixx(x0,y0,t), Ixy(x0,y0,t), and Iyy(x0,y0,t) of intensity I at a particular point P(x0,y0) of the captured images to determine a measurement variable z(t) expressed by an equation (eq1),
the temporal gradient computing unit computes temporal gradients It, Itx, Ity, Itxx, Itxy, Ityy, Itt, Ittx, Itty, Ittxx, Ittxy, and Ittyy from the intensity I and the intensity gradients Ix, Iy, Ixx, Ixy, and Iyy, and
the equation computing unit solves a linear equation (eq2) to simultaneously estimate the optical flow [u v]T and a spatial gradient [ux vx uy vy]T of the optical flow [u v]T:
[
Formula
⁢
⁢
1
]
⁢
z
⁡
(
t
)
=
[
I
⁡
(
x
0
,
y
0
,
t
)
I
x
⁡
(
x
0
,
y
0
,
t
)
I
y
⁡
(
x
0
,
y
0
,
t
)
I
xx
⁡
(
x
0
,
y
0
,
t
)
I
xy
⁡
(
x
0
,
y
0
,
t
)
I
yy
⁡
(
x
0
,
y
0
,
t
)
]
(
eq
⁢
⁢
1
)
[
I
t
I
tx
I
ty
I
txx
I
txy
I
tyy
I
tt
I
ttx
I
tty
I
ttxx
I
ttxy
I
ttyy
]
=
-
[
I
x
I
y
0
0
0
0
I
xx
I
xy
I
x
I
y
0
0
I
xy
I
yy
0
0
I
x
I
y
0
0
2
⁢
I
xx
2
⁢
I
xy
0
0
0
0
I
xy
I
yy
I
xx
I
xy
0
0
0
0
2
⁢
I
xy
2
⁢
I
yy
I
tx
I
ty
0
0
0
0
I
txx
I
txy
I
tx
I
ty
0
0
I
txy
I
tyy
0
0
I
tx
I
ty
0
0
2
⁢
I
txx
2
⁢
I
txy
0
0
0
0
I
txy
I
tyy
I
txx
I
txy
0
0
0
0
2
⁢
I
txy
2
⁢
I
tyy
⁢
]
⁡
[
u
v
u
x
v
x
u
y
v
y
]
.
(
eq
⁢
⁢
2
)

280 more elements

Object motion estimating device, object motion estimating method, program, and recording medium

Life of this patent

Object motion estimating device, object motion estimating method, program, and recording medium

Life of this patent

Abstract

Claims2