US 11,106,900 B2Granted

Person trend recording device, person trend recording method, and program

AssigneeOMRON Corporation

InventorHaruka Taniguchi

Life of this patent

Estimating the term…

Loading patent…

Abstract

A person trend recording device includes: an image acquisition unit configured to acquire moving image data obtained by capturing an image of a target area; a person detector configured to detect and track a person, who acts within the target area, by analyzing the moving image data; an event detector configured to detect an event that occurs with the person during a period of tracking the person, by analyzing the moving image data; a physio-psychological index detector configured to detect a temporal change in a physio-psychological index of the person during the tracking period, by analyzing the moving image data; and a trend record generator configured to generate, as trend record data on the person, data in which at least one or more events that occur with the person are temporally associated with the temporal change in the physio-psychological index of the person.

Claims15

Dependent claims

A person trend recording device comprising a processor, and a memory storing a program that, when executed by the processor, causes the processor to:

operate as an image acquisition circuitry configured to acquire moving image data obtained by capturing an image of a target area;
operate as a person detector configured to detect and track a person, who acts within the target area, by analyzing the moving image data;
operate as an event detector configured to detect an event that occurs with the person during a period of tracking the person, by analyzing the moving image data;
operate as a physio-psychological index detector configured to detect a temporal change in a physio-psychological index of the person during the tracking period, by analyzing the moving image data;
operate as a satisfaction level estimator configured to estimate a satisfaction level of the person based on the temporal change in the physio-psychological index;
and
operate as a trend record generator configured to generate, as trend record data on the person, data in which at least one or more events that occur with the person, the temporal change in the physio-psychological index of the person, and information on the satisfaction level estimated by the satisfaction level estimator are associated temporally,
wherein the satisfaction level estimator estimates the satisfaction level by using two types of physio-psychological indices including a physio-psychological index consciously controllable by the person, and a physio-psychological index unconsciously controllable by the person,
wherein the satisfaction level estimator estimates the satisfaction level S(t) by Formula (1),
Satisfaction
⁢
⁢
level
⁢
⁢
S
⁡
(
t
)
=
Sm
⁡
(
t
)
×
PR
⁡
(
t
)
-
PR
⁢
min
PR
⁢
max
-
PR
⁢
min
(
1
)
where
Sm(t) is a smile level at time t,
PR(t) is a pulse rate at time t,
PRmin is a minimum value of the pulse rate in a predetermined period including time t, and
PRmax is a maximum value of the pulse rate in the predetermined period,
wherein the smile level is the physio-psychological index consciously controllable by the person, and the pulse rate is the physio-psychological index unconsciously controllable by the person.

49 more elements

A person trend recording device comprising a processor, and a memory storing a program that, when executed by the processor, causes the processor to:

operate as an image acquisition circuitry configured to acquire moving image data obtained by capturing an image of a target area;
operate as a person detector configured to detect and track a person, who acts within the target area, by analyzing the moving image data;
operate as an event detector configured to detect an event that occurs with the person during a period of tracking the person, by analyzing the moving image data;
operate as a physio-psychological index detector configured to detect a temporal change in a physio-psychological index of the person during the tracking period;
operate as an interest level estimator configured to estimate an interest level of the person for an object in the target area based on a temporal change in the physio-psychological index;
and
operate as a trend record generator configured to generate, as trend record data on the person, data in which at least one or more events that occur with the person, the temporal change in the physio-psychological index of the person, and information on the interest level estimated by the interest level estimator are associated temporally,
wherein the interest level estimator estimates the interest level by using two types of physio-psychological indices including a physio-psychological index consciously controllable by the person, and a physio-psychological index unconsciously controllable by the person,
wherein the interest level estimator estimates the interest level l(t) by Formula (2),
Interest
⁢
⁢
level
⁢
⁢
I
⁡
(
t
)
=
At
⁡
(
t
)
×
BF
⁡
(
t
)
-
BF
⁢
min
BF
⁢
max
-
BF
⁢
min
(
2
)
where
At(t) is a gaze level at time t,
BF(t) is a blinking frequency at time t,
BFmin is a minimum value of the blinking frequency in a predetermined period including time t, and
BFmax is a maximum value of the blinking frequency in the predetermined period,
wherein the gaze level is the physio-psychological index consciously controllable by the person, and the blinking frequency is the physio-psychological index unconsciously controllable by the person.

49 more elements

A person trend recording method comprising the steps, performed by a computer, of:

acquiring a moving image data obtained by capturing an image of a target area;
detecting and tracking a person, who acts within the target area, by analyzing the moving image data;
detecting an event that occurs with the person during a period of tracking the person, by analyzing the moving image data;
detecting a temporal change in a physio-psychological index of the person during the tracking period;
estimating an interest level of the person for an object in the target area based on the temporal change in the physio-psychological index;
and
generating, as trend record data on the person, data in which at least one or more events that occur with the person, the temporal change in the physio-psychological index of the person, and information on the estimated interest level are associated temporally, and storing the generated data in a memory,
wherein the interest level is estimated by using two types of physio-psychological indices including a physio-psychological index consciously controllable by the person, and a physio-psychological index unconsciously controllable by the person,
wherein the interest level estimator estimates the interest level l(t) by Formula (2),
Interest
⁢
⁢
level
⁢
⁢
I
⁡
(
t
)
=
At
⁡
(
t
)
×
BF
⁡
(
t
)
-
BF
⁢
min
BF
⁢
max
-
BF
⁢
min
(
2
)
where
At(t) is a gaze level at time t,
BF(t) is a blinking frequency at time t,
BFmin is a minimum value of the blinking frequency in a predetermined period including time t, and
BFmax is a maximum value of the blinking frequency in the predetermined period,
wherein the gaze level is the physio-psychological index consciously controllable by the person, and the blinking frequency is the physio-psychological index unconsciously controllable by the person.

49 more elements

Dependencies are inferred from the claim text.

Facts

Application: 16/461,912Filed 28 Nov 2017
Priority: JP JP2017-04882814 Mar 2017
Published: US 2019/0332855 A131 Oct 2019
Granted: 31 Aug 2021
Term ends: Not estimatedThe source records don’t support an estimate.

Classification11 codes

G06V 40/20
Recognition of biometric, human-related or animal-related patterns in image or video data: movements or behaviour, e.g. gesture recognition
G06Q 30/02
Commerce: marketing; Price estimation or determination; Fundraising
G06T 7/20
Image analysis: analysis of motion
G06V 40/168
Human faces, e.g. facial parts, sketches or expressions: feature extraction; Face representation