Operation estimating apparatus and related article of manufacture

An operation estimating apparatus comprising:

• an image obtaining means for repeatedly obtaining images, each of the images having a plurality of operational objects and an operator that is positioned to be able to perform operations on the plurality of operational objects;
• a human body feature point specifying means for specifying a predetermined human body feature point of the operator in each of the images that are repeatedly obtained by the image obtaining means;
• an operation estimating means for estimating one of the operations, which the operator is going to perform, based on the human body feature points specified by the human body feature point specifying means in the images;
• and
• an operation supporting means for supporting the operation on a corresponding one of the plurality of operational objects that corresponds to a result of estimation by the operation estimating means, wherein:
• the operation estimating means compares an actual posture locus of the operator with a transitional estimation model for each of the operations by the operator to obtain a degree of approximation of the transitional estimation model to the actual posture locus, the transitional estimation model for the each of the operations being formed based on an estimated posture locus of the each of the operations, the operator being estimated to track the estimated posture locus to operate one of the plurality of operational objects that corresponds to the estimated posture locus, the actual posture locus of the operator being obtained based on the human body feature points specified by the human body feature point specifying means in the images;
• the operation estimating means estimates that the operator is going to perform the one of the operations that corresponds to the estimated posture locus of the transitional estimation model having the degree of approximation that satisfies a predetermined threshold;
• the plurality of operational objects includes an enclosing operational object, which has a cover, and which encloses a matter by opening and closing the cover, the enclosing operational object having an opening/closing mechanism that opens and closes the cover;
• and
• the operation supporting means controls the opening/closing mechanism of the enclosing operational object to open the cover such that the operation supporting means supports the operation on the enclosing operational object when the corresponding one of the plurality of operational objects that corresponds to the result of estimation by the operation estimating means is the enclosing operational object;
• and
• the operation supporting means controls the opening/closing mechanism of the enclosing operational object to close the cover when the corresponding one of the plurality of operational objects that corresponds to the result of estimation by the operation estimating means is different from the enclosing operational object and simultaneously when the cover of the enclosing operational object has already been opened.

An operation estimating apparatus comprising:

• an image obtaining means for repeatedly obtaining images, each of the images having a plurality of operational objects and an operator that is positioned to be able to perform operations on the plurality of operational objects;
• a human body feature point specifying means for specifying a predetermined human body feature point of the operator in each of the images that are repeatedly obtained by the image obtaining means;
• an operation estimating means for estimating one of the operations, which the operator is going to perform, based on the human body feature points specified by the human body feature point specifying means in the images;
• an approach detecting means for determining whether or not there is a matter that relatively approaches the vehicle;
• and
• a first interruption notifying means for sending a notification to stop operating the corresponding one of the plurality of operational objects when the approach detecting means determines that the matter approaches the vehicle in a case, where the corresponding one of the plurality of operational objects that corresponds to the result of estimation by the operation estimating means is different from the steering wheel and the shift lever, wherein:
• the operation estimating means compares an actual posture locus of the operator with a transitional estimation model for each of the operations by the operator to obtain a degree of approximation of the transitional estimation model to the actual posture locus, the transitional estimation model for the each of the operations being formed based on an estimated posture locus of the each of the operations, the operator being estimated to track the estimated posture locus to operate one of the plurality of operational objects that corresponds to the estimated posture locus, the actual posture locus of the operator being obtained based on the human body feature points specified by the human body feature point specifying means in the images;
• the operation estimating means estimates that the operator is going to perform the one of the operations that corresponds to the estimated posture locus of the transitional estimation model having the degree of approximation that satisfies a predetermined threshold
• the operation estimating apparatus is mounted on a vehicle;
• the plurality of operational objects includes at least one operational object of the vehicle;
• the operator exists in a seat of the vehicle;
• and
• the at least one operational object includes a steering wheel and a shift lever of the vehicle.

An operation estimating apparatus comprising:

• an image obtaining means for repeatedly obtaining images, each of the images having a plurality of operational objects and an operator that is positioned to be able to perform operations on the plurality of operational objects;
• a human body feature point specifying means for specifying a predetermined human body feature point of the operator in each of the images that are repeatedly obtained by the image obtaining means;
• and
• an operation estimating means for estimating one of the operations, which the operator is going to perform, based on the human body feature points specified by the human body feature point specifying means in the images;
• a travel state detecting means for detecting a travel state of the vehicle;
• a traffic information obtaining means for obtaining information about a traffic condition around the vehicle;
• and
• a second interruption notifying means for sending a notification to stop operating the corresponding one of the plurality of operational objects when the travel state detected by the travel state detecting means indicates that the vehicle stops and simultaneously when the information obtained by the traffic information obtaining means indicates the traffic condition that requires the vehicle to start in a case, where the corresponding one of the plurality of operational objects that corresponds to the result of estimation by the operation estimating means is different from the steering wheel and the shift lever, wherein:
• the operation estimating means compares an actual posture locus of the operator with a transitional estimation model for each of the operations by the operator to obtain a degree of approximation of the transitional estimation model to the actual posture locus, the transitional estimation model for the each of the operations being formed based on an estimated posture locus of the each of the operations, the operator being estimated to track the estimated posture locus to operate one of the plurality of operational objects that corresponds to the estimated posture locus, the actual posture locus of the operator being obtained based on the human body feature points specified by the human body feature point specifying means in the images;
• the operation estimating means estimates that the operator is going to perform the one of the operations that corresponds to the estimated posture locus of the transitional estimation model having the degree of approximation that satisfies a predetermined threshold;
• the operation estimating apparatus is mounted on a vehicle;
• the plurality of operational objects includes at least one operational object of the vehicle;
• the operator exists in a seat of the vehicle;
• and
• the at least one operational object includes a steering wheel and a shift lever of the vehicle.

An operation estimating apparatus comprising:

• an image obtaining means for repeatedly obtaining images, each of the images having a plurality of operational objects and an operator that is positioned to be able to perform operations on the plurality of operational objects;
• a human body feature point specifying means for specifying a predetermined human body feature point of the operator in each of the images that are repeatedly obtained by the image obtaining means;
• an operation estimating means for estimating one of the operations, which the operator is going to perform, based on the human body feature points specified by the human body feature point specifying means in the images;
• an operation state monitoring means for detecting an operational state of at least one of the plurality of operational objects to determine that the at least one of the plurality of operational objects is operated;
• and
• a stationary estimation model correcting means for correcting a stationary estimation model that corresponds to the at least one of the plurality of operational objects in accordance with an actual posture point at a time when the operation state monitoring means determines that the at least one of the plurality of operational objects is operated, the actual posture point being obtained based on the human body feature point specified by the human body feature point specifying means, the stationary estimation model being formed based on an estimated posture point of the operator, the operator being estimated to take the estimated posture point to continuously operate the at least one of the plurality of operational objects, wherein:
• the operation estimating means compares an actual posture locus of the operator with a transitional estimation model for each of the operations by the operator to obtain a degree of approximation of the transitional estimation model to the actual posture locus, the transitional estimation model for the each of the operations being formed based on an estimated posture locus of the each of the operations, the operator being estimated to track the estimated posture locus to operate one of the plurality of operational objects that corresponds to the estimated posture locus, the actual posture locus of the operator being obtained based on the human body feature points specified by the human body feature point specifying means in the images;
• and
• the operation estimating means estimates that the operator is going to perform the one of the operations that corresponds to the estimated posture locus of the transitional estimation model having the degree of approximation that satisfies a predetermined threshold.

An operation estimating apparatus comprising:

• an image obtaining means for repeatedly obtaining images, each of the images having a plurality of operational objects and an operator that is positioned to be able to perform operations on the plurality of operational objects;
• a human body feature point specifying means for specifying a predetermined human body feature point of the operator in each of the images that are repeatedly obtained by the image obtaining means;
• and
• an operation estimating means for estimating one of the operations, which the operator is going to perform, based on the human body feature points specified by the human body feature point specifying means in the images, wherein:
• the operation estimating means compares an actual posture locus of the operator with a transitional estimation model for each of the operations by the operator to obtain a degree of approximation of the transitional estimation model to the actual posture locus, the transitional estimation model for the each of the operations being formed based on an estimated posture locus of the each of the operations, the operator being estimated to track the estimated posture locus to operate one of the plurality of operational objects that corresponds to the estimated posture locus, the actual posture locus of the operator being obtained based on the human body feature points specified by the human body feature point specifying means in the images;
• the operation estimating means estimates that the operator is going to perform the one of the operations that corresponds to the estimated posture locus of the transitional estimation model having the degree of approximation that satisfies a predetermined threshold;
• the transitional estimation model includes an estimation model that is formed based on the estimated posture locus tracked by the operator that performs a transitional operation as one of the operations;
• and
• the operator finishes operating a certain operational object of the plurality of operational objects, and then, starts operating another operational object of the plurality of operational objects in the transitional operation, the operation estimating apparatus further comprising:
• an operation state monitoring means for detecting an operational state of at least one of the plurality of operational objects to determine that the at least one of the plurality of operation objects is operated;
• and
• a transitional estimation model correcting means for correcting the transitional estimation model that corresponds to the transitional operation based on the actual posture locus that is obtained based on the human body feature points specified by the human body feature point specifying means specified during an interval between (a) a time, at which the operation state monitoring means becomes unable to detect the operation on the certain operational object, which has been detected to be operated, and (b) a time, at which the operation state monitoring means detects the operation on the another operational object.