Recording medium, image processing device, image processing method, and in-vehicle interface device

An image processing device comprising

• a controller,
• wherein the controller is configured to:
• acquire an image in which a plurality of indicators is captured;
• set a pointer designating a position represented by ends of a first indicator and a second indicator at a position between the end of the first indicator and the end of the second indicator specified based on the acquired image;
• store a history of the position of the set pointer in association with a frame;
• adjust a number of frames for calculating a moving average according to a moving speed of an end of the indicator;
• calculate a moving average of the position of the pointer in each of the adjusted number of frames upon receipt of a current frame;
• and
• set the position of the pointer in the current frame based on the calculated moving average.

An image processing method comprising:

• acquiring an image in which a plurality of indicators is captured;
• setting a pointer designating a position represented by ends of a first indicator and a second indicator at a position between the end of the first indicator and the end of the second indicator specified based on the acquired image;
• storing a history of the position of the set pointer in association with a frame;
• adjusting a number of frames for calculating a moving average according to a moving speed of an end of the indicator;
• calculating a moving average of the position of the pointer in each of the adjusted number of frames upon receipt of a current frame;
• and
• setting the position of the pointer in the current frame based on the calculated moving average.

An in-vehicle interface device comprising

• a controller,
• wherein the controller is configured to:
• acquire an image in which a plurality of indicators is captured;
• set a pointer designating a position represented by ends of a first indicator and a second indicator at a position between the end of the first indicator and the end of the second indicator specified based on the acquired image;
• store a history of the position of the set pointer in association with a frame;
• adjust a number of frames for calculating a moving average according to a moving speed of an end of the indicator;
• calculate a moving average of the position of the pointer in each of the adjusted number of frames upon receipt of a current frame;
• set the position of the pointer in the current frame based on the calculated moving average;
• and
• accept an operation by the pointer displayed on an in-vehicle device operation screen.

A computer readable non-transitory recording medium recording a computer program causing a computer to execute processes of:

• acquiring an image in which a plurality of indicators is captured;
• setting a pointer designating a position represented by ends of a first indicator and a second indicator at a position between the end of the first indicator and the end of the second indicator specified based on the acquired image, wherein:
• the image in which the plurality of indicators is captured is an image of a hand or finger,
• the end of the first indicator and the end of the second indicator are a fingertip of a first finger and a fingertip of a second finger,
• the computer program causes the computer to execute a process of setting the pointer at a middle position between the fingertip of the first finger and the fingertip of the second finger;
• detecting a tip of a fingertip in a current frame by performing a search along a search axis in a predetermined direction on the acquired image of the hand or finger;
• setting a perpendicular axis separated from the tip by a predetermined distance and perpendicular to the search axis;
• detecting an outer boundary of each of the first finger and the second finger by performing a search along the set perpendicular axis;
• specifying a midpoint of the detected outer boundary;
• and
• setting the position of the pointer in the current frame based on the specified midpoint.

A computer readable non-transitory recording medium recording a computer program causing a computer to execute processes of:

• acquiring an image in which a plurality of indicators is captured;
• setting a pointer in a non-selected state and designating a position represented by ends of a first indicator and a second indicator at a position between the end of the first indicator and the end of the second indicator specified based on the acquired image, wherein:
• the image in which the plurality of indicators is captured is an image of a hand or finger,
• the end of the first indicator and the end of the second indicator are a fingertip of a first finger and a fingertip of a second finger,
• the computer program causes the computer to execute a process of setting the pointer at a middle position between the fingertip of the first finger and the fingertip of the second finger;
• inputting the image of the hand or finger to a learning model generated to output classification information including a first shape indicating a selected state in which the first finger and the second finger are in contact or a second shape indicating a non-selected state in which the first finger and the second finger are separated from each other when the image of the hand or finger is input;
• and
• switching a display mode of a pointer located between a tip of the first finger and a tip of the second finger depending on the classification information output by the learning model, wherein the classification indicates a selected state.

A computer readable non-transitory recording medium recording a computer program causing a computer to execute processes of:

• acquiring an image in which a plurality of indicators is captured;
• setting a pointer designating a position represented by ends of a first indicator and a second indicator at a position between the end of the first indicator and the end of the second indicator specified based on the acquired image, wherein:
• the image in which the plurality of indicators is captured is an image of a hand or finger,
• the end of the first indicator and the end of the second indicator are a fingertip of a first finger and a fingertip of a second finger;
• detecting a tip of a fingertip by performing a search along a search axis in a predetermined direction on the acquired image of the hand or finger;
• setting a perpendicular axis separated from the tip by a predetermined distance and perpendicular to the search axis;
• detecting a first outer boundary point and a second outer boundary point of the first finger and the second finger, respectively, by performing a search along the set perpendicular axis;
• correcting the direction in a current frame of the search axis based on a first tangent line tangent to a boundary of the first finger at the detected first outer boundary point and a second tangent line tangent to a boundary of the second finger at the detected second outer boundary point;
• and
• setting the position of the pointer in the current frame based on the corrected search axis.

A computer readable non-transitory recording medium recording a computer program causing a computer to execute processes of:

• acquiring an image in which a plurality of indicators is captured;
• setting a pointer designating a position represented by ends of a first indicator and a second indicator at a position between the end of the first indicator and the end of the second indicator specified based on the acquired image, wherein:
• the image in which the plurality of indicators is captured is an image of a hand or finger,
• the end of the first indicator and the end of the second indicator are a fingertip of a first finger and a fingertip of a second finger;
• storing a position of a fingertip specified when the position of the pointer is set and a direction of a search axis in association with a frame;
• and
• setting the position of the pointer by detecting a tip of a fingertip by performing a search along a search axis associated with a current frame based on a position of the fingertip associated with the current frame when the position of the fingertip and a direction of the search axis associated with the current frame are stored.

A computer readable non-transitory recording medium recording a computer program causing a computer to execute processes of:

• acquiring an image in which a plurality of indicators is captured;
• setting a pointer designating a position represented by ends of a first indicator and a second indicator at a position between the end of the first indicator and the end of the second indicator specified based on the acquired image, wherein:
• the image in which the plurality of indicators is captured is an image of a hand or finger,
• the end of the first indicator and the end of the second indicator are a fingertip of a first finger and a fingertip of a second finger;
• specifying a contour of the hand or finger based on the acquired image of the hand or finger;
• associating each point on the specified contour with a distance to a search axis in a predetermined direction;
• specifying a point on the contour in a current frame corresponding to a local minimum point of the distance as a fingertip end;
• and
• setting the position of the pointer in the current frame based on the local minimum point.

An image processing device comprising

• a controller,
• wherein the controller is configured to:
• acquire an image in which a plurality of indicators is captured;
• set a pointer designating a position represented by ends of a first indicator and a second indicator at a position between the end of the first indicator and the end of the second indicator specified based on the acquired image, wherein:
• the image in which the plurality of indicators is captured is an image of a hand or finger,
• the end of the first indicator and the end of the second indicator are a fingertip of a first finger and a fingertip of a second finger;
• setting the pointer at a middle position between the fingertip of the first finger and the fingertip of the second finger;
• detecting a tip of a fingertip in a current frame by performing a search along a search axis in a predetermined direction on the acquired image of the hand or finger;
• setting a perpendicular axis separated from the tip by a predetermined distance and perpendicular to the search axis;
• detecting an outer boundary of each of the first finger and the second finger by performing a search along the set perpendicular axis;
• specifying a midpoint of the detected outer boundary;
• and
• setting the position of the pointer in the current frame based on the specified midpoint.

An image processing device comprising

• a controller,
• wherein the controller is configured to:
• acquire an image in which a plurality of indicators is captured;
• set a pointer designating a position represented by ends of a first indicator and a second indicator at a position between the end of the first indicator and the end of the second indicator specified based on the acquired image, wherein:
• the image in which the plurality of indicators is captured is an image of a hand or finger,
• the end of the first indicator and the end of the second indicator are a fingertip of a first finger and a fingertip of a second finger;
• setting the pointer at a middle position between the fingertip of the first finger and the fingertip of the second finger;
• inputting the image of the hand or finger to a learning model generated to output classification information including a first shape indicating a selected state in which the first finger and the second finger are in contact or a second shape indicating a non-selected state in which the first finger and the second finger are separated from each other when the image of the hand or finger is input;
• and
• switching a display mode of a pointer located between a tip of the first finger and a tip of the second finger depending on the classification information output by the learning model, wherein the classification indicates a selected state.

An image processing device comprising

• a controller,
• wherein the controller is configured to:
• acquire an image in which a plurality of indicators is captured;
• set a pointer designating a position represented by ends of a first indicator and a second indicator at a position between the end of the first indicator and the end of the second indicator specified based on the acquired image, wherein:
• the image in which the plurality of indicators is captured is an image of a hand or finger,
• the end of the first indicator and the end of the second indicator are a fingertip of a first finger and a fingertip of a second finger;
• setting the pointer at a middle position between the fingertip of the first finger and the fingertip of the second finger;
• detecting a tip of a fingertip by performing a search along a search axis in a predetermined direction on the acquired image of the hand or finger;
• setting a perpendicular axis separated from the tip by a predetermined distance and perpendicular to the search axis;
• detecting a first outer boundary point and a second outer boundary point of the first finger and the second finger, respectively, by performing a search along the set perpendicular axis;
• correcting the direction in a current frame of the search axis based on a first tangent line tangent to a boundary of the first finger at the detected first outer boundary point and a second tangent line tangent to a boundary of the second finger at the detected second outer boundary point;
• and
• setting the position of the pointer in the current frame based on the corrected search axis.

An image processing device comprising

• a controller,
• wherein the controller is configured to:
• acquire an image in which a plurality of indicators is captured;
• set a pointer designating a position represented by ends of a first indicator and a second indicator at a position between the end of the first indicator and the end of the second indicator specified based on the acquired image, wherein:
• the image in which the plurality of indicators is captured is an image of a hand or finger,
• the end of the first indicator and the end of the second indicator are a fingertip of a first finger and a fingertip of a second finger;
• setting the pointer at a middle position between the fingertip of the first finger and the fingertip of the second finger;
• storing a position of a fingertip specified when the position of the pointer is set and a direction of a search axis in association with a frame;
• and
• setting the position of the pointer by detecting a tip of a fingertip by performing a search along a search axis associated with a current frame based on a position of the fingertip associated with the current frame when the position of the fingertip and a direction of the search axis associated with the current frame are stored.

An image processing device comprising

• a controller,
• wherein the controller is configured to:
• acquire an image in which a plurality of indicators is captured;
• set a pointer designating a position represented by ends of a first indicator and a second indicator at a position between the end of the first indicator and the end of the second indicator specified based on the acquired image, wherein:
• the image in which the plurality of indicators is captured is an image of a hand or finger,
• the end of the first indicator and the end of the second indicator are a fingertip of a first finger and a fingertip of a second finger;
• setting the pointer at a middle position between the fingertip of the first finger and the fingertip of the second finger;
• specifying a contour of the hand or finger based on the acquired image of the hand or finger;
• associating each point on the specified contour with a distance to a search axis in a predetermined direction;
• specifying a point on the contour in a current frame corresponding to a local minimum point of the distance as a fingertip end;
• and
• setting the position of the pointer in the current frame based on the local minimum point.

An image processing method comprising:

• acquiring an image in which a plurality of indicators is captured;
• setting a pointer designating a position represented by ends of a first indicator and a second indicator at a position between the end of the first indicator and the end of the second indicator specified based on the acquired image, wherein:
• the image in which the plurality of indicators is captured is an image of a hand or finger,
• the end of the first indicator and the end of the second indicator are a fingertip of a first finger and a fingertip of a second finger;
• setting the pointer at a middle position between the fingertip of the first finger and the fingertip of the second finger;
• detecting a tip of a fingertip in a current frame by performing a search along a search axis in a predetermined direction on the acquired image of the hand or finger;
• setting a perpendicular axis separated from the tip by a predetermined distance and perpendicular to the search axis;
• detecting an outer boundary of each of the first finger and the second finger by performing a search along the set perpendicular axis;
• specifying a midpoint of the detected outer boundary;
• and
• setting the position of the pointer in the current frame based on the specified midpoint.

An image processing method comprising:

• acquiring an image in which a plurality of indicators is captured;
• setting a pointer designating a position represented by ends of a first indicator and a second indicator at a position between the end of the first indicator and the end of the second indicator specified based on the acquired image, wherein:
• the image in which the plurality of indicators is captured is an image of a hand or finger,
• the end of the first indicator and the end of the second indicator are a fingertip of a first finger and a fingertip of a second finger;
• setting the pointer at a middle position between the fingertip of the first finger and the fingertip of the second finger;
• inputting the image of the hand or finger to a learning model generated to output classification information including a first shape indicating a selected state in which the first finger and the second finger are in contact or a second shape indicating a non-selected state in which the first finger and the second finger are separated from each other when the image of the hand or finger is input;
• and
• switching a display mode of a pointer located between a tip of the first finger and a tip of the second finger depending on the classification information output by the learning model, wherein the classification indicates a selected state.

An image processing method comprising:

• acquiring an image in which a plurality of indicators is captured;
• setting a pointer designating a position represented by ends of a first indicator and a second indicator at a position between the end of the first indicator and the end of the second indicator specified based on the acquired image, wherein:
• the image in which the plurality of indicators is captured is an image of a hand or finger,
• the end of the first indicator and the end of the second indicator are a fingertip of a first finger and a fingertip of a second finger;
• setting the pointer at a middle position between the fingertip of the first finger and the fingertip of the second finger;
• detecting a tip of a fingertip by performing a search along a search axis in a predetermined direction on the acquired image of the hand or finger;
• setting a perpendicular axis separated from the tip by a predetermined distance and perpendicular to the search axis;
• detecting a first outer boundary point and a second outer boundary point of the first finger and the second finger, respectively, by performing a search along the set perpendicular axis;
• and
• correcting the direction in a current frame of the search axis based on a first tangent line tangent to a boundary of the first finger at the detected first outer boundary point and a second tangent line tangent to a boundary of the second finger at the detected second outer boundary point;
• and
• setting the position of the pointer in the current frame based on the corrected search axis.

An image processing method comprising:

• acquiring an image in which a plurality of indicators is captured;
• setting a pointer designating a position represented by ends of a first indicator and a second indicator at a position between the end of the first indicator and the end of the second indicator specified based on the acquired image, wherein:
• the image in which the plurality of indicators is captured is an image of a hand or finger,
• the end of the first indicator and the end of the second indicator are a fingertip of a first finger and a fingertip of a second finger;
• setting the pointer at a middle position between the fingertip of the first finger and the fingertip of the second finger;
• storing a position of a fingertip specified when the position of the pointer is set and a direction of a search axis in association with a frame;
• and
• setting the position of the pointer by detecting a tip of a fingertip by performing a search along a search axis associated with a current frame based on a position of the fingertip associated with the current frame when the position of the fingertip and a direction of the search axis associated with the current frame are stored.

An image processing method comprising:

• acquiring an image in which a plurality of indicators is captured;
• setting a pointer designating a position represented by ends of a first indicator and a second indicator at a position between the end of the first indicator and the end of the second indicator specified based on the acquired image, wherein:
• the image in which the plurality of indicators is captured is an image of a hand or finger,
• the end of the first indicator and the end of the second indicator are a fingertip of a first finger and a fingertip of a second finger;
• setting the pointer at a middle position between the fingertip of the first finger and the fingertip of the second finger;
• specifying a contour of the hand or finger based on the acquired image of the hand or finger;
• associating each point on the specified contour with a distance to a search axis in a predetermined direction;
• specifying a point on the contour in a current frame corresponding to a local minimum point of the distance as a fingertip end;
• and
• setting the position of the pointer in the current frame based on the local minimum point.

An in-vehicle interface device comprising

• a controller,
• wherein the controller is configured to:
• acquire an image in which a plurality of indicators is captured;
• set a pointer designating a position represented by ends of a first indicator and a second indicator at a position between the end of the first indicator and the end of the second indicator specified based on the acquired image;
• accept an operation by the pointer displayed on an in-vehicle device operation screen, wherein:
• the image in which the plurality of indicators is captured is an image of a hand or finger,
• the end of the first indicator and the end of the second indicator are a fingertip of a first finger and a fingertip of a second finger;
• setting the pointer at a middle position between the fingertip of the first finger and the fingertip of the second finger;
• detecting a tip of a fingertip in a current frame by performing a search along a search axis in a predetermined direction on the acquired image of the hand or finger;
• setting a perpendicular axis separated from the tip by a predetermined distance and perpendicular to the search axis;
• detecting an outer boundary of each of the first finger and the second finger by performing a search along the set perpendicular axis;
• specifying a midpoint of the detected outer boundary;
• and
• setting the position of the pointer in the current frame based on the specified midpoint.

An in-vehicle interface device comprising

• a controller,
• wherein the controller is configured to:
• acquire an image in which a plurality of indicators is captured;
• set a pointer designating a position represented by ends of a first indicator and a second indicator at a position between the end of the first indicator and the end of the second indicator specified based on the acquired image;
• accept an operation by the pointer displayed on an in-vehicle device operation screen, wherein:
• the image in which the plurality of indicators is captured is an image of a hand or finger,
• the end of the first indicator and the end of the second indicator are a fingertip of a first finger and a fingertip of a second finger;
• setting the pointer at a middle position between the fingertip of the first finger and the fingertip of the second finger;
• inputting the image of the hand or finger to a learning model generated to output classification information including a first shape indicating a selected state in which the first finger and the second finger are in contact or a second shape indicating a non-selected state in which the first finger and the second finger are separated from each other when the image of the hand or finger is input;
• and
• switching a display mode of a pointer located between a tip of the first finger and a tip of the second finger depending on the classification information output by the learning model, wherein the classification indicates a selected state.

An in-vehicle interface device comprising

• a controller,
• wherein the controller is configured to:
• acquire an image in which a plurality of indicators is captured;
• set a pointer designating a position represented by ends of a first indicator and a second indicator at a position between the end of the first indicator and the end of the second indicator specified based on the acquired image;
• accept an operation by the pointer displayed on an in-vehicle device operation screen, wherein:
• the image in which the plurality of indicators is captured is an image of a hand or finger,
• the end of the first indicator and the end of the second indicator are a fingertip of a first finger and a fingertip of a second finger;
• setting the pointer at a middle position between the fingertip of the first finger and the fingertip of the second finger;
• detecting a tip of a fingertip by performing a search along a search axis in a predetermined direction on the acquired image of the hand or finger;
• setting a perpendicular axis separated from the tip by a predetermined distance and perpendicular to the search axis;
• detecting a first outer boundary point and a second outer boundary point of the first finger and the second finger, respectively, by performing a search along the set perpendicular axis;
• correcting the direction in a current frame of the search axis based on a first tangent line tangent to a boundary of the first finger at the detected first outer boundary point and a second tangent line tangent to a boundary of the second finger at the detected second outer boundary point;
• and
• setting the position of the pointer in the current frame based on the corrected search axis.

An in-vehicle interface device comprising

• a controller,
• wherein the controller is configured to:
• acquire an image in which a plurality of indicators is captured;
• set a pointer designating a position represented by ends of a first indicator and a second indicator at a position between the end of the first indicator and the end of the second indicator specified based on the acquired image;
• accept an operation by the pointer displayed on an in-vehicle device operation screen, wherein:
• the image in which the plurality of indicators is captured is an image of a hand or finger,
• the end of the first indicator and the end of the second indicator are a fingertip of a first finger and a fingertip of a second finger;
• setting the pointer at a middle position between the fingertip of the first finger and the fingertip of the second finger;
• storing a position of a fingertip specified when the position of the pointer is set and a direction of a search axis in association with a frame;
• and
• setting the position of the pointer by detecting a tip of a fingertip by performing a search along a search axis associated with a current frame based on a position of the fingertip associated with the current frame when the position of the fingertip and a direction of the search axis associated with the current frame are stored.

An in-vehicle interface device comprising

• a controller,
• wherein the controller is configured to:
• acquire an image in which a plurality of indicators is captured;
• set a pointer designating a position represented by ends of a first indicator and a second indicator at a position between the end of the first indicator and the end of the second indicator specified based on the acquired image;
• accept an operation by the pointer displayed on an in-vehicle device operation screen, wherein:
• the image in which the plurality of indicators is captured is an image of a hand or finger,
• the end of the first indicator and the end of the second indicator are a fingertip of a first finger and a fingertip of a second finger;
• setting the pointer at a middle position between the fingertip of the first finger and the fingertip of the second finger;
• specifying a contour of the hand or finger based on the acquired image of the hand or finger;
• associating each point on the specified contour with a distance to a search axis in a predetermined direction;
• specifying a point on the contour in a current frame corresponding to a local minimum point of the distance as a fingertip end;
• and
• setting the position of the pointer in the current frame based on the local minimum point.