Apparatus and Method for Determination of Medication Location

A method for confirming a medication pill in a mouth of a user, comprising:

• predefining one or more steps associated with confirming the medication pill in the mouth of a user, the one or more steps comprising at least proper identification of the medication pill, proper positioning of an open mouth of the user, and proper showing of whether the medication pill is in the mouth of the user;
• displaying, on a display, one or more guidance instructions for positioning the medication pill relative to an image capture device and within a field of view of the image capture device;
• capturing, by the image capture device, a set of one or more video frames;
• determining, by one or more processors in near real time, based on the set of one or more video frames, that the user has properly positioned the medication pill within the field of view in response to the one or more guidance instructions;
• identifying, by the one or more processors in near real time, based on the set of one or more video frames, one or more visual characteristics of the medication pill;
• setting a bottom portion of a face of the user as a region of interest, the region of interest comprising the mouth of the user;
• determining, by the one or more processors in near real time, based on the set of one or more video frames, that the mouth of the user in the region of interest is open;
• subsequent to determining that the mouth of the user is open, extracting an image of the open mouth from the set of one or more video frames, and classifying, by the one or more processors, the open mouth from the extracted image as a mouth without a pill inside or as a mouth with a pill inside;
• and
• in response to classifying the open mouth as the mouth without a pill inside,
• displaying, on the display, guidance for positioning the open mouth of the user relative to the image capture device, and
• displaying, on the display, guidance to the user, the guidance comprising an instruction to place the medication pill in the mouth of the user.

A system for confirming a medication pill in a mouth of a user, the system comprising:

• an image capture device;
• a display;
• a processor;
• and
• a memory storing one or more instructions that, when executed by the processor, cause the processor to perform operations comprising:
• causing the display to output one or more guidance instructions for positioning an open mouth of the user relative to the image capture device;
• receiving, from the image capture device, one or more video frames of the user;
• determining, based on the one or more video frames, in near real time, whether the user has properly positioned the medication pill in response to the guidance instructions;
• setting a bottom portion of a face of the user as a region of interest, the region of interest comprising the mouth of the user;
• classifying, in near real time, based on the one or more video frames, the mouth of the user in the region of interest as open;
• subsequent to classifying the mouth of the user as open, extracting an image of the open mouth from the one or more video frames, and classifying the open mouth in the extracted image as a mouth without a pill inside or as a mouth with a pill inside;
• and
• in response to classifying the open mouth in the extracted image as the mouth without a pill inside, causing the display to output guidance to the user to place the medication pill in the mouth of the user.

A non-volatile computer readable storage medium storing a computer program, the computer program comprising one or more instructions that, when executed by a computer, cause the computer to perform operations comprising:

• predefining one or more steps associated with confirming a medication pill in a mouth of a user, the one or more steps comprising at least proper identification of the medication pill, proper positioning of an open mouth of the user, and proper showing of whether the medication pill is in the mouth of the user;
• displaying, on a display, one or more guidance instructions for positioning the medication pill relative to an image capture device associated with the display and within a field of view of the image capture device;
• capturing, using the image capture device, a first set of one or more video frames of the user;
• determining, in near real time, based on the first set of one or more video frames, whether the user has properly positioned the medication pill within the field of view in response to the one or more guidance instructions,
• identifying, based on the first set of one or more video frames, one or more visual characteristics of the medication pill;
• setting a bottom portion of a face of the user as a region of interest, the region of interest comprising the mouth of the user;
• capturing, using the image capture device, a second set of one or more video frames;
• determining, in near real time, that the mouth of the user is open in the second set of one or more video frames;
• subsequent to determining that the mouth of the user is open in the second set of one or more video frames, extracting an image of the open mouth of the user from the second set of one or more video frames, and classifying the open mouth in the extracted image as a mouth without a pill inside or as a mouth with a pill inside;
• and
• responsive to classifying the open mouth of the user as the mouth without a pill inside,
• displaying, on the display, guidance for positioning the open mouth of the user relative to the image capture device, and
• displaying, on the display, guidance to the user, the guidance comprising an instruction to place the medication pill in the mouth of the user.