System and method for augmented reality detection of loose pharmacy items

An augmented reality apparatus for fulfilling a pharmacy order, the augmented reality apparatus comprising:

• an image-capturing device configured to capture video data that includes a one or more images of at least a portion of a pharmacy workstation;
• a processor in communication with at least one memory that includes instructions that, when executed by the processor, cause the processor to:
• retrieve pharmacy order information indicating at least one pharmacy item corresponding to the pharmacy order, the pharmacy order corresponding to a pharmacy customer;
• read a first image of the one or more images of the video data;
• identify objects of interest in the first image, the objects of interest corresponding to at least the at least one pharmacy item;
• classify the detected objects of interest;
• identify a virtual boundary defining an opening of a container in a second image of the one or more images;
• superimpose, on the display device, a threshold that defines a virtual opening bound by the virtual boundary on the container;
• count detected objects of interest passing the threshold into the container;
• update an objects in container list based on a determination that at least one of the classified objects passed the threshold;
• and
• generate information corresponding to the pharmacy order based on, at least, the objects in container list;
• and
• a display device configured to display the information, the information indicating, at least, the objects in container list.

A system for fulfilling a pharmacy order, the system comprising:

• a wearable augmented reality device configured to:
• capture video data that includes a one or more images of at least a portion of a pharmacy workstation;
• retrieve pharmacy order information indicating at least one pharmacy item corresponding to the pharmacy order, the pharmacy order corresponding to a pharmacy customer;
• identify objects of interest in the one or more images, the objects of interest corresponding to at least the at least one pharmacy item;
• and
• identify a virtual boundary defining an opening of a container in the one or more images;
• a processor configured to classify the objects of interest identified by the wearable augmented reality device, wherein the wearable augmented reality device:
• identifies, in the one or more images, classified objects of interest that pass the virtual boundary of the container;
• superimposes, on a display of the wearable augmented reality device, a threshold that defines a virtual opening bound by the virtual boundary on the container;
• counts classified objects of interest that pass the threshold into the container;
• and
• provides, to a wearer of the wearable augmented reality device on the display of the wearable augmented reality device, information corresponding to the pharmacy order in response to counting classified objects of interest that pass the virtual opening of the container.