Image-based self-serve beverage detection and assignment

A method, comprising:

• receiving an image feed comprising frames of angled-view images generated by an image sensor, wherein the image sensor is positioned such that a field-of-view of the image sensor encompasses at least a portion of a beverage machine operable to dispense a beverage, wherein the image sensor is configured to generate the angled-view images of at least a portion of the beverage machine, wherein the portion includes a first zone associated with operating the beverage machine to dispense the beverage and a second zone in which a cup is placed to receive the beverage from the beverage machine;
• detecting an event associated with an object entering one or both of the first zone and the second zone;
• determining, in a first one or more image frames associated with a start of the detected event, that both a hand of a person enters the first zone and the cup is placed in the second zone;
• detecting that the cup is removed from the second zone;
• following detecting that the cup is removed from the second zone, determining a length of time during which the cup remained in the second zone;
• if the cup remained in the second zone for at least a threshold time, assigning the beverage to the person whose hand entered the first zone;
• if the cup did not remain in in the second zone for at least the threshold time, not assigning the beverage to the person whose hand entered the first zone;
• in response to detecting the event, identifying frames of images from the image feed associated with the detected event, wherein the identified image frames include:
• the first one or more image frames associated with the start of the detected event, wherein the first one or more image frames include images from the image feed from a first predefined time period before the detected event until a second predefined time period after the detected event;
• and
• a second one or more image frames associated with removal of the cup from the second zone, wherein the second one or more image frames include images from the image feed from a third predefined time period before the cup is removed from the second zone until a fourth predefined time period after the cup is removed from the second zone.

A beverage assignment subsystem comprising at least one processor configured to:

• receive an image feed comprising frames of angled-view images generated by an image sensor, wherein the image sensor is positioned such that a field-of-view of the image sensor encompasses at least a portion of a beverage machine operable to dispense a beverage, wherein the image sensor is configured to generate the angled-view images of at least a portion of the beverage machine, wherein the portion includes a first zone associated with operating the beverage machine to dispense the beverage and a second zone in which a cup is placed to receive the beverage from the beverage machine;
• detect an event associated with an object entering one or both of the first zone and the second zone;
• determine, in a first one or more image frames associated with a start of the detected event, that both a hand of a person enters the first zone and the cup is placed in the second zone;
• detect that the cup is removed from the second zone;
• following detecting that the cup is removed from the second zone, determine a length of time during which the cup remained in the second zone;
• if the cup remained in the second zone for at least a threshold time, assign the beverage to the person whose hand entered the first zone;
• if the cup did not remain in in the second zone for at least the threshold time, do not assign the beverage to the person whose hand entered the first zone;
• wherein the processor is further configured to, in response to detecting the event, identify frames of images from the image feed associated with the detected event, wherein the identified image frames include:
• the first one or more image frames associated with the start of the detected event, wherein the first one or more image frames include images from the image feed from a first predefined time period before the detected event until a second predefined time period after the detected event;
• and
• a second one or more image frames associated with removal of the cup from the second zone, wherein the second one or more image frames include images from the image feed from a third predefined time period before the cup is removed from the second zone until a fourth predefined time period after the cup is removed from the second zone.