Fast multi-modal object recognition system

An interactive information system (IIS), comprising:

• an imaging device;
• a computer processor;
• and
• memory storing instructions, when executed by the computer processor, comprising functionalities for:
• capturing, using the imaging device of the IIS, a first image of a first representative object, wherein the first representative object represents a first one or more object disposed about the IIS;
• analyzing, based on a category model, the first image to determine a first representative category of the first one or more object, wherein the first representative category represents a first one or more category;
• retrieving, based on the first representative category, a first representative object model of a plurality of object models that are stored on a remote server, wherein the first representative object model represents a first one or more object model;
• analyzing, based on the first representative object model, the first image to determine a first representative inventory identifier of the first representative object, wherein the first representative inventory identifier represents a first one or more inventory identifier corresponding to the first one or more object respectively;
• capturing, using the imaging device of the IIS, a third image of a third representative object, wherein the third representative object represents a third one or more object disposed about the IIS;
• and
• in response to a user activation or a failure of determining a third representative inventory identifier of the third representative object based on the first representative object model:
• analyzing, based on the category model, the third image to determine a second representative category of the third one or more object, wherein the second representative category represents a second one or more category;
• retrieving, based on the second representative category, a second representative object model of the plurality of object models that are stored on the remote server, wherein the second representative object model represents a second one or more object model;
• and
• analyzing, based on the second representative object model, the third image to determine the third representative inventory identifier of the third representative object, wherein the third representative inventory identifier represents a third one or more inventory identifier corresponding to the third one or more object respectively.

A non-transitory computer readable medium (CRM) storing computer readable program code for object recognition at an interactive information system (IIS), wherein the computer readable program code, when executed by a computer processor of the IIS, comprises functionalities for:

• capturing, using an imaging device of the IIS, a first image of a first representative object, wherein the first representative object represents a first one or more objects disposed about the IIS;
• analyzing, based on a category model, the first image to determine a first representative category of the first one or more object, wherein the first representative category represents a first one or more category;
• retrieving, based on the first representative category, a first representative object model of a plurality of object models that are stored on a remote server, wherein the first representative object model represents a first one or more object model;
• analyzing, based on the first representative object model, the first image to determine a first representative inventory identifier of the first representative object, wherein the first representative inventory identifier represents a first one or more inventory identifier corresponding to the first one or more object respectively;
• capturing, using the imaging device of the IIS, a third image of a third representative object, wherein the third representative object represents a third one or more object disposed about the IIS;
• and
• in response to a user activation or a failure of determining a third representative inventory identifier of the third representative object based on the first representative object model:
• analyzing, based on the category model, the third image to determine a second representative category of the third one or more object, wherein the second representative category represents a second one or more category;
• retrieving, based on the second representative category, a second representative object model of the plurality of object models that are stored on the remote server, wherein the second representative object model represents a second one or more object mode;
• and
• analyzing, based on the second representative object model, the third image to determine the third representative inventory identifier of the third representative object, wherein the third representative inventory identifier represents a third one or more inventory identifier corresponding to the third one or more object respectively.

A method for object recognition at an interactive information system (IIS), the method comprising:

• capturing, using an imaging device of the IIS, a first image of a first representative object, wherein the first representative object represents a first one or more object disposed about the IIS;
• analyzing, by a computer processor of the IIS and based on a category model, the first image to determine a first representative category of the first one or more object, wherein the first representative category represents a first one or more category;
• retrieving, by the computer processor of the IIS and based on the first representative category, a first representative object model of a plurality of object models that are stored on a remote server, wherein the first representative object model represents a first one or more object model;
• and
• analyzing, by the computer processor of the IIS and based on the first representative object model, the first image to determine a first representative inventory identifier of the first representative object, wherein the first representative inventory identifier represents a first one or more inventory identifier corresponding to the first one or more object respectively,
• wherein determining the first representative inventory identifier of the first one or more object further comprises:
• determining, by the computer processor of the IIS and based on a result of analyzing the first image based on the first representative object model, that a part of the first representative object is associated with a first representative subcategory of the first representative category, wherein the first representative subcategory represents a first one or more subcategory;
• and
• retrieving, by the computer processor of the IIS and based on the first representative subcategory, a part of the first representative object model corresponding to the first representative subcategory.

An interactive information system (IIS), comprising:

• an imaging device;
• a computer processor;
• and
• memory storing instructions, when executed by the computer processor, comprising functionalities for:
• capturing, using the imaging device of the ITS, a first image of a first representative object, wherein the first representative object represents a first one or more object disposed about the ITS;
• analyzing, based on a category model, the first image to determine a first representative category of the first one or more object, wherein the first representative category represents a first one or more category;
• retrieving, based on the first representative category, a first representative object model of a plurality of object models that are stored on a remote server, wherein the first representative object model represents a first one or more object model;
• and
• analyzing, based on the first representative object model, the first image to determine a first representative inventory identifier of the first representative object, wherein the first representative inventory identifier represents a first one or more inventory identifier corresponding to the first one or more object respectively;
• wherein determining the first representative inventory identifier of the first one or more object further comprises:
• determining, based on a result of analyzing the first image based on the first representative object model, that a part of the first representative object is associated with a first representative subcategory of the first representative category, wherein the first representative subcategory represents a first one or more subcategory;
• and
• retrieving, based on the first representative subcategory, a part of the first representative object model corresponding to the first representative subcategory.

A non-transitory computer readable medium (CRM) storing computer readable program code for object recognition at an interactive information system (IIS), wherein the computer readable program code, when executed by a computer processor of the ITS, comprises functionalities for:

• capturing, using an imaging device of the ITS, a first image of a first representative object, wherein the first representative object represents a first one or more objects disposed about the ITS;
• analyzing, based on a category model, the first image to determine a first representative category of the first one or more object, wherein the first representative category represents a first one or more category;
• retrieving, based on the first representative category, a first representative object model of a plurality of object models that are stored on a remote server, wherein the first representative object model represents a first one or more object model;
• and
• analyzing, based on the first representative object model, the first image to determine a first representative inventory identifier of the first representative object, wherein the first representative inventory identifier represents a first one or more inventory identifier corresponding to the first one or more object respectively,
• wherein determining the first representative inventory identifier of the first one or more object further comprises:
• determining, based on a result of analyzing the first image based on the first representative object model, that a part of the first representative object is associated with a first representative subcategory of the first representative category, wherein the first representative subcategory represents a first one or more subcategory;
• and
• retrieving, based on the first representative subcategory, a part of the first representative object model corresponding to the first representative subcategory.

A method for object recognition at an interactive information system (IIS), the method comprising:

• capturing, using an imaging device of the ITS, a first image of a first representative object, wherein the first representative object represents a first one or more object disposed about the ITS;
• analyzing, by a computer processor of the IIS and based on a category model, the first image to determine a first representative category of the first one or more object, wherein the first representative category represents a first one or more category;
• retrieving, by the computer processor of the IIS and based on the first representative category, a first representative object model of a plurality of object models that are stored on a remote server, wherein the first representative object model represents a first one or more object model;
• analyzing, by the computer processor of the IIS and based on the first representative object model, the first image to determine a first representative inventory identifier of the first representative object, wherein the first representative inventory identifier represents a first one or more inventory identifier corresponding to the first one or more object respectively;
• generating, by the computer processor of the IIS and using a first machine learning algorithm, the category model based on a first learning data set comprising a first plurality of training objects corresponding to a plurality of categories;
• and
• generating, by the computer processor of the IIS and using a second machine learning algorithm, an object model based on a second learning data set comprising a second plurality of training objects corresponding to a category of a plurality of objects.

An interactive information system (IIS), comprising:

• an imaging device;
• a computer processor;
• and
• memory storing instructions, when executed by the computer processor, comprising functionalities for:
• capturing, using the imaging device of the ITS, a first image of a first representative object, wherein the first representative object represents a first one or more object disposed about the ITS;
• analyzing, based on a category model, the first image to determine a first representative category of the first one or more object, wherein the first representative category represents a first one or more category;
• retrieving, based on the first representative category, a first representative object model of a plurality of object models that are stored on a remote server, wherein the first representative object model represents a first one or more object model;
• analyzing, based on the first representative object model, the first image to determine a first representative inventory identifier of the first representative object, wherein the first representative inventory identifier represents a first one or more inventory identifier corresponding to the first one or more object respectively;
• generating, using a first machine learning algorithm, the category model based on a first learning data set comprising a first plurality of training objects corresponding to a plurality of categories;
• and
• generating, using a second machine learning algorithm, an object model based on a second learning data set comprising a second plurality of training objects corresponding to a category of a plurality of objects.

A non-transitory computer readable medium (CRM) storing computer readable program code for object recognition at an interactive information system (IIS), wherein the computer readable program code, when executed by a computer processor of the ITS, comprises functionalities for:

• capturing, using an imaging device of the ITS, a first image of a first representative object, wherein the first representative object represents a first one or more objects disposed about the ITS;
• analyzing, based on a category model, the first image to determine a first representative category of the first one or more object, wherein the first representative category represents a first one or more category;
• retrieving, based on the first representative category, a first representative object model of a plurality of object models that are stored on a remote server, wherein the first representative object model represents a first one or more object model;
• analyzing, based on the first representative object model, the first image to determine a first representative inventory identifier of the first representative object, wherein the first representative inventory identifier represents a first one or more inventory identifier corresponding to the first one or more object respectively;
• generating, using a first machine learning algorithm, the category model based on a first learning data set comprising a first plurality of training objects corresponding to a plurality of categories;
• and
• generating, using a second machine learning algorithm, an object model based on a second learning data set comprising a second plurality of training objects corresponding to a category of a plurality of objects.