Automatic identification of food substance

A non-transitory computer readable medium comprising computer executable instructions which when executed by a computer cause the computer to perform a method of displaying information relating to an allergen present in a food substance together with a user profile, the method comprising:

• capturing an image of a scene;
• segmenting the image to determining at least a segmentation of the food substance in the image;
• determining a classification of the food substance using the segmentation;
• determining a presence of the allergen in the food substance using the classification of the food substance and a database of allergens and food substances;
• determining a risk to a user using the user profile specifying a user sensitivity to the allergen;
• and
• displaying an output including a view of the user profile and the image, wherein the image is augmented to identify the presence of the allergen.

A method of displaying information relating to an allergen present in a food substance together with a user profile, the method comprising:

• capturing, by a camera of a mobile device, an image of a scene;
• segmenting, by a processor of the mobile device, the image to determining at least a segmentation of the food substance in the image;
• determining, by the processor of the mobile device, a classification of the food substance using the segmentation;
• determining, by the processor of the mobile device, a presence of the allergen in the food substance using the classification of the food substance and a database of allergens and food substances;
• determining, by the processor of the mobile device, a risk to a user using the user profile specifying a user sensitivity to the allergen;
• and
• displaying, by a display of the mobile device, an output including a view of the user profile and the image, wherein the image is augmented to identify the presence of the allergen.