Managing multimodal annotations of an image

A non-transitory computer readable medium for storing a computer executable program for managing multimodal annotations of images, the computer readable medium comprising:

• computer readable code for establishing a first communication channel between a first machine and a mobile device;
• computer readable code for receiving a first hierarchical data structure based on an image captured by the mobile device at the first machine, the image being annotated with at least one multimodal annotation, wherein the first hierarchical data structure comprises the image designated as a base image, the base image comprising the at least one multimodal annotation being represented by at least one multimodal annotation object, and a base image of the at least one multimodal object comprising any additional multimodal annotations being represented as additional multimodal objects;
• computer readable code for rendering a representation of the first hierarchical data structure at the first machine;
• computer readable code for receiving at least one multimodal annotation modification associated with the image at the first machine;
• computer readable code for incorporating the at least one multimodal annotation modification into the first hierarchical data structure thereby generating a second hierarchical data structure;
• computer readable code for establishing a second communication channel between the first machine and the mobile device;
• and
• computer readable code for transmitting the second hierarchical data structure from the first machine to the mobile device via the second communication channel.

A method of managing multimodal annotations of images, the method comprising:

• receiving an image captured by a mobile device;
• receiving at least one multimodal annotation of the image captured by the mobile device;
• generating a first hierarchical data structure including the image and the at least one multimodal annotation at the mobile device;
• transmitting the first hierarchical data structure from the mobile device to a first machine;
• receiving a second hierarchical data structure including the image and at least one multimodal annotation of the image from the first machine at the mobile device, the second hierarchical data having been generated by the incorporation of at least one multimodal annotation modification received at the first machine into the first hierarchical data structure, wherein the first hierarchical data structure includes a non-annotation object associated with the image, wherein the non-annotation object includes a coupon.