Multimedia system and method for AR/VR smart contact lens

A smart contact lens (SCL) based multimedia system comprising:

• a contact lens substrate;
• a multimedia processing device operatively connected to the smart contact lens and configured to receive and process multimedia data, wherein the multimedia processing device combines one or more of an audio stream, an image stream, a video stream, and a text stream to form a mixed multimedia dataset;
• an image capture device or a video capture device positioned to acquire visual data from a forward-facing field of view and configured to transmit said visual data to the multimedia processing device;
• a communication interface configured to exchange multimedia data with an external communication unit;
• and
• the external communication unit comprising a multimedia transceiver configured to transmit or receive image or video data.

A method of operating a smart contact lens-based multimedia system, the method comprising:

• presenting visual content to a user via a display component embedded in a contact lens substrate;
• emitting audio content via an audio output device;
• receiving or sending data wirelessly to and from an external communication unit via at least one communication unit comprising a data receiver or transceiver;
• and
• receiving, by a multimedia processor operatively connected to the embedded display component and the audio output device, a multimedia stream comprising at least video and audio content;
• demultiplexing, by the multimedia processor, the multimedia stream into separate video and audio streams;
• and
• directing the video stream to the embedded display component and the audio stream to the audio output device in temporal synchronization.

A method for operating a smart contact lens-based multimedia system, the method comprising:

• capturing, via an audio sensor, an audio stream;
• capturing, via an image or video capture sensor integrated into the smart contact lens, one or more of an image stream or a video stream;
• combining one or more of the audio stream, the image stream, the video stream, and a text stream to form a mixed multimedia dataset;
• and
• storing the mixed multimedia dataset in memory or transmitting the mixed multimedia dataset to an external communication unit.

A non-transitory machine-readable medium storing one or more sequences of instructions which, when executed by one or more multimedia processors operatively connected to a smart contact lens-based multimedia system, cause the system to:

• receive, from an external communication unit via a wireless interface, a multimedia data stream comprising at least video data and audio data;
• demultiplex the multimedia data stream into separate data-type-specific streams including a video stream and an audio stream;
• and
• route the video stream to a display component embedded in a contact lens substrate, and route the audio stream to an audio output device in temporal synchronization.

A computer-implemented method executed by one or more multimedia processors operatively connected to a smart contact lens-based multimedia system, the method comprising:

• receiving, from an external communication unit via a wireless interface, a multimedia data stream comprising at least two types selected from video data, audio data, and text data;
• demultiplexing the multimedia data stream into separate data-type-specific streams including a video stream and an audio stream;
• and
• routing the video stream to a display component embedded in a contact lens substrate, and routing the audio stream to an audio output device in temporal synchronization.