Methods and apparatus to optimize artificial intelligence inference workloads on audio and video data streams

A non-transitory machine readable storage medium to reduce a use of a compute engine executing dense layers in a model comprising machine readable instructions to cause programmable circuitry to at least:

• obtain a first vector generated by an initial layer of the model, the first vector corresponding to a first data frame, wherein the initial layer of the model utilizes less computation resources than the dense layers are to utilize;
• measure a similarity between the first vector and a prior vector, wherein the prior vector has been classified by the model and corresponds to second data frames occurring before the first data frame represented by the first vector;
• determine that the first vector satisfies a similarity threshold to the prior vector;
• and
• instruct the compute engine to enter an idle state, the idle state to suspend execution of the dense layers in the model.

A server to distribute first software instructions on a network to reduce a use of a compute engine executing dense layers in a model, the server comprising:

• at least one storage device including second instructions;
• and
• at least one processor to execute the second instructions to transmit the first software instructions over the network, the first software instructions, when executed, to cause at least one device to:
• obtain a first vector generated by an initial layer of the model, the first vector corresponding to a first data frame, wherein the initial layer of the model utilizes less computation resources than the dense layers are to utilize;
• measure a similarity between the first vector and a prior vector, wherein the prior vector has been classified by the model and corresponds to second data frames occurring before the first data frame represented by the first vector;
• determine that the first vector satisfies a similarity threshold to the prior vector;
• and
• instruct the compute engine to enter an idle state, the idle state to suspend execution of the dense layers in the model.