Strategic Distribution of AI Inference Processing in Distributed Computing Environments

A method for performing generative AI inference computations in a distributed computing environment comprising the steps of:

• receiving from a user device a received message comprising a request to generate a generative AI model output response at a first computing device located at a first location and comprised by the distributed computing environment;
• selecting by the first computing device a second location based on one or more criteria;
• responsive to selecting the second location, at least one of:
• generating by the first computing device a second message comprising information from the received message;
• and
• generating by the first computing device the second message by:
• deriving derived information by the first computing device from the received message;
• and
• generating by the first computing device the second message comprising the derived information;
• and
• sending the second message by the first computing device to the second location;
• wherein the second message is provided as an input to at least one of a generative AI inference system or a portion of a generative AI inference system operating on a second computing device located at the second location and comprised by the distributed computing environment.

A method for performing generative AI inference computations in a distributed computing environment comprising the steps of:

• receiving a received message by a second computing device at a second location from a first computing device at a first location, the first and second computing devices comprised by the distributed computing environment, the received message being generated in response to a first computing system receiving from a user device a message comprising a request to generate a generative AI model output response;
• providing the received message as an input to at least one of a generative AI inference system or a portion of a generative AI inference system operating on the second computing device;
• generating a generated output by executing on the second computing device the at least one of the generative AI inference system or the portion of the generative AI inference system;
• and
• sending the generated output by the second computing device to the user device.