Method and system for performing generative artificial intelligence and fine tuning the data model

A system comprising:

• one or more processors;
• and
• a memory coupled to the one or more processors, the memory storing a plurality of instructions, executable by the one or more processors, which, when executed by the one or more processors cause the one or more processors to perform a set of operations comprising:
• receiving a request for allocating graphical processing unit (GPU) resource for performing an operation, wherein the request includes metadata identifying a client identifier (ID) associated with a client, a throughput and a latency of the operation;
• determining a resource limit for performing the operation based on the metadata;
• obtaining at least one attribute associated with each GPU resource of a plurality of GPU resources available for assignment in the system, wherein the at least one attribute indicates capacity of a corresponding GPU resource;
• analyzing the at least one attribute associated with each GPU resource with respect to the resource limit;
• identifying a set of GPU resources from the plurality of GPU resources based on the analysis;
• generating a dedicated AI cluster by patching the set of GPU resources within a single cluster, wherein the dedicated AI cluster reserves a portion of a computation capacity of a computing system for a period of time;
• and
• allocating the dedicated AI cluster to the client associated with the client ID.

A non-transitory computer-readable medium storing a plurality of instructions executable by one or more processors to cause the one or more processors to perform a set of operations comprising:

• receiving a request for allocating graphical processing unit (GPU) resource for performing an operation, wherein the request includes metadata identifying a client identifier (ID) associated with a client, a throughput and a latency of the operation;
• determining a resource limit for performing the operation based on the metadata;
• obtaining at least one attribute associated with each GPU resource of a plurality of GPU resources available for assignment in a computing system, wherein the at least one attribute indicates capacity of a corresponding GPU resource;
• analyzing the at least one attribute associated with each GPU resource with respect to the resource limit;
• identifying a set of GPU resources from the plurality of GPU resources based on the analysis;
• generating a dedicated AI cluster by patching the set of GPU resources within a single cluster, wherein the dedicated AI cluster reserves a portion of a computation capacity of a computing system for a period of time;
• and
• allocating the dedicated AI cluster to the client associated with the client ID.