Application programming interface for spinning up machine learning inferencing server on demand

A non-transitory machine-readable storage medium that provides instructions that, if executed by one or more processors of one or more electronic devices, are configurable to cause said one or more electronic devices to perform operations for spinning up a scoring container on demand, the operations comprising:

• receiving, from an orchestrator component via an application programming interface (API), a request to spin up the scoring container, wherein the scoring container is configured to provide scoring functionality;
• spinning up the scoring container responsive to receiving the request to spin up the scoring container;
• and
• providing, to the orchestrator component via the API, a response to the request to spin up the scoring container, wherein the response includes a job identifier (ID) of the scoring container and a uniform resource locator (URL) to use to submit scoring requests to the scoring container, wherein the URL includes the job ID.

An apparatus comprising:

• one or more processors;
• and
• a non-transitory machine-readable storage medium that provides instructions that, if executed by the one or more processors, are configurable to cause the apparatus to perform operations for spinning up a scoring container on demand, the operations comprising:
• receiving, from an orchestrator component via an application programming interface (API), a request to spin up the scoring container, wherein the scoring container is configured to provide scoring functionality;
• spinning up the scoring container responsive to receiving the request to spin up the scoring container;
• and
• providing, to the orchestrator component via the API, a response to the request to spin up the scoring container, wherein the response includes a job identifier (ID) of the scoring container and a uniform resource locator (URL) to use to submit scoring requests to the scoring container, wherein the URL includes the job ID.