Loading patent…
AssigneeMicrosoft Technology Licensing, LLC
InventorsBharadwaj PUDIPEDDI, Marc TREMBLAY, Gautham POPURI, Layali RASHID, Tiyasa MITRA, Mohit MITTAL, Maral MESMAKHOSROSHAHI
Estimating the term…
Methods, systems, apparatuses, and computer program products are described herein that enable execution of a large AI model on a memory-constrained target device that is communicatively connected to a parameter server, which stores a master copy of the AI model. The AI model may be dissected into smaller portions (e.g., layers or sub-layers), and each portion may be executed as efficiently as possible on the target device. After execution of one portion of the AI model is finished, another portion of the AI model may be downloaded and executed at the target device. To improve efficiency, the input samples may be divided into microbatches, and a plurality of microbatches executing in sequential order may form a minibatch. The size of the group of microbatches or minibatch can be manually or automatically adjusted to reduce the communication overhead.
A method performed by a system that comprises a parameter server, the parameter server being communicatively connected to a group of target devices and including a data manager, a batch manager, a transmitter, and a weight updater, the group of target devices comprising one or more target devices, the group of target devices being configured to run an instance of an artificial intelligence (AI) model, at least one target device comprising an integrated circuit chip having an on-chip memory of a size less than an entirety of the AI model, the method comprising:
A parameter server communicatively coupled to a group of target devices comprising at least a first target device and a second target device, the group of target devices configured to run an instance of an artificial intelligence (AI) model, the parameter server configured to:
A system comprising:
Dependencies are inferred from the claim text.