Electronic device and operating method with model co-location
AssigneesSamsung Electronics Co., Ltd., Korea Advanced Institute of Science and Technology
InventorsMinsoo Rhu, Yujeong Choi
Life of this patent
Estimating the term…
Abstract
An electronic device to co-locating models and a method of operating the electronic device is provided. The electronic device includes one or more of processors configured to analyze computational characteristics in response to a plurality of models being located to an accelerator, determine an affinity representing a utilization of the accelerator in response to two models among the plurality of models being co-located based on the computational characteristics of the plurality of models, and co-locate the two models among the plurality of models to the accelerator based on the affinity.
Claims15
An electronic device, comprising:
- one or more of processors, configured to:
- analyze computational characteristics in response to a plurality of models being located to an accelerator, the computational characteristics including bandwidths of the plurality of models and a bandwidth of the accelerator;
- determine an affinity representing a utilization of the accelerator in response to two models among the plurality of models being co-located to the accelerator, the determining being based on a sum of bandwidths of the two models among the plurality of models in relation to the bandwidth of the accelerator to determine a maximum bandwidth coefficient reflected in the affinity in response to the sum of the bandwidths of the two models among the plurality of models being equal to, or less than, the bandwidth of the accelerator;
- and
- co-locate the two models among the plurality of models to the accelerator based on the affinity at a same time to share memory resources of the accelerator.
A processor-implemented method, the method comprising:
- analyzing computational characteristics in response to a plurality of models being located to an accelerator, the computational characteristics including performances of the plurality of models and a bandwidth of the accelerator;
- determining an affinity representing a utilization of the accelerator in response to two models among the plurality of models being co-located to the accelerator based on a sum of performances of two models among the plurality of models in response to an intermediate buffer of the accelerator being distributed to the two models among the plurality of models to determine an average of a sum of maximum performances of the two models as an intermediate buffer coefficient reflected in the affinity in response to the intermediate buffer of the accelerator being distributed to the two models among the pluralily of models;
- and
- co-locating the two models among the plurality of models to the accelerator based on the affinity at a same time to share memory resources of the accelerator.
Dependencies are inferred from the claim text.