# US 12,602,257 B2 · Electronic device and operating method with model co-location > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,602,257 B2 - Document ID: US12602257B2 - Status: Granted - Kind: B2 - Application number: 17/736,312 - Filed: 2022-05-04 - Published: 2023-03-30 - Granted: 2026-04-14 - Earliest claimed: 2021-09-24 - Assignees: Samsung Electronics Co., Ltd.; Korea Advanced Institute of Science and Technology - Applicants: Samsung Electronics Co., Ltd.; Korea Advanced Institute of Science and Technology - Inventors: Minsoo Rhu; Yujeong Choi - Web page: https://patentgrep.com/patent/US12602257B2 ## CPC - G06F 9/5033: {considering data affinity} - G06F 9/5044: {considering hardware capabilities} - G06F 9/5016: {the resource being the memory} - G06N 3/063: using electronic means - G06F 2209/5019: Workload prediction - G06F 2209/503: Resource availability - G06N 3/044: Recurrent networks, e.g. Hopfield networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract An electronic device to co-locating models and a method of operating the electronic device is provided. The electronic device includes one or more of processors configured to analyze computational characteristics in response to a plurality of models being located to an accelerator, determine an affinity representing a utilization of the accelerator in response to two models among the plurality of models being co-located based on the computational characteristics of the plurality of models, and co-locate the two models among the plurality of models to the accelerator based on the affinity. ## Claims ### Claim 1 An electronic device, comprising: one or more of processors, configured to: analyze computational characteristics in response to a plurality of models being located to an accelerator, the computational characteristics including bandwidths of the plurality of models and a bandwidth of the accelerator; determine an affinity representing a utilization of the accelerator in response to two models among the plurality of models being co-located to the accelerator, the determining being based on a sum of bandwidths of the two models among the plurality of models in relation to the bandwidth of the accelerator to determine a maximum bandwidth coefficient reflected in the affinity in response to the sum of the bandwidths of the two models among the plurality of models being equal to, or less than, the bandwidth of the accelerator; and co-locate the two models among the plurality of models to the accelerator based on the affinity at a same time to share memory resources of the accelerator. ### Claim 2 (depends on 1) The electronic device of claim 1, wherein the one or more of processors is further configured to: analyze a memory capacity and a bandwidth needed by each of the plurality of models, and a performance of each of the plurality of models based on a size of an allocated intermediate buffer as the computational characteristics. ### Claim 3 (depends on 1) The electronic device of claim 1, wherein the one or more of processors is further configured to: determine the affinity based on a determination that a sum of memory capacities of the two models among the plurality of models is greater than a memory capacity of the accelerator. ### Claim 4 (depends on 1) The electronic device of claim 1, wherein the one or more of processors is further configured to: determine a result of dividing the bandwidth of the accelerator by the sum of the bandwidths of the two models among the plurality of models as a bandwidth coefficient reflected in the affinity in response to the sum of the bandwidths of the two models being greater than the bandwidth of the accelerator. ### Claim 5 (depends on 1) The electronic device of claim 1, wherein the one or more of processors is further configured to: determine the affinity based on a sum of performances of the two models in response to an intermediate buffer of the accelerator being distributed to the two models among the plurality of models. ### Claim 6 (depends on 5) The electronic device of claim 5, wherein the one or more of processors is further configured to: determine an average of a sum of maximum performances of the two models among the plurality of models as an intermediate buffer coefficient reflected in the affinity in response to the intermediate buffer of the accelerator being distributed to the two models among the plurality of models. ### Claim 7 (depends on 1) The electronic device of claim 1, wherein the one or more of processors is further configured to: co-locate the two models among the plurality of models to the accelerator in response to the affinity being equal to, or greater than, a preset threshold. ### Claim 8 (depends on 1) The electronic device of claim 1, wherein the one or more of processors is further configured to: co-locate a first model in which a memory resource bottleneck occurs, and a second model in which the affinity with the first model is equal to, or greater than, a preset threshold among the plurality of models. ### Claim 9 A processor-implemented method, the method comprising: analyzing computational characteristics in response to a plurality of models being located to an accelerator, the computational characteristics including performances of the plurality of models and a bandwidth of the accelerator; determining an affinity representing a utilization of the accelerator in response to two models among the plurality of models being co-located to the accelerator based on a sum of performances of two models among the plurality of models in response to an intermediate buffer of the accelerator being distributed to the two models among the plurality of models to determine an average of a sum of maximum performances of the two models as an intermediate buffer coefficient reflected in the affinity in response to the intermediate buffer of the accelerator being distributed to the two models among the pluralily of models; and co-locating the two models among the plurality of models to the accelerator based on the affinity at a same time to share memory resources of the accelerator. ### Claim 10 (depends on 9) The method of claim 9, wherein the analyzing of the computational characteristics comprises: analyzing a memory capacity and a bandwidth needed by each of the plurality of models, and a performance of each of the plurality of models based on a size of an allocated intermediate buffer as the computational characteristics. ### Claim 11 (depends on 9) The method of claim 9, wherein the determining of the affinity comprises: determining the affinity based on a determination that a sum of memory capacities of the two models among the plurality of models is greater than a memory capacity of the accelerator. ### Claim 12 (depends on 9) The method of claim 9, wherein the determining of the affinity comprises: determining a result of dividing the bandwidth of the accelerator by the sum of the bandwidths of the two models among the plurality of models as a bandwidth coefficient reflected in the affinity in response to the sum of the bandwidths of the two models being greater than the bandwidth of the accelerator. ### Claim 13 (depends on 9) The method of claim 9, wherein the determining of the affinity comprises: determining the bandwidth coefficient reflected in the affinity in response to the sum of the bandwidths of the two models among the plurality of models being equal to, or less than, the bandwidth of the accelerator. ### Claim 14 (depends on 9) The method of claim 9, wherein the co-locating comprises: co-locating a first model in which a memory resource bottleneck occurs and a second model in which an affinity with the first model is equal to, or greater than, a preset threshold among the plurality of models. ### Claim 15 (depends on 9) A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform the method of claim 9.