# US 2024/0394059 A1 · Systems and methods for balancing computing resources

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0394059 A1
- Document ID: US20240394059A1
- Status: Published application
- Kind: A1
- Application number: 18/368,972
- Filed: 2023-09-15
- Published: 2024-11-28
- Earliest claimed: 2023-05-25
- Assignees: Huawei Technologies Co., Ltd.
- Applicants: Huawei Technologies Co., Ltd.
- Inventors: Henry Fangli KAO; José Nelson AMARAL
- Web page: https://patentgrep.com/patent/US20240394059A1

## CPC

- G06F 9/30065: {Loop control instructions; iterative instructions, e.g. LOOP, REPEAT}
- G06F 9/325: {for loops, e.g. loop detection or loop counter}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Methods and systems for balancing computing resources is described. According to an aspect, a method for balancing computing resources may be provided. The method includes generating multiple vectorized loops of a scalar code. The method may further include interleaving each of the multiple vectorized loops to generate multiple interleaved vectorized loops. The interleaving may be based on available vector resources. The method may further include interpolating one or more scalar iterations of the scalar loop into each of the multiple interleaved vectorized loops to generate multiple interleaved and scalar interpolated vectorized loops. The method may further include selecting one of the multiple interleaved and scalar interpolated vectorized loops based on a cost mode.

## Claims

### Claim 1

A method comprising:
generating multiple vectorized loops of a scalar code;
interpolating one or more scalar iterations of the scalar loop into each of the multiple vectorized loops to generate multiple scalar interpolated vectorized loops; and
selecting one of the multiple scalar interpolated vectorized loops based on a cost model.

### Claim 2 (depends on 1)

The method of claim 1 further comprising, determining that the scalar loop is legal to vectorize.

### Claim 3 (depends on 1)

The method of claim 1 further comprising, determining that scalar interpolation is legal for each of the multiple vectorized loops.

### Claim 4 (depends on 1)

The method of claim 1 further comprising:
for each of the multiple vectorized loops, determining a number of scalar iterations to interpolate based on the available scalar resources, wherein the one or more scalar iterations interpolated into each of the multiple vectorized loops is based on the determined number of scalar iterations.

### Claim 5 (depends on 1)

The method of claim 1 to wherein the cost model is based on one or more of:
heuristics calculated based on number of instructions in each of the multiple interleaved and scalar interpolated vectorized loops;
latency of instructions in each of the multiple interleaved and scalar interpolated vectorized loops; and
instruction-Level Parallelism (ILP) present in each of the multiple interleaved and scalar interpolated vectorized loops.

### Claim 6 (depends on 1)

The method of claim 1 further comprising:
interleaving one or more of the multiple vectorized loops to generate one or more interleaved vectorized loops, the interleaving being based on available vector resources, wherein:
interpolating one or more scalar iterations of the scalar loop into each of the multiple vectorized loops to generate multiple scalar interpolated vectorized loops further comprises: interpolating one or more scalar iterations of the scalar loop into each of the one or more interleaved vectorized loops to generate one or more interleaved and scalar interpolated vectorized loops; and
selecting one of the multiple scalar interpolated vectorized loops based on a cost model comprises selecting, based on the cost model, one of: the multiple scalar interpolated vectorized loops and the one or more interleaved and scalar interpolated vectorized loops.

### Claim 7

A method comprising:
obtaining a vectorized loop from a program; and
interpolating one or more scalar iterations of the vectorized loop into the vectorized loop to generate a scalar interpolated vectorized loop.

### Claim 8 (depends on 7)

The method of claim 7, wherein obtaining the vectorized loop from a program comprises vectorizing the program.

### Claim 9 (depends on 7)

The method of claim 7, wherein the program is a pre-compiled program.

### Claim 10 (depends on 7)

The method of claim 7 further comprising:
determining that scalar interpolation is legal for the vectorized loop.

### Claim 11 (depends on 7)

The method of claim 7 further comprising:
obtaining runtime data of the vectorized loop.

### Claim 12 (depends on 11)

The method of claim 11 further comprising: determining, based on the obtained runtime data, that scalar interpolation is beneficial to the vectorized loop in terms of resource utilization, wherein the obtained runtime data is performance data indicating resource utilization.

### Claim 13 (depends on 11)

The method of claim 11 wherein interpolating one or more scalar iterations of the vectorized loop into the vectorized loop comprises interpolating based on the obtained runtime data.

### Claim 14 (depends on 7)

The method of claim 7 wherein interpolating one or more scalar iterations of the vectorized loop into the vectorized loop comprises:
generating one or more equivalent scalar iterations of the vectorized loop; and
interpolating the one or more of the generated equivalent scalar iterations into the vectorized loop.

### Claim 15 (depends on 13)

The method of claim 13 further comprising, determining the available scalar resources for scalar interpolation.

### Claim 16 (depends on 15)

The method of claim 15 further comprising:
determining a number of scalar iterations to interpolate into the vectorized loop based on one or more of: the obtained runtime data and the available scalar resources, wherein the one or more scalar iterations interpolated into the vectorized loop is based on the determined number of scalar iterations.

### Claim 17 (depends on 7)

The method of claim 7 further comprising:
scheduling, according to an order of performance based on execution time, vector instructions and scalar instructions in the scalar interpolated vectorized loop to generate a scheduled scalar interpolated vectorized loop.

### Claim 18 (depends on 7)

The method of claim 7 further comprising:
unrolling one or more iterations of the vectorized loop to generate an unrolled and scalar interpolated vectorized loop.

### Claim 19 (depends on 18)

The method of claim 18 further comprising:
determining a number of iterations of the vectorized loop to unroll based on one or more of: the obtained runtime data, the available scalar resources and available vector resources, wherein the unrolled one or more iterations of the vectorized loop is based on the determined number of iterations of the vectorized loop to unroll.

### Claim 20 (depends on 18)

The method of claim 18 further comprising: scheduling, according to an order of performance based on execution time, vector instructions and scalar instructions in the unrolled and scalar interpolated vectorized loop to generate a scheduled unrolled and scalar interpolated vectorized loop.

### Claim 21 (depends on 12)

The method of claim 12 wherein determining, based on the obtained runtime data, that scalar interpolation is beneficial to the vectorized loop in terms of resource utilization comprises: determining, via a first machine-learned model, that scalar interpolation is beneficial to the vectorized loop in terms of resource utilization.

### Claim 22 (depends on 21)

The method of claims 21 wherein determining the number of scalar iterations to interpolate into the vectorized loop comprises: determining, via a second machine-learned model, the number of scalar iterations.

### Claim 23 (depends on 22)

The method of claim 22, wherein determining a number of iterations of the vectorized loop to unroll comprises determining, via a third machine-learned model, the number of iterations of the vectorized loop to unroll, wherein the third machine-learned model is different from or same as the second machine-learned model.

### Claim 24 (depends on 23)

The method of claim 23 wherein scheduling, according to an order, vector instructions and scalar instructions comprises scheduling, via a fourth machine-learned model, the vector instructions and scalar instructions, wherein the fourth machine-learned model is different from or same as the third machine-learned model.

### Claim 25

A method comprising:
generating multiple versions of a vectorized loop;
interpolating one or more scalar iterations of the vectorized loop into each of the multiple versions of the vectorized loop to generate multiple scalar interpolated vectorized loops; and
selecting one version of the vectorized loop from: the vectorized loop and the multiple scalar interpolated vectorized loops.

### Claim 26 (depends on 25)

The method of claim 25 further comprising:
for k iterations of the vectorized loop, executing:
the vectorized loop; and
each of the multiple scalar interpolated vectorized loops, wherein the vectorized loop terminates at nth iteration and n>k.

### Claim 27 (depends on 25)

The method of claim 25 further comprising:
measuring execution time, based on the k iterations, for each of: the vectorized loop, and multiple scalar interpolated vectorized loops.

### Claim 28 (depends on 27)

The method of claim 27, wherein selecting one version of the vectorized loop is based on the measured execution time.

### Claim 29 (depends on 25)

The method of claim 25 further comprising:
obtaining runtime data of the vectorized loop; and
determining that scalar interpolation is legal for the vectorized loop based on the obtained runtime data.

### Claim 30 (depends on 1)

An apparatus comprising:
at least one processor; and
at least one machine-readable medium storing executable instructions which when executed by the at least one processor configure the apparatus to perform the method as described in claim 1.

### Claim 31 (depends on 1)

A computer device comprising a non-transitory computer readable medium having instructions stored thereon which, when executed by a computer processor, causes the computer to perform the method as described in claim 1.
