# US 10,372,456 B2 · Tensor processor instruction set architecture > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,372,456 B2 - Document ID: US10372456B2 - Status: Granted - Kind: B2 - Application number: 15/604,301 - Filed: 2017-05-24 - Published: 2018-11-29 - Granted: 2019-08-06 - Earliest claimed: 2017-05-24 - Assignees: Microsoft Technology Licensing, LLC - Applicants: Microsoft Technology Licensing, LLC - Inventors: Jeremy Halden Fowers; Kalin Ovtcharov; Steven Karl Reinhardt; Eric Sen Chung; Ming Gang Liu - Web page: https://patentgrep.com/patent/US10372456B2 ## CPC - G06F 9/3016: {Decoding the operand specifier, e.g. specifier format} - G06F 9/30036: {Instructions to perform operations on packed data, e.g. vector, tile or matrix operations} - G06F 9/30076: {to perform miscellaneous control operations, e.g. NOP} - G06F 9/30098: {Register arrangements} - G06F 9/3012: {Organisation of register space, e.g. banked or distributed register file} - G06F 9/30123: {according to context, e.g. thread buffers} - G06F 9/30163: {with implied specifier, e.g. top of stack} - G06F 9/3828: {with global bypass, e.g. between pipelines, between clusters} - G06F 9/3836: {Instruction issuing, e.g. dynamic instruction scheduling or out of order instruction execution} - G06F 9/3893: {controlled in tandem, e.g. multiplier-accumulator} - G06F 17/16: Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)} - G06N 3/063: using electronic means - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract A hardware accelerator having an efficient instruction set is disclosed. An apparatus may comprise logic configured to access a first and a second machine instruction. The second machine instruction may be missing a tensor operand needed to execute the second machine instruction. The logic may be further configured to execute the first machine instruction, resulting in a tensor. The logic may be further configured to execute the second machine instruction using the resultant tensor as the missing tensor operand. ## Claims ### Claim 1 An apparatus comprising: a first tensor operation calculator that is configured to execute a first tensor operation; a second, different tensor operation calculator that is configured to execute a second tensor operation differing from the first tensor operation; and logic configured to: access a first machine instruction in a set of machine instructions; access a second machine instruction in the set of machine instructions, wherein the second machine instruction is missing a tensor operand needed to execute the second machine instruction; execute the first machine instruction in the first tensor operation calculator, resulting in a tensor; route, based on the second machine instruction having the missing tensor operand, the resultant tensor from the first tensor operation calculator to the second tensor operation calculator for use as the missing tensor operand; and execute the second machine instruction using the resultant tensor as the missing tensor operand. ### Claim 2 (depends on 1) The apparatus of claim 1, wherein the logic is configured to use a resultant tensor of executing the second machine instruction to execute a third machine instruction immediately following the second machine instruction in response to the third machine instruction missing a tensor operand needed to execute the third machine instruction. ### Claim 3 (depends on 1) The apparatus of claim 1, wherein the second machine instruction is for a tensor operation between at least a first tensor and a second tensor, wherein the second machine instruction at least partially specifies the second tensor but does not specify the first tensor; wherein the logic is configured to route the resultant tensor from executing the first machine instruction in the first tensor operation calculator to the second tensor operation calculator for use as the first tensor and to use the tensor at least partially specified in the second machine instruction as the second tensor. ### Claim 4 (depends on 1) The apparatus of claim 1, wherein the logic is further configured to: access a third machine instruction in the set of machine instructions, wherein the third machine instruction is at an end of the set and instructs to store a tensor to memory but does not specify the tensor; and store to memory a final tensor, resulting from the executing the second machine instruction, as the unspecified tensor. ### Claim 5 (depends on 1) The apparatus of claim 1, wherein the logic is further configured to: access a third machine instruction, the third machine instruction being for a first type of tensor operation, of a plurality of types of tensor operations, wherein the third machine instruction does not specify a first tensor operand and only partially specifies a second tensor; and access a tensor register file that is implied by the first type of tensor operation in order to access the second tensor operand. ### Claim 6 (depends on 1) The apparatus of claim 1, wherein the logic is further configured to: decode the second machine instruction to determine a type of tensor operation instructed by the second machine instruction and an index; select the second tensor operation calculator to perform the type of tensor operation instructed by the second machine instruction; and based on the index, provide a tensor from a tensor register file that is dedicated to the second tensor operation calculator as a tensor input for executing the second machine instruction. ### Claim 7 (depends on 1) The apparatus of claim 1, wherein the logic is configured to: alter a native size of tensor operations executed in the apparatus based on a tiling factor associated with the set of machine instructions. ### Claim 8 A method of executing machine instructions in a tensor processor, comprising: accessing a first machine instruction in a chain of machine instructions; accessing a second machine instruction in the chain of machine instructions, wherein the second machine instruction is missing a tensor operand needed to execute the second machine instruction; executing the first machine instruction by a first tensor operation calculator in the tensor processor to result in a resultant tensor routing the resultant tensor from the first tensor operation calculator to a second tensor operation calculator in the tensor processor for use as the missing tensor operand without intermediate storage of the resultant tensor; and executing the second machine instruction by the second tensor operation calculator in the tensor processor using the resultant tensor as the missing tensor operand. ### Claim 9 (depends on 8) The method of claim 8, further comprising using logic in the tensor processor, to select, by default, a resultant tensor, from executing one machine instruction in the chain, as an input tensor to execute the next machine instruction in the chain. ### Claim 10 (depends on 8) The method of claim 8, further comprising: executing the second machine instruction by the second tensor operation calculator using a tensor at least partially specified in the second machine instruction as a first tensor and the resultant tensor as a second tensor in a tensor operation. ### Claim 11 (depends on 8) The method of claim 8, wherein executing the first machine instruction by the first tensor operation calculator in the tensor processor to result in the resultant tensor comprises loading an initial tensor into a tensor register file in the tensor processor. ### Claim 12 (depends on 11) The method of claim 11, further comprising: routing the tensor from the tensor register file to the second tensor operation calculator in the tensor processor for use as the missing tensor operand without intermediate storage of the resultant tensor. ### Claim 13 (depends on 8) The method of claim 8, further comprising: storing a final tensor, resulting from the execution of the second machine instruction, to memory external to the tensor processor. ### Claim 14 (depends on 8) The method of claim 8, further comprising: altering a native size of tensor operations in the tensor processor based on a tiling factor specified in the chain of machine instructions. ### Claim 15 An apparatus comprising: a tensor register file; a tensor operation calculator that is configured to execute a tensor operation; and logic configured to: access a first machine instruction in a set of machine instructions, the first machine instruction being an instruction to load an initial tensor at a start of the set of machine instructions; access a second machine instruction in the set of machine instructions, wherein the second machine instruction is missing a tensor operand needed to execute the second machine instruction; execute the first machine instruction to load the initial tensor into the tensor register file; route the initial tensor from the tensor register file to the tensor operation calculator for use as the missing tensor operand without intermediate storage of the initial tensor; and execute the second machine instruction using the initial tensor as the missing tensor operand. ### Claim 16 (depends on 15) The apparatus of claim 15, wherein the logic is configured to use a resultant tensor of executing the second machine instruction to execute a third machine instruction immediately following the second machine instruction in response to the third machine instruction missing a tensor operand needed to execute the third machine instruction. ### Claim 17 (depends on 15) The apparatus of claim 15, wherein the logic is further configured to: access a third machine instruction in the set of machine instructions, wherein the third machine instruction is at an end of the set and instructs to store a tensor to memory but does not specify the tensor; and store to memory a final tensor, resulting from the executing the second machine instruction, as the unspecified tensor. ### Claim 18 (depends on 15) The apparatus of claim 15, wherein the logic is further configured to: access a third machine instruction, the third machine instruction being for a first type of tensor operation, of a plurality of types of tensor operations, wherein the third machine instruction does not specify a first tensor operand and only partially specifies a second tensor; and access a tensor register file that is implied by the first type of tensor operation in order to access the second tensor operand. ### Claim 19 (depends on 15) The apparatus of claim 15, wherein the logic is further configured to: decode the second machine instruction to determine a type of tensor operation instructed by the second machine instruction and an index; select the second tensor operation calculator to perform the type of tensor operation instructed by the second machine instruction; and based on the index, provide a tensor from a tensor register file that is dedicated to the second tensor operation calculator as a tensor input for executing the second machine instruction. ### Claim 20 (depends on 15) The apparatus of claim 15, wherein the logic is configured to: alter a native size of tensor operations executed in the apparatus based on a tiling factor associated with the set of machine instructions.