# US 11,308,396 B2 · Neural network layer-by-layer debugging > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,308,396 B2 - Document ID: US11308396B2 - Status: Granted - Kind: B2 - Application number: 16/455,329 - Filed: 2019-06-27 - Published: 2020-12-31 - Granted: 2022-04-19 - Earliest claimed: 2019-06-27 - Assignees: Amazon Technologies, Inc. - Applicants: Amazon Technologies, Inc. - Inventors: Jindrich Zejda; Jeffrey T. Huynh; Drazen Borkovic; Se jong Oh; Ron Diamant; Randy Renfu Huang - Web page: https://patentgrep.com/patent/US11308396B2 ## CPC - G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections - G06N 3/105: {Shells for specifying net layout} - G06F 9/3861: {Recovery, e.g. branch miss-prediction, exception handling (error detection or correction G06F11/00)} - G06F 11/3636: {by tracing the execution of the program} - G06N 3/048: Activation functions - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Techniques are disclosed for debugging a neural network execution on a target processor. A reference processor may generate a plurality of first reference tensors for the neural network. The neural network may be repeatedly reduced to produce a plurality of lengths. For each of the lengths, a compiler converts the neural network into first machine instructions, the target processor executes the first machine instructions to generate a first device tensor, and the debugger program determines whether the first device tensor matches a first reference tensor. A shortest length is identified for which the first device tensor does not match the first reference tensor. Tensor output is enabled for a lower-level intermediate representation of the shortest neural network, and the neural network is converted into second machine instructions, which are executed by the target processor to generate a second device tensor. ## Claims ### Claim 1 A method of debugging a neural network execution on a target processor, the method comprising: receiving, by a debugger program operating on a host system, a request to debug an execution of a neural network on the target processor, the neural network comprising a plurality of layers; generating, using a reference processor on the host system and based on a first sample input, a plurality of first reference tensors for the neural network; repeatedly reducing the plurality of layers of the neural network to produce a plurality of reduced lengths, and for each particular reduced length of the plurality of reduced lengths: converting, by a compiler operating on the host system, the neural network having the particular reduced length into first machine instructions; executing, using the target processor and based on the first sample input or on one of the plurality of first reference tensors, the first machine instructions to generate a first device tensor; and determining, by the debugger program, whether the first device tensor matches a first reference tensor of the plurality of first reference tensors; identifying a shortest length of the plurality of reduced lengths for which the first device tensor does not match the first reference tensor; generating, using the reference processor and based on a second sample input, a plurality of second reference tensors for a representation of the neural network having the shortest length; converting, by the compiler, the neural network having the shortest length into second machine instructions, wherein the second machine instructions includes additional instructions that enable tensor output for the representation; executing, using the target processor and based on the second sample input or on one of the plurality of second reference tensors, the second machine instructions to generate a second device tensor for the representation; and determining, by the debugger program, whether the second device tensor matches a second reference tensor of the plurality of second reference tensors. ### Claim 2 (depends on 1) The method of claim 1, wherein the additional instructions enable tensor output for multiple representations of the neural network. ### Claim 3 (depends on 2) The method of claim 2, wherein executing the second machine instructions further generates a third device tensor for a second representation of the neural network, wherein the representation is a first representation. ### Claim 4 (depends on 3) The method of claim 3, further comprising: determining, by the debugger program, whether the third device tensor matches a third reference tensor of the plurality of second reference tensors. ### Claim 5 (depends on 1) The method of claim 1, wherein the plurality of first reference tensors and the plurality of second reference tensors are generated by the debugger program. ### Claim 6 A method of debugging a neural network execution on a target processor, the method comprising: receiving a plurality of first reference tensors for a neural network; repeatedly reducing a plurality of layers of the neural network to produce a plurality of reduced lengths, and for each particular reduced length of the plurality of reduced lengths: converting, by a compiler, the neural network having the particular reduced length into first machine instructions; executing, using the target processor, the first machine instructions to generate a first device tensor; and determining whether the first device tensor matches a first reference tensor of the plurality of first reference tensors; identifying a shortened length of the plurality of reduced lengths for which the first device tensor does not match the first reference tensor; generating a plurality of second reference tensors for a representation of the neural network having the shortened length; converting, by the compiler, the neural network having the shortened length into second machine instructions; and executing, using the target processor, the second machine instructions to generate a second device tensor for the representation. ### Claim 7 (depends on 6) The method of claim 6, wherein the shortened length is a shortest length of the plurality of reduced lengths. ### Claim 8 (depends on 6) The method of claim 6, further comprising: determining whether the second device tensor matches a second reference tensor of the plurality of second reference tensors. ### Claim 9 (depends on 6) The method of claim 6, wherein the second machine instructions include additional instructions that enable tensor output for the representation. ### Claim 10 (depends on 9) The method of claim 9, wherein the additional instructions enable tensor output for multiple representations of the neural network. ### Claim 11 (depends on 10) The method of claim 10, wherein executing the second machine instructions further generates a third device tensor for a second representation of the neural network, wherein the representation is a first representation. ### Claim 12 (depends on 11) The method of claim 11, further comprising: determining whether the third device tensor matches a third reference tensor of the plurality of second reference tensors. ### Claim 13 (depends on 6) The method of claim 6, wherein the plurality of first reference tensors and the plurality of second reference tensors are generated by a debugger program. ### Claim 14 A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform operations including: receiving a plurality of first reference tensors for a neural network; repeatedly reducing a plurality of layers of the neural network to produce a plurality of reduced lengths, and for each particular reduced length of the plurality of reduced lengths: converting, by a compiler, the neural network having the particular reduced length into first machine instructions; executing, using a target processor, the first machine instructions to generate a first device tensor; and determining whether the first device tensor matches a first reference tensor of the plurality of first reference tensors; identifying a shortened length of the plurality of reduced lengths for which the first device tensor does not match the first reference tensor; generating a plurality of second reference tensors for a representation of the neural network having the shortened length; converting, by the compiler, the neural network having the shortened length into second machine instructions; and executing, using a target processor, the second machine instructions to generate a second device tensor for the representation. ### Claim 15 (depends on 14) The non-transitory computer-readable medium of claim 14, wherein the shortened length is a shortest length of the plurality of reduced lengths. ### Claim 16 (depends on 14) The non-transitory computer-readable medium of claim 14, wherein the operations further comprise: determining whether the second device tensor matches a second reference tensor of the plurality of second reference tensors. ### Claim 17 (depends on 14) The non-transitory computer-readable medium of claim 14, wherein the second machine instructions include additional instructions that enable tensor output for the representation. ### Claim 18 (depends on 17) The non-transitory computer-readable medium of claim 17, wherein the additional instructions enable tensor output for multiple representations of the neural network. ### Claim 19 (depends on 18) The non-transitory computer-readable medium of claim 18, wherein executing the second machine instructions further generates a third device tensor for a second representation of the neural network, wherein the representation is a first representation. ### Claim 20 (depends on 19) The non-transitory computer-readable medium of claim 19, wherein the operations further comprise: determining whether the third device tensor matches a third reference tensor of the plurality of second reference tensors.