US 12,586,153 B2 · Methods of batch-based DNN processing for efficient analytics

Abstract

Technology is disclosed herein to execute an inference model by a processor which includes a reshape layer. In an implementation, the reshape layer of the inference model receives an output produced by a previous layer of the inference model and inserts padding into the output, then supplies the padded output as an input to a next layer of the inference model. In an implementation, the inference model includes a stitching layer at the beginning of the inference model and an un-stitch layer at the end of the model. The stitching layer of the inference model stitches together multiple input images into an image batch and supplies the image batch as an input to a subsequent layer. The un-stitch layer receives output from a penultimate layer of the inference model and unstitches the output to produce multiple output images corresponding to the multiple input images.

Claims20

Dependent claims

A method of executing an inference model by a processor, the method comprising:

by a stitching layer of the inference model:
receiving multiple images;
stitching together the multiple images to form a batch input;
and
supplying the batch input to a subsequent layer of the inference model;
and
by a reshape layer of the inference model:
receiving an output produced by a previous layer of the inference model, wherein the output comprises multiple feature maps corresponding to the multiple images;
inserting padding into the output, resulting in a padded output, including by inserting side padding between the multiple feature maps;
and
supplying the padded output as an input to a next layer of the inference model.

8 more elements

A computing apparatus comprising:

one or more computer-readable storage media;
one or more processors operatively coupled with the one or more computer-readable storage media;
and
program instructions stored on the one or more computer-readable storage media that, when executed by the one or more processors, direct the computing apparatus to at least:
execute an inference model, wherein the inference model includes a stitching layer and a reshape layer, and wherein to execute the inference model, the program instructions direct the computing apparatus to:
at the stitching layer of the inference model:
receive multiple images;
stitch together the multiple images to form a batch input;
and
supply the batch input to a subsequent layer of the inference model;
and
at the reshape layer of the inference model:
receive an output produced by a previous layer of the inference model, wherein the output comprises multiple feature maps corresponding to the multiple images;
insert padding into the output, resulting in a padded output, wherein to insert the padding into the output, the program instructions direct the computing apparatus to insert side padding between the multiple feature maps;
and
supply the padded output as an input to a next layer of the inference model.

13 more elements

A system comprising:

memory circuitry configured to store an inference model that includes a stitching layer and a reshape layer;
and
processing circuitry coupled with the memory circuitry and configured to execute the inference model;
wherein the stitching layer of the inference model, when executed by the processing circuitry, causes the processing circuitry to:
receive multiple images;
stitch together the multiple images to form a batch input;
and
supply the batch input to a subsequent layer of the inference model;
and
wherein the reshape layer of the inference model, when executed by the processing circuitry, causes the processing circuitry to:
receive an output produced by a previous layer of the inference model, wherein the output comprises multiple feature maps corresponding to the multiple images;
insert padding into the output, resulting in a padded output, wherein to insert the padding into the output, the reshape layer causes the processing circuitry to, when executed, insert side padding between the multiple feature maps;
and
supply the padded output as an input to a next layer of the inference model.

11 more elements

Dependencies are inferred from the claim text.

Methods of batch-based DNN processing for efficient analytics

Life of this patent

Methods of batch-based DNN processing for efficient analytics

Life of this patent

Abstract

Claims20