Inference processing system in which server and edge device cooperate to perform computation, server, edge device, and control method thereof, and storage medium

An inference processing system that performs, by a server and an edge device, computation of an inference model including an input layer, L intermediate layers, and an output layer,

• wherein the server comprises:
• a first estimation unit that sets a server-side inference model that is the same as the inference model;
• and
• a server communication unit capable of communicating data with the edge device, and
• the edge device comprises:
• a second estimation unit that sets an edge-side inference model including the input layer and first to n-th (L>n>1) intermediate layers among the L intermediate layers;
• an edge-side communication unit capable of communicating data with the server;
• and
• an image identification unit that detects specific information included in an input image input to an input layer of the edge-side inference model, and generates, according to the detected specific information, first layer position information indicating a position of a specific layer that processes the input image in the edge-side inference model, and second layer position information indicating a position of a layer that is deeper than the specific layer by i (≥1),
• wherein the edge device inputs the input image to an input layer of the edge-side inference model, and transmits, to the server using the edge-side communication unit, first intermediate data that is a computation result obtained by performing processing from the input layer to a layer indicated by the first layer position information, and second intermediate data that is a computation result obtained by performing processing from the input layer to a layer indicated by the second layer position information, and
• the server
• receives the first intermediate data and the second intermediate data using the server communication unit, inputs the first intermediate data to an intermediate layer at a layer position X (1≤X≤L−i) in the server-side inference model, and acquires third intermediate data that is a computation result of an intermediate layer at a layer position X+i−1 obtained by performing processing from an intermediate layer at the layer position X to the intermediate layer at the layer position X+i−1,
• when the third intermediate data and the second intermediate data match, inputs the second intermediate data to an intermediate data at a layer position X+i, and performs processing from the intermediate layer at the layer position X+i to the output layer, and
• when the third intermediate data and the second intermediate data do not match, repeats processing of acquiring the third intermediate data again after incrementing a value of X by 1 until the third intermediate data acquired again and the second intermediate data match.

A server that performs computation of an inference model including an input layer, L intermediate layers, and an output layer in cooperation with an edge device in which an edge-side inference model including the input layer and first to n-th (L>n>1) intermediate layers among the L intermediate layers is set, the server comprising:

• an estimation unit that sets a server-side inference model that is the same as the inference model;
• and
• a server communication unit capable of communicating data with the edge device,
• wherein, when the server receives, using the server communication unit, layer position information indicating a position of a last layer at which an input image was processed in the edge-side inference model, and layer data that is a computation result obtained when the input image is input to an input layer of the edge-side inference model and processing is performed from the input layer to a layer indicated by the layer position information, the server inputs the received layer data to a layer subsequent to a layer indicated by the received layer position information in the server-side inference model and performs processing up to the output layer.

An edge device that cooperates with a server to perform computation of an inference model including an input layer, L intermediate layers, and an output layer, the edge device comprising:

• an estimation unit that sets an edge-side inference model including the input layer and first to n-th intermediate layers among the L intermediate layers;
• an edge-side communication unit capable of communicating data with the server;
• and
• an image identification unit that detects specific information included in an input image input to an input layer of the edge-side inference model, and generates, according to the detected specific information, layer position information indicating a position of a last layer to process the input image in the edge-side inference model,
• wherein the edge device transmits, to the server using the edge-side communication unit, layer data that is a computation result obtained when the input image is input to an input layer of the edge-side inference model and processing is performed from the input layer to a layer indicated by the layer position information, and the layer position information.

A server that performs computation of an inference model including an input layer, L intermediate layers, and an output layer in cooperation with an edge device in which an edge-side inference model including the input layer and first to n-th (L>n>1) intermediate layers among the L intermediate layers is set, the server comprising:

• an estimation unit that sets a server-side inference model that is the same as the inference model;
• and
• a server communication unit capable of communicating data with the edge device,
• wherein, when the server receives, using the server communication unit, first intermediate data that is a computation result obtained when an input image is input to the input layer in the edge-side inference model and processing is performed from the input layer to a specific layer at which the input image was processed, and second intermediate data that is a computation result obtained when processing is performed from the input layer to a layer that is deeper than the specific layer by i (≥1), the server inputs the first intermediate data to an intermediate layer at a layer position X (1≤X≤L−i) in the server-side inference model, and acquires third intermediate data that is a computation result of an intermediate layer at a layer position X+i−1 obtained by performing processing from an intermediate layer at the layer position X to the intermediate layer at the layer position X+i−1,
• when the third intermediate data and the second intermediate data match, the server inputs the computation result to an intermediate data at a layer position X+i, and performs processing from the intermediate layer at the layer position X+i to the output layer, and
• when the third intermediate data and the second intermediate data do not match, the server repeats processing of acquiring the third intermediate data again after incrementing a value of X by 1 until the third intermediate data acquired again and the computation result match.

An edge device that cooperates with a server to perform computation of an inference model including an input layer, L intermediate layers, and an output layer, the edge device comprising:

• an estimation unit that sets an edge-side inference model including the input layer and first to n-th (L>n>1) intermediate layers among the L intermediate layers;
• an edge-side communication unit capable of communicating data with the server;
• and
• an image identification unit that detects specific information included in an input image input to an input layer of the edge-side inference model, and generates, according to the detected specific information, first layer position information indicating a position of a specific layer that processes the input image in the edge-side inference model, and second layer position information indicating a position of a layer that is deeper than the specific layer by i (≥1),
• wherein the edge device inputs the input image to an input layer of the edge-side inference model, and transmits, to the server using the edge-side communication unit, first intermediate data that is a computation result obtained by performing processing from the input layer to a layer indicated by the first layer position information, and second intermediate data that is a computation result obtained by performing processing from the input layer to a layer indicated by the second layer position information.

A control method for an inference processing system that performs, by a server and an edge device, computation of an inference model including an input layer, L intermediate layers, and an output layer,

• wherein the server comprises:
• a first estimation unit that sets a server-side inference model that is the same as the inference model;
• and
• a server communication unit capable of communicating data with the edge device, and
• the edge device comprises:
• a second estimation unit that sets an edge-side inference model including the input layer and first to n-th (L>n>1) intermediate layers among the L intermediate layers;
• an edge-side communication unit capable of communicating data with the server;
• and
• an image identification unit that detects specific information included in an input image input to an input layer of the edge-side inference model, and generates, according to the detected specific information, layer position information indicating a position of a last layer to process the input image in the edge-side inference model,
• the control method comprising:
• a first input step of inputting the input image to an input layer of the edge-side inference model;
• a transmitting step of transmitting, from the edge device to the server using the edge-side communication unit, layer data that is a computation result obtained by performing processing from the input layer to a layer indicated by the layer position information, and the layer position information;
• a receiving step of receiving the layer data and the layer position information at the server using the server communication unit;
• a second input step of inputting the received layer data to a layer subsequent to a layer indicated by the received layer position information in the server-side inference model;
• and
• a processing step of performing processing from the layer, where the received layer data is input in the second input step, to the output layer.

A control method for an inference processing system that performs, by a server and an edge device, computation of an inference model including an input layer, L intermediate layers, and an output layer,

• wherein the server comprises:
• a first estimation unit that sets a server-side inference model that is the same as the inference model;
• and
• a server communication unit capable of communicating data with the edge device, and
• the edge device comprises:
• a second estimation unit that sets an edge-side inference model including the input layer and first to n-th (L>n>1) intermediate layers among the L intermediate layers;
• an edge-side communication unit capable of communicating data with the server;
• and
• an image identification unit that detects specific information included in an input image input to an input layer of the edge-side inference model, and generates, according to the detected specific information, first layer position information indicating a position of a specific layer that processes the input image in the edge-side inference model, and second layer position information indicating a position of a layer that is deeper than the specific layer by i (≥1),
• the control method comprising:
• a first input step of inputting the input image to an input layer of the edge-side inference model;
• a transmitting step of transmitting, from the edge device to the server using the edge-side communication unit, first intermediate data that is a computation result obtained by performing processing from the input layer to a layer indicated by the first layer position information, and second intermediate data that is a computation result obtained by performing processing from the input layer to a layer indicated by the second layer position information;
• a receiving step of receiving the first intermediate data and the second intermediate data at the server using the server communication unit;
• a second input step of inputting the first intermediate data to an intermediate layer at a layer position X (1≤X≤L−i) in the server-side inference model;
• an acquiring step of acquiring third intermediate data that is a computation result of an intermediate layer at a layer position X+i−1 obtained by performing processing from an intermediate layer at the layer position X to the intermediate layer at the layer position X+i−1 in the server-side inference model;
• a first processing step of inputting the second intermediate data to an intermediate data at a layer position X+i in the server-side inference model when the third intermediate data and the second intermediate data match, and performing processing from the intermediate layer at the layer position X+i to the output layer;
• and
• a second processing step of repeating processing of acquiring the third intermediate data again after incrementing a value of X by 1 when the third intermediate data and the second intermediate data do not match, until the third intermediate data acquired again and the second intermediate data match.

A non-transitory computer-readable storage medium storing a program for causing a computer to execute a control method for an inference processing system, the inference processing system performing, by a server and an edge device, computation of an inference model including an input layer, L intermediate layers, and an output layer,

• wherein the server comprises:
• a first estimation unit that sets a server-side inference model that is the same as the inference model;
• and
• a server communication unit capable of communicating data with the edge device, and
• the edge device comprises:
• a second estimation unit that sets an edge-side inference model including the input layer and first to n-th (L>n>1) intermediate layers among the L intermediate layers;
• an edge-side communication unit capable of communicating data with the server;
• and
• an image identification unit that detects specific information included in an input image input to an input layer of the edge-side inference model, and generates, according to the detected specific information, layer position information indicating a position of a last layer to process the input image in the edge-side inference model,
• the control method comprising:
• a first input step of inputting the input image to an input layer of the edge-side inference model;
• a transmitting step of transmitting, from the edge device to the server using the edge-side communication unit, layer data that is a computation result obtained by performing processing from the input layer to a layer indicated by the layer position information, and the layer position information;
• a receiving step of receiving the layer data and the layer position information at the server using the server communication unit;
• a second input step of inputting the received layer data to a layer subsequent to a layer indicated by the received layer position information in the server-side inference model;
• and
• a processing step of performing processing from the layer, where the received layer data is input in the second input step, to the output layer.

A non-transitory computer-readable storage medium storing a program for causing a computer to execute a control method for an inference processing system, the inference processing system performing, by a server and an edge device, computation of an inference model including an input layer, L intermediate layers, and an output layer,

• wherein the server comprises:
• a first estimation unit that sets a server-side inference model that is the same as the inference model;
• and
• a server communication unit capable of communicating data with the edge device, and
• the edge device comprises:
• a second estimation unit that sets an edge-side inference model including the input layer and first to n-th (L>n>1) intermediate layers among the L intermediate layers;
• an edge-side communication unit capable of communicating data with the server;
• and
• an image identification unit that detects specific information included in an input image input to an input layer of the edge-side inference model, and generates, according to the detected specific information, first layer position information indicating a position of a specific layer that processes the input image in the edge-side inference model, and second layer position information indicating a position of a layer that is deeper than the specific layer by i (≥1),
• the control method comprising:
• a first input step of inputting the input image to an input layer of the edge-side inference model;
• a transmitting step of transmitting, from the edge device to the server using the edge-side communication unit, first intermediate data that is a computation result obtained by performing processing from the input layer to a layer indicated by the first layer position information, and second intermediate data that is a computation result obtained by performing processing from the input layer to a layer indicated by the second layer position information;
• a receiving step of receiving the first intermediate data and the second intermediate data at the server using the server communication unit;
• a second input step of inputting the first intermediate data to an intermediate layer at a layer position X (1≤X≤L−i) in the server-side inference model;
• an acquiring step of acquiring third intermediate data that is a computation result of an intermediate layer at a layer position X+i−1 obtained by performing processing from an intermediate layer at the layer position X to the intermediate layer at the layer position X+i−1 in the server-side inference model;
• a first processing step of inputting the second intermediate data to an intermediate data at a layer position X+i in the server-side inference model when the third intermediate data and the second intermediate data match, and performing processing from the intermediate layer at the layer position X+i to the output layer;
• and
• a second processing step of repeating processing of acquiring the third intermediate data again after incrementing a value of X by 1 when the third intermediate data and the second intermediate data do not match, until the third intermediate data acquired again and the second intermediate data match.