Device and method for providing benchmark result of artificial intelligence based model

A method for providing a benchmark result, performed by a computing device, comprising:

• obtaining a first input data including an inference task;
• determining a target model which is a subject a benchmark for the inference task and determining at least one target node at which the inference task of the target model is to be executed, based on a candidate node list including candidate nodes recommended for the benchmark for the inference task, wherein the determined target model corresponds to an artificial intelligence-based model of which the benchmark for the inference task is performed at the at least one target node based on a dataset, and
• providing the benchmark result obtained by executing the target model at the at least one target node,
• wherein the providing the benchmark result when a third node unidentifiable from outside is included in the at least one target node comprises:
• transmitting to the third node a first acknowledgement message including a benchmark task for benchmarking the target model at the third node, in response to receiving a first low power wireless signal from the third node;
• and
• receiving from the third node a second low power wireless signal including a benchmark result generated by the third node.

A method for providing a benchmark result, performed by a computing device, comprising:

• obtaining a first input data including an inference task;
• determining a target model which is a subject of a benchmark for the inference task and determining at least one target node at which the inference task of the target model is to be executed, based on a candidate node list including candidate nodes recommended for the benchmark for the inference task, wherein the determined target model corresponds to an artificial intelligence-based model of which the benchmark for the inference task is performed at the at least one target node based on a dataset, and
• providing the benchmark result obtained by executing the target model at the at least one target node,
• wherein in response to receiving second input data selecting the at least one target node in the candidate node list, i) the at least one target node is determined and ii) information for recommending a target model supportable by the at least one target node to determine the target model is provided.

A computer program stored in a computer readable medium, wherein the computer program allows a computing device to perform following operations to provide a benchmark result when executed by the computing device, and wherein the operations comprise:

• obtaining a first input data including an inference task;
• determining a target model which is a subject of a benchmark for the inference task and determining at least one target node at which the inference task of the target model is to be executed, based on a candidate node list including candidate nodes recommended for the benchmark for the inference task, wherein the determined target model corresponds to an artificial intelligence-based model of which the benchmark for the inference task is performed at the at least one target node based on a dataset;
• and
• providing the benchmark result obtained by executing the target model at the at least one target node,
• wherein the candidate node list comprises:
• first nodes as the candidate nodes wherein the first nodes have an execution environment that supports a first operator included in an inputted source model;
• or
• second nodes as the candidate nodes, wherein the second nodes have an execution environment that does not support a first operator included in an inputted source model but supports a second operator that is able to replace the first operator, and wherein the second operator is different from the first operator.

A computing device for generating a benchmark result, comprising:

• at least one processor;
• and
• a memory,
• wherein the at least one processor:
• obtains a first input data including an inference task and a dataset;
• determine a target model which is a subject of a benchmark for the inference task and determining at least one target node at which the inference task of the target model is to be executed, based on a candidate node list including candidate nodes recommended for the benchmark for the inference task, wherein the determined target model corresponds to an artificial intelligence-based model of which the benchmark for the inference task is performed at the at least one target node based on a dataset;
• and
• provides the benchmark result obtained by executing the target model at the at least one target node,
• wherein the candidate node list comprises:
• first nodes as the candidate nodes wherein the first nodes have an execution environment that supports a first operator included in an inputted source model;
• or
• second nodes as the candidate nodes, wherein the second nodes have an execution environment that does not support a first operator included in an inputted source model but supports a second operator that is able to replace the first operator, and wherein the second operator is different from the first operator.