# US 12,216,647 B1 · Systems and methods for machine learning evaluation pipeline > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,216,647 B1 - Document ID: US12216647B1 - Status: Granted - Kind: B1 - Application number: 18/353,401 - Filed: 2023-07-17 - Published: 2025-01-23 - Granted: 2025-02-04 - Earliest claimed: 2023-07-17 - Assignees: Woven By Toyota, Inc. - Applicants: Woven By Toyota, Inc. - Inventors: Koichiro Yamaguchi - Web page: https://patentgrep.com/patent/US12216647B1 ## CPC - G06F 16/2425: {Iterative querying; Query formulation based on the results of a preceding query} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Provided are a method, system, and device for a neural architecture search (NAS) pipeline for performing an optimized neural architecture search (NAS). The method may include obtaining a first search space comprising a plurality of candidate layers for a neural network architecture; performing a training-free NAS in the first search space to obtain a first set of architectures; obtaining a second search space based on the first set of architectures; performing a gradient-based search in the second search space to obtain a second set of architectures; performing a sampling method search utilizing the second set of architectures as an initial sample; and obtaining an output architecture as an output of the sampling method search. ## Claims ### Claim 1 A method for performing a neural architecture search (NAS), the method comprising: obtaining a first search space comprising a plurality of candidate layers for a neural network architecture; performing a training-free NAS in the first search space to obtain a first set of architectures; obtaining a second search space based on the first set of architectures; performing a gradient-based search in the second search space to obtain a second set of architectures; performing a sampling method search utilizing the second set of architectures as an initial sample; and obtaining an output architecture as an output of the sampling method search, wherein the sampling method search is performed iteratively, wherein the sampling method search comprises an evolutionary search algorithm, wherein the evolutionary search algorithm of the sampling method selects, from an entirety of the search space, component blocks of a given candidate architecture, trains the given candidate architecture, scores the given candidate architecture, replaces a lowest scoring architecture in the candidate pool with the given candidate architecture, and forms a Pareto-front set from the candidate pool, and wherein the obtaining comprises obtaining the output architecture from the Pareto-front set. ### Claim 2 (depends on 1) The method according to claim 1, wherein the first search space is obtained based on a set of architecture parameters. ### Claim 3 (depends on 1) The method according to claim 1, wherein the second search space is obtained based on a sub-space which contain one or more architectures in the first set of architectures. ### Claim 4 (depends on 1) The method according to claim 1, wherein a number of iterations of the sampling method search is based on a predetermined threshold. ### Claim 5 (depends on 1) The method according to claim 1, wherein the training-free NAS is performed based on one or more metrics. ### Claim 6 An apparatus for performing a neural architecture search (NAS), the apparatus comprising: at least one memory storing computer-executable instructions; and at least one processor configured to execute the computer-executable instructions to: obtain a first search space comprising a plurality of candidate layers for a neural network architecture; perform a training-free NAS in the first search space to obtain a first set of architectures; obtain a second search space based on the first set of architectures; perform a gradient-based search in the second search space to obtain a second set of architectures; perform a sampling method search utilizing the second set of architectures as an initial sample; and obtain an output architecture as an output of the sampling method search, wherein the sampling method search is performed iteratively, wherein the sampling method search comprises an evolutionary search algorithm, wherein the evolutionary search algorithm of the sampling method selects, from an entirety of the search space, component blocks of a given candidate architecture, trains the given candidate architecture, scores the given candidate architecture, replaces a lowest scoring architecture in the candidate pool with the given candidate architecture, and forms a Pareto-front set from the candidate pool, and wherein the obtaining comprises obtaining the output architecture from the Pareto-front set. ### Claim 7 (depends on 6) The apparatus according to claim 6, wherein the first search space is obtained based on a set of architecture parameters. ### Claim 8 (depends on 6) The apparatus according to claim 6, wherein the second search space is obtained based on a sub-space which contain one or more architectures in the first set of architectures. ### Claim 9 (depends on 6) The apparatus according to claim 6, wherein a number of iterations of the sampling method search is based on a predetermined threshold. ### Claim 10 (depends on 6) The apparatus according to claim 6, wherein the training-free NAS is performed based on one or more metrics. ### Claim 11 A non-transitory computer-readable recording medium having recorded thereon instructions executable by at least one processor to cause the at least one processor to perform a method comprising: obtaining a second search space based on a first set of architectures; performing a gradient-based search in the second search space to obtain a second set of architectures; performing a sampling method search utilizing the second set of architectures as an initial sample; and obtaining an output architecture as an output of the sampling method search, wherein the sampling method search is performed iteratively, wherein the sampling method search comprises an evolutionary search algorithm, wherein the evolutionary search algorithm of the sampling method selects, from an entirety of the search space, component blocks of a given candidate architecture, trains the given candidate architecture, scores the given candidate architecture, replaces a lowest scoring architecture in the candidate pool with the given candidate architecture, and forms a Pareto-front set from the candidate pool, and wherein the obtaining comprises obtaining the output architecture from the Pareto-front set. ### Claim 12 (depends on 11) The non-transitory computer-readable recording medium according to claim 11, wherein the first search space is obtained based on a set of architecture parameters. ### Claim 13 (depends on 11) The non-transitory computer-readable recording medium according to claim 11, wherein the second search space is obtained based on a sub-space which contain one or more architectures in the first set of architectures. ### Claim 14 (depends on 11) The non-transitory computer-readable recording medium according to claim 11, wherein a number of iterations of the sampling method search is based on a predetermined threshold.