Systems and methods for machine learning evaluation pipeline
AssigneeWoven By Toyota, Inc.
InventorKoichiro YAMAGUCHI
Life of this patent
Estimating the term…
Abstract
Provided are a method, system, and device for a neural architecture search (NAS) pipeline for performing an optimized neural architecture search (NAS). The method may include obtaining a first search space comprising a plurality of candidate layers for a neural network architecture; performing a training-free NAS in the first search space to obtain a first set of architectures; obtaining a second search space based on the first set of architectures; performing a gradient-based search in the second search space to obtain a second set of architectures; performing a sampling method search utilizing the second set of architectures as an initial sample; and obtaining an output architecture as an output of the sampling method search.
Claims20
A method for performing a neural architecture search (NAS), the method comprising:
- obtaining a first search space comprising a plurality of candidate layers for a neural network architecture;
- performing a training-free NAS in the first search space to obtain a first set of architectures;
- obtaining a second search space based on the first set of architectures;
- performing a gradient-based search in the second search space to obtain a second set of architectures;
- performing a sampling method search utilizing the second set of architectures as an initial sample;
- and
- obtaining an output architecture as an output of the sampling method search,
- wherein the sampling method search is performed iteratively,
- wherein the sampling method search comprises an evolutionary search algorithm,
- wherein the evolutionary search algorithm of the sampling method selects, from an entirety of the search space, component blocks of a given candidate architecture, trains the given candidate architecture, scores the given candidate architecture, replaces a lowest scoring architecture in the candidate pool with the given candidate architecture, and forms a Pareto-front set from the candidate pool, and
- wherein the obtaining comprises obtaining the output architecture from the Pareto-front set.
(canceled)
(canceled)
An apparatus for performing a neural architecture search (NAS), the apparatus comprising:
- at least one memory storing computer-executable instructions;
- and
- at least one processor configured to execute the computer-executable instructions to:
- obtain a first search space comprising a plurality of candidate layers for a neural network architecture;
- perform a training-free NAS in the first search space to obtain a first set of architectures;
- obtain a second search space based on the first set of architectures;
- perform a gradient-based search in the second search space to obtain a second set of architectures;
- perform a sampling method search utilizing the second set of architectures as an initial sample;
- and
- obtain an output architecture as an output of the sampling method search,
- wherein the sampling method search is performed iteratively,
- wherein the sampling method search comprises an evolutionary search algorithm,
- wherein the evolutionary search algorithm of the sampling method selects, from an entirety of the search space, component blocks of a given candidate architecture, trains the given candidate architecture, scores the given candidate architecture, replaces a lowest scoring architecture in the candidate pool with the given candidate architecture, and forms a Pareto-front set from the candidate pool, and
- wherein the obtaining comprises obtaining the output architecture from the Pareto-front set.
(canceled)
(canceled)
A non-transitory computer-readable recording medium having recorded thereon instructions executable by at least one processor to cause the at least one processor to perform a method comprising:
- obtaining a second search space based on the first set of architectures;
- performing a gradient-based search in the second search space to obtain a second set of architectures;
- performing a sampling method search utilizing the second set of architectures as an initial sample;
- and
- obtaining an output architecture as an output of the sampling method search,
- wherein the sampling method search is performed iteratively,
- wherein the sampling method search comprises an evolutionary search algorithm,
- wherein the evolutionary search algorithm of the sampling method selects, from an entirety of the search space, component blocks of a given candidate architecture, trains the given candidate architecture, scores the given candidate architecture, replaces a lowest scoring architecture in the candidate pool with the given candidate architecture, and forms a Pareto-front set from the candidate pool, and
- wherein the obtaining comprises obtaining the output architecture from the Pareto-front set.
(canceled)
(canceled)
Dependencies are inferred from the claim text.