# US 12,602,571 B2 · Network partitioning for sensor-based systems > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,602,571 B2 - Document ID: US12602571B2 - Status: Granted - Kind: B2 - Application number: 17/819,047 - Filed: 2022-08-11 - Published: 2023-05-18 - Granted: 2026-04-14 - Earliest claimed: 2021-11-12 - Assignees: Meta Platforms Technologies, LLC - Applicants: Meta Platforms Technologies, LLC - Inventors: Ziyun Li; Xin Dong; Barbara De Salvo; Xinqiao Liu - Web page: https://patentgrep.com/patent/US12602571B2 ## CPC - G06N 3/045: Combinations of networks - G06N 3/044: Recurrent networks, e.g. Hopfield networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/0499: Feedforward networks - G06N 3/063: using electronic means - G06N 3/08: Learning methods - G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections - G06N 3/084: Backpropagation, e.g. using gradient descent - G06N 3/09: Supervised learning - G06N 3/0985: Hyperparameter optimisation; Meta-learning; Learning-to-learn - G06N 3/048: Activation functions - G06N 3/086: using evolutionary algorithms, e.g. genetic algorithms or genetic programming - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract Methods, systems, and media for partitioning neural networks are provided. In some embodiments, a method comprises obtaining a training set. The method comprises training a plurality of neural networks using the training set, wherein neural networks differ based on dimensions of one or more layers of the neural networks and a location of a compression block positioned between a first set of layers of a neural network and a second set of layers of the neural network. The method comprises selecting a neural network based on hardware constraints of a system on which the neural network is to be implemented, wherein the first set of layers of the selected neural network are executed by one or more sensor devices of the system, and wherein the second set of layers of the selected neural network are executed by an aggregator computing device of the system. ## Claims ### Claim 1 A method of partitioning neural networks, comprising: obtaining a training set, wherein the training set comprises a set of input training images and corresponding target outputs; training a plurality of neural networks, each neural network of the plurality of neural networks trained using the training set, and wherein neural networks of the plurality of neural networks differ based at least in part on dimensions of one or more layers of the neural networks and a location of a compression block positioned between a first set of layers of a neural network and a second set of layers of the neural network; and selecting a neural network of the plurality of trained neural networks based on performance of the plurality of neural networks with respect to the training set and hardware constraints of a system on which the neural network is to be implemented, wherein the first set of layers of the selected neural network are executed by sensor computing devices of one or more sensor devices of the system, and wherein the second set of layers of the selected neural network are executed by an aggregator computing device of the system. ### Claim 2 (depends on 1) The method of claim 1, wherein training the plurality of neural networks comprises, for each iteration of a series of iterations, training a second plurality of neural networks, wherein the second plurality of neural networks comprise: at least one neural network with all potential parameters of a plurality of potential parameters included, at least one neural network with no compression block, at least one neural network with multiple compression blocks, and at least one neural network with a random sampling of the potential parameters. ### Claim 3 (depends on 2) The method of claim 2, wherein the at least one neural network with all potential parameters of the plurality of potential parameters included comprises at least one neural network with no compression blocks. ### Claim 4 (depends on 2) The method of claim 2, wherein the at least one neural network with all potential parameters of the plurality of potential parameters included comprises at least one neural network with multiple compression blocks. ### Claim 5 (depends on 2) The method of claim 2, wherein the at least one neural network with multiple compression blocks comprises a subset of the potential parameters. ### Claim 6 (depends on 1) The method of claim 1, wherein the one or more sensor devices comprise a plurality of sensor devices, and wherein the selected neural network comprises: a set of head models, each head model of the set of head models configured to take, as input, sensor data from a sensor of the plurality of sensor devices; and a tail model configured to generate a final output, wherein the tail model corresponds to the second set of layers, wherein the compression block comprises a fusion block configured to aggregate outputs of the set of head models and provide the aggregated outputs to the tail model. ### Claim 7 (depends on 6) The method of claim 6, wherein the selected neural network is selected based at least in part on a location of the fusion block. ### Claim 8 (depends on 1) The method of claim 1, wherein selecting the neural network comprises: (a) randomly sampling the plurality of trained neural networks to identify a subset of the plurality of trained neural networks; (b) evaluating the subset of the plurality of trained neural networks based on the performance of the subset of the plurality of trained neural networks and the hardware constraints of the system; (c) generating a second plurality of neural networks based at least in part on the evaluation of the subset of the plurality of trained neural networks; and (d) repeating (a)-(c) until the neural network has been identified. ### Claim 9 (depends on 8) The method of claim 8, wherein the second plurality of neural networks comprises a portion of the subset of the plurality of trained neural networks selected based on the evaluation of the subset of the plurality of trained neural networks. ### Claim 10 (depends on 8) The method of claim 8, wherein the second plurality of neural networks comprises at least one neural network of the plurality of trained neural networks not included in the subset of the plurality of trained neural networks. ### Claim 11 (depends on 8) The method of claim 8, wherein the random sampling in (a) comprises a random sampling of a first generation of the plurality of trained neural networks to identify the subset of the plurality of neural networks, and wherein (c) generating the second plurality of neural networks comprises generating the second plurality of neural networks using random sampling, mutation, and crossover from top-performers of the randomly sampled first generation of the plurality of trained neural networks identified in the evaluation in (b). ### Claim 12 (depends on 1) The method of claim 1, wherein the hardware constraints of the system comprise a communication latency to transmit feature information generated from the first set of layers on the one or more sensor devices to the second set of layers on the aggregator computing device. ### Claim 13 (depends on 1) The method of claim 1, wherein the hardware constraints of the system comprise memory usage, bandwidth, or any combination thereof. ### Claim 14 (depends on 1) The method of claim 1, wherein at least one neural network of the plurality of neural networks comprises at least 100 layers. ### Claim 15 (depends on 1) The method of claim 1, wherein the compression block comprises a compression that reduces feature dimensions generated by the first set of layers and an expansion that increases the feature dimensions prior to providing features to the second set of layers. ### Claim 16 A system, comprising: a sensor associated with a sensor computing device; and an aggregator computing device, wherein: the sensor computing device is configured to implement a first set of layers of a neural network, and wherein the aggregator computing device is configured to implement a second set of layers of a neural network, and wherein there is a compression block between the first set of layers and the second set of layers that reduces a feature dimension size of a feature set generated by the first set of layers prior to providing the feature set to the second set of layers, and the neural network was selected from a plurality of neural networks based at least in part on an effect of a location of the compression block with respect to the first set of layers and the second set of layers on hardware usage of the sensor computing device and the aggregator computing device. ### Claim 17 (depends on 16) The system of claim 16, wherein the sensor comprises a camera. ### Claim 18 (depends on 16) The system of claim 16, wherein the system comprises a plurality of sensors including the sensor, each associated with a sensor computing device, and wherein the compression block occurs subsequent to a fusion layer that combines feature sets associated with each sensor of the plurality of sensors. ### Claim 19 (depends on 16) The system of claim 16, wherein the hardware usage comprises a communication latency to transmit the feature set from the sensor computing device to the aggregator computing device. ### Claim 20 (depends on 16) The system of claim 16, wherein the hardware usage comprises memory usage, bandwidth, or any combination thereof of the sensor computing device and/or the aggregator computing device.