Collaborative sensor data processing by deep learning accelerators with integrated random access memory

A device, comprising:

• random access memory;
• at least one interface configured to receive data from a sensor as an input to at least a first portion of an artificial neural network and to receive an input generated outside of the device according to a second portion of the artificial neural network, the artificial neural network configured to generate at least one output based on a plurality of sensors, including the sensor connected to the device;
• and
• at least one processing unit coupled to the random access memory and the at least one interface to execute instructions having matrix operands to implement the matrix computations of the first portion of the artificial neural network, the first portion of the artificial neural network configured to process the data from the sensor to generate a first output independent of the input generated outside of the device according to the second portion of the artificial neural network and to generate a second output based on a combination of the data from the sensor and the input generated outside of the device according to the second portion of the artificial neural network.

An apparatus, comprising:

• memory configured to store first data representative of parameters of a first portion of an artificial neural network, second data representative of instructions executable to implement matrix computations of the first portion the artificial neural network using at least the first data stored in the memory, third data from a sensor, and fourth data generated outside of the apparatus according to a second portion of the artificial neural network;
• at least one interface configured to receive the fourth data from outside of the apparatus and store the fourth data into the memory;
• a connection to the memory;
• and
• a Field-Programmable Gate Array (FPGA) or Application Specific Integrated circuit (ASIC) having:
• a memory interface configure to access the memory via the connection;
• and
• at least one processing unit configured to execute the instructions having matrix operands to implement the matrix computations of first portion of the artificial neural network, the first portion of the artificial neural network configured to process the third data from the sensor to generate a first output independent of the fourth data and to generate a second output based on a combination of the third data from the sensor and the fourth data generated according to the second portion of the artificial neural network.