# US 10,311,861 B1 · System and method for encoding data in a voice recognition integrated circuit solution > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,311,861 B1 - Document ID: US10311861B1 - Status: Granted - Kind: B1 - Application number: 15/871,933 - Filed: 2018-01-15 - Granted: 2019-06-04 - Earliest claimed: 2018-01-15 - Assignees: Gyrfalcon Technology Inc. - Applicants: Gyrfalcon Technology Inc. - Inventors: Xiang Gao; Lin Yang; Wenhan Zhang - Web page: https://patentgrep.com/patent/US10311861B1 ## CPC - G10L 15/16: using artificial neural networks - G06N 3/045: Combinations of networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/0495: Quantised networks; Sparse networks; Compressed networks - G06N 3/08: Learning methods - G06N 3/09: Supervised learning - G10L 15/02: Feature extraction for speech recognition; Selection of recognition unit - G10L 15/063: {Training} - G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue - G10L 17/00: Speaker identification or verification techniques - G10L 25/18: the extracted parameters being spectral information of each sub-band - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract Methods of encoding voice data for loading into an artificial intelligence (AI) integrated circuit are provided. The AI integrated circuit may have an embedded cellular neural network for implementing AI tasks based on the loaded voice data. An encoding method may generate a two-dimensional (2D) frequency-time array from an audio waveform, apply a probability function to the 2D frequency-time array to generate a set of 2D arrays, load the set of 2D arrays into the AI integrated circuit, execute programming instructions contained in the AI integrated circuit to feed the set of 2D arrays into the embedded cellular neural network in the AI integrated circuit to generate a voice recognition result, and output the voice recognition result. The encoding method also trains a convolution neural network (CNN) and loads the weights of the CNN into the AI integrated circuit for implementing the AI tasks. ## Claims ### Claim 1 A method of encoding voice data for loading into an artificial intelligence (AI) integrated circuit, the method comprising: receiving, by a processor, voice data comprising at least a segment of an audio waveform; generating, by the processor, a two-dimensional (2D) frequency-time array comprising a plurality of pixels, each pixel having a value that represents an audio intensity of the segment of the audio waveform at a time in the segment and a frequency in the audio waveform; applying, by the processor, a probability function to the 2D frequency-time array to generate a set of 2D arrays comprising a number of 2D arrays, each 2D array having a plurality of pixels, each pixel having a value and corresponding to one of the plurality of pixels in the 2D frequency-time array, wherein: the probability function comprises a plurality of entries representing a range of intensity values of the audio waveform, and for each entry, a probability value for each of a number of sub-functions, each sub-function representing a sub-intensity band of the audio waveform, the number of 2D arrays in the set of 2D arrays is the number of sub-functions in the probability function, and generating the set of 2D arrays comprises, for each pixel in the 2D frequency-time array, applying the probability function to a value of each pixel to determine values of corresponding pixels in the set of 2D arrays; loading the set of 2D arrays into the AI integrated circuit; by the AI integrated circuit, executing one or more programming instructions contained in the AI integrated circuit to feed the set of 2D arrays into an embedded cellular neural network architecture in the AI integrated circuit; generating a voice recognition result from the cellular neural network architecture based on the set of 2D arrays; and outputting the voice recognition result. ### Claim 2 (depends on 1) The method of claim 1, further comprising: receiving a set of sample training voice data comprising at least one sample segment of an audio waveform; using the set of sample training voice data to generate one or more sample 2D frequency-time arrays each comprising a plurality of pixels, each pixel having a value that represents an audio intensity of the sample segment of the audio waveform at a time in the sample segment and a frequency in the audio waveform; applying the probability function to each of the one or more sample 2D frequency-time arrays to generate a set of 2D training arrays, each 2D training array having a plurality of pixels, each pixel having a value and corresponding to one of the plurality of pixels in each sample 2D frequency-time array; using the set of 2D training arrays to train one or more weights of a convolutional neural network; and loading the one or more trained weights into the embedded cellular neural network architecture of the AI integrated circuit. ### Claim 3 (depends on 1) The method of claim 1, wherein the 2D frequency-time array is a 2D spectrogram. ### Claim 4 (depends on 1) The method of claim 1, further comprising scaling the values of each pixel in the set of 2D arrays to an integer within a range of a depth of each channel in the AI integrated circuit. ### Claim 5 A method of encoding voice data for loading into an artificial intelligence (AI) integrated circuit, the method comprising: receiving, by a processor, voice data comprising at least a segment of an audio waveform; generating, by the processor, a two-dimensional (2D) frequency-time array comprising a plurality of pixels, each pixel having a value that represents an audio intensity of the segment of the audio waveform at a time in the segment and a frequency in the audio waveform; applying, by the processor, a probability function to the 2D frequency-time array to generate a set of 2D arrays comprising a number of 2D arrays, each 2D array having a plurality of pixels, each pixel having a value and corresponding to one of the plurality of pixels in the 2D frequency-time array, wherein: the probability function comprises a plurality of entries representing a range of frequency values of the audio waveform, and for each entry, a probability value for each of a plurality of sub-functions, each sub-function representing a sub-frequency band of the audio waveform, the number of 2D arrays in the set of 2D arrays is the number of sub-functions in the probability function, and generating the set of 2D arrays comprises: for each pixel in the 2D frequency-time, applying the probability function to a value of each pixel to determine values of corresponding pixels in the set of 2D arrays; loading the set of 2D arrays into the AI integrated circuit; by the AI integrated circuit, executing one or more programming instructions contained in the AI integrated circuit to feed the set of 2D arrays into an embedded cellular neural network architecture in the AI integrated circuit; generating a voice recognition result from the cellular neural network architecture based on the set of 2D arrays; and outputting the voice recognition result. ### Claim 6 (depends on 5) The method of claim 5, wherein the 2D frequency-time array is a 2D spectrogram. ### Claim 7 (depends on 5) The method of claim 5, further comprising scaling the values of each pixel in the set of 2D arrays to a an integer within a range of a depth of each channel in the AI integrated circuit. ### Claim 8 (depends on 6) The method of claim 6, further comprising scaling each value of the pixel in the 2D spectrogram by using a Mel scale formula before generating the set of 2D arrays. ### Claim 9 A system for encoding voice data for loading into an artificial intelligence (AI) integrated circuit, the system comprising: a processor; and a non-transitory computer readable medium containing programming instructions that, when executed, will cause the processor to: receive voice data comprising at least a segment of an audio waveform, generate a two-dimensional (2D) frequency-time array comprising a plurality of pixels, each pixel having a value that represents an audio intensity of the segment of the audio waveform at a time in the segment and a frequency in the audio waveform, apply a probability function to the 2D frequency-time array to generate a set of 2D arrays comprising a number of 2D arrays, each 2D array having a plurality of pixels, each pixel having a value and corresponding to one of the plurality of pixels in the 2D frequency-time array, wherein: the probability function comprises a plurality of entries representing a range of intensity values of the audio waveform, and for each entry, a probability value for each of a number of sub-functions, each sub-function representing a sub-intensity band of the audio waveform; the number of 2D arrays in the set of 2D arrays is the number of sub-functions in the probability function; and programming instructions for generating the set of 2D arrays comprise programming instructions configured to, for each pixel in the 2D frequency-time array, apply the probability function to a value of each pixel to determine values of corresponding pixels in the set of 2D arrays, and load the set of 2D arrays into the AI integrated circuit; wherein the AI integrated circuit comprises: an embedded cellular neural network architecture, and one or more programming instructions configured to: feed the set of 2D arrays into the embedded cellular neural network architecture in the AI integrated circuit; generate a voice recognition result from the cellular neural network architecture based on the set of 2D arrays; and output the voice recognition result. ### Claim 10 (depends on 9) The system of claim 9, further comprising additional programming instructions configured to: receive a set of sample training voice data comprising at least one sample segment of an audio waveform; use the set of sample training voice data to generate one or more sample 2D frequency-time arrays each comprising a plurality of pixels, each pixel having a value that represents an audio intensity of the sample segment of the audio waveform at a time in the sample segment and a frequency in the audio waveform; apply the probability function to each of the one or more sample 2D frequency-time arrays to generate a set of 2D training arrays, each 2D training array having a plurality of pixels, each pixel having a value and corresponding to one of the plurality of pixels in each sample 2D frequency-time array; use the set of 2D training arrays to train one or more weights of a convolutional neural network; and load the one or more trained weights into the embedded cellular neural network architecture of the AI integrated circuit. ### Claim 11 (depends on 9) The system of claim 9, wherein the 2D frequency-time array is a 2D spectrogram. ### Claim 12 (depends on 9) The system of claim 9, further comprising additional programming instructions configured to scale the values of each pixel in the set of 2D arrays to an integer within a range of a depth of each channel in the AI integrated circuit. ### Claim 13 A system for encoding voice data for loading into an artificial intelligence (AI) integrated circuit, the system comprising: a processor; and a non-transitory computer readable medium containing programming instructions that, when executed, will cause the processor to: receive voice data comprising at least a segment of an audio waveform, generate a two-dimensional (2D) frequency-time array comprising a plurality of pixels, each pixel having a value that represents an audio intensity of the segment of the audio waveform at a time in the segment and a frequency in the audio waveform, apply a probability function to the 2D frequency-time array to generate a set of 2D arrays comprising a number of 2D arrays, each 2D array having a plurality of pixels, each pixel having a value and corresponding to one of the plurality of pixels in the 2D frequency-time array, wherein: the probability function comprises a plurality of entries representing a range of frequency values of the audio waveform, and for each entry, a probability value for each of a plurality of sub-functions, each sub-function representing a sub-frequency band of the audio waveform; the number of 2D arrays in the set of 2D arrays is the number of sub-functions in the probability function; and programming instructions for generating the set of 2D arrays comprise programming instructions configured to, for each pixel in the 2D frequency-time, apply the probability function to a value of each pixel to determine values of corresponding pixels in the set of 2D arrays, and load the set of 2D arrays into the AI integrated circuit; wherein the AI integrated circuit comprises: an embedded cellular neural network architecture, and one or more programming instructions configured to: feed the set of 2D arrays into the embedded cellular neural network architecture in the AI integrated circuit; generate a voice recognition result from the cellular neural network architecture based on the set of 2D arrays; and output the voice recognition result. ### Claim 14 (depends on 13) The system of claim 13, wherein the 2D frequency-time array is a 2D spectrogram. ### Claim 15 (depends on 13) The system of claim 13, further comprising additional programming instructions configured to scale the values of each pixel in the set of 2D arrays to a an integer within a range of a depth of each channel in the AI integrated circuit. ### Claim 16 (depends on 14) The system of claim 14, further comprising additional programming instructions configured to scale each value of the pixel in the 2D spectrogram by using a Mel scale formula before generating the set of 2D arrays.