# US 10,504,009 B2 · Image hash codes generated by a neural network > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,504,009 B2 - Document ID: US10504009B2 - Status: Granted - Kind: B2 - Application number: 15/655,145 - Filed: 2017-07-20 - Published: 2017-11-02 - Granted: 2019-12-10 - Earliest claimed: 2015-06-05 - Assignees: AT&T Intellectual Property I, L.P. - Applicants: At&T Intellectual Property I, L.P. - Inventors: Yadong Mu; Zhu Liu - Web page: https://patentgrep.com/patent/US10504009B2 ## CPC - G06F 16/51: Indexing; Data structures therefor; Storage structures - G06F 16/285: {Clustering or classification} - G06F 16/5838: {using colour} - G06F 18/21: Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation - G06N 3/04: Architecture, e.g. interconnection topology - G06N 3/045: Combinations of networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/09: Supervised learning - G06V 10/70: using pattern recognition or machine learning (optical pattern recognition or electronic computations therefor G06V10/88) - G06V 30/194: References adjustable by an adaptive method, e.g. learning - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A method for classifying images using a neural network is described. The method utilizes the neural network to generate a deep hash data structure. In one embodiment, the method includes receiving, at a neural network, first data corresponding to a plurality of images of a training image set. The method includes adjusting parameters of the neural network based on concurrent application at the neural network of multiple loss functions to second data corresponding to the plurality of images to generate adjusted parameters. The method includes generating a deep hash data structure based on the adjusted parameters of the neural network. The method further includes sending, via a transmitter to a device, third data corresponding to the deep hash data structure. ## Claims ### Claim 1 A method comprising: at a processor that executes operations corresponding to a neural network: receiving first data corresponding to a plurality of images of a training image set; during a training stage, adjusting parameters of the neural network based on concurrent application at the neural network of multiple loss functions to second data corresponding to the plurality of images to generate adjusted parameters, wherein the multiple loss functions include a classification loss function and a hashing loss function, and wherein the concurrent application of the multiple loss functions reduces parameter instability associated with the hashing loss function; generating a deep hash data structure based on the adjusted parameters of the neural network determined from the training image set; and sending, via a transmitter to a device, third data corresponding to the deep hash data structure. ### Claim 2 (depends on 1) The method of claim 1, wherein the deep hash data structure includes a plurality of deep hash parameters, wherein the device is configured to execute an image hashing application based on the plurality of deep hash parameters to generate an image signature for an image file stored at the device, and wherein the image signature has a first number of bits that is less than a second number of bits associated with the image file. ### Claim 3 (depends on 1) The method of claim 1, wherein the device comprises a server that is configured to utilize the deep hash data structure to perform an image service operation and to send a set of images to a mobile device, the set of images retrieved from an image search database based on a comparison of an image signature received from the mobile device to a plurality of image signatures generated at the server based on the deep hash data structure. ### Claim 4 (depends on 3) The method of claim 3, wherein the image search database includes an index generated based on the deep hash data structure, and wherein the index is utilized to identify the set of images based on the image signature. ### Claim 5 (depends on 1) The method of claim 1, wherein the classification loss function is associated with an image classification function that extracts image features from each image of the plurality of images, and wherein the hashing loss function is associated with a hashing function that generates a hash code associated with each image of the plurality of images. ### Claim 6 (depends on 5) The method of claim 5, wherein a number of parameter adjustment iterations for an individual image subset of the plurality of images is determined based on a parameter adjustment iteration threshold. ### Claim 7 (depends on 5) The method of claim 5, wherein, during the training stage, a first task and a second task are performed concurrently in top layers of the neural network, wherein the first task is associated with a hashing loss layer corresponding to the hashing function and the hashing loss function, and wherein the second task is associated with an image classification layer corresponding to the image classification function and the classification loss function. ### Claim 8 (depends on 7) The method of claim 7, wherein the first task includes adjusting a first set of parameters associated with the hashing function based on a first loss value generated by the hashing loss function, and wherein the second task includes adjusting a second set of parameters associated with the image classification function based on a second loss value generated by the classification loss function. ### Claim 9 (depends on 8) The method of claim 8, further comprising, after the training stage, generating the deep hash data structure, wherein the deep hash data structure includes the first set of parameters associated with the hashing function. ### Claim 10 (depends on 8) The method of claim 8, wherein a subset of parameters of the second set of parameters associated with the image classification function is not utilized for generating hash codes during the training stage, and wherein the deep hash data structure does not include the subset of parameters. ### Claim 11 (depends on 1) The method of claim 1, wherein the first data is input into an input layer of the neural network, wherein the second data is input into an output layer of the neural network, the output layer configured to apply the multiple loss functions to the second data. ### Claim 12 A system comprising: a processor; and a memory coupled to the processor, the memory storing instructions that, when executed by the processor, cause the processor to perform operations, the operations including: during a training stage of a neural network, concurrently performing a first task and a second task at the neural network on data associated with multiple subsets of images of a plurality of images of a training image set, wherein the first task includes adjusting a first set of parameters based on a classification loss function to generate an adjusted first set of parameters, wherein the second task includes adjusting a second set of parameters based on a hashing loss function, and wherein the concurrently performing the first task and the second task reduces parameter instability associated with the hashing loss function; and after the training stage, generating a deep hash data structure based on the first set of parameters and the second set of parameters determined from the training image set. ### Claim 13 (depends on 12) The system of claim 12, wherein the first set of parameters is associated with a classification function, and wherein the first set of parameters is adjusted based on a first loss value generated by a classification loss function for a particular image of the training image set. ### Claim 14 (depends on 13) The system of claim 13, wherein the second set of parameters is associated with a hashing function, and wherein the second set of parameters is adjusted based on a second loss value generated by the hashing loss function for the particular image. ### Claim 15 (depends on 12) The system of claim 12, wherein the neural network comprises: convolution layers; pooling layers coupled to the convolution layers; inner product layers coupled to the pooling layers; and top layers coupled to the inner product layers, the top layers to concurrently perform the first task and the second task based on the multiple subsets of images. ### Claim 16 A device comprising: a processor; a memory coupled to the processor, the memory storing instructions that, when executed by the processor, cause the processor to perform operations, the operations including: receiving a plurality of deep hash parameters of a deep hash data structure; and utilizing an image hashing application to generate, based on the plurality of deep hash parameters, an image signature for an image file, the image signature having a first number of bits that is less than a second number of bits associated with the image file, wherein the plurality of deep hash parameters is obtained, at a neural network during a training stage, by iteratively adjusting parameters of the neural network based on concurrent application of multiple loss functions to data associated with a plurality of images of a training image set, the multiple loss functions including a classification loss function and a hashing loss function, and the concurrent application of the multiple loss functions causing a reduction of parameter instability associated with the hashing loss function. ### Claim 17 (depends on 16) The device of claim 16, wherein the operations further comprise sending the image signature, instead of the image file, to an image search server via a wireless communication network to reduce network bandwidth consumption associated with image search operations. ### Claim 18 (depends on 17) The device of claim 17, wherein the operations further comprise receiving a set of images from the image search server, wherein the set of images is retrieved from an image search database based on a comparison of the image signature to a plurality of image signatures generated at the image search server based on the plurality of deep hash parameters. ### Claim 19 (depends on 1) The method of claim 1, wherein the processor adjusts the parameters according to a lazy learning algorithm. ### Claim 20 (depends on 1) The method of claim 1, wherein the device comprises a mobile device, the mobile device configured to generate an image signature of an image file based on the third data.