# US 12,223,412 B2 · Dynamic matrix convolution with channel fusion > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,223,412 B2 - Document ID: US12223412B2 - Status: Granted - Kind: B2 - Application number: 17/123,697 - Filed: 2020-12-16 - Published: 2022-06-16 - Granted: 2025-02-11 - Earliest claimed: 2020-12-16 - Assignees: Microsoft Technology Licensing, LLC - Applicants: Microsoft Technology Licensing, LLC - Inventors: Yinpeng Chen; Xiyang Dai; Mengchen Liu; Dongdong Chen; Lu Yuan; Zicheng Liu; Ye Yu; Mei Chen; Yunsheng Li - Web page: https://patentgrep.com/patent/US12223412B2 ## CPC - G06N 3/04: Architecture, e.g. interconnection topology - G06N 3/084: Backpropagation, e.g. using gradient descent - G06F 17/153: {Multidimensional correlation or convolution} - G06F 17/16: Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)} - G06N 3/045: Combinations of networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/0495: Quantised networks; Sparse networks; Compressed networks - G06N 3/09: Supervised learning - G06V 10/70: using pattern recognition or machine learning (optical pattern recognition or electronic computations therefor G06V10/88) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A computer device for automatic feature detection comprises a processor, a communication device, and a memory configured to hold instructions executable by the processor to instantiate a dynamic convolution neural network, receive input data via the communication network, and execute the dynamic convolution neural network to automatically detect features in the input data. The dynamic convolution neural network compresses the input data from an input space having a dimensionality equal to a predetermined number of channels into an intermediate space having a dimensionality less than the number of channels. The dynamic convolution neural network dynamically fuses the channels into an intermediate representation within the intermediate space and expands the intermediate representation from the intermediate space to an expanded representation in an output space having a higher dimensionality than the dimensionality of the intermediate space. The features in the input data are automatically detected based on the expanded representation. ## Claims ### Claim 1 A computer system for automatic feature detection, comprising: a processor; a communication interface; and a memory configured to hold instructions executable by the processor to: instantiate a dynamic convolution neural network; receive input data via the communication interface; execute the dynamic convolution neural network to: compress the input data from an input space having a dimensionality equal to a predetermined number of channels into an intermediate space having a dimensionality less than the number of channels; dynamically fuse the channels into an intermediate representation within the intermediate space; and expand the intermediate representation from the intermediate space to an expanded representation in an output space having a higher dimensionality than the dimensionality of the intermediate space; and automatically detect features in the input data based on the expanded representation in the output space. ### Claim 2 A method of computer-detecting features in an image, comprising: instantiating a dynamic convolution neural network configured to: compress an image from an input space having a dimensionality equal to a predetermined number of channels into an intermediate space having a dimensionality less than the number of channels; dynamically fuse the channels into an intermediate representation within the intermediate space; and expand the intermediate representation from the intermediate space to an output space having a higher dimensionality than the dimensionality of the intermediate space; and executing the dynamic convolution neural network to detect the features in the image. ### Claim 3 A method of computer-detecting features in input data, comprising: instantiating a dynamic convolution neural network configured to: compress the input data from an input space having a dimensionality equal to a predetermined number of channels into an intermediate space having a dimensionality less than the number of channels; dynamically fuse the channels into an intermediate representation within the intermediate space; and expand the intermediate representation from the intermediate space to an output space having a higher dimensionality than the dimensionality of the intermediate space; and executing the dynamic convolution neural network to detect the features in the input data. ### Claim 4 (depends on 3) The method of claim 3, wherein the input data is compressed from the input space into the intermediate space via a static compression matrix. ### Claim 5 (depends on 3) The method of claim 3, wherein the channels are dynamically fused into the intermediate representation within the intermediate space via a dynamic channel fusion matrix having a plurality of elements that are each a function of the input data. ### Claim 6 (depends on 3) The method of claim 3, wherein the intermediate representation is expanded from the intermediate space to the output space via a static expansion matrix. ### Claim 7 (depends on 3) The method of claim 3, wherein: the input data is compressed from the input space into the intermediate space via a static compression matrix; the channels are dynamically fused into the intermediate representation within the intermediate space via a dynamic channel fusion matrix having a plurality of elements that are each a function of the input data; the intermediate representation is expanded from the intermediate space to the output space via a static expansion matrix; and the static compression matrix and the static expansion matrix represent a plurality of matrices, each matrix of the plurality of matrices corresponding to a column of the static compression matrix and a row of the static expansion matrix, and the dynamic channel fusion matrix represents an attention function aggregating attention scores for the plurality of matrices in the intermediate space. ### Claim 8 (depends on 7) The method of claim 7, wherein the dynamic convolution neural network is configured to compose the static compression matrix, the dynamic channel fusion matrix, and the static expansion matrix into a dynamic residual matrix. ### Claim 9 (depends on 8) The method of claim 8, wherein the dynamic convolution neural network is configured to compute a convolution of the input data based on a sum of the dynamic residual matrix with a static average kernel matrix. ### Claim 10 (depends on 9) The method of claim 9, wherein the static average kernel matrix is weighted by a dynamic channel-wise attention diagonal matrix. ### Claim 11 (depends on 10) The method of claim 10, wherein the dynamic convolution neural network has a dynamic convolution decomposition layer including a dynamic branch configured to compute elements of the dynamic channel fusion matrix and the dynamic channel-wise attention diagonal matrix. ### Claim 12 (depends on 11) The method of claim 11, wherein the dynamic branch is configured to compress the input data via a pooling function. ### Claim 13 (depends on 11) The method of claim 11, wherein the dynamic branch is configured to process the input data via a fully connected neural network layer configured to output the elements of the dynamic channel fusion matrix. ### Claim 14 (depends on 11) The method of claim 11, wherein the dynamic branch is further configured to compute an activation function for the input data based on the sum of the dynamic residual matrix and the static average kernel matrix. ### Claim 15 (depends on 14) The method of claim 14, further comprising training the dynamic convolution neural network by adjusting the dynamic branch based on the activation function. ### Claim 16 (depends on 3) The method of claim 3, wherein the dimensionality of the intermediate space is less than a square root of the number of channels. ### Claim 17 (depends on 3) The method of claim 3, wherein the dynamic convolution neural network is configured to compute a pointwise convolution with dynamic attention across channels. ### Claim 18 (depends on 3) The method of claim 3, wherein the dynamic convolution neural network is configured to compute a depthwise convolution with dynamic attention across neural network layers and dynamic fusion of latent kernel elements. ### Claim 19 (depends on 3) The method of claim 3, wherein the dynamic convolution neural network is configured to compute a spatial convolution with dynamic fusion of channels. ### Claim 20 (depends on 19) The method of claim 19, wherein the dynamic fusion of channels includes dynamic joint fusion of channels and latent kernel elements.