# US 12,225,105 B1 · System and method for multi-modal homomorphic compression > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,225,105 B1 - Document ID: US12225105B1 - Status: Granted - Kind: B1 - Application number: 18/890,774 - Filed: 2024-09-20 - Granted: 2025-02-11 - Earliest claimed: 2023-12-12 - Assignees: Atombeam Technologies Inc. - Applicants: AtomBeam Technologies Inc. - Inventors: Brian Galvin - Web page: https://patentgrep.com/patent/US12225105B1 ## CPC - G06N 3/0455: Auto-encoder networks; Encoder-decoder networks - H04L 9/008: {involving homomorphic encryption} - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/047: Probabilistic or stochastic networks - G06N 3/0475: Generative networks - G06N 3/088: Non-supervised learning, e.g. competitive learning - H03M 7/30: Compression (speech analysis-synthesis for redundancy reduction G10L19/00; for image communication H04N); Expansion; Suppression of unnecessary data, e.g. redundancy reduction - H03M 7/3059: {Digital compression and data reduction techniques where the original information is represented by a subset or similar information, e.g. lossy compression} - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - H03M: CODING; DECODING; CODE CONVERSION IN GENERAL (using fluidic means F15C4/00; optical analogue/digital converters G02F7/00; coding, decoding or code conversion, specially adapted for particular applications, see the relevant subclasses, e.g. G01D, G01R, G06F, G06T, G09G, G10L, G11B, G11C, H04B, H04L, H04M, H04N; ciphering or deciphering for cryptography or other purposes involving the need for secrecy G09C) ## Abstract A system and method for compressing and restoring multi-modal data utilizing a variational autoencoder to enable homomorphic compression techniques is disclosed. Multi-modal input data, comprising at least two different data types, is compressed into a unified latent space using an encoder network of a multi-modal variational autoencoder. Homomorphic operations are performed on compressed data in the latent space. The latent space compressed data is decompressed using a decoder network of the multi-modal variational autoencoder. The system utilizes modality-specific layers and cross-modal attention mechanisms to effectively process diverse data types. The homomorphic operations enable performing computations while the data is in a compressed form, preserving results of those operations in the decompressed output. This approach allows for efficient storage, transmission, and analysis of multi-modal data while maintaining privacy and data integrity across different modalities. ## Claims ### Claim 1 A system for compressing and restoring multi-modal data, comprising: a computing device comprising at least a memory and a processor; a plurality of programming instructions stored in the memory and operable on the processor that, when executing on the processor, cause the computing device to: preprocess raw multi-modal data to generate a plurality of input data sets, wherein the multi-modal data comprises at least two different data types; compress the plurality of input data sets into a plurality of compressed data sets using an encoder within a multi-modal variational autoencoder; decompress the plurality of compressed data sets using a decoder located within the multi-modal variational autoencoder to obtain a plurality of reduced output data sets; and process the plurality of reduced output data sets through a decompressed output organizer, thereby generating a reconstructed multi-modal output. ### Claim 2 (depends on 1) The system of claim 1, wherein the encoder and decoder of the multi-modal variational autoencoder modality-specific layers for processing different data types, shared layers for integrating information across modalities, and activation functions. ### Claim 3 (depends on 2) The system of claim 2, wherein the modality-specific layers comprise at least one of convolutional layers for image data, recurrent layers for sequential data, and fully connected layers for numerical data. ### Claim 4 (depends on 1) The system of claim 1, wherein the multi-modal data comprises at least two of: image data, audio data, text data, time-series data, and sensor data. ### Claim 5 (depends on 1) The system of claim 1, wherein the plurality of programming instructions further includes instructions that, when executing on the processor, cause the multi-modal variational autoencoder to perform partially homomorphic compression in a unified latent space representing all data modalities. ### Claim 6 (depends on 5) The system of claim 5, wherein the multi-modal variational autoencoder comprises a unified latent space, and wherein the plurality of programming instructions further includes instructions that, when executing on the processor, cause the multi-modal variational autoencoder to perform one or more linear operations in the unified latent space. ### Claim 7 (depends on 1) The system of claim 1, wherein the plurality of programming instructions further includes instructions that, when executing on the processor, cause the multi-modal variational autoencoder to utilize cross-modal attention mechanisms to capture relationships between different data types. ### Claim 8 (depends on 1) The system of claim 1, further comprising a multi-modal correlation network, and wherein the output of the decompressed output organizer is input to the multi-modal correlation network, and wherein the output of the multi-modal correlation network provides the reconstructed multi-modal output. ### Claim 9 A method for compressing and restoring multi-modal data, comprising steps of: preprocessing raw multi-modal data to generate a plurality of input data sets, wherein the multi-modal data comprises at least two different data types; compressing the plurality of input data sets into a plurality of compressed data sets using an encoder within a multi-modal variational autoencoder; decompressing the plurality of compressed data sets using a decoder located within the multi-modal variational autoencoder to obtain a plurality of reduced output data sets; and processing the plurality of reduced output data sets through a decompressed output organizer, thereby generating a reconstructed multi-modal output. ### Claim 10 (depends on 9) The method of claim 9, wherein the multi-modal data comprises at least two of: image data, audio data, text data, time-series data, and sensor data. ### Claim 11 (depends on 9) The method of claim 9, further comprising using the multi-modal variational autoencoder to perform partially homomorphic compression in a unified latent space representing all data modalities. ### Claim 12 (depends on 11) The method of claim 11, wherein the multi-modal variational autoencoder comprises a unified latent space, and further comprising using the multi-modal variational autoencoder to perform one or more linear operations in the unified latent space. ### Claim 13 (depends on 9) The method of claim 9, further comprising utilizing cross-modal attention mechanisms to capture relationships between different data types within the multi-modal variational autoencoder. ### Claim 14 (depends on 9) The method of claim 9, further comprising aligning and synchronizing the different data types during preprocessing and reconstruction. ### Claim 15 (depends on 9) The method of claim 9, further comprising training the multi-modal variational autoencoder using a loss function that accounts for reconstruction quality across all data modalities.