Generating three-dimensional gaussian reconstructions utilizing a long-sequence large reconstruction model

A system comprising:

• one or more memory devices;
• and
• one or more processors coupled to the one or more memory devices that cause the system to perform operations comprising:
• determining an initial set of token sequences representing a set of digital images depicting a set of views of a three-dimensional scene according to corresponding camera pose parameters;
• generating a transformed set of token sequences by merging one or more tokens from the initial set of token sequences;
• generating, utilizing a set of parallel state space blocks and a transformer block in a hybrid feed-forward neural network, a set of output token sequences from the transformed set of token sequences;
• and
• generating, from the set of output token sequences, a three-dimensional Gaussian reconstruction of the three-dimensional scene for synthesizing digital images depicting the three-dimensional scene with novel views.

A non-transitory computer-readable medium storing instructions thereon that, when executed by at least one processor, cause the at least one processor to perform operations comprising:

• determining an initial set of token sequences representing a set of digital images depicting a set of views of a three-dimensional scene according to corresponding camera pose parameters;
• generating, utilizing a set of parallel state space blocks in a hybrid feed-forward neural network, a set of modified token sequences from the initial set of token sequences by mixing token information in the initial set of token sequences;
• generating, utilizing a transformer block in the hybrid feed-forward neural network, a set of output token sequences from the set of modified token sequences;
• and
• generating, from the set of output token sequences, a three-dimensional Gaussian reconstruction of the three-dimensional scene for synthesizing digital images depicting the three-dimensional scene with novel views.