Multi-modal hierarchical tokenization deep neural network
AssigneeD. E. Shaw Research, LLC
InventorsPaul Maragakis, Andreas Kraemer, James P. Roney, Peter Skopp
Life of this patent
Estimating the term…
Loading patent…
AssigneeD. E. Shaw Research, LLC
InventorsPaul Maragakis, Andreas Kraemer, James P. Roney, Peter Skopp
Estimating the term…
A system is disclosed for encoding a data string of a first modality into a hierarchical tokenized representation for processing by a text-based deep neural network (DNN) trained on a second modality. The data string comprises multiple units, each having one or more attributes. Each attribute is represented in the tokenized string as a sequence of hierarchical tokens, with a first hierarchical token encoding one or more most significant bits and a subsequent hierarchical token encoding one or more less significant bits. The DNN processes the data string bidirectionally, across the sequence of units and within the token hierarchy, to select tokens that capture attribute information. The selected hierarchical tokens output by the DNN from a representation of the original data string that preserves attribute detail while enabling cross-modal processing using models trained on text.
A computer-implemented method, comprising:
A computer-implemented method comprising:
Dependencies are inferred from the claim text.