Generating embeddings and extracting content attributes from long documents using artificial intelligence
AssigneeHulu, LLC
InventorsVahidreza Arbab, Tuo Li, Yavuz Sunor
Life of this patent
Estimating the term…
Abstract
A method includes determining embeddings in an embedding space for segments of a plurality of documents. A cluster is determined for respective segments based on a set of clusters. The cluster is determined based on a position of respective embeddings in the embedding space. The method determines a weight for the cluster for respective embeddings. The respective embeddings are weighted for a document in the plurality of documents using the weight of the cluster for the respective embeddings to generate weighted embeddings. A set of attributes from the weighted embeddings is determined for the document.
Claims20
A method comprising:
- determining embeddings in an embedding space for segments of a plurality of documents;
- determining a cluster for respective segments based on a set of clusters, wherein the cluster is determined based on a position of respective embeddings in the embedding space;
- determining a weight for the cluster for respective embeddings;
- weighting the respective embeddings for a document in the plurality of documents using the weight of the cluster for the respective embeddings to generate weighted embeddings;
- and
- determining a set of attributes from the weighted embeddings for the document.
A non-transitory computer-readable storage medium having stored thereon computer executable instructions, which when executed by a computing device, cause the computing device to be operable for:
- determining embeddings in an embedding space for segments of a plurality of documents;
- determining a cluster for respective segments based on a set of clusters, wherein the cluster is determined based on a position of respective embeddings in the embedding space;
- determining a weight for the cluster for respective embeddings;
- weighting the respective embeddings for a document in the plurality of documents using the weight of the cluster for the respective embeddings to generate weighted embeddings;
- and
- determining a set of attributes from the weighted embeddings for the document.
An apparatus comprising:
- one or more computer processors;
- and
- a computer-readable storage medium comprising instructions for controlling the one or more computer processors to be operable for:
- determining embeddings in an embedding space for segments of a plurality of documents;
- determining a cluster for respective segments based on a set of clusters, wherein the cluster is determined based on a position of respective embeddings in the embedding space;
- determining a weight for the cluster for respective embeddings;
- weighting the respective embeddings for a document in the plurality of documents using the weight of the cluster for the respective embeddings to generate weighted embeddings;
- and
- determining a set of attributes from the weighted embeddings for the document.
Dependencies are inferred from the claim text.