# US 12,762,599 B2 · System, method, and non-transitory computer readable medium for generating electronic communications using transformer-based sequential and real-time top in type models > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,762,599 B2 - Document ID: US12762599B2 - Status: Granted - Kind: B2 - Application number: 18/437,525 - Filed: 2024-02-09 - Published: 2025-08-14 - Granted: 2026-10-13 - Earliest claimed: 2024-02-09 - Assignees: Walmart Apollo, LLC - Applicants: Walmart Apollo, LLC - Inventors: Sayan Putatunda; Nisha Choudhary; Girish Thiruvenkadam; Rahul Ghosh; Swetha Neralur Ashwathnarayana; Suman Kumar - Web page: https://patentgrep.com/patent/US12762599B2 ## CPC - G06Q 30/0631: {Recommending goods or services} - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR ## Abstract Systems and methods for automatically generating electronic communications including relevant content elements are disclosed. A request to generate an electronic communication including a user identifier and a prior interaction is received. A hybrid sequential model is implemented to generate a set of hybrid sequential element recommendations selected from a catalog of items. The hybrid sequential model includes a sequential sub-model to select a set of sequential elements and a re-ranking sub-model to re-rank the sequential elements to generate the hybrid sequential elements. An affinity model is implemented to generate affinity element recommendations and a set of relevant content elements including at least one of the set of hybrid sequential element recommendations and at least one of the set of affinity element recommendations is generated. The electronic communication is generated including the set of relevant content elements and is transmitted to a user device associated with the user identifier. ## Claims ### Claim 1 A system, comprising: a non-transitory memory; a processor communicatively coupled to the non-transitory memory, wherein the processor is configured to read a set of instructions to: receive a request to generate an electronic communication, wherein the request includes a user identifier and a prior interaction; obtain historical interaction data associated with the user identifier; input at least a portion of the historical interaction data into a hybrid sequential model to generate a set of hybrid sequential element recommendations selected from a catalog of items, the hybrid sequential model comprising a sequential sub-model that includes one or more self-attentive transformer layers trained to select a set of sequential elements and a re-ranking sub-model that includes one or more affinity layers trained to re-rank the set of sequential elements based on an affinity score for the user identifier and the set of sequential elements to generate the set of hybrid sequential elements; input the prior interaction into an affinity model that includes one or more affinity layers trained to generate a set of affinity element recommendations; generate, using an intent model having one or more trained intent layers, a set of relevant content elements including at least one of the set of hybrid sequential element recommendations and at least one of the set of affinity element recommendations; generate the electronic communication to include (i) a first portion comprising interface elements that present information related to the prior interaction and (ii) a second portion comprising the set of relevant content elements that are simultaneously relevant to the first portion and to the user identifier; and transmit the electronic communication to a user device associated with the user identifier. ### Claim 2 (depends on 1) The system of claim 1, wherein the re-ranking sub-model comprises an affinity framework. ### Claim 3 (depends on 2) The system of claim 2, wherein the affinity framework comprises a first affinity framework, and wherein the affinity model comprises a second affinity framework. ### Claim 4 (depends on 3) The system of claim 3, wherein the first affinity framework and the second affinity framework comprise a single trained affinity model. ### Claim 5 (depends on 1) The system of claim 1, wherein the electronic communication is generated, in part, from an electronic communication template. ### Claim 6 (depends on 1) The system of claim 1, wherein each content element of the second portion is encoded as a programmatically selected navigational link to a corresponding interface page of a website to enable bypassing multi-step browsing or search navigation. ### Claim 7 (depends on 1) The system of claim 1, wherein the one or more self-attentive transformer layers comprise an embedding layer to convert at least a portion of the historical interaction data into one or more vector representations projected into a relevant vector space. ### Claim 8 (depends on 7) The system of claim 7, wherein the embedding layer utilizes a text embedding generation process to convert textual elements of the historical interaction data into vectors in the relevant vector space. ### Claim 9 (depends on 1) The system of claim 1, wherein the one or more self-attentive transformer layers further comprise a point-wise feedforward network to receive an output of a self-attention layer and generate an output for use by one or more prediction layers. ### Claim 10 (depends on 1) The system of claim 1, wherein the one or more affinity layers of the re-ranking sub-model comprise a trained multi-label classifier to generate a ranking of the set of sequential elements. ### Claim 11 (depends on 1) The system of claim 1, wherein the intent model generates an intent score for each element in the set of hybrid sequential element recommendations and the set of affinity element recommendations, and wherein the set of relevant content elements comprises elements having a highest intent score in each respective set. ### Claim 12 (depends on 1) The system of claim 1, wherein the affinity model comprises a top in type model to generate a set of top trending elements based on a determined user affinity for a given element type. ### Claim 13 (depends on 1) The system of claim 1, wherein the processor is further configured to receive feedback data representative of one or more interactions with the electronic communication and generate an updated machine learning model based at least in part on the feedback data. ### Claim 14 A computer-implemented method, comprising: receiving a request to generate an electronic communication, wherein the request includes a user identifier and a prior interaction; obtaining historical interaction data associated with the user identifier; inputting at least a portion of the historical interaction data into a hybrid sequential model to generate a set of hybrid sequential element recommendations selected from a catalog of items, the hybrid sequential model comprising a sequential sub-model that includes one or more self-attentive transformer layers trained to select a set of sequential elements and a re-ranking sub-model that includes one or more affinity layers trained to re-rank the set of sequential elements based on an affinity score for the user identifier and the set of sequential elements to generate the set of hybrid sequential elements; inputting the prior interaction into an affinity model that includes one or more affinity layers trained to generate a set of affinity element recommendations; generating, using an intent model having one or more trained intent layers, a set of relevant content elements including at least one of the set of hybrid sequential element recommendations and at least one of the set of affinity element recommendations; generating the electronic communication to include (i) a first portion comprising interface elements that present information related to the prior interaction and (ii) a second portion comprising the set of relevant content elements that are simultaneously relevant to the first portion and to the user identifier; and transmitting the electronic communication to a user device associated with the user identifier. ### Claim 15 (depends on 14) The computer-implemented method of claim 14, wherein the re-ranking sub-model comprises an affinity framework. ### Claim 16 (depends on 15) The computer-implemented method of claim 15, wherein the affinity framework comprises a first affinity framework, and wherein the affinity model comprises a second affinity framework. ### Claim 17 (depends on 16) The computer-implemented method of claim 16, wherein the first affinity framework and the second affinity framework comprise a single trained affinity model. ### Claim 18 (depends on 14) The computer-implemented method of claim 14, wherein the electronic communication is generated, in part, from an electronic communication template. ### Claim 19 A non-transitory computer readable medium having instructions stored thereon, wherein the instructions, when executed by at least one processor, cause at least one device to perform operations comprising: receiving a request to generate an electronic communication, wherein the request includes a user identifier and a prior interaction; obtaining historical interaction data associated with the user identifier; inputting at least a portion of the historical interaction data into a hybrid sequential model to generate a set of hybrid sequential element recommendations selected from a catalog of items, the hybrid sequential model comprising a sequential sub-model that includes one or more self-attentive transformer layers trained to select a set of sequential elements and a re-ranking sub-model that includes one or more affinity layers trained to re-rank the set of sequential elements based on an affinity score for the user identifier and the set of sequential elements to generate the set of hybrid sequential elements; inputting the prior interaction into an affinity model that includes one or more affinity layers trained to generate a set of affinity element recommendations; generating, using an intent model having one or more trained intent layers, a set of relevant content elements including at least one of the set of hybrid sequential element recommendations and at least one of the set of affinity element recommendations; generating the electronic communication to include (i) a first portion comprising interface elements that present information related to the prior interaction and (ii) a second portion comprising the set of relevant content elements that are simultaneously relevant to the first portion and to the user identifier; and transmitting the electronic communication to a user device associated with the user identifier. ### Claim 20 (depends on 19) The non-transitory computer readable medium of claim 19, wherein the electronic communication is generated, in part, from an electronic communication template.