# US 2021/0241320 A1 · Automatic modeling of online learning propensity for target identification > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0241320 A1 - Document ID: US20210241320A1 - Status: Published application - Kind: A1 - Application number: 16/782,054 - Filed: 2020-02-04 - Published: 2021-08-05 - Earliest claimed: 2020-02-04 - Assignees: Microsoft Technology Licensing, LLC - Applicants: Microsoft Technology Licensing, LLC - Inventors: Sneha S. Chaudhari; Mahesh S. Joshi; Gungor Polatkan; Deepak Kumar Dileep Kumar - Web page: https://patentgrep.com/patent/US20210241320A1 ## CPC - G06Q 30/0269: {based on user profile or attribute} - G06F 9/542: {Event management; Broadcasting; Multicasting; Notifications} - G06N 5/04: Inference or reasoning models - G06N 20/00: Machine learning - G06N 20/20: Ensemble learning - G06Q 30/0204: Market segmentation - G06Q 30/0255: {based on user history} - G06Q 50/205: {Education administration or guidance} - G09B 5/00: Electrically-operated educational appliances (working with questions and answers G09B7/00; simulators G09B9/00; advertising or displaying in general G09F) - G06F 3/0484: for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range - G06Q 30/0246: {Traffic} - G09B 5/02: with visual presentation of the material to be studied, e.g. using film strip - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B) ## Abstract In an embodiment, the disclosed technologies include determining a digital identifier, computing, using aggregate digital event data obtained from at least one computing device, digital feature data relating to the digital identifier, inputting the digital feature data relating to the digital identifier into a digital model that has machine-learned correlations between digital feature data and digital propensity prediction values, receiving, from the digital model, a predicted propensity value associated with the digital identifier, determining a propensity score based on the predicted propensity value, causing a digital content item to be displayed on a user interface of a network-based software application associated with the digital identifier if the propensity score satisfies a propensity criterion. ## Claims ### Claim 1 A method comprising: determining a digital identifier associated with a user of a network-based software application; computing, using aggregate digital event data obtained from at least one computing device associated with the network-based software application, digital feature data relating to the digital identifier; inputting the digital feature data relating to the digital identifier into a digital model that has machine-learned correlations between digital feature data and digital propensity prediction values relating to use of the online learning service; receiving, from the digital model, a predicted propensity value associated with the digital identifier; determining a propensity score based on the predicted propensity value; causing a digital content item relating to an online learning service that is operably coupled to the network-based software application to be displayed on a user interface of the network-based software application associated with the digital identifier only if the propensity score satisfies a propensity criterion. ### Claim 2 (depends on 1) The method of claim 1, further comprising computing, as part of the digital feature data, a count of digital events associated with at least one second digital identifier, wherein a second digital identifier of the at least one second digital identifier is associated with another user of the network-based software application who has used the online learning service, and at least one item of profile data associated with the second digital identifier by the network-based software application matches at least one item of profile data associated with the digital identifier by the network-based software application. ### Claim 3 (depends on 2) The method of claim 2, further comprising computing the count of digital events comprises by computing a count of online courses digitally offered through the online learning service that have been digitally completed by at least one user associated with the at least one second digital identifier. ### Claim 4 (depends on 2) The method of claim 2, further comprising computing the count of digital events by computing a count of online courses digitally offered through the online learning service that have been digitally accessed by at least one user associated with the at least one second digital identifier. ### Claim 5 (depends on 1) The method of claim 1, further comprising determining the propensity score associated with the digital identifier by converting the predicted propensity value to a binary value. ### Claim 6 (depends on 5) The method of claim 5, further comprising programmatically determining that the predicted propensity value associated with the digital identifier is greater than predicted propensity values associated with at least a majority of other users of the network-based software application. ### Claim 7 (depends on 1) The method of claim 1, further comprising inputting digital training data into the digital model, wherein the digital training data comprises digital event data collected during at least two different types of online experiments, wherein the at least two different types of online experiments include at least one of a random targeting experiment and a contextual unlocking experiment. ### Claim 8 (depends on 1) The method of claim 1, further comprising programmatically validating the digital model using an A/B testing technique and training data sets that are not mutually exclusive, wherein the A/B testing technique has been configured for the training data sets that are not mutually exclusive. ### Claim 9 (depends on 1) The method of claim 1, further comprising programmatically adjusting a model parameter of the digital model by decreasing a number of trees model parameter value or by decreasing a tree depth model parameter value. ### Claim 10 (depends on 1) The method of claim 1, wherein the network-based software application comprises an online connection network. ### Claim 11 A system, comprising: at least one processor; and at least one storage media storing instructions that, when executed by the at least one processor, cause: determining a digital identifier associated with a user of a network-based software application; computing, using aggregate digital event data obtained from at least one computing device associated with the network-based software application, digital feature data relating to the digital identifier; inputting the digital feature data relating to the digital identifier into a digital model that has machine-learned correlations between digital feature data and digital propensity prediction values relating to use of an online learning service; receiving, from the digital model, a predicted propensity value associated with the digital identifier; determining a propensity score based on the predicted propensity value; causing a digital content item relating to the online learning service to be displayed on a user interface of the network-based software application associated with the digital identifier only if the propensity score satisfies a propensity criterion. ### Claim 12 (depends on 11) The system of claim 11, wherein the instructions, when executed by the at least one processor, further cause computing, as part of the digital feature data, a count of digital events associated with at least one second digital identifier, wherein a second digital identifier of the at least one second digital identifier is associated with another user of the network-based software application who has used the online learning service, and at least one item of profile data associated with the second digital identifier by the network-based software application matches at least one item of profile data associated with the digital identifier by the network-based software application. ### Claim 13 (depends on 12) The system of claim 12, wherein the instructions, when executed by the at least one processor, further cause computing the count of digital events by computing a count of online courses digitally offered through the online learning service that have been digitally completed by at least one user associated with the at least one second digital identifier. ### Claim 14 (depends on 12) The system of claim 12, wherein the instructions, when executed by the at least one processor, further cause computing the count of digital events by computing a count of online courses digitally offered through the online learning service that have been digitally accessed by at least one user associated with the at least one second digital identifier. ### Claim 15 (depends on 11) The system of claim 11, wherein the instructions, when executed by the at least one processor, further cause determining the propensity score associated with the digital identifier by converting the predicted propensity value to a binary value. ### Claim 16 (depends on 15) The system of claim 15, wherein the instructions, when executed by the at least one processor, further cause programmatically determining that the predicted propensity value associated with the digital identifier is greater than predicted propensity values associated with at least a majority of other users of the network-based software application. ### Claim 17 (depends on 11) The system of claim 11, wherein the instructions, when executed by the at least one processor, further cause inputting digital training data into the digital model, wherein the digital training data comprises digital event data collected during at least two different types of online experiments, wherein the at least two different types of online experiments include at least one of a random targeting experiment and a contextual unlocking experiment. ### Claim 18 (depends on 11) The system of claim 11, wherein the instructions, when executed by the at least one processor, further cause programmatically validating the digital model using an A/B testing technique and training data sets that are not mutually exclusive, wherein the A/B testing technique has been configured for the training data sets that are not mutually exclusive. ### Claim 19 (depends on 11) The system of claim 11, wherein the instructions, when executed by the at least one processor, further cause programmatically adjusting a model parameter of the digital model by decreasing a number of trees model parameter value or by decreasing a tree depth model parameter value. ### Claim 20 (depends on 11) The system of claim 11, wherein the network-based software application comprises an online connection network.