# US 2014/0156566 A1 · Customized predictors for user actions in an online system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2014/0156566 A1 - Document ID: US20140156566A1 - Status: Published application - Kind: A1 - Application number: 13/689,969 - Filed: 2012-11-30 - Published: 2014-06-05 - Earliest claimed: 2012-11-30 - Assignees: Facebook, Inc. - Applicants: Facebook, Inc. - Inventors: Igor Kabiljo; Aleksandar Ilic; Ming Hua; Hong Yan - Web page: https://patentgrep.com/patent/US20140156566A1 ## CPC - G06N 5/04: Inference or reasoning models - G06N 20/00: Machine learning - G06Q 10/40: Business processes related to social networking or social networking services - G06Q 30/02: Marketing; Price estimation or determination; Fundraising - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR ## Abstract Online systems generate predictors for predicting actions of users of the online system. The online system receives requests to generate predictor models for predicting whether a user is likely to take an action of a particular action type. The request specifies the type of action and criteria for identifying a successful instance of the action type and a failure instance of the action type. The online system collects data including successful and failure instances of the action type. The online system generates one or more predictors of different types using the generated data. The online system evaluates and compares the performance of the different predictors generated and selects a predictor based on the performance. The online system returns a handle to access the generated predictor to the requester of the predictor. ## Claims ### Claim 1 A computer-implemented method comprising: receiving, by an online system, a plurality of requests for generating predictors, each request specifying a type of action performed by users of the online system and criteria for identifying a successful instance of the type of action; for each request associated with a type of action, generating, by the online system, a predictor for the type of action, the generating comprising: identifying a training data set comprising a plurality of successful instances and a plurality of failure instances of the type of action performed by users of the online system; training one or more candidate predictors using the identified data set, wherein each candidate predictor comprises a machine learned model; evaluating performance of each candidate predictor by executing each predictor on a test data set comprising a plurality of successful instances and a plurality of failure instances of the type of action performed by users of the online system; and selecting a predictor for the type of action by comparing the performance of the one or more candidate predictors; and returning a handle corresponding to the generated predictor to a requester of the selected predictor. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, further comprising: receiving a request for a predictor from a process running in the online system, the request specifying a handle and an instance of user information; executing the predictor corresponding to the specified handle on the instance of user information; and returning a result of the predictor to the requesting process. ### Claim 3 (depends on 1) The computer-implemented method of claim 1, wherein a successful instance of a type of action corresponds to information stored in the online system indicating a user taking an action of the type responsive to the user being presented with a user interface that allows users to take actions of the type of action. ### Claim 4 (depends on 1) The computer-implemented method of claim 1, wherein the request further specifies criteria for failure instance, wherein a failure instance of a type of action corresponds to information stored in the online system indicating a user's failure to take an action of the type responsive to the user being presented with information that allows the user to take action of the type of action. ### Claim 5 (depends on 4) The computer-implemented method of claim 4, wherein a failure instance of a type of action corresponds to information stored in the online system indicating a user providing input indicating a dislike for information presented in a user interface associated with the type of action. ### Claim 6 (depends on 1) The computer-implemented method of claim 1, wherein each type of action is targeted to an entity represented in the online system. ### Claim 7 (depends on 6) The computer-implemented method of claim 6, wherein the entity corresponding to the type of action is one of a user of the online system, a social group, or an image. ### Claim 8 (depends on 1) The computer-implemented method of claim 1, wherein a candidate predictor is a machine learning model of a type based on one of a linear regression, a logistic regression, boosting tree, or weighted decision tree. ### Claim 9 (depends on 1) The computer-implemented method of claim 1, further comprising, for each generated predictor for a type of action: comparing performance of the generated predictor against performance of an existing predictor of the type of action; and responsive to the performance of the generated predictor exceeding the performance of the existing predictor, replacing the existing predictor with the generated predictor. ### Claim 10 (depends on 9) The computer-implemented method of claim 9, further comprising, for each type of action, periodically generating a new predictor: comparing performance of the new predictor against performance of a previously generated predictor; and responsive to the performance of the new predictor exceeding the performance of the previously generated predictor, replacing the previously generated predictor with the new predictor. ### Claim 11 (depends on 10) The computer-implemented method of claim 10, wherein a rate of periodically generating new predictors for each action type is determined based on a comparison of performance of the new predictors with performance of previously generated predictors. ### Claim 12 (depends on 11) The computer-implemented method of claim 11, wherein the rate of periodically generating new predictors for each action type is reduced responsive to an improvement in performance of the new predictor compared to the performance of the existing predictor being below a threshold value. ### Claim 13 A computer-implemented method comprising: storing, by an online system, a plurality of predictors, each predictor for predicting whether a user of the online system performs a type of action, each predictor associated with criteria for determining a successful instance of the type of action; selecting an existing predictor corresponding to a type of action; identifying a data set comprising successful instances and failure instances of the type of action performed by users of the online system; training one or more candidate predictors using the identified data set, wherein each candidate predictor comprises a machine learned model; evaluating performance of each candidate predictor by executing each predictor on a test data set comprising a plurality of successful instances and a plurality of failure instances of the type of action performed by users of the online system; selecting a candidate predictor from the one or more candidate predictors by comparing the performance of the one or more candidate predictors; comparing performance of the selected candidate predictor with performance of the existing predictors; and replacing the existing predictor by the selected candidate predictor responsive to performance of the selected candidate predictor exceeding the performance of the existing predictor. ### Claim 14 (depends on 13) The computer-implemented method of claim 13, wherein a successful instance of a type of action corresponds to information stored in the online system indicating a user taking an action of the type responsive to the user being presented with a user interface that allows users to take actions of the type of action. ### Claim 15 (depends on 13) The computer-implemented method of claim 13, wherein a failure instance of a type of action corresponds to information stored in the online system indicating a user failing to take an action of the type responsive to the user being presented with a user interface that allows users to take actions of the type of action. ### Claim 16 (depends on 13) The computer-implemented method of claim 13, wherein a candidate predictor is a machine learning model of a type based on one of a linear regression, a logistic regression, boosting tree, or weighted decision tree. ### Claim 17 (depends on 13) The computer-implemented method of claim 13, further comprising, periodically generating a new predictor: comparing performance of the new predictor against performance of a previously generated predictor; and responsive to the performance of the new predictor exceeding the performance of the previously generated predictor, replacing the previously generated predictor with the new predictor. ### Claim 18 (depends on 17) The computer-implemented method of claim 17, wherein a rate of periodically generating new predictors for each action type is reduced responsive to an improvement in performance of the new predictor compared to the performance of the existing predictor being below a threshold value. ### Claim 19 A computer program product having a non-transitory computer-readable storage medium storing computer-executable code, the code comprising: a predictor factory module of an online system configured to: store a plurality of predictors, each predictor for predicting whether a user of the online system performs a type of action, each predictor associated with criteria for determining a successful instance of the type of action; select an existing predictor corresponding to a type of action; identify a data set comprising successful instances and failure instances of the type of action performed by users of the online system; train one or more candidate predictors using the identified data set, wherein each candidate predictor comprises a machine learned model; evaluate performance of each candidate predictor by executing each predictor on a test data set comprising a plurality of successful instances and a plurality of failure instances of the type of action performed by users of the online system; select a candidate predictor from the one or more candidate predictors by comparing the performance of the one or more candidate predictors; compare performance of the selected candidate predictor with performance of the existing predictors; and replace the existing predictor by the selected candidate predictor responsive to performance of the selected candidate predictor exceeding the performance of the existing predictor. ### Claim 20 (depends on 19) The computer program product of claim 19, wherein a successful instance of a type of action corresponds to information stored in the online system indicating a user taking an action of the type responsive to the user being presented with a user interface that allows users to take actions of the type of action. ### Claim 21 (depends on 19) The computer program product of claim 19, wherein a failure instance of a type of action corresponds to information stored in the online system indicating a user failing to take an action of the type responsive to the user being presented with a user interface that allows users to take actions of the type of action.