Recommender systems and methods using cascaded machine learning models

A computing system for providing personalized recommendations to a user of items available, the computing system comprising:

• a processor;
• at least one memory device that is accessible by the processor;
• and
• a communications interface that is accessible by the processor,
• wherein the at least one memory device contains program instructions which, when executed by the processor, cause the computing system to implement a method comprising:
• training a first-level machine learning model based on a first recommendation relationship process for each iteration of a first plurality of iterations by:
• obtaining feature information corresponding with past recommendations for a plurality of users;
• obtaining historical data comprising logs of the past recommendations and corresponding user actions for the plurality of users;
• evaluating the feature information and the historical data corresponding to one or more past recommendations associated with at least one traveler;
• counting a number of times a characteristic of the at least one traveler is associated with a certain recommendation;
• and
• selecting, as an output of the first-level machine learning model, one or more recommendation relationship features for inclusion in machine-learned recommendation lists using a pre-defined statistical threshold that defines a minimal causal relationship that exists between a particular recommendation relationship feature and patterns before that particular recommendation relationship feature is included in one or more of the machine-learned recommendation lists;
• training a second-level machine learning model based on a first recommendation relationship process for each iteration of a first plurality of iterations by:
• obtaining the feature information and the historical data;
• obtaining the output of the first-level machine learning model;
• evaluating the feature information and the historical data corresponding to one or more recommendations associated with at least one traveler based on the output of the first-level machine learning model;
• counting a number of times a characteristic of the at least one traveler is associated with a certain recommendation;
• and
• selecting, as an output of the second-level machine learning model and based on the output of the first-level machine learning model, one or more recommendation relationship features for updating the machine-learned recommendation lists using a pre-defined statistical threshold that defines a minimal causal relationship that exists between a particular recommendation relationship feature and patterns before that particular recommendation relationship feature is included in the one or more machine-learned recommendation lists;
• receiving, via the communications interface, context data comprising user information;
• computing a plurality of first-level features comprising context features based upon the context data;
• generating, by the first-level machine learning model using the first-level features, predictions of conversion rates in relation to a plurality of individual items that are available;
• constructing a first list of proposed item recommendations of a plurality of lists of proposed item recommendations based upon the predictions generated by the first-level machine learning model;
• determining a personalized list of item recommendations based on iteratively evaluating one or more of the plurality of lists of proposed item recommendations by the second-level machine learning model, wherein each iteration comprises:
• generating a second list of proposed item recommendations of the plurality of lists of proposed item recommendations by swapping a proposed item recommendation in the first list of proposed item recommendations based upon the predictions generated by the first-level machine learning model;
• computing a plurality of second-level features comprising context features based upon the context data and list features based upon the second list of proposed item recommendations and the corresponding predictions generated by the first-level machine learning model;
• generating, by the second-level machine learning model using the second-level features, a prediction of conversion rates in relation to the second list of proposed item recommendations;
• and
• determining the personalized list of item recommendations based upon the prediction generated by the second-level machine learning model by:
• comparing the predicted conversion rates generated on a current iteration in relation to the second list of proposed item recommendations with a predicted conversion rate generated on a prior iteration, and
• in an event that the comparison determines that the predicted conversion rates generated on the current iteration is greater than the predicted conversion rate generated on the prior iteration, replacing the first list of proposed item recommendations with the second list of proposed item recommendations;
• and
• providing, via the communications interface, the personalized list of item recommendations.

A computer program product comprising a computer-readable medium having instructions stored thereon which, when executed by a processor implement a method of providing personalized recommendations to a user of items available the method comprising:

• at a system comprising a processor:
• training a first-level machine learning model based on a first recommendation relationship process for each iteration of a first plurality of iterations by:
• obtaining feature information corresponding with past recommendations for a plurality of users;
• obtaining historical data comprising logs of the past recommendations and corresponding user actions for the plurality of users;
• evaluating the feature information and the historical data corresponding to one or more past recommendations associated with at least one traveler;
• counting a number of times a characteristic of the at least one traveler is associated with a certain recommendation;
• and
• selecting, as an output of the first-level machine learning model, one or more recommendation relationship features for inclusion in machine-learned recommendation lists using a pre-defined statistical threshold that defines a minimal causal relationship that exists between a particular recommendation relationship feature and patterns before that particular recommendation relationship feature is included in one or more of the machine-learned recommendation lists;
• training a second-level machine learning model based on a first recommendation relationship process for each iteration of a first plurality of iterations by:
• obtaining the feature information and the historical data;
• obtaining the output of the first-level machine learning model;
• evaluating the feature information and the historical data corresponding to one or more recommendations associated with at least one traveler based on the output of the first-level machine learning model;
• counting a number of times a characteristic of the at least one traveler is associated with a certain recommendation;
• and
• selecting, as an output of the second-level machine learning model and based on the output of the first-level machine learning model, one or more recommendation relationship features for updating the machine-learned recommendation lists using a pre-defined statistical threshold that defines a minimal causal relationship that exists between a particular recommendation relationship feature and patterns before that particular recommendation relationship feature is included in the one or more machine-learned recommendation lists;
• receiving, via a communications channel, context data comprising user information;
• computing a plurality of first-level features comprising context features based upon the context data;
• generating, by the first-level machine learning model using the first-level features, predictions of conversion rates in relation to a plurality of individual items that are available;
• constructing a first list of proposed item recommendations of a plurality of lists of proposed item recommendations based upon the predictions generated by the first-level machine learning model;
• determining a personalized list of item recommendations based on iteratively evaluating one or more of the plurality of lists of proposed item recommendations by the second-level machine learning model, wherein each iteration comprises:
• generating a second list of proposed item recommendations of the plurality of lists of proposed item recommendations by swapping a proposed item recommendation in the first list of proposed item recommendations based upon the predictions generated by the first-level machine learning model;
• computing a plurality of second-level features comprising context features based upon the context data and list features based upon the second list of proposed item recommendations and the corresponding predictions generated by the first-level machine learning model;
• generating, by the second-level machine learning model using the second-level features, a prediction of conversion rates in relation to the second list of proposed item recommendations;
• and
• determining the personalized list of item recommendations based upon the prediction generated by the second-level machine learning model by:
• comparing the predicted conversion rates generated on a current iteration in relation to the second list of proposed item recommendations with a predicted conversion rate generated on a prior iteration, and
• in an event that the comparison determines that the predicted conversion rates generated on the current iteration is greater than the predicted conversion rate generated on the prior iteration, replacing the first list of proposed item recommendations with the second list of proposed item recommendations;
• and
• providing, via the communications channel, the personalized list of item recommendations.