# US 2021/0241907 A1 · Multi-model member outreach system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0241907 A1 - Document ID: US20210241907A1 - Status: Published application - Kind: A1 - Application number: 16/881,981 - Filed: 2020-05-22 - Published: 2021-08-05 - Earliest claimed: 2020-02-03 - Assignees: CollectiveHealth, Inc. - Applicants: CollectiveHealth, Inc. - Inventors: Sanjay Basu - Web page: https://patentgrep.com/patent/US20210241907A1 ## CPC - G16H 50/30: for calculating health indices; for individual health risk assessment - G06N 3/0499: Feedforward networks - G06N 3/08: Learning methods - G06N 3/09: Supervised learning - G06N 3/0985: Hyperparameter optimisation; Meta-learning; Learning-to-learn - G06N 20/20: Ensemble learning - G06Q 10/10: Office automation; Time management - G06Q 30/0203: Market surveys; Market polls - G06Q 30/0205: {based on location or geographical consideration} - G06Q 30/0206: {Price or cost determination based on market factors} - G06Q 40/08: Insurance - G16H 10/60: for patient-specific data, e.g. for electronic patient records - G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems - G06N 3/044: Recurrent networks, e.g. Hopfield networks - G06N 3/045: Combinations of networks - G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR ## Abstract Member outreach services to be offered to members of a health plan based on predictions generated by a prediction engine using data associated with the members. The prediction engine can include an ensemble of different base predictive models, and the prediction engine can use a weighted combination of base predictions produced by the different base predictive models to generate a final prediction associated with a member. A representative can attempt to contact the member based on an outreach ticket associated with the final prediction. The representative can also provide ticket feedback associated with the outreach, and the prediction engine can use the ticket feedback to adjust the weights associated with the different base predictive models. ## Claims ### Claim 1 A computer-implemented method, comprising: training, by one or more processors, a set of base predictive models, wherein individual ones of the set of base predictive models are machine learning models and the training is based on historical healthcare data; generating, by the one or more processors and using the set of base predictive models, a set of base predictions associated with a member of a health plan based on healthcare data associated with the member, wherein: different base predictive models of the set of base predictive models generate different base predictions of the set of base predictions, the set of base predictions indicate at least one of: predicted healthcare costs associated with the member, predicted healthcare utilizations associated with the member, or predicted risk metrics associated with the member, and the different base predictions indicate one or more outreach priority levels associated with the member; generating, by the one or more processors, a final prediction associated with the member by combining the set of base predictions based at least in part on a set of weights that correspond to the set of base predictive models; generating, by the one or more processors, an outreach ticket associated with the member based on the final prediction; causing, by the one or more processors, the outreach ticket to be sent to a computing device operable by a representative; receiving, by the one or more processors, ticket feedback associated with the outreach ticket from the computing device, wherein the ticket feedback indicates at least one score, subjectively assigned by the representative, including one or more of: an outreach intervenability score of the member, or an interpretability score of the outreach ticket; and adjusting, by the one or more processors, the set of weights that correspond to the set of base predictive models, based on at least one comparison between the at least one score indicated in the ticket feedback and the one or more outreach priority levels indicated by the different base predictions generated by the set of base predictive models. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, wherein the healthcare data includes enrollment data associated with the member, claim data associated with the member, and at least one of demographic data for a geographic location associated with the member or engagement data associated with a use by the member of a website or mobile application associated with the health plan. ### Claim 3 (depends on 1) The computer-implemented method of claim 1, wherein at least one of the historical healthcare data and the healthcare data associated with the member is stored in a healthcare data repository indexed at one or more of a member level and a provider level. ### Claim 4 (depends on 1) The computer-implemented method of claim 1, wherein the set of base predictive models includes at least two types of base predictive models. ### Claim 5 (depends on 4) The computer-implemented method of claim 4, wherein the at least two types of base predictive models include two or more of regularized linear models, Gradient Boosted Machine Models, Random Forest Models, or deep learning models. ### Claim 6 (depends on 1) The computer-implemented method of claim 1, wherein adjusting the set of weights is performed by a Gradient Boosted Machine meta-learner. ### Claim 7 (canceled) ### Claim 8 (canceled) ### Claim 9 (depends on 1) The computer-implemented method of claim 1, wherein the final prediction indicates at least one of a total cost estimate for the member over a period of time, an estimated change in costs for the member over the period of time, a utilization value indicating an estimated number of visits to one or more healthcare providers over the period of time, or a predicted change to one or more risk metrics associated with the member. ### Claim 10 (depends on 9) The computer-implemented method of claim 9, further comprising: determining, by the one or more processors, that the total cost estimate, the estimated change in costs, the utilization value, or the predicted change to the one or more risk metrics exceeds a threshold value; and generating, by the one or more processors, the outreach ticket based on determining that the total cost estimate, the estimated change in costs, the utilization value, or the predicted change to the one or more risk metrics exceeds the threshold value. ### Claim 11 (depends on 1) The computer-implemented method of claim 1, wherein the outreach ticket is provided to the representative in a queue of outreach tickets ordered based on outreach priorities determined based on the final prediction and corresponding final predictions of other outreach tickets in the queue. ### Claim 12 (depends on 1) The computer-implemented method of claim 1, wherein the outreach ticket identifies one or more attributes of the member, based on key variables identified as being predictive in the set of base predictions or the final prediction. ### Claim 13 (depends on 12) The computer-implemented method of claim 12, wherein generating the outreach ticket comprises selecting, by the one or more processors, values for an outreach ticket template based on the key variables. ### Claim 14 A system, comprising: one or more processors; and memory storing a healthcare data repository of data including historical healthcare data and healthcare data associated with a member of a health plan, the memory further storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising: training a set of base predictive models, wherein individual ones of the set of base predictive models are machine learning models and the training is based on the historical healthcare data; generating, using the set of base predictive models, a set of base predictions associated with the member based on the healthcare data associated with the member, wherein: different base predictive models of the set of base predictive models generate different base predictions of the set of base predictions, the set of base predictions indicate at least one of: predicted healthcare costs associated with the member, predicted healthcare utilizations associated with the member, or predicted risk metrics associated with the member, and the different base predictions indicate one or more outreach priority levels associated with the member; generating a final prediction associated with the member by combining the set of base predictions based at least in part on a set of weights that correspond to the set of base predictive models; generating an outreach ticket associated with the member based on the final prediction; providing the outreach ticket to a computing device operable by a representative; receiving ticket feedback associated with the outreach ticket from the computing device, wherein the ticket feedback indicates at least one score, subjectively assigned by the representative, including one or more of; an outreach intervenability score of the member, or an interpretability score of the outreach ticket; and adjusting the set of weights that correspond to the set of base predictive models, based on at least one comparison between the at least one score indicated in the ticket feedback and the one or more outreach priority levels indicated by the different base predictions generated by the set of base predictive models. ### Claim 15 (depends on 14) The system of claim 14, wherein the healthcare data associated with the member includes enrollment data associated with the member, claim data associated with the member, and at least one of demographic data for a geographic location associated with the member or engagement data associated with a use by the member of a website or mobile application associated with the health plan. ### Claim 16 (canceled) ### Claim 17 (depends on 14) The system of claim 14, wherein the outreach ticket identifies one or more attributes of the member, based on key variables identified as being predictive in the set of base predictions or the final prediction. ### Claim 18 One or more non-transitory computer-readable media storing computer-executable instructions that, when executed by one or more processors of at least one computing device, cause the at least one computing device to perform operations comprising: training a set of base predictive models, wherein individual ones of the set of base predictive models are machine learning models and the training is based on historical healthcare data; generating a set of base predictions associated with a member of a health plan using the set of base predictive models, wherein; the set of base predictive models generate the set of base predictions using healthcare data associated with the member, the set of base predictions indicate at least one of: predicted healthcare costs associated with the member, predicted healthcare utilizations associated with the member, or predicted risk metrics associated with the member, and individual base predictions, of the set of base predictions, indicate corresponding outreach priority levels associated with the member; generating a final prediction associated with the member by combining the set of base predictions, based at least in part on a set of weights that correspond to the set of base predictive models, wherein the set of weights are determined based at least in part on comparisons between: subjective representative feedback associated with past member outreach services offered by representatives to other members, the subjective representative feedback including outreach intervenability scores associated with the past member outreach services offered to the other members, and previous outreach priority levels associated with previous base predictions, associated with the other members, generated by the set of base predictive models; and generating a notification associated with the member based on the final prediction. ### Claim 19 (depends on 18) The one or more non-transitory computer-readable media of claim 18, wherein the notification is an outreach ticket, and the operations further comprise: causing the outreach ticket to be sent to a representative computing device operable by a representative; receiving new subjective representative feedback associated with the outreach ticket from the representative computing device; and adjusting the set of weights based on at least one comparison between a new outreach intervenability score indicated in the new subjective representative feedback and the outreach priority levels indicated by the individual base predictions generated by the set of base predictive models. ### Claim 20 (depends on 18) The one or more non-transitory computer-readable media of claim 18, wherein the notification is an automated notification, and the operations further comprise sending the automated notification to a computing device associated with the member. ### Claim 21 (depends on 1) The computer-implemented method of claim 1, wherein the different base predictive models are trained on different subsets of the historical healthcare data. ### Claim 22 (depends on 14) The system of claim 14, wherein the set of base predictive models includes at least two types of base predictive models. ### Claim 23 (depends on 18) The one or more non-transitory computer-readable media of claim 18, wherein the set of base predictive models includes at least two types of base predictive models.