# US 11,113,289 B2 · Blending learning models for search support > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,113,289 B2 - Document ID: US11113289B2 - Status: Granted - Kind: B2 - Application number: 15/648,374 - Filed: 2017-07-12 - Published: 2018-05-03 - Granted: 2021-09-07 - Earliest claimed: 2016-10-28 - Assignees: Apple Inc. - Applicants: Apple Inc. - Inventors: Hon Yuk Chan; John M. Hörnkvist; Lun Cui; Vipul Ved Prakash; Anubhav Malhotra; Stanley N. Hung; Julien Freudiger - Web page: https://patentgrep.com/patent/US11113289B2 ## CPC - G06F 16/24578: {using ranking} - G06N 3/08: Learning methods - G06F 16/248: Presentation of query results - G06F 16/335: Filtering based on additional data, e.g. user or group profiles (filtering in web context G06F16/9535, G06F16/9536) - G06F 16/951: Indexing; Web crawling techniques - G06F 16/9535: Search customisation based on user profiles and personalisation - G06F 16/9538: Presentation of query results - G06N 3/045: Combinations of networks - G06N 3/09: Supervised learning - G06N 20/00: Machine learning - G06N 20/20: Ensemble learning - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract A method and apparatus of a device that generates a re-ranking model used to re-rank a plurality of search results on a client device is described. In an exemplary embodiment, the device receives a crowd-sourced intra-domain model from a server, where the intra-domain model is a search result re-ranking model generated based on at least device interactions of a plurality of users interacting with a plurality of other devices. The device further generates a re-ranking model from the crowd-sourced intra-domain model and a local model, where the local model includes private data representing a device user's interaction with that device and the re-ranking model is used to re-rank a plurality of search results. ## Claims ### Claim 1 A non-transitory machine-readable medium having executable instructions to cause one or more processing units to perform a method to generate a re-ranking model used to re-rank a plurality of search results on a client device, the method comprising: sending a set of features to the server, wherein the set of features is anonymized data that includes a type of an interaction with the client device by a user of the client device and a discovery of one or more of the features will not lead to a discovery of which device contributed the set of features; receiving a crowd-sourced intra-domain model from a server, wherein the intra-domain model is a search result re-ranking model generated based on at least device interactions of a plurality of users interacting with a plurality of other devices; and generating, by the client device, a merged re-ranking model from the crowd-sourced intra-domain model and a local model, wherein the local model includes private data representing a device user's interaction with that device and the merged re-ranking model is used to re-rank the plurality of search results retrieved from a search domain that is one of a plurality of search domains using a search query, the search domain corresponding to the crowd-sourced intra-domain model. ### Claim 2 (depends on 1) The machine-readable medium of claim 1, wherein there are a plurality of intra-domain models corresponding to a plurality of search domains that are used to generate the plurality of search results. ### Claim 3 (depends on 2) The machine-readable medium of claim 2, further comprising: receiving a crowd-sourced inter-domain model from the server, wherein the crowd-sourced inter-domain model indicates which of the plurality of search domains are to be ranked higher than others in the plurality of search domains; and further generating the merged re-ranking model from the crowd-sourced inter-domain model and the local model and the crowd-sourced inter-domain model is used to blend the plurality of search results from the plurality of search domains. ### Claim 4 (depends on 1) The machine-readable medium of claim 1, wherein a choice of the crowd-sourced intra-domain model is based on at least a local domain model associated with the client device. ### Claim 5 (depends on 1) The machine-readable medium of claim 1, wherein the set of features does not reveal private data about the client device user. ### Claim 6 (depends on 1) The machine-readable medium of claim 1, wherein the merged re-ranking model is a neural network model. ### Claim 7 (depends on 6) The machine-readable medium of claim 6, wherein the neural network model is trained based on at least the set of features. ### Claim 8 (depends on 1) The machine-readable medium of claim 1, wherein the set of features includes an indication of how the user interacted with a result of a query. ### Claim 9 (depends on 8) The machine-readable medium of claim 8, wherein the query result user interaction can be one of a user engaging with the result and abandoned the result. ### Claim 10 (depends on 1) The machine-readable medium of claim 1, wherein the set of features includes an indication of how the user interacts with a set of installed applications on the client device. ### Claim 11 (depends on 1) The machine-readable medium of claim 1, wherein the set of features includes an application use history. ### Claim 12 A non-transitory machine-readable medium having executable instructions to cause one or more processing units to perform a method to generate a crowd-source model used to re-rank a plurality of search results on a client device, the method comprising: receiving features from a plurality of devices, wherein the features are data that indicates how a plurality of users of the plurality of devices use the plurality of devices and the features do not reveal private data about the plurality of users, wherein each of the features is anonymized data that includes a type of an interaction with a corresponding one of the plurality of devices by a corresponding user of that device and a discovery of one or more of the features will not lead to a discovery of which of the plurality of devices contributed the set of features; and generating a crowd-sourced model from the features, wherein the crowd-sourced model includes a crowd-sourced intra-domain model used to re-rank a first set of search results of a search query from one of a plurality of search domains and a crowd-sourced inter-domain used to re-rank a second set of results from the plurality of search domains from the search query; and sending the crowd-sourced model to the client device in response to the client device requesting the crowd-sourced model, wherein the client device uses the crowd-sourced model to locally re-rank a set of search results. ### Claim 13 (depends on 12) The machine-readable medium of claim 12, wherein the client device request is an opt-in by the client device to receive the crowd-sourced model. ### Claim 14 (depends on 13) The machine-readable medium of claim 13, wherein the client device opt-in request is an opt-in request to allow diagnostic data collection. ### Claim 15 A method used to re-rank a plurality of search results on a client device, the method comprising: sending a set of features to the server, wherein the set of features is anonymized data that includes a type of an interaction with the client device by a user of the client device and a discovery of one or more of the features will not lead to a discovery of which device contributed the set of features; receiving a crowd-sourced intra-domain model from a server, wherein the intra-domain model is a search result re-ranking model generated based on at least device interactions of a plurality of users interacting with a plurality of other devices; and generating, by the client device, a merged re-ranking model from the crowd-sourced intra-domain model and a local model, wherein the local model includes private data representing a device user's interaction with that device and the merged re-ranking model is used to re-rank a plurality of search results retrieved from a search domain that is one of a plurality of search domains using a search query, the search domain corresponding to the crowd-sourced intra-domain model. ### Claim 16 (depends on 15) The method of claim 15, wherein there are a plurality of intra-domain models corresponding to the plurality of search domains that are used to generate the plurality of search results. ### Claim 17 (depends on 16) The method of claim 16, further comprising: receiving a crowd-sourced inter-domain model from the server, wherein the crowd-sourced inter-domain model indicates which of the plurality of search domains are to be ranked higher than others in the plurality of search domains; and further generating the merged re-ranking model from the crowd-sourced inter-domain model and the local model and the crowd-sourced inter-domain model is used to blend the plurality of search results from the plurality of search domains. ### Claim 18 (depends on 15) The method of claim 15, wherein the set of features does not reveal private data about the client device user. ### Claim 19 (depends on 15) The method of claim 15, wherein the merged re-ranking model is a neural network model. ### Claim 20 A method to generate a crowd-source model used to re-rank a plurality of search results on a client device, the method comprising: receiving features from a plurality of devices, wherein the features are data that indicates how a plurality of users of the plurality of devices use the plurality of devices and the features do not reveal private data about the plurality of users, wherein each of the features is anonymized data that includes a type of an interaction with a corresponding one of the plurality of devices by a corresponding user of that device and a discovery of one or more of the features will not lead to a discovery of which of the plurality of devices contributed the set of features; and generating a crowd-sourced model from the features, wherein the crowd-sourced model includes a crowd-sourced intra-domain model used to re-rank a first set of search results of a search query from one of a plurality of search domains and a crowd-sourced inter-domain used to re-rank a second set of results from the search results of the search query; and sending the crowd-sourced model to the client device in response to the client device requesting the crowd-sourced model, wherein the client device uses the crowd-sourced model to locally re-rank a set of search results. ### Claim 21 (depends on 20) The method of claim 20, wherein the client device request is an opt-in by the client device to receive the crowd-sourced model. ### Claim 22 (depends on 21) The method of claim 21, wherein the client device opt-in request is an opt-in request to allow diagnostic data collection.