US 11,144,601 B2Granted

Mobile web scraping

AssigneeYodlee, Inc.

InventorSateesh Kavuri

Life of this patent

Estimating the term…

Loading patent…

Abstract

Methods, systems and computer program products implementing data aggregation using distributed Web scraping are disclosed. A mobile device can scrape one or more target sites to collect data from accounts of a particular user. The scraping can occur under scraping conditions as specified by the user. The scraping conditions can include conditions based on time, power, bandwidth, usage, or any combination of the above. The scraping conditions can ensure that the scraping occurs at time that is most convenient to the user, e.g., when sufficient bandwidth is available to the mobile device or the mobile device is not performing other tasks. The mobile device can upload the scraped data to a data aggregation server under submission conditions as specified by the user. The data aggregation server can aggregate the scraped data, enrich the aggregated data, and provide the enriched data to the user through Web access.

Claims13

Dependent claims

A method comprising:

receiving, by a mobile client device of a user and from a data aggregation server, a request to scrape account data associated with the user at one or more target sites corresponding to one or more service providers;
determining, by the mobile client device of the user based on a target database, a number of target sites from which to scrape the account data associated with the user of the mobile client device;
determining, by the mobile client device and at a first time, that an amount of time for scraping the number of target sites exceeds a pre-specified time;
in response to determining that the amount of time for scraping the number of target sites exceeds the pre-specified time, tracking, by the mobile client device, usage of the mobile client device;
determining, by the mobile client device and based on the usage, a second time for scraping the number of target sites;
in response to determining the second time for scraping the number of target sites, scraping, by the mobile client device and at the second time, the account data associated with the user of the mobile client device from the target sites, including providing respective user credentials to each target site, navigating pages of each target site, and retrieving the account data from the pages;
determining, by the mobile client device, whether a data submission condition is satisfied;
and
in response to determining that the data submission condition is satisfied, providing the scraped account data associated with the user of the mobile client device from the mobile client device of the user to the data aggregation server.

6 more elements

A non-transitory computer-readable medium storing instructions that, when executed by one or more processors of a mobile client device, cause the one or more processors to perform operations comprising:

receiving, by the mobile client device of a user and from a data aggregation server, a request to scrape account data associated with the user at one or more target sites corresponding to one or more service providers;
determining, by the mobile client device of the user based on a target database, a number of target sites from which to scrape the account data associated with the user of the mobile client device;
determining, by the mobile client device and at a first time, that an amount of time for scraping the number of target sites exceeds a pre-specified time;
in response to determining that the amount of time for scraping the number of target sites exceeds the pre-specified time, tracking, by the mobile client device, usage of the mobile device;
determining, by the mobile client device and based on the usage, a second time for scraping the number of target sites;
in response to determining the second time for scraping the number of target sites, scraping, by the mobile client device and at the second time, the account data associated with the user of the mobile client device from the one or more target sites, including providing respective user credentials to each target site, navigating pages of each target site, and retrieving the account data from the pages;
determining, by the mobile client device whether a data submission condition is satisfied;
and
in response to determining that the data submission condition is satisfied, providing the scraped account data associated with the user of the mobile client device from the mobile client device of the user to the data aggregation server.

6 more elements

Dependencies are inferred from the claim text.

Facts

Application: 15/624,578Filed 15 Jun 2017
Priority: IN 20174101354617 Apr 2017
Published: US 2018/0300408 A118 Oct 2018
Granted: 12 Oct 2021
Term ends: Not estimatedThe source records don’t support an estimate.

Classification15 codes

G06F 16/2471
Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries: distributed queries
G06F 16/951
Retrieval from the web: indexing; Web crawling techniques
H04L 67/02
Protocols: based on web technology, e.g. hypertext transfer protocol [HTTP]
H04L 67/12
Protocols: specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in…