Method for providing scraping-based service and application for executing the same
AssigneeKakaoBank Corp.
InventorsDohyoung Keedi Kim, Hyeon Seung Kim
Life of this patent
Estimating the term…
Abstract
The disclosure relates to a method for providing a scraping-based service. The method for providing a scraping-based service includes: requesting and receiving at least one of scraping rule information defining a series of tasks for obtaining external data; collecting the external data by executing tasks according to the received rule information; and providing service information to a user using the collected external data. According to the disclosure, by providing a scraping service through a client terminal and a remote server in a distributed manner, it is possible to increase efficiency of a scraping task and actively cope with changes in a scraping target.
Claims12
A system for providing a scraping-based service comprising:
- a client terminal including a task executor configured to request and receive rule information for scraping and obtain external data by executing tasks within the rule information;
- and
- a remote server including a rule server configured to manage the rule information and transmit the rule information according to the request from the client terminal, a command server configured to execute a command according to a remote procedure call received from the client terminal and return an execution result of the command, and a context server configured to record task result values according to the rule information as log information,
- wherein the rule information includes information defining a type and an execution order of the tasks,
- the tasks are configured in the rule information in error message units for error detection, and some independent tasks are executed in parallel according to a predefined dependency relationship with reference to context data values including task names and variable names at the time of executing the tasks,
- the command is a command included in the task, and includes a local command executable by the client terminal, a remote command executed through the remote server, and a test command periodically executing verification, and
- the remote command has a dynamic form in which it is modifiable through the command server.
A remote server for providing a scraping-based service comprising:
- a rule server configured to manage rule information for scraping and transmit the rule information according to a request from a client terminal;
- a command server configured to execute a command according to a remote procedure call received from the client terminal and return an execution result of the command;
- and
- a context server configured to record task result values according to the rule information as log information,
- wherein the rule information includes information defining a type and an execution order of tasks,
- the tasks are configured in the rule information in error message units for error detection, and some independent tasks are executed in parallel according to a predefined dependency relationship with reference to context data values including task names and variable names at the time of executing the tasks,
- the command is a command included in the task, and includes a local command executable by the client terminal, a remote command executed through the remote server, and a test command periodically executing verification, and
- the remote command has a dynamic form in which it is modifiable through the command server.
Dependencies are inferred from the claim text.