US 9,430,301 B2Granted

Distributed system for downloading internet information and a method thereof

AssigneesPeking University Founder Group Co., Ltd., Peking University, Beijing Founder Electronics Co., Ltd.

InventorsLei Yan, Xiao Liang, Xiaoming Yu, Jianwu Yang

Life of this patent

Estimating the term…

Loading patent…

Abstract

A distributed system for downloading distributed Internet information and a method thereof are disclosed. A distribution subsystem receives downloading task information from a user, and sends the downloading task information to a downloading subsystem. The downloading subsystem downloads task result information according to the downloading task information, and sends the task result information to the distribution subsystem. The distribution subsystem receives the task result information, and returns the task result information to the user made the request. The physical environment of the system can include a distribution server and a downloading subsystem group, the internal function of the system is to assign the required tasks to the downloading subsystem group based on a queue model, the downloading subsystem group can process the assigned tasks concurrently. The system and the method is adapt to a mass-data acquisition over Internet, which may significantly improve the efficiency and timeliness for obtaining information.

Claims11

Dependent claims

A distributed system for downloading Internet information, comprising:

a distribution server and a plurality of downloading subsystem workstations;
wherein the downloading subsystem workstations are distributed over an internet and in communication with the distribution server through a local area network:
wherein the distribution server is configured to receive downloading task information from a user, assign corresponding tasks to the downloading subsystem workstations, respectively, according to the downloading task information based on a queue model, and return task result information to the user;
and
the downloading subsystem workstations are configured to receive the corresponding tasks assigned by the distribution server, respectively, simultaneously download internet data corresponding to the corresponding tasks, and return the internet data corresponding to the corresponding tasks to the distribution server;
wherein all the internet data corresponding to the corresponding tasks forms the task result information;
wherein a processor of the distribution server is further configured to:
monitor distributing of the corresponding tasks assigned to the downloading subsystem workstations, respectively, if the distributing of the corresponding tasks assigned to the downloading subsystem workstations, respectively, is completed or overtime, delete the corresponding tasks assigned to the downloading subsystem workstations, respectively, and withdraw the distribution memory space;
and
save the corresponding tasks assigned to the downloading subsystem workstations in a distribution memory queue, and create a unique distribution task number for each piece of the corresponding tasks assigned to the downloading subsystem workstations;
and
analyze and verify validity of the downloading task information and the task result information.

9 more elements

A method for distributed downloading Internet information by using a distributed system for downloading Internet information, wherein the distributed system for downloading Internet information comprises a distribution server and a plurality of downloading subsystem workstations;

the downloading subsystem workstations are distributed over an internet and in communication with the distribution server through a local area network;
the method comprising steps:
(1) receiving, by the distribution server, downloading task information from a user, assigning, by the distribution server, corresponding tasks to the downloading subsystem workstations, respectively, according to the downloading task information based on a queue model;
(2) saving the corresponding tasks assigned to the downloading subsystem workstations in a distribution memory queue, and creating a unique distribution task number for each piece of the corresponding tasks assigned to the downloading subsystem workstations;
(3) monitoring distribution of the corresponding tasks assigned to the downloading subsystem workstations, respectively, if the distribution of the corresponding tasks assigned to the downloading subsystem workstations, respectively, is completed or overtime, delete the corresponding tasks assigned to the downloading subsystem workstations, respectively, and withdraw the distribution memory space;
(4) simultaneously downloading, by the downloading subsystem workstations, Internet data corresponding to the corresponding tasks, and return the internet data corresponding to the corresponding tasks to the distribution server;
(5) receiving, by the distribution server, the internet data corresponding to the corresponding tasks, generating task result information based on all the internet data corresponding to the corresponding tasks, and returning the task result information to the user made the request;
and
(6) analyzing and verifying validity of the downloading task information and the task result information.

6 more elements

Dependencies are inferred from the claim text.