Bandwidth control system and method capable of reducing traffic congestion on content servers

A method of crawling by a crawling device to collect content over a telecommunications network, comprising:

• a requesting step of sending a connection request for establishing a connection between the crawling device and a server serving as a content provider;
• an establishing step of establishing a connection between the crawling device and the server;
• and
• a collecting step of collecting content provided by the server,
• the connection request sent in said requesting step including information that, when there is a shortage of network bandwidth, the connection between the crawling device and the server can be disconnected by priority,
• if the connection between the crawling device and the server is disconnected during collecting content in said collecting step, the location of the content being recorded in a list,
• all of said steps being performed again at a later time for collecting the content with reference to the list.

A method of crawling by a plurality of agent devices to collect content over a telecommunications network, comprising:

• a collection request sending step of sending a collection request to one of the agent devices to collect content;
• a requesting step of sending a connection request for establishing a connection between the one agent device and a server serving as a content provider;
• an establishing step of establishing a connection between the one agent device and the server;
• and
• a collecting step of collecting content from the server by the one agent device,
• the connection request sent in said requesting step including information that, when there is a shortage of network bandwidth, the connection between one of the agent devices and the server can be disconnected by priority,
• if the connection between one of the agent devices and the server is disconnected during collecting content in said collecting step, a collection request being set to another of the agent devices to collect the content.

A program storage medium for storing a computer-readable program, the program causing a computer to implement a method of controlling a network bandwidth used by a communication terminal, comprising:

• a connection request sending step of sending a connection request for connection with a destination from the communication terminal, the connection request including information that, when there is a shortage of network bandwidth, the connection between the communication terminal and the destination can be disconnected by priority;
• a connection request receiving step of receiving the connection request between the communication terminal and the destination by a network bandwidth control system;
• a connection establishing step of establishing a connection between the communication terminal and the destination by the network bandwidth control system;
• and
• a disconnecting step of disconnecting the connection between the communication terminal and the destination by the network bandwidth control system when there is a shortage of network bandwidth.

A program storage medium for storing a computer-readable program, the program causing a computer to implement a method of crawling by a crawling device to collect content over a telecommunications network, comprising:

• a requesting step of sending a connection request for establishing a connection between the crawling device and a server serving as a content provider;
• an establishing step of establishing a connection between the crawling device and the server;
• and
• a collecting step of collecting content provided by the server,
• the connection request sent in said requesting step including information that, when there is a shortage of network bandwidth, the connection between the crawling device and the server can be disconnected by priority,
• if the connection between the crawling device and the server is disconnected during collecting content in said collecting step, the location of the content being recorded in a list,
• all of said steps being performed again at a later time for collecting the content with reference to the list.

A program storage medium for storing a computer-readable program, the program causing a computer to implement a method of crawling by a plurality of agent devices to collect content over a telecommunications network, comprising:

• a collection request sending step of sending a collection request to one of the agent devices to collect content;
• a requesting step of sending a connection request for establishing a connection between the one agent device and a server serving as a content provider;
• an establishing step of establishing a connection between the one agent device and the server;
• and
• a collecting step of collecting content from the server by the one agent device,
• the connection request sent in said requesting step including information that, when there is a shortage of network bandwidth, the connection between one of the agent devices and the server can be disconnected by priority,
• if the connection between one of the agent devices and the server is disconnected during collecting content in said collecting step, a collection request being sent to another of the agent devices to collect the content.

A network bandwidth control system for controlling a network bandwidth used by a communication terminal, comprising:

• a connection request receiving circuit operable to receive a connection request between the communication terminal and a destination together with information that, when there is a shortage of network bandwidth, the connection between the communication terminal and the destination can be disconnected by priority;
• a connection establishing circuit operable to establish a connection between the communication terminal and the destination;
• and
• a disconnecting circuit operable to disconnect the connection between the communication terminal and the destination,
• said disconnecting circuit disconnecting the connection between the communication terminal and the destination when there is a shortage of network bandwidth.

A crawling device for collecting content over a telecommunications network, comprising:

• a requesting circuit operable to send a connection request for establishing a connection between the crawling device and a server serving as a content provider;
• an establishing circuit operable to establish a connection between the crawling device and the server;
• and
• a collecting circuit operable to collect content provided by the server,
• said requesting circuit sending the connection request together with information that, when there is a shortage of network bandwidth, the connection between the crawling device and the server can be disconnected by priority,
• if the connection between the crawling device and the server is disconnected during collecting content, the location of the content being recorded in a list, the list being referenced at a later time for collecting the content.

An agent device for collecting content over telecommunications network, comprising:

• a collection request receiving circuit operable to receive a collection request to collect content;
• a requesting circuit of sending a connection request for establishing a connection between the agent device and a server serving as a content provider together with information that, when there is a shortage of network bandwidth, the connection between the agent device and the server can be disconnected by priority;
• an establishing circuit of establishing a connection between the agent device and the server;
• a collecting circuit of collecting content from the server;
• a requesting circuit operable to send a content collection request to another agent device to collect content;
• and
• a disconnecting circuit operable to disconnect the connection between the communication terminal and the destination when there is a shortage of network bandwidth,
• said requesting circuit sending, if the connection between one of the agent devices and the server is disconnected during collecting content in said collecting circuit, a content collection request to another agent device to collect the content.

A telecommunications network system comprising a network bandwidth control system, a crawling device, a plurality of agent devices, and a plurality of content servers, which are connected over a telecommunications network,

• said network bandwidth control system being provided for controlling the network bandwidth used by said crawling device, and comprising:
• a connection request receiving circuit operable to receive, from said crawling device, a connection request between said crawling device and one of said content servers together with information that, when there is a shortage of network bandwidth, the connection between said crawling device and said one content server can be disconnected by priority;
• a connection establishing circuit operable to establish a connection between said crawling device and said one content server;
• and
• a disconnecting circuit operable to disconnect the connection between said crawling device and said one server,
• said disconnecting circuit disconnecting the connection between said crawling device and said one server when there is a shortage of network bandwidth,
• said crawling device being provided for collecting content over the network, comprising:
• a requesting circuit operable to send a connection request to said network bandwidth control system for establishing a connection between said crawling device and one of said content servers;
• an establishing circuit operable to establish a connection between said crawling device and said one server;
• and
• a collecting circuit operable to collect content provided by said one server,
• said requesting circuit sending the connection request together with information that, when there is a shortage of network bandwidth, the connection between said crawling device and said one server can be disconnected by priority,
• if the connection between said crawling device and said one server is disconnected during collecting content, the location of the content being recorded in a list provided in said network bandwidth control system, the list being referenced at a later time for collecting the content,
• said agent device being provided for collecting content over the network, comprising:
• a collection request receiving circuit operable to receive a collection request from said crawling device to collect content;
• a requesting circuit of sending a connection request to said network bandwidth control system for connection establishment between said agent device and one of said content servers together with information that, when there is a shortage of network bandwidth, the connection between said agent device and said one server can be disconnected by priority;
• an establishing circuit of establishing a connection between said agent device and said one server;
• a collecting circuit of collecting content from said one server;
• a requesting circuit operable to send a content collection request to another agent device to collect content;
• and
• a disconnecting circuit operable to disconnect the connection between said crawling device and said one server when there is a shortage of network bandwidth,
• if the connection between one of said agent devices and said one server is disconnected during collecting content by said collecting circuit, said requesting circuit sending a content collection request to another agent device to collect the content.

A bandwidth control system for controlling bandwidths used by a plurality of network devices which are operable to collect information from the Internet, said system comprising:

• a circuit operable to receive a connection message from each of the network devices for informing a connection established between the network device and a content server;
• a circuit operable to record each network device in association with a content server to which the network device is connected;
• a circuit operable to monitor traffic on the content servers to which the plurality of network devices are connected;
• and
• a circuit operable to disconnect a connection between a network device and a content server when the traffic on the content server becomes heavier.