System and method for efficiently distributing time-sensitive data in bandwidth-limited environments

A method for controlling data transmission in a communications network, the network comprising a central node in communication with a plurality of dispersed elements, each element comprising at least one cache memory for storing data, the elements arranged in sub-nets within the network, a sub-net comprising a defined geographical location, the method, comprising:

• creating by a computer, a logical overlay to the communications network;
• receiving data objects from sources external to the communications network;
• identifying a utility of the received data objects to one or more of the elements;
• determining a perishability of the data objects having the utilities;
• assigning a time to live for each of the perishable data objects;
• assigning a counter with a counter value of 1 to each of the perishable data objects;
• and
• propagating the perishable data objects to selected ones of the one or more elements, comprising:
• through the overlay, identifying a first element having a highest available bandwidth for receiving the perishable data objects in at least one sub-net,
• sending the perishable data objects to the first element, and
• instructing the first element to:
• store the perishable data objects in cache memory;
• compute the time to live has not expired;
• increase the counter value by 1;
• through the overlay, discover a second element with a second highest bandwidth;
• and
• replicate and transmit each perishable data object having a non-expired time to live and an incremented counter value of 2 to the second element.

A method for controlling perishable data transmission in a hierarchical communications network, the network comprising a central node in communication with a plurality of dispersed hosts and dispersed elements, the host and elements arranged in sub-nets within the network, a sub-net comprising a defined geographical location, the method, comprising:

• creating an overlay to the communications network;
• receiving data objects from sources external to the communications network;
• identifying a utility of the received data objects to one or more of the elements;
• determining a perishability of the data objects having the utilities;
• assigning a time to live for each of the perishable data objects;
• and
• propagating the perishable data objects to selected ones of the one or more elements, comprising multicasting the data objects to each selected element in the sub-net until a stopping condition is met, the stopping condition chosen from a group consisting of a pre-set time, expiration of the time to live, and a pre-determined number of elements receiving the data objects.

A data communications system having enhanced data transmission latency and control, comprising:

• a physical communications network;
• a logical overlay to the physical communications network;
• a central computer complex comprising one or more central processors and a non-transitory computer-readable storage medium having encoded thereon machine instructions for computer-controlled operation of the system;
• a data object intake that receives data objects from sources external to the system;
• a plurality of data communications system elements, one or more elements in communication with the central computer complex over the physical communications network and over the logical overlay, each element comprising a non-transitory local data store and local machine instruction storage medium, and a local processor, each element in further communication with selected ones of the elements;
• and
• a hierarchical data object storage system comprising a central data store at the central computer complex and the local data stores for storing data objects received at the intake, wherein to control data object dispersal in the hierarchical data object storage system the central processor executes the machine instructions to:
• ascertain a need to disperse data objects received at the intake to one or more elements in the physical communications network, comprising for the received data objects, classify each data object as one of a perishable data object or a non-perishable data object,
• employ the overlay to locate specific elements requiring the perishable data objects, and
• using the physical communications network, execute a dispersion operation causing one or more specific elements to receive the perishable data objects, comprising:
• choose a first specific element to receive the perishable data objects, comprising determining a specific element having a highest available bandwidth in the physical communications network for receipt of the perishable data objects, and
• send the perishable data objects to the first specific element, each of the perishable data objects having an assigned time to live value and an assigned, incrementing counter, wherein the local processor of the first specific element:
• computes that the assigned time to live has not expired;
• stores the perishable data objects in the local data store and assigns locally-stored perishable data objects an incrementing counter a value of one;
• discovers a second specific element having a next highest available bandwidth in the physical communications network for receipt of the perishable data objects;
• replicates the perishable data objects and assigns each of the replicated perishable data objects a counter value of two;
• and
• transmits the replicated perishable data objects to the second specific element.