Determination of landmarks

A method comprising:

• receiving a file comprising a plurality of portions at a computing device;
• generating a hash value for each of the portions by the computing device, wherein each hash value is associated with a position in the received file;
• for each hash value:
• if the hash value is less than a threshold hash value according to a predefined ordering method:
• adding the hash value to a first data structure;
• and
• removing one or more hash values from the first data structure that are greater than the hash value according to the predefined ordering method;
• if the hash value is greater than the threshold value according to the predefined ordering method:
• adding the hash value to a second data structure;
• if the hash value is a last hash value for a current window:
• if the first data structure is empty, determining a minimum hash value from the second data structure according to the predefined ordering method as a landmark;
• if the first data structure is not empty, determining a minimum hash value from the first data structure according to the predefined ordering method as the landmark;
• and
• associating the landmark with the file;
• and
• providing one or more landmarks associated with the file by the computing device.

A system comprising:

• at least one computing device;
• and
• a landmark engine adapted to:
• add a generated hash value to a first data structure, wherein the generated hash value is associated with a position in a sequence of hash values;
• determine, based on the position associated with the generated hash value, if the generated hash value is a last hash value for a current window, wherein the current window is associated with a range of positions in the sequence of hash values;
• and
• if so, associate a minimum hash value from the first data structure according to a predefined ordering method with the window.