Stream recognition and filtering

A device comprising hardware and software for implementing a method comprising:

• determining a plurality of synch points in a data item, the data item consisting of an arbitrary sequence of bits, and each synch point consisting of a sequence of bits in the data item;
• for each of the plurality of synch points, determining a corresponding block of bits in the data item;
• for each block of bits, determining a corresponding block signature by applying a hash function or message digest function to the block of bits;
• forming a data item signature by associating each of the plurality of synch points with the corresponding block signature of the corresponding block of bits.

A computer-implemented method, implemented by hardware in combination with software, the method comprising:

• (A) obtaining a first data item signature for a first data item, the first data item signature comprising an association between a plurality of synch points in the first data item and a corresponding plurality of block signatures;
• (B) attempting to find a synch point of the plurality of synch points in a second data item;
• and
• (C) if a synch point of the plurality of synch points is found in the second data item, then:
• (C)(1) determining a block signature of a corresponding block of bits in the second data item, wherein the block signature is determined by applying a hash function or message digest function to the corresponding block of bits in the second data item;
• and
• (C)(2) ascertaining whether or not the synch point and the corresponding block signature from the second data item correspond to a synch point and block signature in the first data item signature;
• and
• (C)(3) when the synch point and the corresponding block signature from the second data item correspond to a synch point and block signature in the first data item signature, maintaining information indicative of the correspondence;
• (D) repeating steps (B) and (C) while at least some of the second data item remains unprocessed, until a predetermined number of synch points and corresponding block signatures from the second data item match synch points and corresponding block signatures in the first data item signature;
• (E) indicating a match between the first data item and the second data item when the predetermined number of synch points and corresponding block signatures from the second data item match synch points and corresponding block signatures in the first data item signature.

A computer-implemented method, implemented by hardware in combination with software, the method comprising:

• (A) obtaining a plurality of data item signatures, at least one signature for each of a plurality of data items, the data item signature for each particular data item of the plurality of data items comprising an association between a plurality of synch points in the particular data item and a corresponding plurality of block signatures for the particular data item;
• (B) attempting to find a synch point of the plurality of data item signatures in a second data item;
• and
• (C) if a synch point of the plurality of data item signatures is found in the second data item, then:
• (C)(1) determining a block signature of a corresponding block of bits in the second data item, wherein the block signature is determined by applying a hash function or message digest function to the corresponding block of bits in the second data item;
• and
• (C)(2) ascertaining whether or not the synch point and the corresponding block signature of the second data item correspond to a synch point and block signature in any of the plurality of data item signatures;
• and
• (C)(3) when the synch point and the corresponding block signature of the second data item correspond to a synch point and block signature in one or more of the plurality of data item signatures, maintaining information indicative of the correspondence;
• (D) repeating steps (B) and (C) while at least some of the second data item remains unprocessed, until a predetermined number of synch points and corresponding block signatures from the second data item match synch points and corresponding block signatures in at least one of the plurality of data item signatures;
• (E) when a predetermined number of synch points and corresponding block signatures from the second data item match the predetermined number of synch points and block signatures in a first data item signature of the plurality of data items, indicating a match between the first data item and the second data item.