Using self-maintaining structure information for faster data access

A system comprising:

• at least one programmable processor;
• and
• a non-transitory machine-readable medium storing instructions that, when executed by the at least one programmable processor, cause the at least one programmable processor to perform operations comprising:
• generating a schema representing a structure of an object in a plurality of objects stored in a storage location, each object in the plurality of objects including one or more data elements, the schema identifying, for the object in the plurality of objects, one or more data elements of the object, an offset location of each of the one or more data elements of the object, and a value of each of the one or more data elements of the object;
• receiving a query requesting access to at least one of the one or more data elements of the object in the plurality of objects;
• identifying a generated schema in a plurality of generated schemas representing the object;
• mapping the identified generated schema to a binary representation of the structure of the object, such that each row in the schema is mapped to an offset field in the binary representation, and each offset field is mapped to a value field in the binary representation;
• accessing, using the binary representation of the structure of the object mapped to the identified generated schema, the one or more data elements of the object in the plurality of objects;
• and
• retrieving the one or more data elements of the object in the plurality of objects, wherein the generated schema identifies, using a skip list offset location in the object, one or more skip lists permitting skipping at least a portion of the one or more data elements of the object during the accessing and the retrieving, wherein the one or more skip lists are incorporated into a beginning of the binary representation of the structure of the object.

A computer program product comprising a non-transitory machine-readable medium storing instructions that, when executed by at least one programmable processor, cause the at least one programmable processor to perform operations comprising:

• generating a schema representing a structure of an object in a plurality of objects stored in a storage location, each object in the plurality of objects including one or more data elements, the schema identifying, for the object in the plurality of objects, one or more data elements of the object, an offset location of each of the one or more data elements of the object, and a value of each of the one or more data elements of the object;
• receiving a query requesting access to at least one of the one or more data elements of the object in the plurality of objects;
• identifying a generated schema in a plurality of generated schemas representing the object;
• mapping the identified generated schema to a binary representation of the structure of the object, such that each row in the schema is mapped to an offset field in the binary representation, and each offset field is mapped to a value field in the binary representation;
• accessing, using the binary represent of the structure of the object mapped to the identified generated schema, the one or more data elements of the object in the plurality of objects;
• and
• retrieving the one or more data elements of the object in the plurality of objects, wherein the generated schema identifies, using a skip list offset location in the object, one or more skip lists permitting skipping at least a portion of the one or more data elements of the object during the accessing and the retrieving, wherein the one or more skip lists are incorporated into a beginning of the binary representation of the structure of the object.