Method and apparatus for data exchange in a distributed system

A method in computing device for a receiver to decode messages received in a distributed system with constrained set of resources and progressively build a metadata description data structure, the method comprising the steps of:

• receiving a first encoded message from a sender, wherein the first encoded message includes data for each field of a subset of one or more but less than all of a plurality of fields of a data structure;
• determining that the first encoded message includes a metadata description that defines how to decode the data currently included in each of the subset of fields of the first encoded message, wherein the first encoded message also includes a metadata description identification for the metadata description that identifies only the subset of fields represented by the metadata description, and wherein the metadata description does not include the data currently stored in the subset of fields;
• storing the metadata description according to the metadata description identification in a metadata description data structure, whereby the metadata description data structure is progressively populated with metadata for data structures as messages are received that include metadata descriptions;
• decoding the first encoded message according to the metadata description;
• receiving a second encoded message from the sender, wherein the second encoded messages includes the same combination of fields of the data structure as the first encoded message and includes the same metadata description identification as the first encoded message;
• determining that the second encoded message does not include a metadata description;
• accessing, from among multiple metadata descriptions stored in the metadata description data structure, the metadata description stored responsive to the first encoded message, wherein the accessing is performed with the metadata description identification included in the second encoded message;
• and
• decoding the second encoded message according to the stored metadata description, whereby the second encoded message is decoded without the second encoded message including a metadata description.

An apparatus for exchanging data in a distributed system with constrained resources without symmetric metadata knowledge between senders and receivers, the apparatus comprising:

• a network element including the following:
• a sender including an encoding module, which when encoding messages for a receiver, is operative to:
• generate metadata descriptions for unique combinations of subset of fields of data structures, wherein each subset of fields of each of the data structures includes one or more but less than all of a plurality of fields of that data structure, wherein each metadata description defines for the receiver how to decode data in the corresponding subset of fields of that data structure included in messages, and wherein the metadata description does not include the data currently in those corresponding subset of fields,
• generate a unique metadata description identification for each different metadata description that identifies only the subset of fields represented by that metadata description,
• include metadata descriptions and the corresponding metadata description identifications in messages to the receiver when the receiver has not previously received those metadata descriptions, and
• include metadata description identifications without including metadata descriptions in messages to the receiver when the receiver has previously received those metadata descriptions, whereby these messages are encoded without per field overhead;
• and
• the receiver including a decoding module, which when decoding messages received from the sender, is operative to:
• store metadata descriptions in a metadata description data structure as they are received from the sender and in a manner that they may be looked up using corresponding metadata description identifications, wherein each metadata description is associated with a different metadata description identification, whereby the metadata description data structure is progressively populated with metadata for data structures as messages are received from the sender that include metadata descriptions, and
• decode those messages received from the sender that do not include metadata descriptions based on selecting via a look up in the metadata description data structure those of the stored metadata descriptions that correspond with the metadata description identifications included in these messages.