Methods and apparatus for transmission and rendering of complex 3D models over networks using mixed representations

A computer system for communicating three dimensional models, comprising:

• one or more central processing units (CPU) and one or more memories;
• one or more network connections to one or more networks;
• one or more three dimensional models stored in one or more of the memories, each of the models being divided into one or more components that are viewed from a viewpoint, each component having one or more representations;
• a communication process that adaptively communicates one or more of the representations through the network interface to at least two client computers by performing the following steps:
• building a model hierarchy, with one or more nodes, by clustering together one or more of the model components based on one or more clustering criteria;
• associating one or more representations with each node in the hierarchy;
• associating a cost function with each representation and using the cost function to determine a combined cost for the entire model hierarchy;
• receiving from a requesting client a set of user preferences specifying parameters of an image transmission, including capabilities of client apparatus;
• receiving from said requesting client a request for a transmission of requested image data from said viewpoint;
• dynamically evaluating the transmission context of transmissions over said network at the time of the request including at least estimated parameter values for available bandwidth, server loads and client loads;
• establishing a resource budget for the transmission based on said parameters and estimated parameter values;
• evaluating a visibility of each node in the hierarchy from said viewpoint and associating the visibility with the respective node;
• establishing a prioritization scheme for the resource budget;
• evaluating, with respect to the transmission context, a contribution of each node to an image of the 3D model to be rendered by one or more of the clients;
• and traversing the 3D model hierarchy to select adaptively, in response to the transmission context, a transmission set of one or more of the representations to be transmitted to the client using a prioritization scheme that dynamically selects or rejects a node for transmission to a requesting client based on both the visibility and contribution of each respective nodes and the display and network capabilities of the requesting client and
• transmitting the transmission set of representations to the client.

An article of manufacture comprising a computer program product including computer-readable instruction means which performs the steps of:

• building a model hierarchy, with one or more nodes, by clustering together one or more of the model components based on one or more clustering criteria;
• associating one or more representations with each node in the hierarchy;
• associating a cost function with each representation and using the cost function to determine a combined cost for the entire model hierarchy;
• receiving from a requesting client a set of User preferences specifying parameters of an image transmission, including capabilities of client apparatus;
• receiving from said requesting client a request for a transmission of requested image data from said viewpoint;
• dynamically evaluating the transmission context of transmissions over said network at the time of the request including at least estimated parameter values for available bandwidth, server loads and client loads;
• establishing a resource budget for the transmission based on said parameters and estimated parameter values;
• evaluating a visibility of each node in the hierarchy from said viewpoint and associating the visibility with the respective node;
• establishing a prioritization scheme for the resource budget;
• evaluating, with respect to the transmission context, a contribution of each node to an image of the 3D model to be rendered by one or more of the clients;
• and traversing the 3D model hierarchy to select adaptively, in response to the transmission context, a transmission set of one or more of the representations to be transmitted to the client using a prioritization scheme that dynamically selects or rejects a node for transmission to a requesting client based on both the visibility and contribution of each respective nodes and the display and network capabilities of the requesting client and transmitting the transmission set of representations to the client.