Resource optimization for online services

An optimization engine for an online service, comprising:

• a communication interface configured to receive allocation data describing an allocation of a plurality of objects located on a plurality of network elements associated with the online service, and log data relating to usage of the online service;
• and
• one or more processors configured to:
• infer a relationship graph from the allocation data and the log data, the relationship graph comprising a node for each object and connections between pairs of nodes, each connected pair of nodes corresponding to a pair of objects having at least one associated data item in common, and each of the connections between the pairs of nodes having an assigned weight value relating to a number of common data items between a corresponding pair of objects;
• partition the relationship graph into a set of clusters, each cluster comprising pairs of nodes joined by connections having similar weight values relative to weight values of connections between nodes in different clusters;
• control the communication interface to issue commands to reallocate one or more of the objects to at least one of the plurality of network elements based on the clusters in the relationship graph, objects corresponding to nodes in a same cluster being co-located at a same network element;
• and
• coalesce, subsequent to controlling the interface to issue the commands to reallocate one or more of the objects, a plurality of common data items of objects located at the same network element, resulting in only a single instance of the plurality of common data items being stored at the same network element and a reduction in the number of instances of each of the plurality of common data items across the plurality of network elements, at least a first common data item of the plurality of common data items having been stored on a different network element from the same network element prior to reallocating, a first object having been reallocated to the same network element and associated with the at least first common data item including the at least first common data item and additional data items.

A computer-implemented method associated with an email service, for allocating a plurality of user mailboxes amongst a plurality of servers, comprising:

• collecting log data from each of the plurality of servers, the log data for each server comprising information relating to emails communicated between pairs of user mailboxes located at that server;
• inferring a relationship graph from the log data, the relationship graph comprising a node for each user mailbox of the email service and connections between pairs of nodes, each connected pair of nodes corresponding to a pair of user mailboxes that have communicated by email, and each of the connections between the pairs of nodes having an assigned weight value relating to a number of emails communicated between a corresponding pair of user mailboxes;
• partitioning the relationship graph into a set of clusters, each cluster comprising pairs of nodes joined by connections having similar weight values, relative to weight values of connections between nodes in different clusters;
• distributing the plurality of user mailboxes to at least one of the plurality of servers based on the clusters in the relationship graph, user mailboxes corresponding to nodes in a same cluster being co-located at a same server;
• and
• coalescing, subsequent to distributing, a plurality of common data items of nodes located at the same server, the plurality of common data items including two or more instances of a first common data item, resulting in only a single instance of the first common data item being stored at the same server and a reduction in the number of instances of each of the plurality of common data items across the plurality of servers, at least a first instance of the two or more instances having been stored on a different server from the same server prior to distributing, the first common data item being an e-mail message.