US 10,120,904 B2Granted

Resource management in a distributed computing environment

AssigneeCloudera, Inc.

InventorJairam Ranganathan

Life of this patent

Estimating the term…

Loading patent…

Abstract

Systems and methods are disclosed for resource management in a distributed computing environment. In some embodiments, a resource manager for a large distributed cluster needs to be able to provide resource responses very quickly. But each query may also not be accurate in initial resource request and will often have to come back to the resource manager multiple times. An artifact may provide low latency query responses by using resource request caching that can handle re-requests of resources. According to some embodiments, a queuing mechanism may take into account resources currently expended and any resource requirement estimates available in order to make queuing decisions that meet policies set by an administrator. In some embodiments, scheduling decisions are distribute across a cluster of computing systems while still maintaining approximate compliance with resource management policies set by an administrator.

Claims7

Dependent claims

A computer-implemented method for managing computing resources in a distributed computing cluster operable to execute queries on data stored in the cluster, the computing cluster having multiple data nodes storing the data, the method comprising:

instantiating the plurality of data nodes in the cluster, a respective data node operating an instance of a query engine that is capable of planning a query and executing at least a query portion that corresponds to data stored on the respective data node;
receiving, by a master node in the distributed computing cluster, an administrator usage policy, wherein the administrator usage policy includes limitations on usage of computing resources by a user and/or group of users;
receiving, by the master node, a query and an initial request for computing resources to process the query;
and
determining, by the master node, if sufficient computing resources are available to process the query based on the initial request for computing resources and a limitation on usage of computing resources applicable to the query and resource request based on the administrator usage policy, wherein,
if sufficient resources are not available:
the master node is configured to place the query into a queue until sufficient resources are available.

4 more elements

Dependencies are inferred from the claim text.

Facts

Application: 14/588,250Filed 31 Dec 2014
Published: US 2016/0188594 A130 Jun 2016
Granted: 6 Nov 2018
Term ends: Not estimatedThe source records don’t support an estimate.

Classification6 codes

G06F 16/24552
Query execution: database cache management
G06F 9/5066
Partitioning or combining of resources: algorithms for mapping a plurality of inter-dependent sub-tasks onto a plurality of physical CPUs
G06F 9/5083
Allocation of resources, e.g. of the central processing unit [CPU]: techniques for rebalancing the load in a distributed system
G06F 16/182
File system types: distributed file systems

Family

US 2016/0188594 A1Pre-grant publication · published 30 Jun 2016
US 10,120,904 B2This patent

From the US filing history. Foreign family members aren’t listed.

Timeline

Citations

Open elsewhere

Building an agent or pulling many patents? Use the API or MCP server instead of scraping pages. Get an API key