Back to results
US 9,178,935 B2Granted
Distributed steam processing
AssigneePaypal, Inc.
InventorsChi-Hsien Chiu, Patrick Crane, Alyssa Neckorcuk, Gyanit Singh, Neelakantan Sundaresan
Life of this patent
Estimating the term…
Abstract
A method and system for forming hybrid cluster to process log files are described. In example embodiments, a method configures a node to execute as a first slave node. The first slave node executes in a first operating environment. The method also adds the first slave node to a Hadoop cluster. The Hadoop cluster includes a second slave node that operates in a second and different operating environment.
Claims18
A computer-implemented system comprising:
- at least one processor;
- a configuration module that is executable by the at least one processor to:
- configure a node to execute as a first slave node in a first operating environment that does not support Hadoop, the first slave node with access to a local file system;
- add the first slave node to a Hadoop cluster that includes a second slave node that is configurable to operate in a second operating environment that supports Hadoop and with access to a network file system different from the local file system, the second slave node to operate in a second operating environment that is different from the first operating environment;
- configure the first slave node of the first operating environment that does not support Hadoop to share the access to the network file system with the Hadoop cluster including the second slave node of the second operating environment that supports Hadoop;
- and
- update the network file system to indicate the first slave node as being added to the Hadoop cluster;
- and
- a communication module to:
- receive a request from the first slave node of the first operating environment to access a log file from the network file system, the log file stored by the network file system operated within the second operating environment that supports Hadoop;
- and
- request a server-side runner module to retrieve the log file through the network file system operated by the second operating environment that supports Hadoop.
9 more elements
A method comprising:
- configuring a node to execute as a first slave node, the first slave node executing in a first operating environment that does not support Hadoop and accessing a local file system;
- adding the first slave node to a Hadoop cluster that includes a second slave node operating in a second operating environment that supports Hadoop and with access to a network file system different from the local file system accessible to the first slave node;
- configuring the first slave node of the first operating environment that does not support Hadoop to share the access to the network file system with the Hadoop cluster including the second slave node of the second operating environment that supports Hadoop;
- updating the network file system to indicate the first slave node as being added to the Hadoop cluster;
- receiving a request from the first slave node of the first operating environment to access a log file from the network file system, the log file stored by the network file system operated within the second operating environment that supports Hadoop;
- and
- requesting a server-side runner module to retrieve the log file through the network file system operated by the second operating environment that supports Hadoop.
4 more elements
A machine-readable medium storing instructions which, when executed by a machine, cause the machine to execute a method comprising:
- configuring a node to execute as a first slave node, the first slave node executing in a first operating environment that does not support Hadoop and accessing a local file system;
- adding the first slave node to a Hadoop cluster that includes a second slave node operating in a second operating environment that supports Hadoop and with access to a network file system different from the local file system accessible to the first slave node;
- configuring the first slave node of the first operating environment that does not support Hadoop to share the access to the network file system with the Hadoop cluster including the second slave node of the second operating environment that supports Hadoop;
- updating the network file system to indicate the first slave node as being added to the Hadoop cluster;
- receiving a request from the first slave node of the first operating environment to access a log file from the network file system, the log file stored by the network file system operated within the second operating environment that supports Hadoop;
- and
- requesting a server-side runner module to retrieve the log file through the network file system operated by the second operating environment that supports Hadoop.
4 more elements
Dependencies are inferred from the claim text.