Systems and methods for modeling machine learning and data analytics

#The present disclosure relates generally to systems and techniques for data analysis. Some embodiments relate specifically to systems and techniques for using an interactive service-based platform to model and test data models.

#Many organizations and individuals use electronic data to improve their operations or aid their decision-making. For example, many business enterprises use data management technologies to enhance the efficiency of various business processes, such as executing transactions, tracking inputs and outputs, pricing or marketing products. As another example, many businesses use operational data to evaluate performance of business processes, to measure the effectiveness of efforts to improve processes, or to decide how to adjust processes.

#The sheer volume of data available through transactional logs, social media, web traffic and other sources provides many opportunities to become a data driven organization. Moreover, the ability to model and learn from the data allows entities to adapt to changing environments and situations. However, to capitalize on this data is not that straightforward, and often requires highly-skilled data scientists to build and test models, and the process of using large, diverse and dynamic datasets to derive insights is tedious, costly and time consuming. Further, the process of consuming data from different sources and changing data requirements is an added overhead in the project which can affect development and implementation timelines. As such, many business entities are incapable of modeling the business problem, building and testing models that address the problem, and producing actionable results without highly specialized experts.

#Advantages of the some embodiments may be understood by referring to the following description taken in conjunction with the accompanying drawings. In the drawings, like reference characters generally refer to the same parts throughout the different views. Also, the drawings are not necessarily to scale, emphasis instead generally being placed upon illustrating principles of some embodiments of the invention.

#FIG. 1 is an illustrative architecture of a machine learning lifecycle management platform and system in accordance with various embodiments of the invention.

#FIG. 2 is a screen capture of a user interface for creating and managing projects within the machine learning lifecycle management platform of FIG. 1 in accordance with various embodiments of the invention.

#Referring to FIG. 1, in some embodiments a system and platform 100 for building, managing and analyzing various predictive data includes, at a general level, a collector service 104, an interface layer 108, an execution layer 112, a data layer 116 and storage 120. The collector service 104 may implement data collection tasks by connecting to one or more data sources 122. The data sources 122 may be static historical data (e.g., web logs, transaction logs, etc.), traditional relational databases (e.g., Oracle, MySql), schemaless databases (e.g., MongoDB), and/or cloud services providing access to data via an application programming interface (“API”) from sources such as Salesforce, Marketo and Stubhub. The connector service can connect to the data sources 106 in either push or pull mode. In push mode, the client application using or providing the data source sends data to the collector service 104 through an API. In pull mode, the collector service connects to the data source 106 at some predefined interval to get the data.

#The interface layer 108 provides the user-facing functions for the platform 100. More specifically, the interface layer 108 includes a dashboard 124 that facilitates the creation and organization of projects within the platform 100, and allow s a user to monitor task progress, and see log files for troubleshooting. In some cases, the dashboard 124 also monitors machine utilization and health information for all machines in the deployment of the platform 100. The interface layer 108 also includes a publisher/challenger module 128 that facilitates the viewing and publishing of data produced by execution of the models.

#The execution layer 112 includes a reporting service 132, a publishing service 136, a scheduling service 140, a job server 144 and a worker pool 148. The reporting service 132 provides the functional and support for the dashboard 124 and connects to the dataset layer 116 in read only mode to gather and serve data to the dashboard 124. The publishing service 136 accepts input features from the client machines and data sources and returns results from the predictive models as they are executed. The publishing service 136 can operate in two different modes—caching mode and direct serving mode. In direct serving mode, a training node (described in greater detail below) is paired with a publisher node (also described in greater detail below). In caching mode, the training node is paired with a predictor node, and the predictor node periodically calculates the system state and model output and caches the results, which are provided on demand to the requesting publisher node. Caching mode may also be used for models that include a streaming data service providing near real-time data into the model and requires frequent processing and/or recalculations.