Dynamic data ingestion
AssigneeInternational Business Machines Corporation
InventorsPiotr P. Godowski, Artur Obrzut, Luigi Pichetti, Krzysztof Pienkowski
Life of this patent
Estimating the term…
Abstract
A virtual scan group manager (VSGM) can receive a plurality of respective sets of data from a plurality of respective devices. The VSGM can allocate respective portions of the plurality of respective sets of data to respective virtual scan groups (VSGs). The VSGs can be processed by storing the data contained in each VSG in a data repository. Respective sets of data stored in the data repository can be catalogued in a VSGM logfile.
Claims20
A method comprising:
- receiving, at a virtual scan group manager (VSGM) stored on a software asset management (SAM) server, a plurality of respective device scans from a plurality of respective devices;
- determining, for each respective device scan, a respective size of the respective device scan and a respective interval between a previous device scan and a current device scan for the respective device;
- selecting respective device scans having a respective interval between a previous device scan and a current device scan above a time threshold;
- compiling selected device scans in a plurality of respective virtual scan groups (VSGs);
- processing the plurality of VSGs containing selected device scans by loading respective device scans of each respective VSG into a data repository stored on the SAM server;
- and
- recording each processed device scan in a VSGM logfile stored in the VSGM.
A system comprising:
- a software asset management (SAM) server containing a virtual scan group manager (VSGM) and a data repository, wherein the VSGM is communicatively coupled to a plurality of devices producing a plurality of respective device scans, wherein the VSGM comprises a processor and a memory storing a VSGM logfile, the processor configured to:
- initiate an extract, transform, load (ETL) cycle to retrieve the plurality of respective device scans from the plurality of respective devices;
- quantify a respective size of each respective device scan and a respective interval between a previous device scan and a current device scan for each respective device;
- allocate each respective device scan having a respective interval above an interval threshold to a virtual scan group (VSG) of a plurality of VSGs;
- load each respective VSG containing respective device scans having respective intervals above the interval threshold to the data repository;
- and
- update the VSGM logfile with a respective instance for each respective device scan loaded to the data repository.
A computer program product comprising a computer readable storage medium having program instructions embodied therewith, wherein the computer readable storage medium is not a transitory signal per se, the program instructions executable by a processor to cause the processor to:
- extract respective sets of data from a plurality of devices;
- determine, for each respective set of data, a respective size of the respective set of data and a respective interval between a last time the respective set of data was stored and a current time;
- generate a plurality of virtual scan groups (VSGs) based on a quantity of respective sets of data, respective sizes of the respective sets of data, and respective intervals of the respective sets of data;
- populate each respective VSG with a plurality of respective sets of data having a respective interval above an interval threshold;
- process each respective VSG by storing the plurality of respective sets of data of each respective VSG in a database;
- and
- write, to a virtual scan group manager (VSGM) logfile, information regarding each respective set of data that is stored in the database.
Dependencies are inferred from the claim text.