# US 2021/0240663 A1 · High density time-series data indexing and compression > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0240663 A1 - Document ID: US20210240663A1 - Status: Published application - Kind: A1 - Application number: 17/234,143 - Filed: 2021-04-19 - Published: 2021-08-05 - Earliest claimed: 2019-05-24 - Assignees: Hydrolix Inc. - Applicants: Hydrolix Inc. - Inventors: Hasan Alayli - Web page: https://patentgrep.com/patent/US20210240663A1 ## CPC - G06F 16/13: File access structures, e.g. distributed indices (arrangements of input from, or output to, record carriers G06F3/06) - G06F 16/182: Distributed file systems - G06F 16/183: {Provision of network file services by network file servers, e.g. by using NFS, CIFS (network file access protocols H04L67/1097)} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Time-series columnar-based information is received and indexed in a compute infrastructure for cost-effective cloud-based object storage. The approach leverages a file format that enables highly-performant search and retrieval of the data stored in the cloud. In operation, an indexer receives the time-series information, indexes that information according to the file format, and forwards the indexed information for storage to the object store, where it is stored as a set of time-based partitions. A partition comprises a set of files, namely, a manifest file, a data file, and an index file. These files are structured as a compact instance of a set of raw unstructured data that comprises the given partition. Highly-performant information retrieval is enabled in response to a time-bounded query, because operations at a query peer (with respect to one or more partitions) are carried out in real-time during query processing and without requiring retrieval of the data file as a whole. ## Claims ### Claim 1 A computing system, comprising: at least one hardware processor; computer memory holding computer program code executed on the hardware processor, the program code configured as a service; the service configured (i) to receive from a data source time-series columnar-based information, the information comprising a set of time-based partitions, (ii) to index the information according to a file format comprising a set of files, the set of files comprising a first file, a second file, and a third file, wherein the first file includes a set of data strings seen in a column during indexing of the information together with byte-range data configured to selectively retrieve data from the second and third files, and wherein the third file contains a list structure for each data string in the first file, wherein the first file, the second file and the third file for a given partition together comprise a compact instance of a set of raw unstructured data that comprises the given partition, and (iii) to transfer the indexed information to a cloud-based object store. ### Claim 2 (depends on 1) The computing system as described in claim 1 wherein the program code is also configured as a second service configured to merge one or more partitions into a larger time-based partition prior to transfer to the cloud-based object store. ### Claim 3 (depends on 1) The computing system as described in claim 1 wherein the columnar-based information is stored in the second file in contiguous byte-ranges. ### Claim 3 (depends on 2) The computing system as described in claim 2 wherein the second service executes on-demand, periodically or continuously. ### Claim 4 (depends on 3) The computing system as described in claim 3 6 wherein the one or more partitions are merged based on one of: data volume, network location, and available processing resources. ### Claim 5 (depends on 1) The computing system as described in claim 1 wherein the file format is schema-less. ### Claim 6 (depends on 1) The computing system as described in claim 1 wherein the service is further configured to issue a notification to catalog the indexed information that has been transferred. ### Claim 7 A method for data indexing, comprising: indexing the time-series columnar-based information comprising a set of time-based partitions into a set of files, the set of files comprising a first file, a second file, and a third file, the first file including a set of data strings seen in a column during indexing of the information together with byte-range data configured to selectively retrieve data from the second and third files, the second file storing column data seen during the indexing, and the third file containing a list structure for each data string in the first file, wherein the column data is stored in the second file in contiguous byte-ranges; and transferring the set of files for storage in a cloud-based object store. ### Claim 8 (depends on 7) The method as described in claim 7 further including: merging one or more partitions into a larger time-based partition prior to transfer to the cloud-based object store. ### Claim 9 (depends on 8) The method as described in claim 8 wherein the one or more partitions are generated on-demand, periodically or continuously. ### Claim 10 (depends on 7) The method as described in claim 7 further including issuing a notification upon transfer of the set of files, wherein the notification is an instruction to identifying the set of files in a searchable catalog. ### Claim 11 A computer program product in a non-transitory computer-readable medium, the computer program product comprising program code executed in one or more hardware processors and configured to provide data indexing, the program code comprising code configured to: index the time-series columnar-based information comprising a set of time-based partitions into a set of files, the set of files comprising a first file, a second file, and a third file, the first file including a set of data strings seen in a column during indexing of the information together with byte-range data configured to selectively retrieve data from the second and third files, the second file storing column data seen during the indexing, and the third file containing a list structure for each data string in the first file, wherein the column data is stored in the second file in contiguous byte-ranges; and transfer the set of files for storage in a cloud-based object store. ### Claim 12 (depends on 11) The computer program product as described in claim 11 wherein the program code is further configured to: merge one or more partitions into a larger time-based partition prior to transfer to the cloud-based object store.