Back to results
US 9,542,426 B2Granted
Memory management for in-memory processing computing environments and systems
AssigneeTeradata US, Inc.
InventorJames Shau
Life of this patent
Estimating the term…
Abstract
Data can be stored in a memory for an in-memory processing system such the data is available for processing as soon as it is needed to be processed. A first portion and a second portion of the data can be stored in the memory of the in-memory processing system for processing by the in-memory processing system, such that the second portion of the data is stored in the memory before the in-memory processing system completes the processing of the first portion of the data, thereby allowing the in-memory processing system to process the second portion of the data when the processing system is able to process the second portion of the data.
Claims12
A method, performed at least partly by a device, for processing data in an in-memory processing system that obtains the data from its memory of a first size for the processing of the data, wherein the data has a size that is larger than the first size of the memory of the in-memory processing system, and wherein the method comprises:
- storing a first portion and a second portion of the data in the memory of the in-memory processing system for processing by the in-memory processing system;
- wherein the storing of the second portion of the data stores the second portion of the data in the memory of the in-memory processing system for processing by the in-memory processing system before the in-memory processing system completes the processing of the first portion of the data, thereby allowing the in-memory processing system to process the second portion of the data when the processing system is able to process the second portion of the data;
- generating an execution plan for the processing data by the in-memory processing system, wherein the execution plan includes at least one pipeline-enabled component;
- storing, by the at least one pipeline-enabled component, the first portion of the data in the memory of the in-memory processing system for processing by the in-memory processing system;
- and storing, by the at least one pipeline-enabled component, a second portion of the data in the memory of the in-memory processing system for processing by the in-memory processing system before the in-memory processing system completes the processing of the first portion of the data;
- generating at least one pipelined-aware execution plan component in the execution plan, wherein the pipelined-aware component is configured to interact with the at least one pipeline-enabled component so that the pipeline-enabled execution plan component can effectively determine, at runtime when the execution plan is executed, to store at least one of the first and second portions of the data in-memory processing system for processing by the in-memory processing system;
- wherein the pipelined-aware execution plan component is configured to cause generation of a runtime pipeline operator configured to operate at runtime to allow the in-memory processing system to process data in a pipelined manner;
- and
- wherein the runtime pipeline operator includes one or more instances of a scanning runtime operator configured to scan the data.
6 more elements
An in-memory processing system that includes:
- a memory of a first size;
- and one or more processors adapted to:
- obtain data from the memory of the first size for the processing of the data, wherein the data has a size that is larger than the first size of the memory of the in-memory processing system;
- store a first portion and a second portion of the data in the memory of the in-memory processing system for processing by the in-memory processing system;
- wherein the storing of the second portion of the data stores the second portion of the data in the memory of the in-memory processing system for processing by the in-memory processing system before the in-memory processing system completes the processing of the first portion of the data, thereby allowing the in-memory processing system to process the second portion of the data when the processing system is able to process the second portion of the data;
- generate an execution plan for the processing data by the in-memory processing system, wherein the execution plan includes at least one pipeline-enabled component that:
- stores the first portion of the data in the memory of the in-memory processing system for processing by the in-memory processing system;
- and stores a second portion of the data in the memory of the in-memory processing system for processing by the in-memory processing system before the in-memory processing system completes the processing of the first portion of the data;
- generate at least one pipelined-aware execution plan component in the execution plan, wherein the pipelined-aware component is configured to interact with the at least one pipeline-enabled component so that the pipeline-enabled execution plan component can effectively determine, at runtime when the execution plan is executed, to store at least one of the first and second portions of the data in-memory processing system for processing by the in-memory processing system;
- wherein the pipelined-aware execution plan component is configured to cause generation of a runtime pipeline operator configured to operate at runtime to allow the in-memory processing system to process data in a pipelined mariner;
- and wherein the runtime pipeline oerator includes one or more instances of a scanning runtime operator configured to scan the data.
8 more elements
A non-transitory computer-readable storage medium storing at least executable code for processing data in an in-memory processing system that obtains the data from its memory of a first size for the processing of the data, wherein the data has a size that is larger than the first size of the memory of the in-memory processing system, and wherein the executable code when executed:
- stores a first portion and a second portion of the data in the memory of the in-memory processing system for processing by the in-memory processing system;
- and wherein the storing of the second portion of the data stores the second portion of the data in the memory of the in-memory processing system for processing by the in-memory processing system before the in-memory processing system completes the processing of the first portion of the data, thereby allowing the in-memory processing system to process the second portion of the data when the processing system is able to process the second portion of the data;
- generates an execution plan for the processing data by the in-memory processing system, wherein the execution plan includes at least one pipeline-enabled component configured to:
- store the first portion of the data in the memory of the in-memory processing system for processing by the in-memory processing system;
- and store a second portion of the data in the memory of the in-memory processing system for processing by the in-memory processing system before the in-memory processing system completes the processing of the first portion of the data;
- generates at least one pipelined-aware execution plan component in the execution plan, wherein the pipelined-aware component is configured to interact with the at least one pipeline-enabled component so that the pipeline-enabled execution plan component can effectively determine, at runtime when the execution plan is executed, to store at least one of the first and second portions of the data in-memory processing system for processing by the in-memory processing system;
- wherein the pipelined-aware execution plan component is configured to cause generation of a runtime pipeline operator configured to operate at runtime to allow the in-memory processing system to process data in a pipelined manner;
- and wherein the runtime pipeline operator includes one or more instances of a scanning runtime operator configured to scan the data.
5 more elements
Dependencies are inferred from the claim text.