Systems and methods of streaming data
AssigneeeBay Inc.
InventorsConnie W. Yang, Viswa Vutharkar, Ankur Bansal, Christopher J. Kasten, Michael Chiocca, Dharani Ramanathan, Subash Ramanathan, Padma Nayak, Rajeshwari Muthupandian
Life of this patent
Estimating the term…
Abstract
Systems and methods of stream technologies are disclosed. In some example embodiments, a method and system for a database stream switchover from one data center to another with notification at each component is provided.
Claims20
A system comprising:
- at least one processor;
- and
- a non-transitory computer-readable medium storing executable instructions that, when executed, cause the at least one processor to perform operations comprising:
- detecting a switchover event or a failover event for a first database host of a first data center, the first database host storing a first table of data;
- in response to detecting the switchover event or the failover event, performing an orchestrated stop of a data extraction process, the orchestrated stop comprising:
- stopping a first extraction of a data stream from the first table of data in the first database host at a specific location within the data stream, the data stream comprising change data of the first table, the change data comprising data in the first table that has changed;
- subsequent to stopping the first extraction of the data stream from the first table of data, stopping a first last change record (LCR) mirroring of the change data at the specific location within the data stream, the first LCR mirroring comprising a copying of last change data of the first table from a first LCR cluster of machines in the first data center to a second LCR cluster of machines in a second data center, the second data center having a second database host storing a second table of data;
- subsequent to stopping the first LCR mirroring, stopping a first merging of last change data in the first LCR cluster of machines with a record from the first table that corresponds to the data in the first table that has changed, the first merging forming a reconstructed record;
- and
- subsequent to stopping the first merging, stopping a first FCR mirroring of the reconstructed record at the specific location within the data stream, the first FCR mirroring comprising a copying of the reconstructed record from a first FCR cluster of machines in the first data center to a second FCR cluster of machines in the second data center;
- and
- determining an offset value indicating the specific location within the data stream;
- and
- performing an orchestrated start of the extraction process in the second data center using the offset value, the orchestrated start comprising:
- starting a second extraction of the data stream from the first table of data in the second database host at the specific location within the data stream using the offset value;
- subsequent to starting the second extraction of the data stream from the first table of data in the second database host, starting a second LCR mirroring of the change data at the specific location within the data stream, the second LCR mirroring comprising a copying of last change data of the first table from the second LCR cluster of machines in the second data center to another LCR cluster of machines in another data center;
- subsequent to starting the second LCR mirroring, starting a second merging of last change data in the second LCR cluster of machines with the reconstructed record stored in a second FCR cluster of machines in the second data center;
- and
- subsequent to starting the second merging, starting a second FCR mirroring of the harvested last change data at the specific location within the data stream, the second FCR mirroring comprising a copying of the harvested last change data from the second FCR cluster of machines in the second data center to another FCR cluster of machines in the other data center.
A computer-implemented method comprising:
- detecting a switchover event or a failover event for a first database host of a first data center, the first database host storing a first table of data;
- in response to detecting the switchover event or the failover event, performing an orchestrated stop of a data extraction process, the orchestrated stop comprising:
- stopping a first extraction of a data stream from the first table of data in the first database host at a specific location within the data stream, the data stream comprising change data of the first table, the change data comprising data in the first table that has changed;
- subsequent to stopping the first extraction of the data stream from the first table of data, stopping a first last change record (LCR) mirroring of the change data at the specific location within the data stream, the first LCR mirroring comprising a copying of last change data of the first table from a first LCR cluster of machines in the first data center to a second LCR cluster of machines in a second data center, the second data center having a second database host storing a second table of data;
- subsequent to stopping the first LCR mirroring, stopping a first merging of last change data in the first LCR cluster of machines with a record from the first table that corresponds to the data in the first table that has changed, the first merging forming a reconstructed record;
- and
- subsequent to stopping the first merging, stopping a first FCR mirroring of the harvested last change data at the specific location within the data stream, the first FCR mirroring comprising a copying of the harvested last change data from a first FCR cluster of machines in the first data center to a second FCR cluster of machines in the second data center;
- and
- determining an offset value indicating the specific location within the data stream;
- and
- performing an orchestrated start of the extraction process in the second data center using the offset value, the orchestrated start comprising:
- starting a second extraction of the data stream from the first table of data in the second database host at the specific location within the data stream using the offset value;
- subsequent to starting the second extraction of the data stream from the first table of data in the second database host, starting a second LCR mirroring of the change data at the specific location within the data stream, the second LCR mirroring comprising a copying of last change data of the first table from the second LCR cluster of machines in the second data center to another LCR cluster of machines in another data center;
- subsequent to starting the second LCR mirroring, starting a second merging of last change data in the second LCR cluster of machines with the reconstructed record stored in a second FCR cluster of machines in the second data center;
- and
- subsequent to starting the second merging, starting a second FCR mirroring of the harvested last change data at the specific location within the data stream, the second FCR mirroring comprising a copying of the harvested last change data from the second FCR cluster of machines in the second data center to another FCR cluster of machines in the other data center.
A non-transitory machine-readable storage device storing a set of instructions that, when executed by at least one processor, causes the at least one processor to perform performing operations comprising:
- detecting a switchover event or a failover event for a first database host of a first data center, the first database host storing a first table of data;
- in response to detecting the switchover event or the failover event, performing an orchestrated stop of a data extraction process, the orchestrated stop comprising:
- stopping a first extraction of a data stream from the first table of data in the first database host at a specific location within the data stream, the data stream comprising change data of the first table, the change data comprising data in the first table that has changed;
- subsequent to stopping the first extraction of the data stream from the first table of data, stopping a first last change record (LCR) mirroring of the change data at the specific location within the data stream, the first LCR mirroring comprising a copying of last change data of the first table from a first LCR cluster of machines in the first data center to a second LCR cluster of machines in a second data center, the second data center having a second database host storing a second table of data;
- subsequent to stopping the first LCR mirroring, stopping a first merging of last change data in the first LCR cluster of machines with a record from the first table that corresponds to the data in the first table that has changed, the first merging forming a reconstructed record;
- and
- subsequent to stopping the first merging, stopping a first FCR mirroring of the harvested last change data at the specific location within the data stream, the first FCR mirroring comprising a copying of the harvested last change data from a first FCR cluster of machines in the first data center to a second FCR cluster of machines in the second data center;
- and
- determining an offset value indicating the specific location within the data stream;
- and
- performing an orchestrated start of the extraction process in the second data center using the offset value, the orchestrated start comprising:
- starting a second extraction of the data stream from the first table of data in the second database host at the specific location within the data stream using the offset value;
- subsequent to starting the second extraction of the data stream from the first table of data in the second database host, starting a second LCR mirroring of the change data at the specific location within the data stream, the second LCR mirroring comprising a copying of last change data of the first table from the second LCR cluster of machines in the second data center to another LCR cluster of machines in another data center;
- subsequent to starting the second LCR mirroring, starting a second merging of last change data in the second LCR cluster of machines with the reconstructed record stored in a second FCR cluster of machines in the second data center;
- and
- subsequent to starting the second merging, starting a second FCR mirroring of the harvested last change data at the specific location within the data stream, the second FCR mirroring comprising a copying of the harvested last change data from the second FCR cluster of machines in the second data center to another FCR cluster of machines in the other data center.
Dependencies are inferred from the claim text.