Message processing method and apparatus in distributed system
AssigneeHuawei Cloud Computing Technologies Co., Ltd.
InventorsXiaoqin Xie, Kun Li
Life of this patent
Estimating the term…
Abstract
In a message processing method, a message stream platform receives a plurality of log messages from a production platform, and the plurality of log messages are used to record information about a first service executed by the production platform. The message stream platform stores the plurality of log messages in a log file based on time segments. Then, the message stream platform may send messages in a same time segment in the plurality of log files to a consumption platform based on the time segment mark. According to the message processing method, the messages in the same time segment in the plurality of log files may be sent to the consumption platform to ensure that the consumption platform obtains the log messages generated in the same time segment.
Claims20
A method implemented by a message stream platform and comprising:
- receiving, from a production platform, log messages that record information about a first service executed by the production platform;
- allocating time segment marks to the log messages, wherein the time segment marks identify segments of time;
- storing the log messages in a log file based on time segments and the time segment marks, wherein the time segments comprise a first time segment;
- sending, to a consumption platform, the log messages that are related to the first service and that are in the first time segment;
- receiving, from the production platform, an association request comprising a shard identifier of a second shard, wherein the second shard is based on splitting a first shard or aggregating the first shard and a third shard, and wherein the first shard and the third shard are a subset formed by the log messages during execution of the first service;
- and
- setting, according to the association request, new time segment marks for subsequently received log messages related to the first service.
A message stream platform comprising:
- a memory configured to store instructions;
- and
- one or more processors coupled to the memory and configured to execute the instructions to cause the message stream platform to:
- receive, from a production platform, log messages that record information about a first service executed by the production platform;
- allocate time segment marks to the log messages, wherein the time segment marks identify segments of time;
- store the log messages in a log file based on time segments and the time segment marks, wherein the time segments comprise a first time segment;
- send, to a consumption platform, the log messages that are related to the first service and that are in the first time segment;
- receive, from the production platform, an association request comprising a shard identifier of a second shard, wherein the second shard is based on splitting a first shard or aggregating the first shard and a third shard, and wherein the first shard and the third shard are a subset formed by the log messages during execution of the first service;
- and
- set, according to the association request, new time segment marks for subsequently received log messages related to the first service.
A computer program product comprising instructions that are stored on a non-transitory computer-readable medium and that, when executed by one or more processors, cause a message stream platform to:
- receive, from a production platform, log messages that record information about a first service executed by the production platform;
- allocate time segment marks to the log messages, wherein the time segment marks identify segments of time;
- store the log messages in a log file based on time segments and the time segment marks, wherein the time segments comprise a first time segment;
- send, to a consumption platform, the log messages that are related to the first service and that are in the first time segment;
- receive, from the production platform, an association request comprising a shard identifier of a second shard, wherein the second shard is based on splitting a first shard or aggregating the first shard and a third shard, and wherein the first shard and the third shard are a subset formed by the log messages during execution of the first service;
- and
- set, according to the association request, new time segment marks for subsequently received log messages related to the first service.
A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, cause a message stream platform to:
- receive, from a production platform, log messages that record information about a first service executed by the production platform;
- allocate time segment marks to the log messages, wherein the time segment marks identify segments of time;
- store the log messages in a log file based on time segments and the time segment marks, wherein the time segments comprise a first time segment;
- send, to a consumption platform, the log messages that are related to the first service and that are in the first time segment;
- receive, from the production platform, an association request comprising a shard identifier of a second shard, wherein the second shard is based on splitting a first shard or aggregating the first shard and a third shard, and wherein the first shard and the third shard are a subset formed by the log messages during execution of the first service;
- and
- set, according to the association request, new time segment marks for subsequently received log messages related to the first service.
Dependencies are inferred from the claim text.