# US 2025/0094348 A1 · Data processing method and apparatus, electronic device, and storage medium > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2025/0094348 A1 - Document ID: US20250094348A1 - Status: Published application - Kind: A1 - Application number: 18/967,484 - Filed: 2024-12-03 - Published: 2025-03-20 - Earliest claimed: 2023-12-12 - Assignees: Beijing Baidu Netcom Science Technology Co., Ltd. - Applicants: Beijing Baidu Netcom Science Technology Co., Ltd. - Inventors: Mingyuan LIANG - Web page: https://patentgrep.com/patent/US20250094348A1 ## CPC - G06F 12/0806: Multiuser, multiprocessor or multiprocessing cache systems - G06F 12/0895: of parts of caches, e.g. directory or tag array - G06F 3/06: Digital input from, or digital output to, record carriers {, e.g. RAID, emulated record carriers or networked record carriers} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Data processing method and apparatus, an electronic device, and a storage medium are disclosed, which is in the fields of artificial intelligence, such as distributed storage and cloud computing. The method includes: determining a priority of each placement group in a cache pool respectively, and dividing placement groups with the same priority into a same waiting queue; constructing a target queue which is initially empty, and in response to determining that a supplementary trigger condition is met, determining placement groups to be retrieved based on the principle that a placement group in a waiting queue with higher priority is retrieved first, retrieving the placement groups to be retrieved from the corresponding waiting queue and adding the placement groups to be retrieved to the target queue; and in response to determining that the target queue is not empty, iteratively traversing each placement group in the target queue, wherein when traversing each placement group, the placement group is used as a target placement group respectively, and the number of writable objects is determined as a first quantity, and the first quantity of objects retrieved from the target placement group is written to a backend pool. ## Claims ### Claim 1 A method for data processing, comprising: determining a priority of each placement group in a cache pool respectively, and dividing placement groups with the same priority into a same waiting queue; constructing a target queue which is initially empty, and in response to determining that a supplementary trigger condition is met, determining placement groups to be retrieved based on a principle that a placement group in a waiting queue with higher priority is retrieved first, retrieving the placement groups to be retrieved from the corresponding waiting queue and adding the placement groups to be retrieved to the target queue; and in response to determining that the target queue is not empty, iteratively traversing each placement group in the target queue, wherein when traversing each placement group, the placement group is used as a target placement group respectively, and the number of writable objects is determined as a first quantity, and the first quantity of objects retrieved from the target placement group is written to a backend pool. ### Claim 2 (depends on 1) The method of claim 1, wherein the determining the priority of each placement group in the cache pool respectively comprises: in response to determining that the amount of data in the cache pool is greater than a first threshold, determining that the cache pool meets a write trigger condition, and performing the following processing for each placement group in the cache pool respectively: comparing the amount of data in the placement group with a reference data amount corresponding to the placement group, the reference data amount being a data amount threshold determined based on the first threshold, and determining the priority of the placement group based on the comparison result, wherein the larger the ratio of the amount of data in the placement group to the reference data amount, the higher the priority of the placement group. ### Claim 3 (depends on 1) The method of claim 1, wherein the response to determining that the supplementary trigger condition is met comprises: in response to determining that there are no unprocessed placement groups in the target queue, determining that the supplementary trigger condition is met, wherein a processed placement group is the one when all objects in the placement group have been retrieved, and the processed placement group is deleted from the target queue; the determining the group to be retrieved comprises: using placement groups in the waiting queue with the highest priority from waiting queues where the placement groups have not been retrieved as the placement groups to be retrieved. ### Claim 4 (depends on 1) The method of claim 1, wherein the response to determining that the supplementary trigger condition is met comprises: in response to determining that the number of unprocessed placement groups in the target queue is less than a second threshold, determining that the supplementary trigger condition is met, wherein the completion of the processing comprises: all objects have been retrieved, and the processed placement group is deleted from the target queue; the determining the placement groups to be retrieved comprises: in response to determining that it is the first time the supplementary trigger condition is met, using each placement group in the waiting queue with the highest priority as the group to be retrieved, and in response to determining that it is not the first time the supplementary trigger condition is met, determining the number of placement groups to be retrieved as the second quantity, and using the second quantity of placement groups determined from the unretrieved placement groups in each waiting queue based on the principle as the placement groups to be retrieved. ### Claim 5 (depends on 4) The method of claim 4, wherein the determining the number of placement groups to be retrieved as the second quantity comprises: obtaining the difference between the second threshold and the number of unprocessed placement groups in the target queue, and using the difference as the second quantity. ### Claim 6 (depends on 1) The method of claim 1, wherein the retrieving the first quantity of objects from the target placement group comprises: calling a list thread, using the list thread to retrieve the first quantity of objects from the target placement group, and adding the retrieved objects as objects to be written to a write queue; the writing to the backend pool comprises: in response to determining that the write queue is not empty, writing each object to be processed in the write queue to the backend pool in the order of the time each object was added to the write queue, from first to last. ### Claim 7 (depends on 6) The method of claim 6, wherein the iteratively traversing each placement group in the target queue comprises: monitoring the number of unwritten objects to be processed in the write queue, and in response to determining based on the monitoring result that the number of unwritten objects to be processed is less than or equal to a third threshold, traversing the next placement group, wherein the third threshold is less than a fourth threshold, and the fourth threshold is the maximum number of objects to be processed that the write queue is allowed to include. ### Claim 8 (depends on 7) The method of claim 7, wherein the determining the number of writable objects as the first quantity, comprises: obtaining the difference between the fourth threshold and the number of unwritten objects to be processed, and using the difference as the first quantity. ### Claim 9 (depends on 6) The method of claim 6, wherein the using the list thread to retrieve the first quantity of objects from the target placement group comprises: using the list thread to retrieve the top M objects from the unretrieved objects in the target placement group, wherein the objects in the target placement group are arranged in a predetermined order, M is a positive integer greater than or equal to the first quantity, and the retrieved M objects meet the following condition: after filtering out objects from the M objects with a time difference between the write time and the current time that is less than a fifth threshold, the number of remaining objects equals the first quantity; further comprising: in response to determining that the M objects cannot be retrieved, retrieving all unretrieved objects from the target placement group, filtering out objects with time difference between the write time and the current time that is less than the fifth threshold, and adding the remaining objects as objects to be written to the write queue. ### Claim 10 (depends on 9) The method of claim 9, further comprising: after using the list thread to retrieve objects from the target placement group, controlling the list thread to enter a sleep state, and releasing the lock corresponding to the target placement group occupied by the list thread when retrieving objects from the target placement group. ### Claim 11 An electronic device, comprising: at least one processor; and a memory communicatively connected with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform a method for data processing, wherein the method for data processing comprises: determining a priority of each placement group in a cache pool respectively, and dividing placement groups with the same priority into a same waiting queue; constructing a target queue which is initially empty, and in response to determining that a supplementary trigger condition is met, determining placement groups to be retrieved based on a principle that a placement group in a waiting queue with higher priority is retrieved first, retrieving the placement groups to be retrieved from the corresponding waiting queue and adding the placement groups to be retrieved to the target queue; and in response to determining that the target queue is not empty, iteratively traversing each placement group in the target queue, wherein when traversing each placement group, the placement group is used as a target placement group respectively, and the number of writable objects is determined as a first quantity, and the first quantity of objects retrieved from the target placement group is written to a backend pool. ### Claim 12 (depends on 11) The electronic device of claim 11, wherein the determining the priority of each placement group in the cache pool respectively comprises: in response to determining that the amount of data in the cache pool is greater than a first threshold, determining that the cache pool meets a write trigger condition, and performing the following processing for each placement group in the cache pool respectively: comparing the amount of data in the placement group with a reference data amount corresponding to the placement group, the reference data amount being a data amount threshold determined based on the first threshold, and determining the priority of the placement group based on the comparison result, wherein the larger the ratio of the amount of data in the placement group to the reference data amount, the higher the priority of the placement group. ### Claim 13 (depends on 11) The electronic device of claim 11, wherein the response to determining that the supplementary trigger condition is met comprises: in response to determining that there are no unprocessed placement groups in the target queue, determining that the supplementary trigger condition is met, wherein a processed placement group is the one when all objects in the placement group have been retrieved, and the processed placement group is deleted from the target queue; the determining the group to be retrieved comprises: using placement groups in the waiting queue with the highest priority from waiting queues where the placement groups have not been retrieved as the placement groups to be retrieved. ### Claim 14 (depends on 11) The electronic device of claim 11, wherein the response to determining that the supplementary trigger condition is met comprises: in response to determining that the number of unprocessed placement groups in the target queue is less than a second threshold, determining that the supplementary trigger condition is met, wherein the completion of the processing comprises: all objects have been retrieved, and the processed placement group is deleted from the target queue; the determining the placement groups to be retrieved comprises: in response to determining that it is the first time the supplementary trigger condition is met, using each placement group in the waiting queue with the highest priority as the placement group to be retrieved, and in response to determining that it is not the first time the supplementary trigger condition is met, determining the number of placement groups to be retrieved as the second quantity, and using the second quantity of placement groups determined from the unretrieved placement groups in each waiting queue based on the principle as the placement groups to be retrieved. ### Claim 15 (depends on 14) The electronic device of claim 14, wherein the determining the number of placement groups to be retrieved as the second quantity comprises: obtaining the difference between the second threshold and the number of unprocessed placement groups in the target queue, and uses the difference as the second quantity. ### Claim 16 (depends on 11) The electronic device of claim 11, wherein the retrieving the first quantity of objects from the target placement group comprises: calling a list thread, use the list thread to retrieve the first quantity of objects from the target placement group, and adding the retrieved objects as objects to be written to a write queue; the writing to the backend pool comprises: in response to determining that the write queue is not empty, writing each object to be processed in the write queue to the backend pool in the order of the time each object was added to the write queue, from first to last. ### Claim 17 (depends on 16) The electronic device of claim 16, wherein the iteratively traversing each placement group in the target queue comprises: monitoring the number of unwritten objects to be processed in the write queue, and in response to determining based on the monitoring result that the number of unwritten objects to be processed is less than or equal to a third threshold, traversing the next placement group, wherein the third threshold is less than a fourth threshold, and the fourth threshold is the maximum number of objects to be processed that the write queue is allowed to include. ### Claim 18 (depends on 17) The electronic device of claim 17, wherein the determining the number of writable objects as the first quantity comprises: obtaining the difference between the fourth threshold and the number of unwritten objects to be processed, and using the difference as the first quantity. ### Claim 19 (depends on 16) The electronic device of claim 16, wherein the using the list thread to retrieve the first quantity of objects from the target placement group comprises: using the list thread to retrieve the top M objects from the unretrieved objects in the target placement group, wherein the objects in the target placement group are arranged in a predetermined order, M is a positive integer greater than or equal to the first quantity, and the retrieved M objects meet the following condition: after filtering out objects from the M objects with a time difference between the write time and the current time that is less than a fifth threshold, the number of remaining objects equals the first quantity; further comprising: in response to determining that the M objects cannot be retrieved, retrieving all unretrieved objects from the target placement group, filtering out objects with a time difference between the write time and the current time that is less than the fifth threshold, and adding the remaining objects as objects to be written to the write queue. ### Claim 20 A non-transitory computer readable storage medium with computer instructions stored thereon, wherein the computer instructions are used for causing a method for data processing, wherein the method for data processing comprises: determining a priority of each placement group in a cache pool respectively, and dividing placement groups with the same priority into a same waiting queue; constructing a target queue which is initially empty, and in response to determining that a supplementary trigger condition is met, determining placement groups to be retrieved based on a principle that a placement group in a waiting queue with higher priority is retrieved first, retrieving the placement groups to be retrieved from the corresponding waiting queue and adding the placement groups to be retrieved to the target queue; and in response to determining that the target queue is not empty, iteratively traversing each placement group in the target queue, wherein when traversing each placement group, the placement group is used as a target placement group respectively, and the number of writable objects is determined as a first quantity, and the first quantity of objects retrieved from the target placement group is written to a backend pool.