Memory compaction for append-only formatted data in a distributed storage network

A method for execution by one or more computing devices of a storage unit in a distributed storage network (DSN), the method comprises:

• receiving data from at least one computing device within the DSN, wherein at least a portion of the data is based on append-only formatting;
• storing the data in a memory;
• determining whether the memory includes invalid data;
• based on a determination that the memory includes invalid data, determining whether an amount of invalid data included on the memory compares unfavorably to a predetermined limit;
• and
• based on a determination that the amount of invalid data compares unfavorably to the predetermined limit, determining a first rate that is based on reading at least a first portion of valid data from the first section of the memory and writing the at least the first portion of valid data to a section of the memory, wherein the first rate is also based on a proportion, integral and derivative (PID) function that is based on a target usage level of the memory and a current usage level of the memory.

A storage unit (SU) comprising:

• an interface configured to interface and communicate with a dispersed or distributed storage network (DSN);
• a plurality of memory devices configured to store data received from a plurality of computing devices within the DSN;
• memory that stores operational instructions;
• and
• processing circuitry operably coupled to the interface and to a first memory device of the plurality of memory devices, wherein the processing circuitry is configured to execute the operational instructions to:
• receive data from at least one computing device within the DSN, wherein at least a portion of the data is based on append-only formatting;
• store the data in the first memory device;
• determine whether a first memory device includes invalid data;
• based on a determination that the first memory device includes invalid data, determine whether an amount of invalid data included on the first memory device compares unfavorably to a predetermined limit;
• and
• based on a determination that the amount of invalid data compares unfavorably to the predetermined limit, determine a first rate that is based on reading at least a first portion of valid data from a first section of the first memory device and writing the at least the first portion of valid data to a second memory device, wherein the first rate is also based on a proportion, integral and derivative (PID) function that is based on a target usage level of the memory and a current usage level of the first memory device.