Quality of service (QoS) enforcement with low contention command processing

A data storage system comprising:

• processing circuitry and a memory;
• non-volatile data storage drives;
• and
• wherein the memory has program code stored thereon, wherein the program code, when executed by the processing circuitry, provides Quality of Service (QOS) enforcement by causing the processing circuitry to:
• for each host I/O command received by the data storage system, obtain a QoS bucket for the host I/O command, wherein the QoS bucket for each host I/O command corresponds to a storage volume indicated by that host I/O command, and wherein the QoS bucket includes i) a QoS credit count, ii) a QoS wait queue, and iii) a lock flag;
• for each host I/O command received by the data storage system for which i) the QoS wait queue in the QoS bucket for the host I/O command is empty and ii) the QoS credit count in the QoS bucket for the host I/O command is at least as large as a number of credits required to process the host I/O command, i) perform an atomic operation that subtracts the number of credits required to process the host I/O command from the QoS credit count in the QoS bucket for the host I/O command without locking the QoS bucket for the host I/O command, and ii) process the host I/O command;
• and
• for each host I/O command received by the data storage system for which the QoS wait queue is non-empty and for each one of the host I/O commands for which the number of credits required to process the host I/O command is larger than the QoS credit count in the QoS bucket for the host I/O command:
• read the lock flag in the QoS bucket for the host I/O command to determine whether the lock flag is set,
• in response to a determination that the lock flag in the QoS bucket for the host I/O command is set, enqueue a command descriptor for the host I/O command onto the QoS wait queue in the QoS bucket for the host I/O command, and
• in response to a determination that the lock flag in the QoS bucket for the host I/O command is not set, i) perform an atomic operation to set the lock flag in the Qos bucket for the host I/O command, and ii) perform credits generation for the QoS bucket for the host I/O command.

A computer program product including a non-transitory computer readable medium having instructions stored thereon, wherein the instructions, when executed on processing circuitry, cause the processing circuitry to perform steps including:

• for each host I/O command received by the data storage system, obtaining a QoS bucket for the host I/O command, wherein the QoS bucket for each host I/O command corresponds to a storage volume indicated by that host I/O command, and wherein the QoS bucket includes i) a QoS credit count, ii) a QoS wait queue, and iii) a lock flag;
• for each host I/O command received by the data storage system for which i) the QoS wait queue in the QoS bucket for the host I/O command is empty and ii) the QoS credit count in the QoS bucket for the host I/O command is at least as large as a number of credits required to process the host I/O command, i) performing an atomic operation that subtracts the number of credits required to process the host I/O command from the QoS credit count in the QoS bucket for the host I/O command without locking the QoS bucket for the host I/O command, and ii) processing the host I/O command;
• for each host I/O command received by the data storage system for which the QoS wait queue is non-empty and for each one of the host I/O commands for which the number of credits required to process the host I/O command is larger than the QoS credit count in the QoS bucket for the host I/O command:
• read the lock flag in the QoS bucket for the host I/O command to determine whether the lock flag is set,
• in response to a determination that the lock flag in the QoS bucket for the host I/O command is set, enqueue a command descriptor for the host I/O command onto the QoS wait queue in the QoS bucket for the host I/O command, and
• in response to a determination that the lock flag in the Qos bucket for the host I/O command is not set, i) perform an atomic operation to set the lock flag in the QoS bucket for the host I/O command, and ii) perform credits generation for the QoS bucket for the host I/O command.