Real-time message classification quality evaluation for a communication services platform

A system, comprising:

• a processing device;
• and
• a memory, coupled with the processing device, configured to perform operations comprising:
• receiving, by a communication services platform, a first plurality of messages to be transmitted to respective destinations,
• determining, using a first artificial intelligence (AI) model, a corresponding first message category for each message of the first plurality of messages,
• determining, using a second AI model, a corresponding second message category for each message of the first plurality of messages,
• determining a value of a performance metric of the first AI model by comparing, for each message, the corresponding first message category and the second message category, and
• responsive to determining that the value of the performance metric does not satisfy a predefined quality condition, modifying one or more parameters of the first AI model to improve a value of a performance metric of the first AI model.

A non-transitory computer-readable storage medium including instructions that, when executed by a processing device, cause the processing device to perform operations comprising:

• receiving, by a communication services platform, a first plurality of messages to be transmitted to respective destinations;
• determining, using a first artificial intelligence (AI) model, a corresponding first message category for each message of the first plurality of messages;
• determining, using a second AI model, a corresponding second message category for each message of the first plurality of messages;
• determining a value of a performance metric of the first AI model by comparing, for each message, the corresponding first message category and the second message category;
• and
• responsive to determining that the value of the performance metric does not satisfy a predefined quality condition, modifying one or more parameters of the first AI model to improve a value of a performance metric of the first AI model.