Method and system for generating variable training data for artificial intelligence systems

A method comprising:

• categorizing the training data into a plurality of categories;
• generating a set of category groups, each category group of the set of category groups including one or more of the plurality of categories;
• training a first artificial intelligence (AI) system for the training task, the training of the first AI system including using a first subset of the training data, the first subset of the training data corresponding to a set of the one or more of the plurality of categories included in a first category group of the set of category groups;
• training a second AI system for the training task, the training of the second AI system including using a second subset of the training data, the second subset of the training data corresponding to a set of the one or more of the plurality of categories included in a second category group of the set of category groups;
• using the first AI system to generate a first output for the training task and using the second AI system to generate a second output for the training task;
• and modifying the training data according to an instruction specific to the first category group and an instruction specific to the second category group, wherein the instruction specific to the first category group and the instruction specific to the second category group are generated based on a comparison of the first output and the second output, wherein the modifying of the training data includes looping back to modify a type and a number of category groups in the set of category groups;
• retraining the first AI system or the second AI system based on the modifying of the training data;
• and using the first AI system to generate a third output for the training task based on the modified data and using the second AI system to generate a fourth output for the training task based on the modified data.

A non-transitory computer-readable storage medium comprising a set of instructions that, when executed by one or more computer processors, causes the one or more computer processors to perform operations comprising:

• receiving training data associated with a training task;
• categorizing the training data into a plurality of categories;
• generating a set of category groups, each category group of the set of category groups including one or more of the plurality of categories;
• training a first artificial intelligence (AI) system for the training task, the training of the first AI system including using a first subset of the training data, the first subset of the training data corresponding to a set of the one or more of the plurality of categories included in a first category group of the set of category groups;
• training a second AI system for the training task, the training of the second AI system including using a second subset of the training data, the second subset of the training data corresponding to a set of the one or more of the plurality of categories included in a second category group of the set of category groups;
• using the first AI system to generate a first output for the training task and using the second AI system to generate a second output for the training task;
• modifying the training data according to an instruction specific to the first category group and an instruction specific to the second category group, wherein the instruction specific to the first category group and the instruction specific to the second category group are generated based on a comparison of the first output and the second output, wherein the modifying of the training data includes looping back to modify a type and a number of category groups in the set of category groups;
• retraining the first AI system or the second AI system based on the modifying of the training data;
• and using the first AI system to generate a third output for the training task based on the modified data and using the second AI system to generate a fourth output for the training task based on the modified data.