Systems and methods for managing interaction records between ai agents and human evaluators
AssigneeAI Redefined Inc.
InventorFrançois CHABOT
Life of this patent
Estimating the term…
Abstract
Systems and methods of training artificial intelligence (AI) agents for live Human-In-the-Loop Reinforcement Learning system are described. The systems and methods determine, among records of interaction between AI agents and human evaluators within a simulation environment, which of the records of interaction are either preserved or deleted, and on which location within a computer network the preserved records of interaction are stored. Storage available on the human user terminal is leveraged to store personalized datasets, freeing the generalization database to perform aggressive deduplication and decimation.
Claims20
A computer-implemented method for managing storage of reward records on a device of a human-in-the-loop learning system, the method comprising:
- receiving a new record, the new record containing a state of an environment, an action taken by an agent in response to the state, and a reward generated by an evaluator;
- calculating a confidence score for the new record based at least in part on a likelihood that the reward contained in the new record was generated in response to the action contained in the new record;
- calculating a novelty score for the new record based at least in part on similarities between the new record and a plurality of similar stored records contained on a record storage of the device, each of the plurality of similar stored records having an associated confidence score and an associated novelty score;
- determining whether an available storage capacity of the record storage is above a capacity threshold;
- storing the new record in the record storage of the device if the available storage capacity of the record storage is equal to or above the capacity threshold;
- and
- replacing one of the plurality of similar stored records in the record storage with the new record if the available storage capacity of the record storage is below the capacity threshold, the confidence score of the new record is above the lowest confidence score associated with any of the plurality of similar stored records, and the novelty score of the new record is above a novelty threshold based on the novelty scores of the plurality of similar stored records.
A system for managing storage of reward records on a device of a human-in-the-loop learning system, the system comprising:
- a processor;
- and
- a non-transitory computer-readable medium having store thereon instructions which when executed by the processor cause the device to:
- i) receive a new record, the new record containing a state of an environment, an action taken by an agent in response to the state, and a reward generated by an evaluator;
- ii) calculate a confidence score for the new record based at least in part on a likelihood that the reward contained in the new record was generated in response to the action contained in the new record;
- iii) calculate a novelty score for the new record based at least in part on similarities between the new record and a plurality of similar stored records contained on a record storage of the device, each of the plurality of similar stored records having an associated confidence score and an associated novelty score;
- iv) determine whether an available storage capacity of the record storage is above a capacity threshold;
- v) store the new record in the record storage of the device if the available storage capacity of the record storage is equal to or above the capacity threshold;
- and
- vi) replace one of the plurality of similar stored records in the record storage with the new record if the available storage capacity of the record storage is below the capacity threshold, the confidence score of the new record is above the lowest confidence score associated with any of the plurality of similar stored records, and the novelty score of the new record is above a novelty threshold based on the novelty scores of the plurality of similar stored records.
Dependencies are inferred from the claim text.