Apparatus and method for offline preference-based reinforcement learning

Abstract

The embodiments disclosed herein are directed to a reinforcement learning apparatus and method. According to an embodiment, there is provided a reinforcement learning apparatus for performing offline preference-based reinforcement learning, the reinforcement learning apparatus including: memory configured to store a program and a dataset for performing reinforcement learning; and a controller provided with at least one processor, adapted to operate by executing the program stored in the memory, and configured to construct a ranked list of trajectories (RLT) by repeating the tasks of extracting a trajectory segment and adding the trajectory segment to the RLT, in which trajectory segments are sorted by preference level, based on preference feedbacks for a trajectory pair including the trajectory segment a plurality of times, and to train a reward model based on preference pairs each including two trajectory segments extracted from the RLT and a preference label assigned to the two trajectory segments.

Dependencies are inferred from the claim text.