Robotic process automation with resilient playback of recordings

A computer-implemented method for playing back a previously established recording having a plurality of recorded actions undertaken with respect to one or more software programs, the method comprising:

• initiating execution of the previously established recording, the previously established recording including at least properties and previously captured screen images for one or more of the recorded actions within the recording, the previously captured screen images are screen images captured at the time of the recording of the previously established recording;
• determining if an error occurred during execution of the previously established recording;
• identifying, in response to the determining the error occurred, a particular action of the one or more of the recorded actions within the previously established recording in which the error occurred;
• identifying a pre-action captured screen image of the previously captured screen images that corresponds to a screen image occurring just prior to the particular action;
• identifying a post-action captured screen image of the previously captured screen images that corresponds to a screen image occurring just after the particular action;
• determining an amount of change between the pre-action captured screen image and the post-action captured screen image;
• comparing the amount of change with a threshold amount to determining type of processing to be performed to resolve the error that occurred during execution of the previously established recording;
• resolving the particular action associated with the error using a first type of processing technique if the comparing determines that the amount of change is greater than the threshold amount;
• and
• resolving the particular action associated with the error using a second type of processing technique if the comparing determines that the amount of change is less than the threshold amount,
• wherein the second type of processing technique includes at least:
• detecting control objects in the particular captured screen image;
• locating a particular control object within the control objects detected in the particular captured screen image based on at least properties of the particular action from the previously established recording;
• and
• facilitating execution of the particular action using the particular control object.

A non-transitory computer readable medium including at least computer program code tangibly stored therein for playing back a previously established recording having a plurality of recorded actions of a robotic process automation system, the computer readable medium comprising:

• computer program code for initiating execution of the previously established recording, the previously established recording including at least properties and previously captured screen images for one or more of the recorded actions within the recording, the previously captured screen images are screen images captured at the time of the recording of the previously established recording;
• computer program code for determining if an error occurred during execution of the previously established recording;
• computer program code for identifying, in response to the determining the error occurred, a particular action of the plurality of recorded actions of the previously established recording in which the error occurred;
• computer program code for identifying a pre-action captured screen image of the previously captured screen images that corresponds to a screen image occurring just prior to the particular action;
• computer program code for identifying a post-action captured screen image of the previously captured screen images that corresponds to a screen image occurring just after the particular action;
• computer program code for determining an amount of change between the pre-action captured screen image and the post-action captured screen image;
• computer program code for comparing the amount of change with a threshold amount to determining type of processing to be performed to resolve the error that occurred during execution of the previously established recording;
• computer program code for resolving the particular action associated with the error using a first type of processing technique if the comparing determines that the amount of change is greater than the threshold amount;
• and
• computer program code for resolving the particular action associated with the error using a second type of processing technique if the comparing determines that the amount of change is less than the threshold amount,
• wherein the second type of processing technique includes at least:
• detecting one or more control objects in the particular captured screen image;
• identifying a particular control object within the one or more control objects detected in the particular captured screen image based on at least one or more properties of the particular action from the previously established recording;
• and
• facilitating execution of the particular action using the particular control object that has been identified.