Error detection method and robot system based on association identification

A non-transitory computer-readable storage medium for storing at least one instruction that when executed by a computer, causes the computer to perform an error detection method, comprising:

• determining, in a plurality of error detection cycles, a plurality of pose differences of an end of an operating arm;
• accumulating the plurality of the pose differences;
• and
• generating, in response to an accumulated value of the plurality of pose differences meeting an error detection condition, a control signal related to a fault,
• wherein the method further comprises:
• in each error detection cycle,
• obtaining a target pose of the end of the operating arm;
• acquiring a positioning image;
• recognizing, in the positioning image, a plurality of pose identifications located on the end of the operating arm;
• recognizing, based on the plurality of pose identifications, an angle identification located on the end of the operating arm, the angle identification having a position association relationship with a first pose identification of the plurality of pose identifications;
• determining, based on the angle identification and the plurality of pose identifications, an actual pose of the end of the operating arm;
• and
• determining, based on the target pose and the actual pose, a pose difference of the end of the operating arm,
• wherein recognizing the plurality of pose identifications located on the end of the operating arm comprises:
• determining a plurality of candidate pose identifications from the positioning image;
• recognizing, based on a pose pattern matching template, an initial pose identification from the plurality of candidate pose identifications;
• searching for a remaining pose identification in eight searching directions that are associated with the initial pose identification, with an initial pose identification pattern corner of the initial pose identification as a starting point;
• and
• stopping searching for the pose identification, in response to the number of the recognized pose identifications being greater than or equal to a pose identification number threshold, or
• redetermining, in response to the number of the recognized pose identifications being less than the pose identification number threshold and a distance of the searching being greater than twice a distance between previous two pose identification pattern corners of the recognized pose identifications, the initial pose identification from the plurality of candidate pose identifications based on the pose pattern matching template,
• wherein determining the actual pose of the end of the operating arm comprises:
• determining a roll angle of a pose identification coordinate system relative to an end coordinate system of the operating arm based on the angle identification and the plurality of pose identifications;
• determining a pose of the pose identification coordinate system relative to a reference coordinate system based on two-dimensional coordinates of the plurality of pose identifications in the positioning image and three-dimensional coordinates of the plurality of pose identifications in the pose identification coordinate system;
• and
• determining a pose of the end coordinate system of the operating arm relative to the reference coordinate system as the actual pose based on the roll angle of the pose identification coordinate system relative to the end coordinate system of the operating arm and the pose of the pose identification coordinate system relative to the reference coordinate system, or
• wherein determining the actual pose of the end of the operating arm comprises:
• determining a roll angle of a pose identification coordinate system relative to an end coordinate system of the operating arm based on the angle identification and the plurality of pose identifications;
• determining three-dimensional coordinates of the plurality of pose identifications in the end coordinate system of the operating arm based on the roll angle of the pose identification coordinate system relative to the end coordinate system of the operating arm and three-dimensional coordinates of the plurality of pose identifications in the pose identification coordinate system;
• and
• determining a pose of the end coordinate system of the operating arm relative to the reference coordinate system as the actual pose based on two-dimensional coordinates of the plurality of pose identifications in the positioning image and three-dimensional coordinates of the plurality of pose identifications in the end coordinate system of the operating arm,
• wherein the control signal related to the fault comprises a first alarm signal indicating that the control of the operating arm has failed, and the method further comprises:
• receiving status information of at least one drive device for driving the operating arm, in response to the target pose and the actual pose meeting an error detection condition;
• and
• sending out a second alarm signal in response to the status information and drive information of the at least one drive device meeting a fault detection condition, the second alarm signal indicating that a drive device of the operating arm has failed.

A robot system comprising:

• a master manipulator including a robotic arm, a handle disposed on the robotic arm, and at least one master manipulator sensor disposed at one or more joints on the robotic arm, the at least one master manipulator sensor for obtaining joint information of the one or more joints;
• an operating arm provided with at least one angle identification and a plurality of pose identifications at its end;
• at least one drive device for driving the operating arm;
• at least one drive device sensor coupled with the at least one drive device and for obtaining status information of the at least one drive device;
• the image acquisition device for acquiring a positioning image of the operating arm;
• and
• a control device configured to be connected with the master manipulator, the at least one drive device, the at least one drive device sensor, and the image acquisition device, and perform an error detection method comprising:
• determining, in a plurality of error detection cycles, a plurality of pose differences of the end of the operating arm;
• accumulating the plurality of the pose differences;
• and
• generating, in response to an accumulated value of the plurality of pose differences meeting an error detection condition, a control signal related to a fault,
• wherein the method further comprises:
• in each error detection cycle,
• obtaining a target pose of the end of the operating arm;
• acquiring the positioning image;
• recognizing, in the positioning image, the plurality of pose identifications located on the end of the operating arm;
• recognizing, based on the plurality of pose identifications, the at least one angle identification located on the end of the operating arm, the at least one angle identification having a position association relationship with a first pose identification of the plurality of pose identifications;
• determining, based on the at least one angle identification and the plurality of pose identifications, an actual pose of the end of the operating arm;
• and
• determining, based on the target pose and the actual pose, a pose difference of the end of the operating arm,
• wherein recognizing the plurality of pose identifications located on the end of the operating arm comprises:
• determining a plurality of candidate pose identifications from the positioning image;
• recognizing, based on a pose pattern matching template, an initial pose identification from the plurality of candidate pose identifications;
• searching for a remaining pose identification in eight searching directions that are associated with the initial pose identification, with an initial pose identification pattern corner of the initial pose identification as a starting point;
• and
• stopping searching for the pose identification, in response to the number of the recognized pose identifications being greater than or equal to a pose identification number threshold, or
• redetermining, in response to the number of the recognized pose identifications being less than the pose identification number threshold and a distance of the searching being greater than twice a distance between previous two pose identification pattern corners of the recognized pose identifications, the initial pose identification from the plurality of candidate pose identifications based on the pose pattern matching template,
• wherein determining the actual pose of the end of the operating arm comprises:
• determining a roll angle of a pose identification coordinate system relative to an end coordinate system of the operating arm based on the angle identification and the plurality of pose identifications;
• determining a pose of the pose identification coordinate system relative to a reference coordinate system based on two-dimensional coordinates of the plurality of pose identifications in the positioning image and three-dimensional coordinates of the plurality of pose identifications in the pose identification coordinate system;
• and
• determining a pose of the end coordinate system of the operating arm relative to the reference coordinate system as the actual pose based on the roll angle of the pose identification coordinate system relative to the end coordinate system of the operating arm and the pose of the pose identification coordinate system relative to the reference coordinate system, or
• wherein determining the actual pose of the end of the operating arm comprises:
• determining a roll angle of a pose identification coordinate system relative to an end coordinate system of the operating arm based on the angle identification and the plurality of pose identifications;
• determining three-dimensional coordinates of the plurality of pose identifications in the end coordinate system of the operating arm based on the roll angle of the pose identification coordinate system relative to the end coordinate system of the operating arm and three-dimensional coordinates of the plurality of pose identifications in the pose identification coordinate system;
• and
• determining a pose of the end coordinate system of the operating arm relative to the reference coordinate system as the actual pose based on two-dimensional coordinates of the plurality of pose identifications in the positioning image and three-dimensional coordinates of the plurality of pose identifications in the end coordinate system of the operating arm,
• wherein the control signal related to the fault comprises a first alarm signal indicating that the control of the operating arm has failed, and the method further comprises:
• receiving the status information of at least one drive device for driving the operating arm, in response to the target pose and the actual pose meeting an error detection condition;
• and
• sending out a second alarm signal in response to the status information and drive information of the at least one drive device meeting a fault detection condition, the second alarm signal indicating that a drive device of the operating arm has failed.