# US 11,077,555 B1 · Method to minimize collisions of mobile robotic device > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,077,555 B1 - Document ID: US11077555B1 - Status: Granted - Kind: B1 - Application number: 16/241,934 - Filed: 2019-01-07 - Granted: 2021-08-03 - Earliest claimed: 2015-12-07 - Assignees: AI Incorporated - Applicants: Ali Ebrahimi Afrouzi - Inventors: Ali Ebrahimi Afrouzi - Web page: https://patentgrep.com/patent/US11077555B1 ## CPC - B25J 9/1666: {Avoiding collision or forbidden zones} - B25J 9/163: {learning, adaptive, model based, rule based expert control} - B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50}) ## Abstract Included is a method for preventing a mobile robotic device from becoming stuck during a work session including: selecting, by a control system of the mobile robotic device, one or more actions to navigate through a workspace, wherein each action transitions the mobile robotic device from a current state to a next state; actuating, by the control system of the mobile robotic device, the mobile robotic device to execute the selected one or more actions; detecting, by the control system of the mobile robotic device, whether a collision is incurred by the mobile robotic device for each action executed; and, calculating and assigning, by the control system of the mobile robotic device, more than one level of rewards for each action executed based on collisions incurred by the mobile robotic device and completion of the action. ## Claims ### Claim 1 A method for preventing a mobile robotic device from becoming stuck during a work session comprising: selecting, by a control system of the mobile robotic device, one or more actions to navigate through a workspace, wherein each action transitions the mobile robotic device from a current state to a next state; actuating, by the control system of the mobile robotic device, the mobile robotic device to execute the selected one or more actions; detecting, by the control system of the mobile robotic device, whether a collision is incurred by the mobile robotic device for each action executed; calculating and assigning, by the control system of the mobile robotic device, more than one level of rewards for each action executed based on collisions incurred by the mobile robotic device and completion of the action; and assigning, by the control system of the of the mobile robotic device, each collision to a location within a map of the workspace wherein the location corresponds to where the respective collision occurred. ### Claim 2 (depends on 1) The method of claim 1 wherein selecting the one or more actions further comprises: reviewing, by the control system of the mobile robotic device, all previous actions executed from the current state of the mobile robotic device; and selecting, by the control system of the mobile robotic device, the one or more actions that previously resulted in a highest reward. ### Claim 3 (depends on 1) The method of claim 1, wherein selecting the one or more actions further comprises: reviewing, by the control system of the mobile robotic device, all previous actions executed from a collection of states beginning with the current state of the mobile robotic device; calculating, by the control system of the mobile robotic device, a total reward for different collections of states and actions; and selecting, by the control system of the mobile robotic device, a collection of states and actions resulting in a highest reward. ### Claim 4 (depends on 1) The method of claim 1 further comprising: determining, by the control system of the mobile robotic device, a total reward value for the one or more actions executed and corresponding states visited during the work session; assigning, by the control system of the mobile robotic device, the total reward value to a policy comprised of the one or more actions executed and corresponding states visited during the work session; and iteratively generating new policies, by the control system of the mobile robotic device, until a policy that exceeds a predetermined total reward value is generated. ### Claim 5 (depends on 1) The method of claim 1, wherein the one or more actions are selected, in part, based on input from one or more sensors. ### Claim 6 (depends on 1) The method of claim 1, wherein detecting whether a collision is incurred by the mobile robotic device for each action executed, includes detecting whether the mobile robotic device is repeatedly transitioning between a same collection of states. ### Claim 7 (depends on 6) The method of claim 6 wherein the collection of states includes two to ten different states. ### Claim 8 (depends on 6) The method of claim 6 further comprising selecting one or more actions to navigate the robotic device to a state not included in the collection of states. ### Claim 9 (depends on 8) The method of claim 8 wherein the robotic device transitions to one or more states within the collection of states to reach the state not included in the collection of states. ### Claim 10 (depends on 1) The method of claim 1, wherein locations of collisions within the map of the workspace correspond to collisions that occurred during one or more work sessions. ### Claim 11 (depends on 1) The method of claim 1 further comprising: determining, by the control system of the mobile robotic device, a movement path of the mobile robotic device based on locations of previous collisions and the number of previous collisions at different locations within the map of the workspace. ### Claim 12 (depends on 1) The method of claim 1, wherein each state of the mobile robotic device comprises at least a location of the mobile robotic device within the workspace. ### Claim 13 (depends on 1) The method of claim 1, wherein the one or more actions is selected based on at least one of: locations of previous collisions, locations of obstacles, level of debris accumulation in different areas, and floor type of different areas. ### Claim 14 A method for preventing a mobile robotic device from becoming stuck during a work session comprising: selecting, by a system of the mobile robotic device, actions to navigate through a workspace, wherein each action transitions the mobile robotic device from a current state to a next state, wherein selecting the actions comprises: reviewing, by the system of the mobile robotic device, all previous actions executed from a collection of states beginning with the current state of the mobile robotic device; calculating, by the system of the mobile robotic device, a total reward for different collections of states and actions; and selecting, by the system of the mobile robotic device, a collection of states and actions resulting in a highest reward; actuating, by the system of the mobile robotic device, the mobile robotic device to execute the selected actions; detecting, by the system of the mobile robotic device, whether a collision is incurred by the mobile robotic device for each action executed; calculating and assigning, by the system of the mobile robotic device, more than one level of rewards for each action executed based on collisions incurred by the mobile robotic device and completion of the respective action; and assigning, by the system of the of the mobile robotic device, each collision to a location within a map of the workspace wherein the location corresponds to where the respective collision occurred. ### Claim 15 (depends on 14) The method of claim 14 further comprising: determining, by the system of the mobile robotic device, a total reward value for the actions executed and corresponding states visited during the work session; assigning, by the system of the mobile robotic device, the total reward value to a policy comprised of the actions executed and corresponding states visited during the work session; and iteratively generating new policies, by the system of the mobile robotic device, until a policy that exceeds a predetermined total reward value is generated. ### Claim 16 (depends on 14) The method of claim 14 wherein the actions are selected, in part, based on input from one or more sensors. ### Claim 17 (depends on 14) The method of claim 14 further comprising: determining, by the system of the mobile robotic device, a movement path of the mobile robotic device based on locations of previous collisions and number of previous collisions at different locations within the map of the workspace. ### Claim 18 (depends on 14) The method of claim 14 wherein detecting whether a collision is incurred by the mobile robotic device for each action executed further comprises: detecting whether the mobile robotic device is repeatedly transitioning between a same collection of states, the collection of states including two to ten different states; and selecting one or more actions to navigate the robotic device to a state not included in the collection of states, wherein the robotic device transitions to one or more states within the collection of states to reach the state not included in the collection of states. ### Claim 19 A method for preventing a mobile robotic device from becoming stuck during a work session comprising: selecting, by a system of the mobile robotic device, actions to navigate through a workspace, wherein each action transitions the mobile robotic device from a current state to a next state, and selecting each action comprises: reviewing, by the system of the mobile robotic device, all previous actions executed from the current state of the mobile robotic device; and choosing, by the system of the mobile robotic device, an action resulting in a highest reward; actuating, by the system of the mobile robotic device, the mobile robotic device to execute the selected actions; detecting, by the system of the mobile robotic device, whether a collision is incurred by the mobile robotic device for each action executed; calculating and assigning, by the system of the mobile robotic device, more than one level of rewards for each action executed based on collisions incurred by the mobile robotic device and completion of the action; and assigning, by the system of the of the mobile robotic device, each collision to a location within a map of the workspace wherein the location corresponds to where the respective collision occurred. ### Claim 20 (depends on 19) The method of claim 19 further comprising: determining, by the system of the mobile robotic device, a total reward value for the actions executed and corresponding states visited during the work session; assigning, by the system of the mobile robotic device, the total reward value to a policy comprised of the actions executed and corresponding states visited during the work session; and iteratively generating new policies, by the system of the mobile robotic device, until a policy that exceeds a predetermined total reward value is generated.