Systems and methods for virtual environment for reinforcement learning in manufacturing

A computing device having a processor and memory, the computing device configured to:

• receive a first state of a tool of a computer numerical control (CNC) machine;
• determine a reward and a value of the reward based on the received first state, wherein the reward is at least one of:
• positive and negative;
• perform at least one action as part of generating a tool path, wherein the at least one action is limited by a policy comprising data for a set of possible actions available to the CNC machine, wherein the policy is at least one of:
• avoiding a collision and avoiding a part violation in manufacturing of the part;
• proceed to a subsequent state based on the performed at least one action, the subsequent state being a second state, wherein the second state is combined with the first state;
• determine, a series of actions for the tool path until a final state is reached, based on the determined reward and value associated with the performed at least one action at each subsequent state;
• train manufacturing machinery within a virtual environment based on reinforcement learning to determine the series of actions for the tool path;
• and
• wherein each action of the series of actions is determined based on at least one of:
• changing the tool, moving the tool, turning a spindle on, and turning the spindle off.

A computer-aided manufacturing (CAM) system of a computer numerical control (CNC) machine comprising a computing device, wherein the computing device comprising an interpreter logic, an agent logic, and a policy logic;

• wherein the interpreter logic of the computing device is configured to:
• receive a first state of a tool of the CNC machine;
• determine a reward and a value of the reward based on the received first state, wherein the reward is at least one of:
• positive and negative;
• transmit a set of information comprising the determined reward and the value of the reward to the agent logic of the computing device;
• and
• wherein the agent logic of the computing device is configured to:
• perform at least one action to generate a tool path, receive a policy from the policy logic, wherein the at least one action is limited by the policy and the policy comprises data for a set of possible actions available to the CNC machine, wherein the policy is at least one of:
• avoiding a collision and avoiding a part violation in manufacturing of the part;
• proceed to a subsequent state based on the performed at least one action, the subsequent state being a second state, wherein the second state is combined with the first state;
• determine a series of actions for the tool path until a final state is reached, based on the determined reward and value associated with the performed at least one action at each subsequent state;
• train manufacturing machinery within a virtual environment based on reinforcement learning to determine the series of actions for the tool path;
• and
• wherein each action of the series of actions is determined based on at least one of:
• changing the tool, moving the tool, turning a spindle on, and turning the spindle off.