Camera-guided interpretation of neuromuscular signals

A method of a computerized system for training an inference model used to generate a musculoskeletal representation, the method comprising:

• receiving, by at least one processor, a plurality of neuromuscular signals of a user, the plurality of neuromuscular signals being sensed and recorded by a plurality of neuromuscular sensors arranged on one or more wearable devices worn by the user;
• receiving, by the at least one processor, at least one image captured by at least one camera while the plurality of neuromuscular signals are sensed and recorded;
• determining, by the at least one processor based on information obtained from the at least one image, position information describing a spatial relationship between two or more connected musculoskeletal segments of the user;
• determining, by the at least one processor based on the plurality of neuromuscular signals, force information;
• associating, by the at least one processor, the position information with the force information;
• training, by the at least one processor, an inference model to output a musculoskeletal representation consistent with the position information, or the force information, or the position information and the force information when neuromuscular input signals provided to the inference model have at least one predetermined characteristic, to produce an updated inference model;
• and
• causing, by the at least one processor, the updated inference model to be stored in a memory.

A non-transitory computer-readable storage medium storing code that, when executed by at least one computer, causes the at least one computer to perform a method for training an inference model used to generate a musculoskeletal representation, wherein the method comprises:

• receiving a plurality of neuromuscular signals of a user, the plurality of neuromuscular signals being sensed and recorded by a plurality of neuromuscular sensors arranged on one or more wearable devices worn by the user;
• receiving at least one image captured by at least one camera while the plurality of neuromuscular signals are sensed and recorded;
• determining, based on information obtained from the at least one image, position information describing a spatial relationship between two or more connected musculoskeletal segments of the user;
• determining, based on the plurality of neuromuscular signals, force information;
• associating the position information with the force information;
• training an inference model to output a musculoskeletal representation consistent with the position information, or the force information, or the position information and the force information when neuromuscular input signals provided to the inference model have at least one predetermined characteristic, to produce an updated inference model;
• and
• causing the updated inference model to be stored in a memory.