Camera-guided interpretation of neuromuscular signals

A method of a computerized system for providing dynamically-updated musculoskeletal information, the method comprising:

• receiving, by at least one processor, at least one image captured by at least one camera;
• receiving, by the at least one processor, a plurality of neuromuscular signals sensed and recorded by a plurality of neuromuscular sensors configured to sense and record a plurality of neuromuscular signals from a user, wherein the plurality of neuromuscular sensors are arranged on one or more wearable devices structured to be worn by the user to obtain the plurality of neuromuscular signals;
• and
• providing, by the at least one processor, as an input to a trained inference model, information based on the plurality of neuromuscular signals and information based on the at least one image;
• determining, by the at least one processor, based on an output of the trained inference model, position information describing a spatial relationship between two or more connected musculoskeletal segments of the user, or force information describing a force exerted by at least one musculoskeletal segment of the user, or both the position information and the force information, and
• outputting, by the at least one processor, the position information, or the force information, or both the position information and the force information.

A non-transitory computer-readable storage medium storing code that, when executed by at least one computer, causes the at least one computer to perform a method for providing dynamically-updated musculoskeletal information, wherein the method comprises:

• receiving at least one image captured by at least one camera;
• receiving a plurality of neuromuscular signals sensed and recorded by a plurality of neuromuscular sensors configured to sense and record a plurality of neuromuscular signals from a user, wherein the plurality of neuromuscular sensors are arranged on one or more wearable devices structured to be worn by the user to obtain the plurality of neuromuscular signals;
• and
• providing, as an input to a trained inference model, information based on the plurality of neuromuscular signals and information based on the at least one image;
• determining, based on an output of the trained inference model, position information describing a spatial relationship between two or more connected musculoskeletal segments of the user, or force information describing a force exerted by at least one musculoskeletal segment of the user, or both the position information and the force information, and
• outputting the position information, or the force information, or both the position information and the force information.