Back to results
US 12,749,142 B2Granted
Soft robotic sensing and proprioception via cable and microfluidic transmission
AssigneeThe Regents of the University of California
InventorsKeng-Yu Lin, Arturo Gamboa-Gonzalez, Michael Wehner
Life of this patent
Estimating the term…
Abstract
A method and system for sensing using a soft robotic system. The method and system uses displacement and/or deformation of elastomeric components, fibers, or liquids in the soil robotic system to change a visual state which is recordable in images by a digital camera. The displacement or deformation, or force applied to the soft robotic system is measured by analyzing the images using machine vision.
Claims17
A sensor system, comprising:
- a material;
- a plurality of sensors attached to the material, each of the sensors comprising a chamber containing a marker;
- a digital imager positioned for capturing a series of digital images of the markers as a function of time;
- an image processor for image processing the one or more digital images to:
- detect one or more changes in the marker resulting from one or more motions of the chamber in response to one or more forces applied to the material,
- compare relative motion between the marker in different ones of the sensors to differentiate between displacement modes comprising a bending mode, an elongation mode, and a twist mode, and
- from the changes or the relative motion, detect a pressure or the one or more displacement modes of the material in response to the one or more forces;
- a display assembly comprising a window forming a boundary around each of the markers, the boundary delimiting an extent of an image frame for each of the series of images being processed by the image processing, wherein, for each image frame, the image processing:
- obtains the image comprising image data;
- crops the image frame to include only a portion of the image within the boundary;
- converts the image data to gray scale to accentuate differences in light and dark colors and to eliminate possible noise from reflection,
- scales up every pixel value within the image frame to further accentuate a difference between a white background behind the marker;
- detects a line edge of each of the markers using an edge detector algorithm;
- and
- returns at least one end point pixel of each of the line edges using a probability algorithm;
- uses the end point pixel of each of the line edges to calculate the one or more changes comprising a displacement of the marker between successive ones of the image frames.
13 more elements
A sensor system comprising:
- a finger comprising a material;
- a network or array of sensors attached to the material, each of the sensors comprising a chamber containing a marker, and the chambers each comprising a channel comprising a first end and a second end;
- a digital imager positioned for capturing a series of digital images of the markers as a function of time;
- an image processor for image processing the one or more digital images to:
- associate each of the markers with locations of the first ends in the material;
- determines the linear displacements of each of the markers;
- and
- compare the linear displacements of each of the markers, taking into account the locations of the first ends associated with the each of the markers, so as to detect displacement mode, assigning each of a plurality of arrangements of the markers, or arrangements of changes in the markers, to a different one of the displacement modes or combination of the displacement modes;
- wherein:
- the first ends are distributed in three dimensions throughout a volume of the material deforming in response to forces, and the second ends containing the markers arranged in a two dimensional plane imaged in the one or more images by the digital imager;
- the sensors comprise fibers, cables, or fluid moving in the channels, the first ends are distributed in an array, and the markers are configured in a display assembly, so that for the displacement mode comprising,
- a bending mode having a center of curvature, a first set of the markers, attached to the first ends in a first row of array closest to the center of curvature, have the linear displacement in an opposite direction in the one or more images, as compared to a second set of the markers attached to the second ends in a second row of the array furthest from the center of curvature;
- an elongation mode, all the markers have the linear displacement in the same direction in the one or more images;
- and
- a twist mode about a central twist axis, a third set of the markers, attached to the first ends at corners of the array furthest from the twist axis, have the linear displacement that is larger in the one or more images as compared to a fourth set of the markers attached to the first ends closer to the twist axis;
- and the sensors further comprise the network or array of the sensors arranged so that the relative motion of the markers differentiates between the displacement modes.
13 more elements
A method of controlling a tool, comprising:
- capturing, using a single digital camera, one or more digital images of one or more changes of a plurality of sensors in response to application of one or more forces to the one or more sensors attached to a tool, wherein the changes are displayed by motion of markers in a display assembly, each of the markers attached to a different one of the sensors, the sensors comprising fibers, cables, or fluid moving in channels, the channels having first ends distributed in an array and second ends, and the markers configured in the display assembly;
- computing a measurement of the response from the changes captured in the one or more images, the measurement capable of identifying a pressure and the response comprising a bending mode, an elongation mode, and a twist mode;
- and
- controlling the tool using the measurement as proprioceptive feedback;
- wherein each of the sensors comprises a chamber containing the cable or fluid capable of moving along the chamber, or deforming the chamber, in response to the one or more forces, and
- the markers comprise colored portions of the cables or the fluid;
- and wherein:
- the computing comprises:
- determining one or more linear displacements of each of the markers;
- and
- comparing the linear displacements of each of the markers, taking into account the locations of first ends associated with the each of the markers, so as to detect a displacement mode;
- and
- for the displacement mode comprising:
- the bending mode having a center of curvature, a first set of the markers, attached to the first ends in a first row of array closest to the center of curvature, have the one or more linear displacements in an opposite direction in the one or more images, as compared to a second set of the markers attached to the second ends in a second row of the array furthest from the center of curvature;
- the elongation mode, all the markers have the one or more linear displacements in the same direction in the one or more images;
- and
- the twist mode about a central twist axis, a third set of the markers, attached to the first ends at corners of the array furthest from the twist axis, have the one or more linear displacements that is larger in the one or more images as compared to a fourth set of the markers attached to the first ends closer to the twist axis.
14 more elements
A system, comprising:
- a plurality of fluidic sensors attached to a tool, each of the fluidic sensors comprising a marker and a channel having a first end and a second end;
- a camera for capturing one or more images of the markers;
- and
- a computer coupled to the camera;
- the computer processing the images to:
- compute a measurement of a response of markers to one or more forces applied to the tool in each of the fluidic sensors, the measurement capable of identifying the forces comprising a pressure and displacement modes of the tool comprising a bending mode, an elongation mode, and a twist mode by comparing one or more linear displacements of the markers;
- and
- controlling the tool using the measurement as proprioceptive feedback;
- wherein:
- for the displacement mode comprising:
- the bending mode having a center of curvature, a first set of the markers, attached to the first ends in a first row of an array closest to the center of curvature, have the one or more linear displacements in an opposite direction in the one or more images, as compared to a second set of the markers attached to the second ends in a second row of the array furthest from the center of curvature;
- the elongation mode, all the markers have the one or more linear displacements in the same direction in the one or more images;
- and
- the twist mode about a central twist axis, a third set of the markers, attached to the first ends at corners of the array furthest from the twist axis, have the one or more linear displacements that are larger in the one or more images as compared to a fourth set of the markers attached to the first ends closer to the twist axis.
11 more elements
Dependencies are inferred from the claim text.