# US 2023/0264367 A1 · Visuotactile operators for proximity sensing and contact control > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2023/0264367 A1 - Document ID: US20230264367A1 - Status: Published application - Kind: A1 - Application number: 18/103,825 - Filed: 2023-01-31 - Published: 2023-08-24 - Earliest claimed: 2022-02-24 - Assignees: Samsung Electronics Co., Ltd. - Applicants: Samsung Electronics Co., Ltd. - Inventors: Francois HOGAN; Jean-Francois Tremblay; Bobak Hamed Baghi; Michael Jenkin; Kaleem Siddiqi; Gregory Lewis Dudek - Web page: https://patentgrep.com/patent/US20230264367A1 ## CPC - B25J 13/083: {fitted with slippage detectors} - B25J 13/084: {Tactile sensors (in general G01L5/16, G01L5/22)} - B25J 9/1612: {characterised by the hand, wrist, grip control} - B25J 9/1697: {Vision controlled systems} - B25J 19/023: {including video camera means} - G05B 2219/39507: Control of slip motion - G05B 2219/40567: Purpose, workpiece slip sensing - G05B 2219/40575: Camera combined with tactile sensors, for 3-D - G05B 2219/40625: Tactile sensor - B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50}) - G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G) ## Abstract A method for identifying and manipulating objects may include obtaining, from an image sensor, image sensor data; identifying, using the image sensor data, a location of an object; controlling a robotic element, which includes the image sensor, to move towards the location of the object; determining a slippage based on contact between the image sensor and the object; and controlling a movement of the robotic element based on the determined slippage. ## Claims ### Claim 1 A method for identifying and manipulating objects, the method comprising: obtaining, from an image sensor, image sensor data; identifying, using the image sensor data, a location of an object; controlling a robotic element, which includes the image sensor, to move towards the location of the object; determining a slippage based on contact between the image sensor and the object; and controlling a movement of the robotic element based on the determined slippage. ### Claim 2 (depends on 1) The method of claim 1, wherein the identifying the location of the object comprises identifying a bounding box for the object in an image that is represented by the image sensor data. ### Claim 3 (depends on 2) The method of claim 2, wherein the identifying the bounding box comprises predicting coordinates of the bounding box in the image. ### Claim 4 (depends on 2) The method of claim 2, wherein the identifying the bounding box comprises identifying a centroid of the bounding box and determining a distance between the centroid and a center of the image sensor. ### Claim 5 (depends on 1) The method of claim 1, wherein the determining the slippage comprises measuring a deformation of a surface of the image sensor when the image sensor is in contact with the object. ### Claim 6 (depends on 1) The method of claim 1, wherein the determining the slippage comprises determining a marker flow and determining an object flow, and determining a slip field as a difference between the object flow and the marker flow. ### Claim 7 (depends on 6) The method of claim 6, wherein the determining the marker flow comprises identifying movement of at least one marker, and wherein determining the object flow comprises determining a motion of the object in relation to the image sensor. ### Claim 8 (depends on 6) The method of claim 6, further comprising combining the marker flow and the object flow using a convolutional neural network architecture. ### Claim 9 An electronic device for performing image authentication, the electronic device comprising: at least memory storing instructions; and at least one processor configured to execute the instructions to: obtain, from an image sensor, image sensor data; identify, using the image sensor data, a location of an object; control a robotic element, which includes the image sensor, to move towards the location of the object; determine a slippage based on contact between the image sensor and the object; and control a movement of the robotic element based on the determined slippage. ### Claim 10 (depends on 9) The electronic device of claim 9, wherein the at least one processor is further configured to identify a bounding box for the object in an image that is represented by the image sensor data. ### Claim 11 (depends on 10) The electronic device of claim 10, wherein the at least one processor is further configured to predict coordinates of the bounding box in the image. ### Claim 12 (depends on 10) The electronic device of claim 10, wherein the at least one processor is further configured to identify a centroid of the bounding box and determine a distance between the centroid and a center of the image sensor. ### Claim 13 (depends on 9) The electronic device of claim 9, wherein the at least one processor is further configured to measure a deformation of a surface of the image sensor when the image sensor is in contact with the object. ### Claim 14 (depends on 9) The electronic device of claim 9, wherein the at least one processor is further configured to determine a marker flow and determine an object flow, and determine a slip field as a difference between the object flow and the marker flow. ### Claim 15 (depends on 14) The electronic device of claim 14, wherein the at least one processor is further configured to identify movement of at least one marker, and wherein determining the object flow comprises determining a motion of the object in relation to the image sensor. ### Claim 16 (depends on 14) The electronic device of claim 14, wherein the at least one processor is further configured to combine the marker flow and the object flow using a convolutional neural network architecture. ### Claim 17 A non-transitory computer readable storage medium that stores instructions to be executed by at least one processor to perform a method for identifying and manipulating objects, the method comprising: obtaining, from an image sensor, image sensor data; identifying, using the image sensor data, a location of an object; controlling a robotic element, which includes the image sensor, to move towards the location of the object; determining a slippage based on contact between the image sensor and the object; and controlling a movement of the robotic element based on the determined slippage. ### Claim 18 (depends on 17) The non-transitory computer readable storage medium of claim 17, wherein the identifying the location of the object comprises identifying a bounding box for the object in an image that is represented by the image sensor data. ### Claim 19 (depends on 18) The non-transitory computer readable storage medium of claim 18, wherein the identifying the bounding box comprises predicting coordinates of the bounding box in the image. ### Claim 20 (depends on 18) The non-transitory computer readable storage medium of claim 18, wherein the identifying the bounding box comprises identifying a centroid of the bounding box and determining a distance between the centroid and a center of the image sensor.