# US 2019/0054624 A1 · Robot and a method of controlling a robot

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0054624 A1
- Document ID: US20190054624A1
- Status: Published application
- Kind: A1
- Application number: 16/085,290
- Filed: 2017-03-17
- Published: 2019-02-21
- Earliest claimed: 2016-03-17
- Assignees: Flow Robotics A/S
- Applicants: Flow Robotics A/S
- Inventors: Kasper Støy; Andrés Faíña Rodríguez-Vila; Farzad Nejatimoharrami
- Web page: https://patentgrep.com/patent/US20190054624A1

## CPC

- B25J 9/1694: {characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion}
- B25J 9/1679: {characterised by the tasks executed}
- B25J 13/06: Control stands, e.g. consoles, switchboards
- B25J 13/081: {Touching devices, e.g. pressure-sensitive}
- B25J 15/0616: {with vacuum}
- G05B 19/4155: characterised by program execution, i.e. part program or machine function execution, e.g. selection of a program
- G05B 2219/39059: Sensor adaptation for robots by software
- G05B 2219/39396: Manipulator action on screen depends from displayed position on screen
- G05B 2219/40293: Gantry, portal
- G05B 2219/45092: Analysing or chemical synthesis robot, moving samples from station to station
- B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50})
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)

## Abstract

The present invention relates to a robot comprising a horizontal or horizontally slanted transparent experiment layer being adapted to support items at arbitrary positions on or in the experiment layer, and a moveable sensor arranged below the transparent experimental layer said sensor being configured for providing a sensor signal indicative of item(s)' location on the experiment layer, an actuator arranged for being moved into different positions above the horizontal transparent layer a display device being configured for visually representing located item(s) a user input device configured for receiving information as to operation of the actuator.

## Claims

### Claim 1

A robot comprising:
a horizontal or horizontally slanted transparent experiment layer being adapted to support items at arbitrary positions on the experiment layer;
a moveable sensor arranged below the transparent experimental layer said sensor being configured for providing a sensor signal indicative of item(s)' location on the experiment layer;
an actuator arranged for being moved into different positions above the horizontal transparent layer;
a display device being configured for visually representing located item(s);
a user input device;
wherein the robot further comprising a processor to
on the basis of the sensor signal locate an item on the transparent layer;
display the located item(s) or a graphical presentation thereof on the display device;
receive a manipulation input through the user input device, and
in response to said manipulation input, control the actuator to manipulate the content of the displayed item(s).

### Claim at position 2

2-28. (canceled)

### Claim at position 3

29-55. (canceled)

### Claim 56 (depends on 1)

The robot according to claim 1, wherein the location of an item comprising determining the relative position in the robot of the item.

### Claim 57 (depends on 1)

The robot according to claim 1, wherein the sensor further being adapted to determine the content of an item.

### Claim 58 (depends on 1)

The robot according to claim 1, wherein the transparent layer extends in one horizontal plane.

### Claim 59 (depends on 1)

The robot according to claim 1, wherein the manipulation comprising exchange content between items supported by the layer.

### Claim 60 (depends on 1)

The robot according to claim 1, wherein the robot comprising rails and slides arranged below the transparent layer and on which the sensor is arranged so to move the sensor horizontally in a plane parallel-displaced to the experimental layer.

### Claim 61 (depends on 1)

The robot according to claim 1, wherein the sensor comprising a camera, such as an digital camera, such as an optical coherence tomography camera, with a field of view covering at least a section of the experimental layer.

### Claim 62 (depends on 1)

The robot according to claim 1, wherein the robot comprising rails and slides arranged above the transparent layer and on which the actuator is arranged so to move the actuator horizontally in a plane parallel-displaced to the experimental layer and vertically.

### Claim 63 (depends on 1)

The robot according to claim 1, wherein the robot comprising a robotic arm on which the actuator is arranged.

### Claim 64 (depends on 1)

The robot according to claim 1, wherein the actuator comprising an electronically operated suction and dispensing device, such as a syringe, pipette or the like.

### Claim 65 (depends on 1)

The robot according to claim 1, wherein one or more of the items are containers, such as petri dish, well-plate or the like.

### Claim 66 (depends on 1)

The robot according to claim 1, wherein one or more of the items are receptacles having an at least partially transparent bottom and/or a non-transparent marker.

### Claim 67 (depends on 1)

The robot according to claim 1, further comprising an item dispenser configured for dispensing items onto the experimental layer.

### Claim 68 (depends on 1)

The robot according to claim 1, wherein the display device and input device together is a touch sensitive display device.

### Claim 69 (depends on 1)

The robot according to claim 1, further comprising a receptacle being adapted to receive a container and having an open or transparent bottom and/or a non-transparent marker.

### Claim 70 (depends on 1)

A method for controlling the robot according to claim 1, the method comprising the steps of:
obtaining by use of the sensor the location of one or more items, if any present, at the layer,
displaying on the display device, an image representing the one or more located items,
receiving manipulation input through the user input device indicative of desired exchange of content between the displayed items,
controlling the actuator in accordance with the received manipulation input.

### Claim 71 (depends on 70)

The method according to claim 70, wherein the sensor is a camera providing a digital image and the step of obtaining the location of one or more items comprising
obtaining the field of view used for providing the digital image;
for at least one item, recognising in the digital image a pre-defined pattern representing the item and the pattern's position in the field of view.

### Claim 72 (depends on 71)

The method according to claim 71, wherein the pre-defined pattern is a projected contour of an item on the experimental layer.

### Claim 73 (depends on 71)

The method according to claim 71, wherein the pre-defined pattern is an object, preferably not resembling the shape of the item, such as bar-code, a QR-code, a polygon, a circle, an ellipse.

### Claim 74 (depends on 70)

The method according to claim 70, wherein the manipulation input comprising:
selecting one or more items to obtain a manipulated content
instructing the robot to provide the manipulated content.

### Claim 75 (depends on 70)

The method according to claim 70, wherein step of obtaining the location of one or more items is repeated a number of time during the time at which a manipulation is carried out and the thereby obtained locations for each item are compared to identify movements of the items during manipulation.

### Claim 76 (depends on 75)

The method according to claim 75, further comprising generating an error signal if a movement of an item is identified as being larger than a predefined threshold.

### Claim 77 (depends on 70)

The method according to claim 70, the method comprising
obtaining by use of the sensor for an item a signal representing the content of the item, such as a number of bacteria colonies in a petri dish.

### Claim 78 (depends on 77)

The method according to claim 77, wherein the obtaining by use of the sensor for an item a signal representing the content of the item is repeated a plurality of time so as to obtain a plurality of consecutive signals each representing the content of the item at different time instances.
