# US 2022/0222391 A1 · Controlling of Manual Operations Using a Portable Hand Tool during Production of an Assembly of Parts

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0222391 A1
- Document ID: US20220222391A1
- Status: Published application
- Kind: A1
- Application number: 17/606,885
- Filed: 2020-04-23
- Published: 2022-07-14
- Earliest claimed: 2019-05-13
- Assignees: InBolt
- Applicants: InBolt
- Inventors: Louis DUMAS; Rudy COHEN; Albane DERSY
- Web page: https://patentgrep.com/patent/US20220222391A1

## CPC

- G05B 19/41805: {characterised by assembly}
- G06F 30/17: Mechanical parametric or variational design
- B25F 5/02: Construction of casings, bodies or handles {(B25F5/006, B25F5/008 take precedence)}
- G06F 30/12: characterised by design entry means specially adapted for CAD, e.g. graphical user interfaces [GUI] specially adapted for CAD
- G05B 2219/31027: Computer assisted manual assembly CAA, display operation, tool, result
- G05B 2219/31046: Aid for assembly, show display on screen next workpiece, task, position to be assembled, executed
- G05B 2219/45055: Assembly
- G05B 2219/45127: Portable, hand drill
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- B25F: COMBINATION OR MULTI-PURPOSE TOOLS NOT OTHERWISE PROVIDED FOR; DETAILS OR COMPONENTS OF PORTABLE POWER-DRIVEN TOOLS NOT PARTICULARLY RELATED TO THE OPERATIONS PERFORMED AND NOT OTHERWISE PROVIDED FOR {(machines or arrangements of machines for performing specified combinations of different metal-working operations not covered by a single other subclass B23P23/00 and subgroups)}

## Abstract

The invention relates to a system for controlling manual operations during production of an assembly of parts, the system comprising a portable hand tool allowing an assembly technician to perform manual operations on a plurality of elements of said assembly of parts, and locating means integrated at least in part into an electronic module attached to said portable hand tool and able to determine a location of at least one manual operation associated with an element by locating the position of said portable hand tool in a three-dimensional coordinate system associated with a set of modelling data representing a three-dimensional modelled image of said assembly of parts. According to the invention, said locating means comprise a depth camera belonging to said electronic module and a processing module able to estimate the position of said portable hand tool on the basis of images captured by said depth camera.

## Claims

### Claim 1

A system for controlling manual operations during production of an assembly of parts, the system comprising:
a portable hand tool allowing an assembly technician to perform manual operations on a plurality of elements of said assembly of parts, and locating means integrated at least in part into an electronic module attached to said portable hand tool, said locating means being able to determine a location of at least one manual operation associated with an element by locating the position of said portable hand tool in a three-dimensional coordinate system, characterized in that said three-dimensional coordinate system is linked to a three-dimensional model of said assembly of parts,
in that the controlling system comprises a storage memory containing a plurality of CAD files allowing the construction of three-dimensional modelled images of the assembly of parts at various stages in production,
and in that said locating means comprise a depth camera belonging to said electronic module and a processing module able to estimate and save a current position of said portable hand tool, said current position being estimated by aligning point clouds extracted from at least one current image captured by said depth camera with a mesh or point clouds extracted from a three-dimensional modelled image representative of a current state of said assembly of parts, said three-dimensional modelled image representative of a current state being constructed on the basis of one CAD file from among the plurality of CAD files contained in the storage memory.

### Claim 2 (depends on 1)

The system according to claim 1, wherein said locating means further comprise an inertial measurement unit integrated into said electronic module, and in that the processing module is able to estimate a movement of the portable hand tool from a last estimated and saved current position on the basis of measurements delivered by said inertial measurement unit and to estimate at least one updated position of said portable hand tool on the basis of the last current position and said movement.

### Claim 3 (depends on 1)

The system according to claim 1, wherein the processing module is integrated into the electronic module.

### Claim 4 (depends on 1)

The system according to claim 1, wherein the processing module is integrated into a remote apparatus.

### Claim 5 (depends on 4)

The system according to claim 4,
wherein said locating means further comprise an inertial measurement unit integrated into said electronic module, and in that the processing module is able to estimate a movement of the portable hand tool from a last estimated and saved current position on the basis of measurements delivered by said inertial measurement unit and to estimate at least one updated position of said portable hand tool on the basis of the last current position and said movement,
wherein in that said electronic module comprises communication means able to transmit the images captured by the depth camera and the measurements from said inertial measurement unit to the remote apparatus for processing by the processing module.

### Claim 6 (depends on 1)

The system according to claim 1, further comprising at least one portable user interface able to automatically indicate to said assembly technician a setpoint parameter to be applied for said element according to the location of the associated manual operation determined by said locating means.

### Claim 7 (depends on 6)

The system according to claim 6, wherein said at least one portable user interface is integrated into said electronic module, and/or into a mobile telephone and/or into a portable tablet.

### Claim 8 (depends on 6)

The system according to claim 6, wherein the user interface is a display screen.

### Claim 9 (depends on 6)

The system according to claim 6, wherein said setpoint parameter is pre-saved in the storage module of said system saving a set of setpoint parameters to be applied to each of said plurality of elements of said assembly of parts.

### Claim 10 (depends on 1)

The system according to claim 1, wherein the electronic module (3) comprises validation means able to trigger automatic saving of information relating to the manual operation.

### Claim 11 (depends on 10)

The system according to claim 10, characterized in that the validation means is a manually operated button (35), the actuation of which triggers said automatic saving of information relating to the manual operation.

### Claim 12 (depends on 10)

The system according to claim 10, wherein said information comprises the time of validation, and a reference allowing the unique identification of the element associated with the manual operation.

### Claim 13 (depends on 1)

The system according to claim 1, wherein the hand tool is a tightening wrench or screwdriver, a riveting machine, a stapler, a drill, a cutter or a welder.

### Claim 14

A method for controlling manual operations during production of an assembly of parts, the method comprising the following steps:
an assembly technician using a portable hand tool to perform manual operations on a plurality of elements of said assembly of parts; and
determining a location of a manual operation associated with an element by locating the position of said portable hand tool in a three-dimensional coordinate system,
wherein
the three-dimensional coordinate system is linked to a three-dimensional model of said assembly of parts,
in that the method comprises a step of storing in a storage memory a plurality of CAD files allowing the construction of three-dimensional modelled images of the assembly of parts at various stages in production,
and in that the determining step comprises image captures by a depth camera belonging to said electronic module attached to said portable hand tool and a step of estimating and saving a current position of said portable hand tool by aligning point clouds extracted from at least one current image captured by said depth camera with a mesh or point clouds extracted from a three-dimensional modelled image representative of a current state of said assembly of parts, said three-dimensional modelled image representative of a current state being constructed on the basis of one CAD file from among the plurality of CAD files contained in the storage memory.

### Claim 15 (depends on 14)

The method according to claim 14, wherein said method further comprises a step of estimating a movement of the portable hand tool from a last estimated and saved current position on the basis of measurements delivered by an inertial measurement unit belonging to said electronic module, and of estimating at least one updated position of said portable hand tool on the basis of the last current position and said movement.

### Claim 16 (depends on 14)

The method according to claim 14, wherein said method further comprises a step of automatically transmitting, to a portable user interface, a setpoint parameter to be applied for said element according to the location of the associated manual operation from the determining step.

### Claim 17 (depends on 16)

The method according to claim 16, wherein the transmitting step consists in displaying said setpoint parameter on a display screen integrated into said electronic module, into a mobile telephone and/or into a portable tablet.
