# US 2024/0404508 A1 · Method for assisting a worker in a production line, data processing apparatus and computer program

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0404508 A1
- Document ID: US20240404508A1
- Status: Published application
- Kind: A1
- Application number: 18/680,744
- Filed: 2024-05-31
- Published: 2024-12-05
- Earliest claimed: 2023-06-01
- Applicants: Airbus SAS
- Inventors: Alberto COMIN; Nunzio SCIAMMETTA
- Web page: https://patentgrep.com/patent/US20240404508A1

## CPC

- G06Q 50/04: Manufacturing
- G10L 13/08: Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
- G05B 23/027: {Alarm generation, e.g. communication protocol; Forms of alarm}
- G06Q 10/101: Collaborative creation, e.g. joint development of products or services
- G10L 15/16: using artificial neural networks
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L 25/63: for estimating an emotional state
- G06Q 10/0872: {using inventory planning}
- G06Q 10/0877: {by inventory control or reporting using inventory tracking or counting}
- G10L 2015/223: {Execution procedure of a spoken command}
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)

## Abstract

A method for assisting a worker in a production line by: receiving a vocal input message from the worker; converting the vocal input message into a text input message; processing the text input message using a neural network to identify the content of the text input message and extract a request about at least one item related to the production line, wherein the neural network is configured as a large language model, LLM, and trained with information about the items of the production line; generating a response message containing information about the item of the production line using the neural network; converting the response message into a vocal response message; and providing the vocal response message to the worker.

## Claims

### Claim 1

A method for assisting a worker in a production line, comprising:
receiving a vocal input message from a worker;
converting the vocal input message into a text input message;
processing the text input message using a neural network to identify a content of the text input message and extract a request about at least one item related to the production line, wherein the neural network is configured as a large language model, LLM, and trained with information about items of the production line;
generating a response message containing information about the at least one item of the production line using the neural network;
converting the response message into a vocal response message; and
providing the vocal response message to the worker.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the at least one item comprises at least one of a tool, an assembling part, a component of a product which is assembled in the production line, a production robot, and a characteristic of the production line, and
wherein the information about the at least one item includes at least one of” a manual of the at least one item, an inventory of the at least one item, live information about the at least one item or spare parts.

### Claim 3 (depends on 1)

The method according to claim 1, further comprising accessing and searching a database based on the extracted item to determine information related to the at least one item,
wherein the information includes at least a characteristic of the at least one item.

### Claim 4 (depends on 3)

The method according to claim 3, wherein the database includes a plurality of information sources, which includes at least one of at least one manual about an item of the production line, an inventory and a live status of the production line, wherein the live status includes a current position of the at least one item related to the production line.

### Claim 5 (depends on 3)

The method according to claim 3, wherein generating the response message comprises summarizing a plurality of information found in a database.

### Claim 6 (depends on 1)

The method according to claim 1, wherein the neural network is configured as a Generative Pretrained Transformer, GPT, or a generative artificial intelligence.

### Claim 7 (depends on 1)

The method according to claim 1, further comprising:
receiving a confirmation message from the worker based on the vocal response;
evaluating the confirmation message for a level of required modification of the response message, and
when the level of required modification exceeds a predetermined value, accessing and searching a database based on the confirmation message to determine further information related to the at least one item.

### Claim 8 (depends on 1)

The method according to claim 1, further comprising:
retrieving environmental data of the production line to determine a current situation, and
accessing and searching a database based on the extracted item to determine information related to the at least one item based on the retrieved environmental data.

### Claim 9 (depends on 8)

The method according to claim 8, further comprising determining a risk level of the current situation based on the environmental data.

### Claim 10 (depends on 9)

The method according to claim 9, further comprising:
adding an alert message to the response message, when the risk level is above a predetermined value.

### Claim 11 (depends on 9)

The method according to claim 9, further comprising:
sending a message to an assistant terminal, when the risk level is above a predetermined value, and
wherein the neural network preferably is configured to send an alert message independent of the request from the worker, when the risk level is above the predetermined value.

### Claim 12 (depends on 8)

The method according to claim 8, wherein retrieving the environmental data includes accessing wearables of the worker, in particular at least one of a headset and smart glasses of the worker.

### Claim 13 (depends on 1)

The method according to claim 1, further comprising:
analyzing the vocal input message for determining a stress level of the worker using the neural network,
wherein generating the response message is performed based on the determined stress level of the worker.

### Claim 14 (depends on 1)

A data processing apparatus comprising:
an input device,
a data storage, and
a processor configured to perform the method according to claim 1.

### Claim 15 (depends on 1)

A non-transitory computer readable medium storing a computer program comprising instructions, which, when the program is executed by a processor of a computer, cause the computer to carry out the method of claim 1.
