# US 2026/0073457 A1 · Work support system using wearable device for frontline workers

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0073457 A1
- Document ID: US20260073457A1
- Status: Published application
- Kind: A1
- Application number: 18/827,447
- Filed: 2024-09-06
- Published: 2026-03-12
- Earliest claimed: 2024-09-06
- Assignees: Hitachi, Ltd.
- Applicants: HITACHI, Ltd.
- Inventors: Naohiro KOHMU; Riu HIRAI; Takuya HABARA; Sudhanshu GAUR
- Web page: https://patentgrep.com/patent/US20260073457A1

## CPC

- G06Q 50/04: Manufacturing
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G06Q 10/06316: {Sequencing of tasks or work}
- G06V 10/761: {Proximity, similarity or dissimilarity measures}
- G06V 10/7715: {Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods}
- G06V 10/96: Management of image or video recognition tasks
- G06V 20/44: {Event detection}
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G06V 40/28: {Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113)}
- H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
- G06V 2201/06: Recognition of objects for industrial automation
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

An operator support system enhances efficiency of users, such as frontline workers, by using a wearable device equipped with cameras, audio interface, and display that captures hand movements and surrounding conditions, which allows for real-time task monitoring and interaction via natural language. The system integrates time-series analysis into a skill assessment mechanism that evaluates users' proficiency by comparing captured task data with pre-stored data. Based on the assessment, a machine learning system tailors user instructions for performing certain tasks. The system adapts to users' individual skill level, thereby improving workflow and reducing disruptions.

## Claims

### Claim 1

A method for assisting operators using a device, the method comprising:
in response to obtaining task-related features associated with time series data associated with an object in a first set of images captured by one or more cameras, accessing a database to obtain task data associated with time-series patterns corresponding to a plurality of skill levels and representing a sequence of actions associated with a task;
applying a time-series analysis to the task-related features and the task data to determine a time-series similarity representing a degree of match between the task-related features and the task data;
estimating a content of the task based on object recognition results from the time-series similarity;
setting a skill level information of a user based on the time-series similarity and the task data;
using machine learning to generate an instruction based on at least the skill level information and task-related features; and
communicating the instruction to a device coupled to the one or more cameras.

### Claim 2 (depends on 1)

The method of claim 1, wherein the task-related features associated with time series data comprise a transition of the object in the first set of images.

### Claim 3 (depends on 1)

The method of claim 1, wherein the task-related features associated with time series data comprise a time interval between two events in the time series data that represents a duration of the task.

### Claim 4 (depends on 1)

The method of claim 1, wherein setting the skill level information comprises accessing a skill assessment table in the database to calculate or adjust the time-series similarity.

### Claim 5 (depends on 1)

The method of claim 1, wherein generating the instruction comprises using a retrieval-augmented generation (RAG) system that incorporates the skill level information and retrieves information from the database based on task-related features identified in the time-series data

### Claim 6 (depends on 5)

The method of claim 5, wherein the RAG system further uses a user input related to the task to generate the instruction.

### Claim 7 (depends on 6)

The method of claim 6, further comprising:
monitoring a performance of the user during a task execution to gather performance data;
analyzing the performance data to adjust the skill level; and
storing at least one of the performance data or the user input in a knowledge storage system for future reference.

### Claim 8 (depends on 7)

The method of claim 7, wherein the knowledge storage system categorizes the stored data according to user skill levels to facilitate a revision of at least one of an instruction or a manual.

### Claim 9 (depends on 1)

The method of claim 1, wherein the device is a wrist-mounted device, and a first camera among the one or more cameras is a wide-angle camera configured to simultaneously capture, in response to obtaining at an audio interface a user instruction in a natural language format, images comprising hand gestures involving two hands in real time.

### Claim 10 (depends on 9)

The method of claim 9, wherein the device comprises a second camera among the one or more cameras that is configured to capture and display a second set of images that represent a surrounding environment.

### Claim 11

A system for assisting operators using a device, the system comprising:
a device coupled to one or more cameras;
a database configured to store task data associated with time-series patterns corresponding to a plurality of skill levels and representing a sequence of actions associated with a task;
a task estimation unit configured to analyze task-related features associated with time series data from an object in a first set of images captured by the one or more cameras and to estimate a content of the task based on object recognition results from a time-series similarity; and
a computing and communication system configured to couple to the database and at least one of the device or the task estimation unit, the computing and communication system comprising:
a similarity calculation unit that applies to the task-related features and the task data a time-series analysis to obtain the time-series similarity based on a degree of match between the task-related features and the task data;
a skill level determination unit configured to set a skill level information of a user based on the time-series similarity and the task data; and
a work instruction generation unit configured to use machine learning to generate an instruction based on at least the skill level information and task-related features, and to communicate the instruction to the device.

### Claim 12 (depends on 11)

The system of claim 11, further comprising an audio interface configured to obtain a user instruction in a natural language format.

### Claim 13 (depends on 12)

The system of claim 12, wherein the device is a wrist-mounted device and a first camera among the one or more cameras is a wide-angle camera configured to simultaneously capture, in response to the audio interface obtaining the user instruction, images comprising hand gestures involving two hands in real time.

### Claim 14 (depends on 11)

The system of claim 11, wherein the computing and communication system comprises a retrieval-augmented generation (RAG) system that generates the instruction based on the skill level information by retrieving information from the database.

### Claim 15 (depends on 14)

The system of claim 14, wherein the RAG system further uses a user input related to the task to generate the instruction.

### Claim 16 (depends on 11)

The system of claim 11, wherein the device comprises a second camera among the one or more cameras that is configured to capture and display a second set of images that represent a surrounding environment.

### Claim 17 (depends on 11)

The system of claim 11, wherein the task-related features associated with time series data comprise at least one of a transition of the object in the first set of images or a time interval between two events in the time series data that represents a duration of the task.

### Claim 18 (depends on 15)

The system of claim 15, further comprising a knowledge storage system that categorizes the stored data according to user skill levels to facilitate a revision of at least one of an instruction or a manual.

### Claim 19 (depends on 18)

The system of claim 18, wherein the knowledge storage system stores at least one of performance data or the user input for future reference.

### Claim 20 (depends on 19)

The system of claim 19, wherein the computing and communication system is configured to monitor and analyze the performance data during a task execution to adjust the skill level.
