# US 2022/0044494 A1 · Data extraction for machine learning systems and methods

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0044494 A1
- Document ID: US20220044494A1
- Status: Published application
- Kind: A1
- Application number: 17/394,022
- Filed: 2021-08-04
- Published: 2022-02-10
- Earliest claimed: 2020-08-06
- Assignees: Transportation IP Holdings, LLC
- Applicants: Transportation IP Holdings, LLC
- Inventors: Nathan Thomas North
- Web page: https://patentgrep.com/patent/US20220044494A1

## CPC

- G07C 5/006: {Indicating maintenance}
- G05B 23/0243: {model based detection method, e.g. first-principles knowledge model}
- G06F 30/27: using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model
- G07C 5/008: {communicating information to a remotely located station (transmission systems for measured values G08C)}
- G07C 5/0825: {using optical means}
- G07C 5/0833: {using audio means}
- G05B 2219/25204: Translate between different communication protocols
- G05B 2219/2637: Vehicle, car, auto, wheelchair
- G06N 20/00: Machine learning
- G07C: TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A system (e.g., a maintenance system) includes a maintenance task input unit and one or more processors. The maintenance task input unit is configured to generate plural input data groups corresponding to a maintenance task. Each of the input data groups is in a corresponding one of plural formats. The one or more processors are coupled to the maintenance task input unit and configured to obtain the plural input data groups from the maintenance task input unit. The one or more processors are configured to identify a particular format of the formats for each input data group, and to prepare an intermediate representation of the input representation based on the particular format that is identified. Also the one or more processors are configured to modify the intermediate representation to provide a model input having a common format. The one or more processors are configured to modify intermediate representations corresponding to each of the input data groups to the common format, and to provide the model input to a maintenance system artificial intelligence modeler that is configured to use the model input to at least one of develop or use a model using the model input for vehicle maintenance.

## Claims

### Claim 1

A system comprising:
a maintenance task input unit configured to generate plural input data groups corresponding to a maintenance task, each of the input data groups in a corresponding one of plural formats; and
one or more processors configured to obtain the plural input data groups from the maintenance task input unit, identify a particular format of the formats for each input data group, prepare an intermediate representation of the input data based on the particular format that is identified, modify the intermediate representation to provide a model input having a common format, wherein the one or more processors are configured to modify intermediate representations corresponding to each of the input data groups to the common format, and the one or more processors are configured to provide the model input to a maintenance system artificial intelligence modeller configured to use the model input to at least one of develop or use a model using the model input for vehicle maintenance.

### Claim 2 (depends on 1)

The system of claim 1, wherein the one or more processors are configured to prepare the intermediate representation by generating an n-dimensional array having an n-dimensional array format corresponding to the particular format identified.

### Claim 3 (depends on 2)

The system of claim 2, wherein the common format defines a common n-dimensional array format.

### Claim 4 (depends on 1)

The system of claim 1, wherein the plurality of formats include at least one of an image format, a text format, a video format, or an audio format.

### Claim 5 (depends on 1)

The system of claim 1, wherein the one or more processors are configured to modify the intermediate representation to provide the model input by altering at least one of a size or a shape of an n-dimensional array of the intermediate representation.

### Claim 6 (depends on 1)

The system of claim 1, wherein the model is configured to provide an evaluation of the maintenance task.

### Claim 7 (depends on 6)

The system of claim 6, wherein the maintenance task input unit is configured to generate the input data by acquiring an input from an operator performing the maintenance task.

### Claim 8 (depends on 6)

The system of claim 6, wherein the maintenance task input unit is configured to generate the input data by at least one of acquiring image information or acquiring audio information corresponding to the maintenance task.

### Claim 9

A method comprising:
obtaining plural input data groups of input information corresponding to a maintenance task, each of the input data groups in a corresponding one of a plurality of formats;
identifying a particular format of the plurality of formats for each input data group;
preparing, for each input data group, an intermediate representation of the input data based on the particular format that is identified for the input data;
modifying the intermediate representation to provide a model input having a common format, wherein intermediate representations corresponding to each of the plurality of inputs are modified to the common format; and
providing the model input to an artificial intelligence modeller configured to use to the model input to at least one of develop or use a model using the model input.

### Claim 10 (depends on 9)

The method of claim 9, wherein preparing the intermediate representation comprises generating an n-dimensional array having an n-dimensional array format corresponding to the particular format identified.

### Claim 11 (depends on 10)

The method of claim 10, wherein the common format defines a common n-dimensional array format.

### Claim 12 (depends on 9)

The method of claim 9, wherein the plurality of formats include at least one of an image format, a text format, a video format, or an audio format.

### Claim 13 (depends on 9)

The method of claim 9, wherein modifying the intermediate representation to provide the model input comprises altering at least one of a size or a shape of an n-dimensional array of the intermediate representation.

### Claim 14 (depends on 9)

The method of claim 9, further comprising:
performing the maintenance task;
generating the input data including information corresponding to the maintenance task;
evaluating the maintenance task using the model; and
performing a responsive task corresponding to an output of the model.

### Claim 15 (depends on 14)

The method of claim 14, wherein obtaining the input data comprises acquiring an input from an operator performing the maintenance task.

### Claim 16 (depends on 14)

The method of claim 14, wherein obtaining the input data comprises at least one of acquiring an image file or acquiring an audio file corresponding to the maintenance task.

### Claim 17

A method comprising:
performing a maintenance task;
generating an input data including information corresponding to the maintenance task; and
obtaining, with one or more processors, the input data of input information in a corresponding one of a plurality of formats;
identifying a particular format of the plurality of formats for the input data;
preparing an intermediate representation of the input data based on the particular format that is identified;
modifying the intermediate representation to provide a model input having a common format, wherein the one or more processors are configured to modify intermediate representations corresponding to each of the plurality of inputs to the common format; and
providing the model input to an artificial intelligence modeller configured to use to the model input to at least one of develop or use a model using the model input.

### Claim 18 (depends on 17)

The method of claim 17 comprising:
evaluating the maintenance task using the model; and
performing a responsive task corresponding to an output of the model.

### Claim 19 (depends on 17)

The method of claim 17, wherein obtaining the input data comprises acquiring an input from an operator performing the maintenance task.

### Claim 20 (depends on 17)

The method of claim 17, wherein obtaining the input data comprises at least one of acquiring an image file or acquiring an audio file corresponding to the maintenance task.
