Back to results
US 11,688,051 B2Granted
Automating inspection using neural network
AssigneeJohnson Controls Tyco IP Holdings LLP
InventorWalter Martin
Life of this patent
Estimating the term…
Abstract
Aspects of the present disclosure include methods, systems, and non-transitory computer readable media that perform the steps of capturing capture one or more images comprising a plurality of visual features of an inspected device, transmitting the one or more images to a server comprising a neural network, wherein the neural network analyzes the plurality of visual features to identify the inspected device and the server identifies the checklist comprising tasks or questions associated with an inspection process of the inspected device, receiving the checklist associated from the server, and displaying the checklist.
Claims17
A mobile device for inspection, comprising;
- a camera;
- a memory that store instructions;
- and
- a processor configured to execute the instructions to:
- cause the camera to capture one or more images comprising a plurality of visual features of an inspected device, wherein the plurality of visual features are configured to identify the inspected device, and wherein the inspected device is associated with an inspection process;
- identify location information associated with the inspected device;
- transmit the one or more images and the location information to a server comprising a neural network configured to identify the inspected device from the plurality of features and the location information and to identify a checklist of tasks or questions associated with the inspection process for the inspected device, wherein the neural network includes:
- a plurality of feature layers configured to extract the plurality features of the one or more images, and
- a plurality of identification layers configured to identify a plurality of inspected devices;
- receive the checklist associated with the inspected device as identified by the neural network from the server;
- and
- display a formatted user interface of one or more the tasks or the questions from the checklist.
9 more elements
A method of automating a display of a checklist at a computer device, comprising:
- capturing, by a camera of a computer device, one or more images comprising a plurality of visual features of an inspected device, wherein the plurality of visual features are configured to identify the inspected device, and wherein the inspected device is associated with an inspection process;
- identifying location information associated with the inspected device;
- transmitting, by a transmitter of the computer device, the one or more images and the location information to a server comprising a neural network configured to identify the inspected device from the plurality of features and the location information and to identify a checklist of tasks or questions associated with the inspection process for the inspected device, wherein the neural network includes:
- a plurality of feature layers configured to extract the plurality features of the one or more images, and
- a plurality of identification layers configured to identify a plurality of inspected devices;
- receiving, by a receiver of the computer device, the checklist associated with the inspected device as identified by the neural network from the server;
- and
- displaying, by a display of the computer device, a formatted user interface of one or more the tasks or the questions from the checklist.
5 more elements
A system, comprising:
- a neural network configured to identify an inspected device from a plurality of features and location information associated with the inspected device, wherein the neural network includes:
- a plurality of feature layers configured to extract the plurality features of one or more images, and
- a plurality of identification layers configured to identify a plurality of inspected devices;
- a server configured to identify a checklist of tasks or questions associated with an inspection process for the inspected device;
- and
- a mobile device configured to:
- capture, by a camera of the mobile device, the one or more images comprising the plurality of visual features of the inspected device, wherein the plurality of visual features are configured to identify the inspected device, and wherein the inspected device is associated with the inspection process;
- identify the location information associated with the inspected device;
- transmit, by a transmitter of the mobile device, the one or more images and the location information to the server;
- receive, by a receiver of the mobile device, the checklist associated with the inspected device as identified by the neural network;
- and
- displaying, by a display of the mobile device, a formatted user interface of one or more the tasks or the questions from the checklist.
9 more elements
A non-transitory computer readable medium comprising instructions stored therein that, when executed by a processor of a mobile device, cause the processor to:
- cause a camera to capture one or more images comprising a plurality of visual features of an inspected device, wherein the plurality of visual features are configured to identify the inspected device, and wherein the inspected device is associated with an inspection process;
- identify location information associated with the inspected device;
- transmit the one or more images and the location information to a server comprising a neural network configured to identify the inspected device from the plurality of features and the location information and to identify a checklist of tasks or questions associated with the inspection process for the inspected device, wherein the neural network includes:
- a plurality of feature layers configured to extract the plurality features of the one or more images, and
- a plurality of identification layers configured to identify a plurality of inspected devices;
- receive the checklist associated with the inspected device as identified by the neural network from the server;
- and
- display a formatted user interface of one or more the tasks or the questions from the checklist.
5 more elements
Dependencies are inferred from the claim text.