# US 2021/0344833 A1 · Inspection workflow using object recognition and other techniques > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0344833 A1 - Document ID: US20210344833A1 - Status: Published application - Kind: A1 - Application number: 17/348,755 - Filed: 2021-06-15 - Published: 2021-11-04 - Earliest claimed: 2017-11-06 - Assignees: Fluke Corporation - Applicants: Fluke Corporation - Inventors: Peter A. Bergstrom; Brian Knight; Jamie Rhead - Web page: https://patentgrep.com/patent/US20210344833A1 ## CPC - H04N 23/64: {Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image} - G06Q 10/20: Administration of product repair or maintenance - G06V 20/20: in augmented reality scenes - H04N 7/183: {for receiving images from a single remote source} - G06F 3/0482: Interaction with lists of selectable items, e.g. menus - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Systems for guiding a user through a workflow routine can include an inspection tool, a user interface, memory, and a processor. The processor can provide instructions via the user interface to perform a workflow routine using the inspection tool and save acquiring inspection data to memory. Instructions can direct a user to which equipment to inspect and/or how to collect inspection data associated with one or more pieces of equipment. Systems can determine which equipment is available for inspection by the user, such as via image recognition or proximity detection, and instruct a user to acquire inspection data associated with such equipment. Workflow routine instructions can be provided to the user via various devices, such as an inspection tool, smartphone, or a tablet. ## Claims ### Claim 1 A system, comprising: a display; a processor communicatively coupled to the display; and a non-transitory computer readable medium configured to store executable instructions, wherein the executable instructions, when executed by the processor, cause the system to: receive an inspection workflow with a plurality pieces of equipment; determine a guidance related to the inspection workflow at a given time based on information of the plurality pieces of equipment and data collected at the given time by the system; and present, via the display, an indication of the guidance. ### Claim 2 (depends on 1) The system of claim 1, wherein the indication of the guidance comprises a direction, the system further comprising: a location device to detect location data of the system, wherein the executable instructions, when executed by the processor, further cause the system to: retrieve first location data of a piece of equipment among the plurality pieces of equipment; and determine the direction to the piece of equipment based on the first location data and second location data detected by the location device at the given time. ### Claim 3 (depends on 1) The system of claim 1, further comprising: a location device to detect location data of the system, wherein the executable instructions, when executed by the processor, further cause the system to: retrieve first location data of a piece of equipment among the plurality pieces of equipment; detect, based on the first location data and second location data detected by the location device at the given time, the piece of equipment being within a threshold distance from the system; determine the guidance to collect a type of measurement data from the piece of equipment based on an inspection status of the piece of equipment and the piece of equipment being within the threshold distance from the system; and associate the type of measurement data with the piece of equipment in the inspection workflow. ### Claim 4 (depends on 1) The system of claim 1, further comprising: a communication device to wirelessly communicate with a piece of equipment among the plurality pieces of equipment, wherein the executable instructions, when executed by the processor, further cause the system to: detect an establishment of a wireless communication to a piece of equipment among the plurality pieces of equipment; and determine the guidance to collect a type of measurement data from the piece of equipment based on the establishment of the wireless communication and an inspection status of the piece of equipment in the inspection workflow. ### Claim 5 (depends on 4) The system of claim 4, wherein the executable instructions, when executed by the processor, further cause the communication device to: wirelessly collect the type of measurement data from the piece of equipment among the plurality pieces of equipment. ### Claim 6 (depends on 1) The system of claim 1, further comprising: an imaging device to capture images, wherein the executable instructions, when executed by the processor, cause the system to: retrieve a reference image associated with a piece of equipment among the plurality pieces of equipment; detect, based on image data collected at the given time by the imaging device and the reference image, the piece of equipment being within a threshold distance from the system; and generate the guidance to collect measurement data from the piece of equipment based on an inspection status of the piece of equipment and the piece of equipment being within the threshold distance from the imaging device. ### Claim 7 (depends on 6) The system of claim 6, wherein the image data collected at the given time is acoustic image data, infrared image data, or visible light image data. ### Claim 8 (depends on 1) The system of claim 1, further comprising: an imaging device to capture images, wherein the executable instructions, when executed by the processor, cause the system to: retrieve a viewing perspective of a first reference image associated with a piece of equipment among the plurality pieces of equipment; detect, based on image data collected at the given time by the imaging device and the first reference image, the piece of equipment being within a threshold distance from the imaging device; determining the guidance to collect a second reference image based on an inspection status of the piece of equipment, the piece of equipment being within the threshold distance from the imaging device, and the viewing perspective; and associating the second reference image with the piece of equipment in the inspection workflow. ### Claim 9 (depends on 8) The system of claim 8, wherein the viewing perspective comprises an orientation, a position, or a measuring distance. ### Claim 10 A method, comprising: receiving an inspection workflow with a plurality pieces of equipment; determining a guidance for completing the inspection workflow at a given time based on information of the plurality pieces of equipment and data collected at the given time by an inspection device; and revealing, via a user interface, an indication of the guidance. ### Claim 11 (depends on 10) The method of claim 10, further comprising: creating the inspection workflow with the plurality pieces of equipment; and collecting respective measurement data associated with the plurality pieces of equipment. ### Claim 12 (depends on 10) The method of claim 10, further comprising: retrieving first location data of a piece of equipment among the plurality pieces of equipment; detecting, based on the first location data and second location data collected at the given time by an inspection device, the piece of equipment being within a threshold distance from the inspection device; determining the guidance to collect a type of measurement data from the piece of equipment based on an inspection status of the piece of equipment and the piece of equipment being within the threshold distance from the inspection device; and associating the type of measurement data with the piece of equipment in the inspection workflow. ### Claim 13 (depends on 10) The method of claim 10, further comprising: retrieving a viewing perspective of a first reference image associated with a piece of equipment among the plurality pieces of equipment; detecting, based on image data collected at the given time by an inspection device and the first reference image, the piece of equipment being within a threshold distance from the inspection device; determining the guidance to collect a second reference image based on an inspection status of the piece of equipment, the piece of equipment being within the threshold distance from the inspection device, and the viewing perspective; and associating the second reference image with the piece of equipment in the inspection workflow. ### Claim 14 (depends on 13) The method of claim 13, further comprising: updating the inspection status of the piece of equipment based on the second reference image. ### Claim 15 A non-transient computer-readable medium containing executable instructions, wherein the executable instructions, when executed by a processor of a system, cause the system to: receive an inspection workflow with a plurality pieces of equipment; determine a guidance related to the inspection workflow at a given time based on information of the plurality pieces of equipment and data collected at the given time by the system; and present, via a user interface, the guidance related to the inspection workflow. ### Claim 16 (depends on 15) The computer-readable medium of claim 15, wherein the executable instructions, when executed by the processor, further cause the system to: detect a wireless communication between the system and a piece of equipment among the plurality pieces of equipment; and determine the guidance to collect a type of measurement data from the piece of equipment based on the wireless communication and an inspection status of the piece of equipment in the inspection workflow. ### Claim 17 (depends on 16) The computer-readable medium of claim 16, wherein the executable instructions, when executed by the processor, further cause the system to: based on the guidance, wirelessly collect the type of measurement data from the piece of equipment. ### Claim 18 (depends on 15) The computer-readable medium of claim 15, wherein data collected at the given time by the system comprises image data, wherein the executable instructions, when executed by the processor, further cause the system to: recognize, based on image data, a piece of equipment among the plurality pieces of equipment being within a threshold distance from the system; and determine the guidance to collect a type of measurement data from the piece of equipment based on an inspection status of the piece of equipment in the inspection workflow and the piece of equipment being within the threshold distance from the system. ### Claim 19 (depends on 15) The computer-readable medium of claim 15, wherein data collected at the given time by the system comprises location data, wherein the executable instructions, when executed by the processor, further cause the system to: recognize, based on location data, a piece of equipment among the plurality pieces of equipment being within a threshold distance from the system; and determine the guidance to collect a type of measurement data from the piece of equipment based on an inspection status of the piece of equipment in the inspection workflow and the piece of equipment being within the threshold distance from the system. ### Claim 20 (depends on 15) The computer-readable medium of claim 15, wherein the executable instructions, when executed by the processor, further cause the system to: in response to detecting only one piece of equipment being within a threshold range from the system, select the only one piece of equipment for inspection; and present, via the user interface, a reference image and an inspection process of the only one piece of equipment.