# US 12,118,674 B2 · Configuration tool and method for a quality control system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,118,674 B2 - Document ID: US12118674B2 - Status: Granted - Kind: B2 - Application number: 18/318,249 - Filed: 2023-05-16 - Published: 2023-09-14 - Granted: 2024-10-15 - Earliest claimed: 2017-09-27 - Assignees: Arkite NV - Applicants: Arkite NV - Inventors: Ives De Saeger - Web page: https://patentgrep.com/patent/US12118674B2 ## CPC - G06Q 10/06: Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling - G06T 19/003: {Navigation within 3D models or images} - G01F 15/0755: {involving digital counting} - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06T 7/00: Image analysis - G06T 15/08: Volume rendering - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - G06T 2207/10028: Range image; Depth image; 3D point clouds - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G01F: MEASURING VOLUME, VOLUME FLOW, MASS FLOW OR LIQUID LEVEL; METERING BY VOLUME - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A configuration tool adapted to configure a quality control system to monitor and/or guide an operator in a working environment through recognition of objects, events or an operational process, comprises: a volumetric sensor adapted to capture volumetric image frames of the working environment while an object, event or operational process is demonstrated; a display, coupled to the volumetric sensor and configured to live display the volumetric image frames; and a processor configured to: generate a user interface in overlay of the volumetric image frames to enable a user to define a layout zone; and automatically generate a virtual box in the layout zone when an object, event or operational process is detected during demonstration of the object, event or operational process. ## Claims ### Claim 1 A configuration tool adapted to configure a quality control system to monitor and/or guide an operator in a working environment through recognition of objects, events or an operational process, said configuration tool comprising: A—a volumetric sensor adapted to capture volumetric image frames of said working environment while an object, event or operational process is demonstrated; B—a display, coupled to said volumetric sensor and configured to live display said volumetric image frames; and C—a processor configured to: C1—generate a user interface in overlay of said volumetric image frames to enable a user to define a layout zone; C2—automatically generate a virtual box in said layout zone when an object, event or operational process is detected during demonstration of said object, event or operational process; and C3—assign a confidence level to said virtual box representative for the repetitiveness of the detection of said object during plural demonstrations of said operational process. ### Claim 2 (depends on 1) The configuration tool according to claim 1, wherein said processor is further configured to detect presence of a tool, a container or a grid through comparison of subsequent volumetric image frames during demonstration of said operational process, and to automatically generate either: a virtual tool box at the location where said tool has been detected, for sensing presence of said tool during normal operation; or a virtual container box at the location where said container has been detected, for sensing an event in or near said container during normal operation; or a plurality of virtual container boxes at the location where objects have been detected in or near said grid, for sensing an event in or near said grid during normal operation. ### Claim 3 (depends on 1) The configuration tool according to claim 1, wherein said processor is further configured to select a precision parameter for said virtual box, said precision parameter being one of: a fill level; an occlusion level; a standard deviation; or another parameter describing a change in a virtual box, wherein the parameter with highest difference between subsequent volumetric image frames during demonstration of said operational process is selected as precision parameter. ### Claim 4 (depends on 1) The configuration tool according to claim 1, wherein said processor is further configured to: detect one or more valid variants of said object, event or operational process through respective demonstrations; and automatically adapt said virtual box to include detection of said one or more valid variants during normal execution of said operational process. ### Claim 5 (depends on 1) The configuration tool according to claim 1, wherein said processor is further configured to: detect one or more wrong variants of said object, event or operational process through respective demonstrations; and automatically adapt said virtual box or generate a negative virtual box to exclude detection of said one or more wrong variants during normal execution of said operational process. ### Claim 6 (depends on 1) The configuration tool according to claim 1, wherein said processor is further configured to: detect said object or event during normal execution of said operational process; and automatically adapt said virtual box in view of detection of said object or event during normal execution. ### Claim 7 (depends on 1) The configuration tool according to claim 1, wherein said processor is further configured to: automatically re-configure said virtual box upon detection of a position change of said volumetric sensor. ### Claim 8 (depends on 1) The configuration tool according to claim 1, wherein said processor is further configured to: generate a user interface in overlay of said volumetric image frames to enable a user to define a work zone; and automatically generate a process box in said work zone when a sequence of events is detected during demonstration of said operational process. ### Claim 9 (depends on 8) The configuration tool according to claim 8, wherein said processor is further configured to detect an event during said demonstration by: C2a—capturing volumetric image frames during said demonstration; C2b—background filtering said volumetric image frames; C2c—comparing pixels of a volumetric image frame during said demonstration with corresponding pixels in a preceding volumetric image frame during said demonstration to determine a difference level for each pixel; C2d—counting the amount of pixels that differs between said volumetric image frame and said preceding volumetric image frame to obtain a frame difference value; and C2e—detecting a process step when said frame difference value for each one of a first minimum amount of subsequent frames exceeds a first threshold, and the aggregate frame difference value across said first minimum amount of subsequent frames exceeds a second threshold. ### Claim 10 (depends on 1) The configuration tool according to claim 1, wherein said processor is further configured to: automatically generate a process by sequencing multiple virtual boxes similar to said virtual box and/or process boxes similar to said process box, in correspondence with the order wherein events associated with said multiple virtual boxes and/or process boxes have been detected during said demonstration. ### Claim 11 (depends on 1) The configuration tool according to claim 1, wherein said user interface is adapted to enable a user to specify feedback for an operator during normal execution of said operational process. ### Claim 12 (depends on 1) The configuration tool according to claim 1, wherein said user interface is adapted to bring a 3D CAD model of said object into said volumetric image frames. ### Claim 13 A configuration method for configuring a quality control system to monitor and/or guide an operator in a working environment through recognition of events, said configuration method comprising: A—capturing volumetric image frames of said working environment while an object, event or operational process is demonstrated through a volumetric sensor; B—live displaying said volumetric image frames; C1—generating a user interface in overlay of said volumetric image frames to enable a user to define a layout zone; and C2—automatically generating a virtual box when an object, event or operational process is detected during demonstration of said object, event or operational process; and C3—assigning a confidence level to said virtual box representative for the repetitiveness of the detection of said object during plural demonstrations of said operational process. ### Claim 14 A quality control system for monitoring and/or guiding an operator in a working environment, said quality control system being coupled to a volumetric sensor for observing predetermined virtual boxes for presence or absence of a physical object or an operational action, said quality control system comprising: a configuration tool adapted to configure said quality control system, said configuration tool comprising: A—said volumetric sensor adapted to capture volumetric image frames of said working environment while an object, event or operational process is demonstrated; B—a display, coupled to said volumetric sensor and configured to live display said volumetric image frames; C—a processor configured to: C1—generate a user interface in overlay of said volumetric image frames to enable a user to define a layout zone; and C2—automatically generate a virtual box in said layout zone when an object, event or operational process is detected during demonstration of said object, event or operational process; and C3—assign a confidence level to said virtual box representative for the repetitiveness of the detection of said object during plural demonstrations of said operational process. ### Claim 15 (depends on 14) The quality control system according to claim 14, further comprising: a workflow module adapted to retrieve volumetric observational data from said working environment and configured to: D1—observe one or more virtual box generated by said configuration tool based on said volumetric observational data; D2—detect presence or absence of a physical object or an operational action within said one or more virtual box; D3—create an event if presence or absence of a physical object or an operational action is detected in said one or more virtual box; and D4—validate said event.