# US 12,764,483 B2 · Acoustical or vibrational monitoring in a guided assembly system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,764,483 B2 - Document ID: US12764483B2 - Status: Granted - Kind: B2 - Application number: 18/903,401 - Filed: 2024-10-01 - Published: 2025-01-16 - Granted: 2026-10-13 - Earliest claimed: 2019-02-21 - Assignees: LightGuide, Inc. - Applicants: LightGuide, Inc. - Inventors: Paul Ryznar; William T. Sommerville; Ryan J. Feldman - Web page: https://patentgrep.com/patent/US12764483B2 ## CPC - H04R 29/004: {for microphones (H04R29/007 takes precedence)} - G05B 19/41805: {characterised by assembly} - G05B 2219/31027: Computer assisted manual assembly CAA, display operation, tool, result - G05B 2219/31046: Aid for assembly, show display on screen next workpiece, task, position to be assembled, executed - H04R 29/006: {Microphone matching} - H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K) - G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G) ## Abstract A guidance, monitoring, and inspection system for a work area includes a non-visual sensory detection sensor, such as a microphone or vibration detection sensor, visual sensors, other sensors, and a processor. The microphone or vibration sensor is configured to sense sounds or vibrations generated in the work area during the performance of an action that are then received by a processor. The visual sensor and other sensors sense characteristics and identity of objects present in the work area. The processor analyzes the received acoustic and/or vibrational signals and compares the received signals to an expected signal associated with an operational step that was performed to confirm that the operational step has been performed, and that it has been performed properly. The processor identifies a particular operational step as defined by the visual sensors and the other sensors. ## Claims ### Claim 1 A guidance, monitoring, and inspection system for a work area comprises: an operational guide system comprising a light projector, wherein the operational guide system is configured to direct the light projector to selectively provide visual indicators to guide actions of the individual in a work area; a sensor configured to sense sounds and/or vibrations generated in the work area during the performance of an operational step performed in the work area; a processor configured to receive signals from the sensor, wherein the signals comprise acoustic and/or vibrational signals; wherein the processor is configured to analyze the received signals that are received during the performance of the operational step performed in the work area; wherein the processor is configured to verify that the received signals match an identified signal signature associated with the operational step; wherein the processor is further configured to confirm that the operational step is correctly completed by confirming that the received signals match the signal signature associated with the operational step; and wherein the operational guide system is configured to indicate to the individual another operational step after the processor has confirmed completion of the operational step, and wherein the operational guide system is operable to hold indication of the another operational step until the processor has confirmed completion of the operational step. ### Claim 2 (depends on 1) The system of claim 1, wherein the sensor comprises a transducer. ### Claim 3 (depends on 1) The system of claim 1, wherein the operational guide system further comprises a camera configured to image actions of the individual in the work area, and wherein the processor is configured to analyze images of the actions of the individual in the work area to confirm completion of an action by the individual, and wherein the operational guide system is configured to indicate to the individual another operational step after the processor has confirmed both completion of the operational step and completion of the action by the individual. ### Claim 4 (depends on 1) The system of claim 1, wherein the operational step comprises a plurality of steps with each step associated with a particular portion of the associated signal signature, and wherein the particular portions of the associated signal signature are arranged in a particular order. ### Claim 5 (depends on 1) The system of claim 1, wherein the processor is configured to determine an associated time period within a time range for when a particular operational step is being performed during the task, wherein each particular portion of the associated signal signature is associated with a respective portion of the associated time period for when the particular operational step is being performed during the task. ### Claim 6 (depends on 1) The system of claim 1, wherein the operational guide system further comprises a camera configured to image the work area, and wherein the processor is configured to analyze images of the work area to confirm completion of an action by the individual, and wherein the operational guide system is configured to indicate to the individual another operational step after the processor has confirmed both completion of the operational step and completion of the action by the individual. ### Claim 7 A guidance, monitoring, and inspection system for a work area comprises: an operational guide system comprising a light projector, wherein the operational guide system is configured to direct the light projector to selectively provide visual indicators to guide actions of an individual in a work area; a first vibration sensor configured to sense vibration signals; a processor configured to receive the vibration signals from the first vibration sensor; wherein the processor is configured to analyze and compare the received vibration signals to a vibration signature associated with an operational step, wherein the received vibration signals are received during the performance of the operational step performed in the work area; wherein the processor is further configured to confirm that the operational step is correctly completed by confirming that the received vibration signal matches the vibration signature associated with the operational step; and wherein the operational guide system is configured to indicate to the individual another operational step after the processor has confirmed completion of the operational step, and wherein the operational guide system is operable to hold indication of the another operational step until the processor has confirmed completion of the operational step. ### Claim 8 (depends on 7) The guidance, monitoring, and inspection system of claim 7, wherein the operational step comprises a plurality of steps with a plurality of associated vibration signatures, and wherein the vibration signatures of the plurality of associated vibration signatures are arranged in a particular order. ### Claim 9 (depends on 7) The guidance, monitoring, and inspection system of claim 7, wherein the processor is configured to differentiate between two vibration signals based upon at least one of: frequencies of the vibrations signals, identification of particular vibrations associated with a coupling of a component to the work piece, and a quantity of the identified vibrations. ### Claim 10 (depends on 7) The guidance, monitoring, and inspection system of claim 7 further comprising a plurality of vibration sensors arranged around a periphery of the work area, wherein the vibration sensors are each configured to sense vibration signals, and wherein the processor is configured to determine a spatial location of the received vibration signals from the plurality of vibration sensors, wherein the spatial location is with respect to the location of each of the plurality of vibration sensors. ### Claim 11 (depends on 7) The guidance, monitoring, and inspection system of claim 7, wherein the processor is configured to determine an associated time period within a time range for when a particular operational step is being performed during the task. ### Claim 12 (depends on 7) The guidance, monitoring, and inspection system of claim 7 further comprising a network server configured to store a plurality of vibration signatures, each vibration signature corresponding to at least one particular operational step, wherein each operational step is associated with at least one corresponding vibration signature, and wherein the processor is configured to request all vibration signatures associated with a particular operational step from the network server for local storage. ### Claim 13 (depends on 7) The guidance, monitoring, and inspection system of claim 7, wherein the operational step includes the installation of a first component and a second component, wherein installing the first component produces an vibration signal that is different from the vibration signal produced by installing the second component, and wherein the first component is configured to produce a first vibration signal that matches a first vibration signature and the second component is configured to produce a second vibration signal that matches a second vibration signature, and wherein the associated vibration signature comprises the first vibration signature and the second vibration signature. ### Claim 14 A guidance, monitoring, and inspection system for a work area comprises: an operational guide system comprising a light projector, wherein the operational guide system is configured to direct the light projector to selectively provide visual indicators to guide actions of an individual in a work area; one or more sensors configured to sense sounds and/or vibrations, respectively, generated in the work area; a processor configured to receive acoustic signals and/or vibration signals from the one or more sensors; wherein the processor is configured to analyze and compare the received acoustic signals and/or vibration signals to an acoustic signature and/or vibration signature, respectively, associated with an operational step, wherein the received acoustic signals and/or vibrational signals are received during the performance of the operational step performed in the work area; wherein the processor is configured to perform the analysis and comparison of the received acoustic signals and/or vibration signals when the operational guide system indicates that the operational step is being carried out; wherein the operational guide system is configured to determine that the operational step is being carried out as defined by first inputs comprising one or more of: operator input of the individual, received sensor data, and received video data; wherein the processor is further configured to confirm that the operational step is correctly completed by at least one of: (1) confirming that the received acoustic signal and/or vibration signal matches the acoustic signature and/or vibration signature, respectively, associated with the operational step, and (2) receiving a confirmation indication from the operational guide system that the operational step has been completed, wherein the operational guide system is configured to confirm the operational step is correctly completed as defined by subsequent inputs comprising one or more of: additional operator inputs of the individual, additional sensor inputs, and additional video inputs; and wherein the operational guide system is configured to indicate to the individual another operational step after the processor has confirmed completion of the operational step, and wherein the operational guide system is operable to hold indication of the another operational step until the processor has confirmed completion of the operational step. ### Claim 15 (depends on 14) The guidance, monitoring, and inspection system of claim 14, wherein the sensor is a transducer. ### Claim 16 (depends on 14) The guidance, monitoring, and inspection system of claim 14, wherein the processor is further configured to identify the acoustic signature and/or vibration signature associated with the operational step by identifying a particular acoustic signature and/or vibration signature from a plurality of acoustic signatures and/or vibration signatures, respectively, that most closely matches the received acoustic signals and/or vibration signals, and verifying that the identified acoustic signature and/or vibration signature is an acoustic signature and/or vibration signature associated with the operational step. ### Claim 17 (depends on 14) The guidance, monitoring, and inspection system of claim 14, wherein the operational input of the individual comprises selected operator feedback indicating the operational step is taking place, and wherein the additional operator inputs of the individual comprise subsequent operator feedback indicating the operational step has been completed. ### Claim 18 (depends on 14) The guidance, monitoring, and inspection system of claim 14, wherein the received sensor data comprises data associated with a work piece and/or components present in the work area that are associated with the operational step, and wherein the video data comprises spatial data corresponding to expected positioning and orientation of the work piece present in the work area with respect to the operational step. ### Claim 19 (depends on 18) The guidance, monitoring, and inspection system of claim 18, wherein the received sensor data comprises characteristic data and/or identification information associated with the work piece and/or components present in the work area. ### Claim 20 (depends on 19) The guidance, monitoring, and inspection system of claim 19, wherein the additional sensor inputs comprise additional data associated with the work piece and/or components present in the work area and comprising additional characteristic data associated with the work piece and/or components present in the work area and indicating that the operational step has been correctly completed, and wherein the additional video data inputs comprise spatial data corresponding to expected positioning and orientation of the work piece present in the work area with respect to the operational step and indicating that the operational step has been correctly completed.