# US 11,308,604 B2 · Image-based control of wire harness and connector manufacturing processes using artificial intelligence > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,308,604 B2 - Document ID: US11308604B2 - Status: Granted - Kind: B2 - Application number: 16/994,740 - Filed: 2020-08-17 - Published: 2021-02-18 - Granted: 2022-04-19 - Earliest claimed: 2019-08-18 - Assignees: Crimping & Stamping Technologies, Inc. - Applicants: Crimping & Stamping Technologies, Inc. - Inventors: Edgar Iván Sanchez Ortega - Web page: https://patentgrep.com/patent/US11308604B2 ## CPC - G06T 7/0004: {Industrial image inspection} - B25J 9/161: {Hardware, e.g. neural networks, fuzzy logic, interfaces, processor} - B25J 9/163: {learning, adaptive, model based, rule based expert control} - B25J 9/1669: {characterised by special application, e.g. multi-arm co-operation, assembly, grasping} - B25J 9/1697: {Vision controlled systems} - B60R 16/0215: {Protecting, fastening and routing means therefor} - G05B 19/21: using an incremental digital measuring device - G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting - G06F 18/2413: based on distances to training or reference patterns - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/08: Learning methods - G06N 3/09: Supervised learning - G06V 10/764: using classification, e.g. of video objects - G06V 10/82: using neural networks - B60R 16/0207: {Wire harnesses} - G05B 2219/40082: Docking, align object on end effector with target - G05B 2219/40087: Align hand on workpiece to pick up workpiece, peg and hole - G06T 2207/10024: Color image - G06T 2207/20081: Training; Learning - G06T 2207/20084: Artificial neural networks [ANN] - G06T 2207/30164: Workpiece; Machine component - G06V 2201/06: Recognition of objects for industrial automation - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50}) - B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR - G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract An embodiment provides a method of operating wire harness manufacturing equipment, including: adding, using the manufacturing equipment, an element to a wire to form a combination of the element and the wire; capturing, using an imaging device, an upper image and a lower image of the combination; analyzing, using one or more processors operatively coupled to the imaging device, the upper image and the lower image to detect a defect; and thereafter indicating that the defect has been detected. Other embodiments are described and claimed. ## Claims ### Claim 1 A method of operating wire harness manufacturing equipment, comprising: adding, using the wire harness manufacturing equipment, an element to a wire to form a combination of the element and the wire; capturing, using an imaging device, an upper image and a lower image of the combination without alignment referenced to a master or reference image; analyzing, using one or more processors operatively coupled to the imaging device, the upper image and the lower image to detect a defect; and thereafter indicating that the defect has been detected. ### Claim 2 (depends on 1) The method of claim 1, wherein the analyzing does not utilize a master or reference image and data derived therefrom supplied during a teach-in mode for the wire harness manufacturing equipment. ### Claim 3 (depends on 1) The method of claim 1, wherein the analyzing comprises identifying the element in one or more of the upper image and the lower image and thereafter identifying the defect in one or more of the upper image and the lower image. ### Claim 4 (depends on 3) The method of claim 3, wherein the analyzing comprises, after identifying the element, examining one or more predetermined regions in one or more of the upper image and the lower image. ### Claim 5 (depends on 4) The method of claim 4, wherein the one or more predetermined regions are automatically identified in the upper image and the lower image using location information automatically obtained for the element. ### Claim 6 (depends on 1) The method of claim 1, wherein the capturing comprises moving the combination manually. ### Claim 7 (depends on 1) The method of claim 1, wherein the analyzing comprises utilizing an artificial intelligence program to detect the defect. ### Claim 8 (depends on 7) The method of claim 7, wherein the artificial intelligence program comprises a trained neural network. ### Claim 9 (depends on 8) The method of claim 8, wherein the trained neural network has been trained to identify the element. ### Claim 10 (depends on 9) The method of claim 9, wherein the trained neural network has not been trained to identify the defect. ### Claim 11 (depends on 1) The method of claim 1, wherein the element is selected from the group consisting of a crimp, a seal, and another wire. ### Claim 12 (depends on 1) The method of claim 1, wherein the defect is selected from the group consisting of insulation position defects, bell mouth defects, brush defects, cut off tab defects, crimp wing defects, incorrect wing bending defects, cable seal defects, and wire strands pressed on top of a conductor crimp defects. ### Claim 13 A method, comprising: adding, using wire harness manufacturing equipment, an element to a wire to form a combination of the element and the wire; capturing, using an imaging device, an upper image and a lower image of the combination without alignment referenced to a master or reference image; analyzing, using one or more processors operatively coupled to the imaging device, the upper image and the lower image to detect if a defect is present; thereafter indicating that the defect has not been detected; and incorporating at least the combination of the element and the wire into a wire harness. ### Claim 14 (depends on 13) The method of claim 13, wherein the wire harness is one of a vehicle wire harness and an appliance wire harness. ### Claim 15 (depends on 14) The method of claim 14, comprising one of: incorporating the vehicle wire harness into a vehicle; and incorporating the appliance wire harness into a home appliance. ### Claim 16 (depends on 15) The method of claim 15, wherein the vehicle is selected from the group consisting of an automobile, a construction vehicle, and an aircraft. ### Claim 17 A method, comprising: obtaining, from an imaging device, an upper image and a lower image of an element added to a wire to form a combination of the element and the wire, wherein the upper and lower images were captured without alignment referenced to a master or reference image; analyzing, using one or more processors, the upper image and the lower image to detect a defect; and thereafter indicating that the defect has been detected. ### Claim 18 (depends on 17) The method of claim 17, wherein the obtaining comprises obtaining image data of the combination derived from manually waiving the combination in the field of view of the imaging device. ### Claim 19 (depends on 17) The method of claim 17, wherein the analyzing comprises using an artificial neural network to identify the element. ### Claim 20 (depends on 17) The method of claim 17, wherein the analyzing comprises using an artificial neural network to identify the defect directly.