# US 2019/0320519 A1 · Method and system for vision-based automatic fault notification and classification of system lighting > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0320519 A1 - Document ID: US20190320519A1 - Status: Published application - Kind: A1 - Application number: 16/158,673 - Filed: 2018-10-12 - Published: 2019-10-17 - Earliest claimed: 2018-04-17 - Assignees: Electronics and Telecommunications Research Institute - Applicants: Electronics and Telecommunications Research Institute - Inventors: Hyunseok KIM; Tae-Gyu KANG; Dae Ho KIM; You Jin KIM; Ji Hun JEON - Web page: https://patentgrep.com/patent/US20190320519A1 ## CPC - H05B 47/20: Responsive to malfunctions or to light source life; for protection - G05B 23/0281: {Quantitative, e.g. mathematical distance; Clustering; Neural networks; Statistical analysis} - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/08: Learning methods - G06N 3/09: Supervised learning - G06N 5/022: Knowledge engineering; Knowledge acquisition - G06T 7/0004: {Industrial image inspection} - H05B 45/50: responsive to malfunctions or undesirable behaviour of LEDs; responsive to LED life; Protective circuits - H05B 45/59: for reducing or suppressing flicker or glow effects - G06N 3/044: Recurrent networks, e.g. Hopfield networks - G06N 3/045: Combinations of networks - G06T 2207/10016: Video; Image sequence - H05B 47/1965: {using handheld communication devices} - H05B: ELECTRIC HEATING; ELECTRIC LIGHT SOURCES NOT OTHERWISE PROVIDED FOR; CIRCUIT ARRANGEMENTS FOR ELECTRIC LIGHT SOURCES, IN GENERAL - G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A lighting fault diagnosis method includes determining a fault item among a plurality of fault items based on fault symptom data of a test for each fault item, and recommending a repair method suitable for the determined fault item. ## Claims ### Claim 1 A method for fault diagnosis of a lighting, the method comprising: receiving fault symptom data of a test for each of one or more predetermined fault items, the fault symptom data tagged with the fault items, based on the test for each of the one or more predetermined fault items; determining a first fault item among the one or more predetermined fault items; retrieving a fault comparison image corresponding to the first fault item from a fault symptom image database (DB); diagnosing whether the fault symptom data corresponds to the first fault item by comparing fault symptom data tagged with the first fault item to the fault comparison image; and recommending a repair method based on the fault symptom image DB when the fault symptom data corresponds to the first fault item. ### Claim 2 (depends on 1) The method of claim 1, wherein the determining is iteratively performed while a predetermined condition is satisfied. ### Claim 3 (depends on 2) The method of claim 2, wherein the predetermined condition includes a condition to iteratively perform the determining with respect to all of the one or more predetermined fault items in an order of high frequency count. ### Claim 4 (depends on 1) The method of claim 1, wherein the recommending comprises recommending the repair method using a neural network trained with data accumulated in the fault symptom image DB through deep learning. ### Claim 5 (depends on 1) The method of claim 1, further comprising: labeling and storing the fault symptom data in the fault symptom image DB when the fault symptom data corresponds to the first fault item. ### Claim 6 (depends on 1) The method of claim 1, wherein the diagnosing comprises: determining a predetermined determination method corresponding to the first fault item; and comparing the fault symptom data tagged with the first fault item to the fault comparison image based on the determination method. ### Claim 7 (depends on 6) The method of claim 6, wherein the predetermined determination method comprises at least one of a per unit time brightness change data comparison, a per unit time brightness peak-to-peak variation comparison, an image color coordinate comparison, an image audio output comparison, and a brightness change timepoint comparison. ### Claim 8 (depends on 1) The method of claim 1, wherein the diagnosing comprises diagnosing whether a fault occurs using a data comparison algorithm including a mean squared error (MSE) and a correlation analysis. ### Claim 9 (depends on 1) The method of claim 1, wherein the fault symptom data include audio information and image information. ### Claim 10 (depends on 1) The method of claim 1, wherein the one or more predetermined fault items include at least one of dimming, a flicker, a color temperature, noise, and a response delay. ### Claim 11 An apparatus for fault diagnosis of a lighting, the apparatus comprising: a receiver configured to receive fault symptom data of a test for each of one or more predetermined fault items, the fault symptom data tagged with the fault items, based on the test for each of the one or more predetermined fault items; a fault symptom image database (DB) in which a fault symptom image is pre-stored; a diagnoser configured to determine a first fault item among the one or more predetermined fault items, retrieve a fault comparison image corresponding to the first fault item from a fault symptom image DB, and diagnose whether the fault symptom data corresponds to the first fault item by comparing the fault symptom data tagged with the first fault item to the fault comparison image; and a recommender configured to recommend a repair method based on the fault symptom image DB when the fault symptom data corresponds to the first fault item. ### Claim 12 A method of generating fault symptom data, the method comprising: controlling a lighting by executing a test control instruction for each fault item; capturing an image of the lighting; tagging and storing the captured image for each fault item; receiving a user opinion in relation to a fault symptom; generating fault symptom data of a test for each fault item based on device information of the lighting, the tagged image, and the user opinion; and transmitting the fault symptom data of the test for each fault item to a server. ### Claim 13 (depends on 12) The method of claim 12, further comprising: automatically extracting unique information of the lighting by recognizing a quick response (QR) code attached to the lighting. ### Claim 14 (depends on 12) The method of claim 12, further comprising: performing an image segmentation to segment a region corresponding to the lighting in all frames of the captured image; and normalizing the image.