# US 11,908,120 B2 · Fault detection method and system for tunnel dome lights based on improved localization loss function

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,908,120 B2
- Document ID: US11908120B2
- Status: Granted
- Kind: B2
- Application number: 17/895,713
- Filed: 2022-08-25
- Published: 2023-11-16
- Granted: 2024-02-20
- Earliest claimed: 2022-05-11
- Assignees: East China Jiaotong University
- Applicants: East China Jiaotong University
- Inventors: Gang Yang; Cailing Tang; Lizhen Dai; Zhipeng Yang; Yuyu Weng; Hui Yang; Rongxiu Lu; Fangping Xu; Shuo Xu
- Web page: https://patentgrep.com/patent/US11908120B2

## CPC

- G06T 7/0002: {Inspection of images, e.g. flaw detection}
- G06T 7/0004: {Industrial image inspection}
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06T 7/97: {Determining parameters from multiple pictures (depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85)}
- G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V 10/776: Validation; Performance evaluation
- G06V 10/82: using neural networks
- G06V 20/46: {Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames}
- G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle
- G06V 20/70: Labelling scene content, e.g. deriving syntactic or semantic representations
- G06T 2207/10016: Video; Image sequence
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A fault detection method and system for tunnel dome lights based on an improved localization loss function. The method includes: constructing a dataset of tunnel dome light detection images; acquiring a you only look once (YOLO) v5s neural network based on the improved localization loss function; training the YOLO v5s neural network according to the dataset to obtain a trained YOLO v5s neural network; acquiring a to-be-detected tunnel dome light image; detecting, with the trained YOLO v5s neural network, the to-be-detected tunnel dome light image to obtain position coordinates of luminous dome lights; and determining, according to the position coordinates of the luminous dome lights, whether a fault occurs in the tunnel dome lights. The present disclosure can accurately localize the tunnel dome lights and label the positions, and can detect whether the tunnel dome lights work normally according to a relative positional relationship between the labeled dome lights.

## Claims

### Claim 1

A fault detection method for tunnel dome lights based on an improved localization loss function, comprising:
constructing a dataset of tunnel dome light detection images;
acquiring a you only look once (YOLO) v5s neural network based on the improved localization loss function;
training the YOLO v5s neural network according to the dataset to obtain a trained YOLO v5s neural network;
acquiring a to-be-detected tunnel dome light image;
detecting, with the trained YOLO v5s neural network, the to-be-detected tunnel dome light image to obtain position coordinates of luminous dome lights; and
determining, according to the position coordinates of the luminous dome lights, whether a fault occurs in the tunnel dome lights;
wherein the constructing a dataset of tunnel dome light detection images specifically comprises:
acquiring videos of the tunnel dome lights with a multi-angle camera;
extracting video frames from acquired videos at intervals to obtain a tunnel dome light image set,
labeling tunnel dome lights in the tunnel dome light image set with Labeling software; and
processing labeled tunnel dome light images into a Pascal VOC format to obtain the dataset of the tunnel dome light detecting images;
the improved localization loss function is specifically a side, corner and aspect ratio loss for bounding box regression (SCALoss) function, and comprises a side overlap (SO) loss, a corner distance (CD) loss and an aspect ratio (AR) loss; and,
the SCALoss function has a following Eq:
LSCA=LSO+αLCD+LAR
wherein, LSCA is the SCALoss function, LSO is the SO loss, LCD is the CD loss, LAR is the AR loss, and α is a weight coefficient.

### Claim 2 (depends on 1)

The fault detection method for tunnel dome lights based on an improved localization loss function according to claim 1, wherein α is 0.25.

### Claim 3 (depends on 1)

The fault detection method for tunnel dome lights based on an improved localization loss function according to claim 1, wherein the determining, according to the position coordinates of the luminous dome lights, whether a fault occurs in the tunnel dome lights specifically comprises:
acquiring position coordinates of two consecutive luminous dome lights;
calculating a distance difference between the two consecutive luminous dome lights according to the position coordinates;
determining whether the distance difference is greater than a preset threshold; and
determining that the fault occurs in the tunnel dome lights if yes.

### Claim 4

A fault detection system for tunnel dome lights based on an improved localization loss function, comprising:
a dataset construction module, configured to construct a dataset of tunnel dome light detection images;
a neural network acquisition module, configured to acquire a you only look once (YOLO) v5s neural network based on the improved localization loss function;
a training module, configured to train the YOLO v5s neural network according to the dataset to obtain a trained YOLO v5s neural network;
a to-be-detected image acquisition module, configured to acquire a to-be-detected tunnel dome light image;
a position detection module, configured to detect, with the trained YOLO v5s neural network, the to-be-detected tunnel dome light image to obtain position coordinates of luminous dome lights; and
a fault recognition module, configured to determine, according to the position coordinates of the luminous dome lights, whether a fault occurs in the tunnel dome lights;
wherein the dataset construction module specifically comprises:
a video acquisition unit, configured to acquire videos of the tunnel dome lights with a multi-angle camera;
a frame extraction unit, configured to extract video frames from acquired videos at intervals to obtain a tunnel dome light image set;
an image labeling unit, configured to label tunnel dome lights in the tunnel dome light image set with Labelling software; and
a format conversion unit, configured to process labeled tunnel dome light images into a Pascal VOC format to obtain the dataset of the tunnel dome light detection images;
the improved localization loss function is specifically a side, corner and aspect ratio loss for bounding box regression (SCALoss) function, and comprises a side overlap (SO) loss, a corner distance (CD) loss and an aspect ratio (AR) loss; and
the SCALoss function has a following Eq.:
LSCA=LSO+αLCD+LAR
wherein, the LSCA is the SCALoss function, LSO is the SO loss, LCD is the CD loss, LAR is the AR loss, and α is a weight coefficient.

### Claim 5 (depends on 4)

The fault detection system for tunnel dome lights based on an improved localization loss function according to claim 4, wherein the fault recognition module specifically comprises:
a coordinate acquisition unit, configured to acquire position coordinates of two consecutive luminous dome lights;
a calculation unit, configured to calculate a distance difference between the two consecutive luminous dome lights according to the position coordinates; and
a determination unit, configured to determine whether the distance difference is greater than a preset threshold, and determine that the fault occurs in the tunnel dome lights if yes.