# US 2026/0062039 A1 · Systems and methods for machine learning enhanced railway condition monitoring, assessment and prediction

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0062039 A1
- Document ID: US20260062039A1
- Status: Published application
- Kind: A1
- Application number: 19/385,757
- Filed: 2025-11-11
- Published: 2026-03-05
- Earliest claimed: 2021-05-19
- Assignees: Rutgers, The State University of New Jersey
- Applicants: Rutgers, The State University of New Jersey
- Inventors: Xiang Liu
- Web page: https://patentgrep.com/patent/US20260062039A1

## CPC

- B61L 23/041: {Obstacle detection}
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- B61L 23/042: {Track changes detection}
- B61L 25/025: {Absolute localisation, e.g. providing geodetic coordinates}
- B61L 25/026: {Relative localisation, e.g. using odometer}
- G06T 7/0002: {Inspection of images, e.g. flaw detection}
- G06T 7/579: from motion
- G06V 10/82: using neural networks
- G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle
- G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G06V 40/10: Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30184: Infrastructure
- G06T 2207/30196: Human being; Person
- G06T 2207/30242: Counting objects in image
- G06T 2207/30261: Obstacle
- B61L: GUIDING RAILWAY TRAFFIC; ENSURING THE SAFETY OF RAILWAY TRAFFIC (brakes or auxiliary equipment B61H, B61K; point or crossing construction E01B)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Systems and methods enable automated intelligent railway monitoring using railway image data from an imaging device. The railway image data includes image frames produced by the imaging device. A railway object recognition model is used to identify objects within the image frames and a railway condition is determined based on the object. A location and time associated with the railway condition are determined based on railway image data. Railway metrics are generated based on the railway condition and historical railway conditions. A railway condition prediction model is used to predict future railway conditions affecting an operating condition, an operating demand or both based on the railway metrics. An operating recommendation is determined to address the future railway condition and an alert is generated to with an indication of railway condition, the operating recommendation or both.

## Claims

### Claim 1

A method comprising:
receiving, by at least one processor, at least one railway sensor data feed from at least one sensor device;
wherein the at least one railway sensor data feed comprises sensor data of at least one railway geographic location associated with at least one sensor device geographic location of the at least one sensor device;
utilizing, by the at least one processor, at least one first sensor data recognition model to detect, within the sensor data of the at least one railway geographic location, at least one railway geographic location object indicative of at least one object at the at least one railway geographic location;
utilizing, by the at least one processor, at least one second sensor data recognition model to identify, within the sensor data of the at least one railway geographic location object, at least one railway geographic location condition state indicative of at least one particular railway condition associated with the at least one object at the at least one railway geographic location;
generating, by the at least one processor, at least one alert to at least one computing device associated with at least one user based at least in part on the at least one railway geographic location condition state indicative of the at least one particular railway condition, wherein the at least one alert comprises an indication of at least one of the at least one particular railway condition to facilitate preventing delay of at least one train.

### Claim 2 (depends on 1)

The method of claim 1, wherein the at least one first sensor data recognition model is trained to identify at least one presence of at least one train within the at least one railway geographic location.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
wherein the at least one railway geographic location condition state comprises a presence of at least one object in the at least one railway sensor data feed; and
determining, by the at least one processor, a distance from at least one train to the at least one object based at least in part on a motion estimation.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
utilizing, by the at least one processor, the at least one second sensor data recognition model to identify, within the sensor data of the at least one railway geographic location condition state comprising at least one person present within the at least one railway geographic location.

### Claim 5 (depends on 4)

The method of claim 4, wherein the at least one railway geographic location condition state comprises a classification of the at least one person as at least one of:
authorized personnel, or
at least one other person.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
utilizing, by the at least one processor, at least one inferencing algorithm to recognize each passenger on a train platform at the at least one railway geographic location; and
determining, by the at least one processor, a passenger count on the train platform indicative of an amount of passengers based at least in part on each passenger recognized on the train platform.

### Claim 7 (depends on 6)

The method of claim 6, further comprising generating, by the at least one processor, at least one passenger metric based at least in part on the passenger count, the train platform and the at least one railway geographic location.

### Claim 8 (depends on 1)

The method of claim 1, further comprising:
determining, by the at least one processor, a defect railway condition based at least in part on the at least one railway geographic location condition state being a track defect on at least one rail or in at least one track area.

### Claim 9 (depends on 1)

The method of claim 1, further comprising:
determining, by the at least one processor, at least one risk-based mitigation based at least in part on the at least one railway geographic location condition state; and
wherein the at least one risk-based mitigation comprises at least one action associated with at least one of engineering, train operation, construction, personnel management, education, enforcement or evaluation.

### Claim 10 (depends on 1)

The method of claim 1, further comprising:
receiving, by the at least one processor, the at least one railway sensor data feed comprising at least one fixed location image data feed from the at least one sensor device comprising at least one imaging device positioned on in at least one fixed location facing at least one railway grade crossing;
utilizing, by the at least one processor, the at least one first sensor data recognition model to identify, within the at least one fixed location image data feed, at least one object associated with signaling a state of the railway at the location; and
utilizing, by the at least one processor, the at least one second sensor data recognition model, trained on a second training dataset of annotated images, to recognize at least one device status indicative of the state of the railway at the location signaled by the at least one object, the at least one state being associated with presence or operation or both of at least one of: a train, the at least one railway grade crossing, at least one trespassing person, or at least one blockage.

### Claim 11

A system comprising:
at least one processor, wherein the at least one processor, upon execution of computer instructions, is configured to receive at least one railway sensor data feed from at least one sensor device;
wherein the at least one railway sensor data feed comprises sensor data of at least one railway geographic location associated with at least one sensor device geographic location of the at least one sensor device;
utilize at least one first sensor data recognition model to detect, within the sensor data of the at least one railway geographic location, at least one railway geographic location object indicative of at least one object at the at least one railway geographic location;
utilize at least one second sensor data recognition model to identify, within the sensor data of the at least one railway geographic location object, at least one railway geographic location condition state indicative of at least one particular railway condition associated with the at least one object at the at least one railway geographic location;
generate at least one alert to at least one computing device associated with at least one user based at least in part on the at least one railway geographic location condition state indicative of the at least one particular railway condition, wherein the at least one alert comprises an indication of at least one of the at least one particular railway condition to facilitate preventing delay of at least one train.

### Claim 12 (depends on 11)

The system of claim 11, wherein the at least one first sensor data recognition model is trained to identify at least one presence of at least one train within the at least one railway geographic location.

### Claim 13 (depends on 11)

The system of claim 11, wherein the at least one processor, upon execution of the computer instructions, is further configured to:
wherein the at least one railway geographic location condition state comprises a presence of at least one object in the at least one railway sensor data feed; and
determine a distance from at least one train to the at least one object based at least in part on motion estimation.

### Claim 14 (depends on 11)

The system of claim 11, wherein the at least one processor, upon execution of the computer instructions, is further configured to:
utilize the at least one second sensor data recognition model to identify, within the sensor data of the at least one railway geographic location condition state comprising at least one person present within the at least one railway geographic location.

### Claim 15 (depends on 14)

The system of claim 14, wherein the at least one railway geographic location condition state comprises a classification of the at least one person as at least one of:
authorized personnel, or
at least one other person.

### Claim 16 (depends on 11)

The system of claim 11, wherein the at least one processor, upon execution of the computer instructions, is further configured to:
utilize at least one inferencing algorithm to recognize each passenger on a train platform at the at least one railway geographic location; and
determine a passenger count on the train platform indicative of an amount of passengers based at least in part on each passenger recognized on the train platform.

### Claim 17 (depends on 16)

The system of claim 16, wherein the at least one processor, upon execution of the computer instructions, is further configured to generate at least one passenger metric based at least in part on the passenger count, the train platform and the at least one railway geographic location.

### Claim 18 (depends on 11)

The system of claim 11, wherein the at least one processor, upon execution of the computer instructions, is further configured to:
determine a defect railway condition based at least in part on the at least one railway geographic location condition state being a track defect on at least one rail or in at least one track area.

### Claim 19 (depends on 11)

The system of claim 11, wherein the at least one processor, upon execution of the computer instructions, is further configured to:
determine at least one risk-based mitigation based at least in part on the at least one railway geographic location condition state; and
wherein the at least one risk-based mitigation comprises at least one action associated with at least one of engineering, train operation, construction, personnel management, education, enforcement or evaluation.

### Claim 20 (depends on 11)

The system of claim 11, wherein the at least one processor, upon execution of the computer instructions, is further configured to:
receive the at least one railway sensor data feed comprising at least one fixed location image data feed from the at least one sensor device comprising at least one imaging device positioned on in at least one fixed location facing at least one railway grade crossing;
utilize the at least one first sensor data recognition model to identify, within the at least one fixed location image data feed, at least one object associated with signaling a state of the railway at the location; and
utilizing, by the at least one processor, the at least one second sensor data recognition model, trained on a second training dataset of annotated images, to recognize at least one device status indicative of the state of the railway at the location signaled by the at least one object, the at least one state being associated with presence or operation or both of at least one of: a train, the at least one railway grade crossing, at least one trespassing person, or at least one blockage.
