Computer vision system for industrial equipment gauge digitization and alarms
AssigneeRicoh Company, Ltd.
InventorEdward L. Schwartz
Life of this patent
Estimating the term…
Abstract
A system for analog gauge monitoring uses a machine learning model for computer vision that is trained using synthetic training data generated based on one or a few images of the gauge being monitored and a geometric model describing the scale and the indicator of the gauge. In some embodiments, the synthetic training data is generated using an image model implemented as a generative adversarial network (GAN) type neural network and trained to modify an image of a given gauge such that the gauge face is preserved while the gauge indicator is added to or removed from the image of the given gauge for any given gauge.
Claims20
A method for monitoring a gauge having a face including markings denoting a scale for a parameter value being measured by the gauge and an indicator indicating a reading of the parameter value over the scale by displacement of the indicator on the face, the method comprising:
- receiving a set of training images of the gauge;
- receiving data relating to annotations of the training images of the gauge, the annotations denoting the scale and the indicator on the face of the gauge;
- generating a geometric model describing the gauge using geometric information derived from the data relating to the annotations;
- providing an image model previously trained to modify an image of a given gauge such that a gauge face is preserved while a gauge indicator is added to or removed from the image of the given gauge for any given gauge, the previously trained image model being a neural network that is trained using a mask to distinguish the gauge indicator from the gauge face for any given gauge;
- generating, using the geometric model and the previously trained image model, synthetic images of the gauge, the synthetic images having indicator at different positions relative to the scale, the synthetic images forming a training data set, wherein generating the synthetic images of the gauge comprises:
- selecting an image patch from a respective training image of the gauge as input to the previously trained image model;
- providing a reference image patch of the indicator of the gauge to the previously trained image model;
- and
- applying the previously trained image model to erase the indicator from or draw the indicator onto the selected image patch;
- training a machine learning model for computer vision using the training data set;
- receiving operation images of the gauge during normal operation of the gauge;
- and
- applying the trained machine learning model for computer vision to the operation images to predict readings of the gauge.
A system for monitoring a gauge having a face including markings denoting a scale for a parameter value being measured by the gauge and an indicator indicating a reading of the parameter value over the scale by displacement of the indicator on the face, the system comprising:
- a hardware processor;
- and
- a memory coupled with the hardware processor, wherein the memory is configured to provide the processor with instructions which when executed cause the processor to:
- receive a set of training images of the gauge;
- receive data relating to annotations of the training images of the gauge, the annotations denoting the scale and the indicator on the face of the gauge;
- generate a geometric model describing the gauge using geometric information derived from the data relating to the annotations;
- providing an image model previously trained to modify an image of a given gauge such that a gauge face is preserved while a gauge indicator is added to or removed from the image of the given gauge for any given gauge, the previously trained image model being a neural network that is trained using a mask to distinguish the gauge indicator from the gauge face for any given gauge;
- generate, using the geometric model and the previously trained image model, synthetic images of the gauge, the synthetic images having indicator at different positions relative to the scale, the synthetic images forming a training data set, wherein to generate the synthetic images of the gauge, the instructions further cause the processor to:
- select an image patch from a respective training image of the gauge as input to the previously trained image model;
- provide a reference image patch of the indicator of the gauge to the previously trained image model;
- and
- apply the previously trained image model to erase the indicator from or draw the indicator onto the selected image patch;
- train a machine learning model for computer vision using the training data set;
- receive operation images of the gauge during normal operation of the gauge;
- and
- apply the trained machine learning model for computer vision to the operation images to predict readings of the gauge.
A non-transitory computer readable medium storing instructions which, when executed by a computer, cause the computer to:
- receive a set of training images of a gauge;
- receive data relating to annotations of the training images of the gauge, the annotations denoting a scale and an indicator on a face of the gauge;
- generate a geometric model describing the gauge using geometric information derived from the data relating to the annotations;
- provide an image model previously trained to modify an image of a given gauge such that a gauge face is preserved while a gauge indicator is added to or removed from the image of the given gauge for any given gauge, the previously trained image model being a neural network that is trained using a mask to distinguish the gauge indicator from the gauge face for any given gauge;
- generate, using the geometric model and the previously trained image model, synthetic images of the gauge, the synthetic images having indicator at different positions relative to the scale, the synthetic images forming a training data set, wherein to generate the synthetic images of the gauge, the instructions further cause the computer to:
- select an image patch from a respective training image of the gauge as input to the previously trained image model;
- provide a reference image patch of the indicator of the gauge to the previously trained image model;
- and
- apply the previously trained image model to erase the indicator from or draw the indicator onto the selected image patch;
- train a machine learning model for computer vision using the training data set;
- receive operation images of the gauge during normal operation of the gauge;
- and
- apply the trained machine learning model for computer vision to the operation images to predict readings of the gauge.
Dependencies are inferred from the claim text.