User guided iterative frame and scene segmentation via network overtraining
AssigneeMatterport, Inc.
InventorGary Bradski
Life of this patent
Estimating the term…
Abstract
Systems and methods for user guided iterative frame and scene segmentation are disclosed herein. The systems and methods can rely on overtraining a segmentation network on a frame. A disclosed method includes selecting a frame from a scene and generating a frame segmentation using the frame and a segmentation network. The method also includes displaying the frame and frame segmentation overlain on the frame, receiving a correction input on the frame, and training the segmentation network using the correction input. The method includes overtraining the segmentation network for the scene by iterating the above steps on the same frame or a series of frames from the scene.
Claims20
A computer-implemented method comprising:
- selecting a frame from a scene;
- generating a first frame segmentation using:
- (i) the frame;
- and (ii) a segmentation network;
- displaying on a device:
- (i) the frame;
- and (ii) the first frame segmentation overlaid on the frame;
- receiving a correction input directed to the frame;
- training the segmentation network using the correction input;
- generating, after training the segmentation network using the correction input, a revised frame segmentation using:
- (i) the frame;
- and (ii) the segmentation network;
- and
- displaying on the device:
- (i) the frame;
- and (ii) the revised frame segmentation overlaid on the frame.
A computer-implemented method comprising:
- selecting a frame from a scene;
- generating a first frame segmentation using:
- (i) the frame;
- and (ii) a segmentation network;
- displaying on a device:
- (i) the frame;
- and (ii) the first frame segmentation overlaid on the frame;
- receiving a correction input on the frame;
- training the segmentation network using the correction input;
- and
- training the segmentation network for the scene by iterating the selecting, generating, displaying, receiving, and training steps using one of:
- the same frame;
- or a series of frames from the scene.
A device comprising:
- a display;
- a frame selector instantiated on the device, wherein the frame selector is programmed to select a frame from a scene;
- a segmentation editor instantiated on the device, wherein the segmentation editor is programmed to, in response to the frame selector selecting the frame, display on the display:
- (i) the frame;
- and (ii) a frame segmentation overlaid on the frame;
- a correction interface configured to receive a correction input directed to the frame;
- and
- wherein the device is programmed to:
- provide the correction input to a trainer for a segmentation network;
- receive a revised frame segmentation from the segmentation network after the trainer has applied the correction input to the segmentation network;
- and
- display the revised frame segmentation overlaid on the frame.
Dependencies are inferred from the claim text.