Back to results
US 11,144,766 B2Granted
Method for fast visual data annotation
AssigneeSony Group Corporation
InventorCheng-Yi Liu
Life of this patent
Estimating the term…
Abstract
Fast visual data annotation includes automatic detection using an automatic detector to detect subjects and joints in video frames. Then, annotation with sampling is performed, including determining when a frame is a sample (e.g., based on comparison of frames). Replay and refinement is utilized where user is involved with manually annotating subjects and/or joints in only select video frames.
Claims15
A method comprising:
- receiving video content on a device;
- processing the video content with an automatic detector by the device, wherein the automatic detector is configured to detect one or more subjects and one or more joints within the video content, wherein the one or more subjects are detected using a first image processing algorithm which utilizes a first set of templates to detect and match specified shapes, and wherein the one or more joints are detected using a second image processing algorithm which utilizes a second set of templates to detect facial components;
- and
- performing a two-step manual target subject keeping and tuning, wherein the two-step manual target subject keeping and tuning comprises selecting samples of the video content to manually review, wherein the samples are frames selected based on a difference amount between one or more joints of a first frame and a second frame, and interpolating joint information for non-selected frames.
1 more element
An apparatus comprising:
- a non-transitory memory for storing an application, the application for:
- receiving video content;
- processing the video content with an automatic detector, wherein the automatic detector is configured to detect one or more subjects and one or more joints within the video content, wherein the one or more subjects are detected using a first image processing algorithm which utilizes a first set of templates to detect and match specified shapes, and wherein the one or more joints are detected using a second image processing algorithm which utilizes a second set of templates to detect facial components;
- and
- receiving input for performing a two-step manual target subject keeping and tuning, wherein the two-step manual target subject keeping and tuning comprises selecting samples of the video content to manually review, wherein the samples are frames selected based on a difference amount between one or more joints of a first frame and a second frame, and interpolating joint information for non-selected frames;
- and
- a processor coupled to the memory, the processor configured for processing the application.
4 more elements
A system comprising:
- a first device for:
- receiving video content;
- processing the video content with an automatic detector, wherein the automatic detector is configured to detect one or more subjects and one or more joints within the video content, wherein the one or more subjects are detected using a first image processing algorithm which utilizes a first set of templates to detect and match specified shapes, and wherein the one or more joints are detected using a second image processing algorithm which utilizes a second set of templates to detect facial components, wherein the one or more joints are detected within the video content by detecting straight lines with a bend connecting the straight lines;
- and
- a second device for:
- receiving input for performing a two-step manual target subject keeping and tuning, wherein the two-step manual target subject keeping and tuning comprises selecting samples of the video content to manually review, wherein the samples are frames selected based on a difference amount between one or more joints of a first frame and a second frame, wherein the samples are less than 100% of all of the frames, and interpolating joint information for non-selected frames.
3 more elements
Dependencies are inferred from the claim text.