Capturing and processing facial motion data
AssigneesSony Corporation, Sony Pictures Entertainment Inc.
InventorsDemian Gordon, Remington Scott, Parag Havaldar, Dennis J. Hauck, Jr.
Life of this patent
Estimating the term…
Abstract
Capturing and processing facial motion data includes: coupling a plurality of sensors to target points on a facial surface of an actor; capturing frame by frame images of the plurality of sensors disposed on the facial surface of the actor using at least one motion capture camera disposed on a head-mounted system; performing, in the head-mounted system, a tracking function on the frame by frame images of the plurality of sensors to accurately map the plurality of sensors for each frame; and generating, in the head-mounted system, a modeled surface representing the facial surface of the actor.
Claims27
A system, comprising:
- at least one motion capture camera configured to capture motion data from a facial surface of an actor using a plurality of sensors disposed on the facial surface of the actor;
- and
- a motion capture processor coupled to said at least one motion capture camera, said motion capture processor including:
- a storage unit;
- an image capture module configured to control said at least one motion capture camera to capture frame by frame images of said plurality of sensors disposed on the facial surface of the actor, said image capture module receiving and storing said frame by frame images on said storage unit;
- a sensor tracking module configured to retrieve the stored frame by frame images from said storage unit, to perform a tracking function on the images to accurately map said plurality of sensors for each frame, and to provide said motion data from the facial surface;
- and
- a digital modeling module configured to receive said motion data to generate a modeled surface representing the facial surface of the actor;
- and
- a near-IR light configured to project speckle pattern on the facial surface of the actor and generating a 3-D mesh of the facial surface by triangulating the speckle pattern.
A method of capturing and processing facial motion data, comprising:
- coupling a plurality of sensors to target points on a facial surface of an actor;
- capturing frame by frame images of said plurality of sensors disposed on the facial surface of the actor using at least one motion capture camera disposed on a head-mounted system;
- performing, in the head-mounted system, a tracking function on said frame by frame images of said plurality of sensors to accurately map said plurality of sensors for each frame;
- and
- generating, in the head-mounted system, a modeled surface representing the facial surface of the actor;
- and
- projecting a speckle pattern on the facial surface of the actor using a near-IR light and generating a 3-D mesh of the facial surface by triangulating the speckle pattern.
A system for capturing and processing facial motion data, comprising:
- means for coupling a plurality of sensors to target points on a facial surface of an actor;
- means for capturing frame by frame images of said plurality of sensors disposed on the facial surface of the actor using at least one motion capture camera disposed on a head-mounted system;
- means for performing, in the head-mounted system, a tracking function on said frame by frame images of said plurality of sensors to accurately map said plurality of sensors for each frame;
- and
- means for generating, in the head-mounted system, a modeled surface representing the facial surface of the actor;
- and
- means for projecting a speckle pattern on the facial surface of the actor and generating a 3-D mesh of the facial surface by triangulating the speckle pattern.
Dependencies are inferred from the claim text.