Back to results
US 11,080,938 B1Granted
Automatic summarization of remotely-guided augmented reality sessions
AssigneeInternational Business Machines Corporation
InventorsYochay Tzur, Eyal Mush
Life of this patent
Estimating the term…
Abstract
Receiving a recording of a remotely-guided augmented reality (AR) session which includes: images of a scene, captured by a camera of a local user; position and orientation data of the camera; and annotations generated by a remote user at points-of-interest (POIs) in a three-dimensional (3D) representation of the scene. Automatically generating a summary of the session, by: projecting the annotations to matching locations in some of the prominent images, based on the POIs of the annotations and on the position and orientation data of the camera and including, in the summary: the prominent images, including those of the prominent images having the projected annotations.
Claims15
A method comprising operating at least one hardware processor to:
- receive a recording of a remotely-guided augmented reality (AR) session, the recording comprising:
- images of a scene, captured by a camera of a local user,
- position and orientation data of the camera,
- annotations generated by a remote user at points-of-interest (POIs) in a three-dimensional (3D) representation of the scene;
- and
- automatically generate an electronic document which comprises a summary of the remotely-guided AR session, by:
- selecting prominent images out of the images of the scene, by:
- (a) defining the images of the scene as a set of candidate images (G),
- (b) adding, to an empty set (S), those of the images of G on which the annotations were indicated during the remotely-guided AR session,
- (c) removing from G images whose pairwise viewed region distance from any image in S is below a predefined threshold, and
- (d) adding to S additional images from G, using a non-maximal suppression algorithm in which images of G are sorted according to a focus score (F) and are rejected according to viewed region distance,
- such that S includes the prominent images,
- projecting the annotations to matching locations in some of the prominent images, based on the POIs of the annotations and on the position and orientation data of the camera, and
- including, in the summary:
- the prominent images, including those of the prominent images having the projected annotations.
12 more elements
A system comprising:
- (i) at least one hardware processor;
- and
- (ii) a non-transitory computer-readable storage medium having program code embodied therewith, the program code executable by said at least one hardware processor to:
- receive a recording of a remotely-guided augmented reality (AR) session, the recording comprising:
- images of a scene, captured by a camera of a local user,
- position and orientation data of the camera,
- annotations generated by a remote user at points-of-interest (POIs) in a three-dimensional (3D) representation of the scene;
- and
- automatically generate an electronic document which comprises a summary of the remotely-guided AR session, by:
- selecting prominent images out of the images of the scene, by:
- (a) defining the images of the scene as a set of candidate images (G),
- (b) adding, to an empty set (S), those of the images of G on which the annotations were indicated during the remotely-guided AR session,
- (c) removing from G images whose pairwise viewed region distance from any image in S is below a predefined threshold, and
- (d) adding to S additional images from G, using a non-maximal suppression algorithm in which images of G are sorted according to a focus score (F) and are rejected according to viewed region distance,
- such that S includes the prominent images,
- projecting the annotations to matching locations in some of the prominent images, based on the POIs of the annotations and on the position and orientation data of the camera, and
- including, in the summary:
- the prominent images, including those of the prominent images having the projected annotations.
15 more elements
A computer program product comprising a non-transitory computer-readable storage medium having program code embodied therewith, the program code executable by at least one hardware processor to:
- receive a recording of a remotely-guided augmented reality (AR) session, the recording comprising:
- images of a scene, captured by a camera of a local user,
- position and orientation data of the camera,
- annotations generated by a remote user at points-of-interest (POIs) in a three-dimensional (3D) representation of the scene;
- and
- automatically generate an electronic document which comprises a summary of the remotely-guided AR session, by:
- selecting prominent images out of the images of the scene, by:
- (a) defining the images of the scene as a set of candidate images (G),
- (b) adding, to an empty set (S), those of the images of G on which the annotations were indicated during the remotely-guided AR session,
- (c) removing from G images whose pairwise viewed region distance from any image in S is below a predefined threshold, and
- (d) adding to S additional images from G, using a non-maximal suppression algorithm in which images of G are sorted according to a focus score (F) and are rejected according to viewed region distance,
- such that S includes the prominent images,
- projecting the annotations to matching locations in some of the prominent images, based on the POIs of the annotations and on the position and orientation data of the camera, and
- including, in the summary:
- the prominent images, including those of the prominent images having the projected annotations.
12 more elements
Dependencies are inferred from the claim text.