US 2020/0349735 A1Published application

Multiple user simultaneous localization and mapping (slam)

AssigneeApple Inc.

InventorsAbdelhamid Dine, Kuen-Han Lin, Oleg Naroditsky

Life of this patent

Estimating the term…

Loading patent…

Abstract

In some implementations, a first electronic device including a first image sensor uses a processor to perform a method. The method involves obtaining a first set of keyframes based on images of a physical environment captured by the first image sensor. The method generates a mapping defining relative locations of keyframes of the first set of keyframes. The method receives a keyframe corresponding to an image of the physical environment captured at a second, different electronic device and localizes the received keyframe to the mapping. The method then receives an anchor from the second electronic device that defines a position of a virtual object relative to the keyframe. The method displays a CGR environment including the virtual object at a location based on the anchor and the mapping.

Claims20

Dependent claims

A method, comprising:

at a first electronic device having a first image sensor:
capturing a first set of keyframes based on images of a physical environment captured by the first image sensor, the first set of keyframes defined in a first coordinate system;
receiving a second set of keyframes corresponding to images of the physical environment captured by a second image sensor at a second electronic device, the second set of keyframes defined in a second coordinate system different than the first coordinate system;
generating a first mapping defining relative locations of keyframes of the first and second sets of keyframes in the first coordinate system, wherein generating the first mapping comprises
generating an internal mapping defining relative locations of the first set of keyframes in the first coordinate system;
generating an external mapping defining relative locations of the second set of keyframes;
and
pairing one keyframe of the first set of keyframes and one keyframe of the second set of keyframes;
and
transmitting only the first set of keyframes.

7 more elements

A system comprising:

a non-transitory computer-readable storage medium;
and
one or more processors coupled to the non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium comprises program instructions that, when executed on the one or more processors, cause the system to perform operations comprising:
at a first electronic device having a first image sensor:
obtaining a first set of keyframes based on images of a physical environment captured by the first image sensor, the first set of keyframes defined in a first coordinate system;
receiving a second set of keyframes corresponding to images of the physical environment captured by a second image sensor at a second electronic device, the second set of keyframes defined in a second coordinate system different than the first coordinate system;
and
generating a first mapping defining relative locations of keyframes of the first and second sets of keyframes in the first coordinate system, wherein generating the first mapping comprises generating an internal mapping defining relative locations of the first set of keyframes in the first coordinate system;
generating an external mapping defining relative locations of the second set of keyframes;
and pairing one keyframe of the first set of keyframes and one keyframe of the second set of keyframes;
and
at the second electronic device with the second image sensor:
receiving the first set of keyframes corresponding to images of the physical environment captured at the first electronic device;
and
generating a second mapping defining relative locations of keyframes of the first and second sets of keyframes in the second coordinate system, wherein generating the second mapping comprises generating an internal mapping defining relative locations of the first set of keyframes in the first coordinate system;
generating an external mapping defining relative locations of the second set of keyframes;
and pairing a keyframe of the first set of keyframes and a keyframe of the second set of keyframes.

14 more elements

A non-transitory computer-readable storage medium, storing program instructions computer-executable on a computer to perform operations comprising:

at a first electronic device having a first image sensor:
obtaining a first set of keyframes based on images of a physical environment captured by the first image sensor, the first set of keyframes defined in a first coordinate system;
receiving a second set of keyframes corresponding to images of the physical environment captured by a second image sensor at a second electronic device, the second set of keyframes defined in a second coordinate system different than the first coordinate system;
and
generating a first mapping defining relative locations of keyframes of the first and second sets of keyframes in the first coordinate system, wherein generating the first mapping comprises generating an internal mapping defining relative locations of the first set of keyframes in the first coordinate system;
generating an external mapping defining relative locations of the second set of keyframes;
and pairing one keyframe of the first set of keyframes and one keyframe of the second set of keyframes;
and
at the second electronic device with the second image sensor:
receiving the first set of keyframes corresponding to images of the physical environment captured at the first electronic device;
and
generating a second mapping defining relative locations of keyframes of the first and second sets of keyframes in the second coordinate system, wherein generating the second mapping comprises generating an internal mapping defining relative locations of the first set of keyframes in the first coordinate system;
generating an external mapping defining relative locations of the second set of keyframes;
and pairing a keyframe of the first set of keyframes and a keyframe of the second set of keyframes.

11 more elements

Dependencies are inferred from the claim text.

Facts

Application: 16/927,237Filed 13 Jul 2020
Priority: 2 May 2019
Published: 5 Nov 2020
Term ends: Not estimatedThe source records don’t support an estimate.

Classification7 codes

G06T 7/579
From multiple images: from motion
G06T 7/74
Using feature-based methods: involving reference images or patches
G06T 7/97
Image analysis: determining parameters from multiple pictures
G06T 19/00
Manipulating three-dimensional [3d] models or images for computer graphics

Family

Application16/401,287Continuation · filed 2 May 2019
Application16/927,237Continuation
US 10,748,302 B1Related patent
US 11,127,161 B2Related patent
US 2020/0349735 A1This patent

From the US filing history. Foreign family members aren’t listed.