Back to results
US 9,503,681 B1Granted
Simulated transparent display with augmented reality for remote collaboration
AssigneePurdue Research Foundation
InventorsVoicu S Popescu, Juan P Wachs
Life of this patent
Estimating the term…
Abstract
A method and system for remote collaboration and remote instruction utilizing computing devices at trainee site and mentor sites are disclosed. Annotations are superimposed onto a trainee's view of a view field displayed using a simulated transparent display with augmented reality (“AR”). By tracking the position of the trainee's view point, the position of the trainee's computing device, and the geometry of the view field, the annotations remain anchored to the image of the view field when the trainee device moves and/or when the view field deforms or becomes occluded.
Claims20
A system comprising:
- a first computing device, the first display device comprising:
- a first processor;
- a first camera coupled to or operable by the first processor, the first camera configured to capture a reference image and a current image of a view field;
- a display configured to display the current image;
- a depth acquisition module coupled to or operable by the first processor, the depth acquisition module configured to acquire geometry information of the view field;
- and
- a first transceiver coupled to or operable by the first processor, the first transceiver configured to:
- transmit the reference image and the geometry information;
- and
- receive annotation information to be displayed on the first display;
- and
- a second computing device communicatively connectable with the first computing device, the second computing device comprising:
- a second processor;
- a user-operable input device coupled or operable by to the second processor;
- a second transceiver coupled or operable by to the second processor, the second transceiver configured to receive the reference image and the geometry information from the first computing device;
- an annotation module coupled to or operable by the second processor to receive input via the input device and determine at least one annotation based at least in part on the input;
- and
- a second display coupled to or operable by the second processor, the second display configured to display the reference image and to display the at least one annotation superimposed over the reference image,
- wherein:
- the second transceiver is further configured to transmit reference frame data to the first computing device, the reference frame data associated, at least in part, with the reference image, the geometry information, and the at least one annotation;
- and
- the first computing device is further configured to display the at least one annotation anchored to the displayed current image based on the received reference frame data.
19 more elements
A method comprising:
- acquiring a reference image and geometry information of operating view field by a first computing device;
- transmitting the reference image from the first computing device to a second computing device;
- displaying the reference image at the second computing device;
- determining an annotation at the second computing device;
- displaying the annotation over the reference image on a display of at the second computing device;
- generating reference frame data at the second computing device based, at least in part, on the reference image, the geometry information, and the annotation, the reference frame data comprising information associated with a plurality of image features in the reference image;
- transmitting the reference frame data and the annotation from the second computing device to the first computing device;
- receiving the reference frame data and the annotation by the first computing device;
- acquiring a current image of the view field by the first computing device;
- determining a position of the annotation in the current image based on the reference frame data;
- displaying the current image on a display of the first computing device;
- and
- displaying the annotation over the current image at the determined position on the display of the first computing device.
10 more elements
One or more non-transitory computer-readable media storing instructions that are executable by one or more processors to cause a computing system to:
- acquire a reference image and geometry information of a view field;
- transmit the reference image and the geometry information via a communications interface;
- receive reference frame data and an annotation via the communications interface;
- acquire a current image of the view field;
- determine a position of the annotation in the current image of the view field based on the reference frame data and geometry information;
- and
- display the annotation over the current image at the determined position on a computing device.
4 more elements
Dependencies are inferred from the claim text.