# US 2013/0293468 A1 · Collaboration environment using see through displays

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2013/0293468 A1
- Document ID: US20130293468A1
- Status: Published application
- Kind: A1
- Application number: 13/464,941
- Filed: 2012-05-04
- Published: 2013-11-07
- Earliest claimed: 2012-05-04
- Assignees: Microsoft Technology Licensing, LLC
- Inventors: Kathryn Stone Perez; John Clavin; Kevin A. Geisner; Stephen G. Latta; Brian J. Mount; Arthur C. Tomlin; Adam G. Poulos
- Web page: https://patentgrep.com/patent/US20130293468A1

## CPC

- G06F 3/033: Pointing devices displaced or positioned by the user {, e.g. mice, trackballs, pens or joysticks}; Accessories therefor (digitisers characterised by the transducing means G06F3/041)
- G02B 27/017: {Head mounted}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06F 21/00: Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F 21/32: using biometric data, e.g. fingerprints, iris scans or voiceprints
- G06F 21/6218: {to a system of files or objects, e.g. local or distributed file system or database}
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G02B 2027/0138: {comprising image capture systems, e.g. camera}
- G02B 2027/014: {comprising information/image processing systems}
- G02B 2027/0178: {Eyeglass type (eyeglass details G02C)}
- G02B 2027/0187: {slaved to motion of at least a part of the body of the user, e.g. head, eye}
- G06F 2221/2141: Access rights, e.g. capability lists, access control lists, access tables, access matrices
- G06T 2219/024: Multi-user, collaborative environment
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A see-through, near-eye, mixed reality display device and system for collaboration amongst various users of other such devices and personal audio/visual devices of more limited capabilities. One or more wearers of a see through head mounted display apparatus define a collaboration environment. For the collaboration environment, a selection of collaboration data and the scope of the environment are determined. Virtual representations of the collaboration data in the field of view of the wearer, and other device users are rendered. Persons in the wearer's field of view to be included in collaboration environment and who are entitled to share information in the collaboration environment are defined by the wearer. If allowed, input from other users in the collaboration environment on the virtual object may be received and allowed to manipulate a change in the virtual object.

## Claims

### Claim 1

A method for presenting a collaboration experience using a see through head mounted display apparatus, comprising:
determining a three dimensional location of the apparatus, the apparatus includes one or more sensors and a see-through display;
determining an orientation of the apparatus;
determining a gaze of a wearer looking through the see-through display of the apparatus;
determining a three dimensional location of at one or more users in the field of view of the user through the see-through display, the determining of the three dimensional location of the movable object is performed using the one or more sensors;
receiving a selection of collaboration data and a selection of a collaboration environment within the field of view from the wearer;
rendering virtual representations of the collaboration data in the field of view;
determining persons in the wearer's field of view to be included in collaboration environment and who are entitled to share information in the collaboration environment;
outputting shared collaboration data in the form of virtual objects to users in the collaboration environment having audio/visual devices in the collaboration environment;
receiving input from users in the collaboration environment on the virtual object, the input manipulating a change in the virtual object; and
displaying the change in the virtual object to the wearer.

### Claim 2 (depends on 1)

The method of claim 1 further including:
determining the location of real world objects in the field of view of a user;
rendering the virtual objects in relation to the real world objects in the collaboration environment.

### Claim 3 (depends on 1)

The method of claim 1 further including defining a first set of users and a second set of users in the collaboration environment, the first set of users receiving a version of the collaboration objects and the second set of users receiving a more limited version of the collaboration objects.

### Claim 4 (depends on 1)

The method of claim 1 wherein receiving a selection includes presenting the user with a set of virtual objects for the collaboration environment and receiving a selection of the virtual objects.

### Claim 5 (depends on 1)

The method of claim 1 further including presenting a set of object controls in the field of view of the wearer, the object controls including at least collaboration permission controls for users in the collaboration environment.

### Claim 6 (depends on 1)

The method of claim 1 further including displaying collaboration permissions for each person in the field of view of the wearer.

### Claim 7 (depends on 1)

The method of claim 1 further including determining an associated audio/visual device for each user in the collaboration environment, including determining the audio/visual capabilities of the audio visual device associated with each user.

### Claim 8 (depends on 7)

The method of claim 7 further including displaying the audio/visual capabilities of each device associated with each user and wherein the step of outputting includes outputting objects suitable for the audio visual capability of each device.

### Claim 9 (depends on 8)

The method of claim 8 further including translating changes for a virtual object in received input from users to a format suitable for the audio/visual capability of each device.

### Claim 10

A method for presenting a collaboration experience, comprising:
creating a collaboration environment defined by at least one wearer of a see through head mounted display apparatus by
receiving from the wearer a selection of collaboration data and a selection of a collaboration environment within the field of view and a definition of which users in the collaboration environment are entitled to share information in the collaboration environment;
determining three dimensional locations of users in the collaboration environment including, for users having see through head mounted display apparatuses:
automatically determining an orientation of the apparatus;
automatically determining a gaze of a user looking through the see-through display of the apparatus;
receiving from users in the collaboration environment shared virtual objects
outputting the shared virtual objects to users having audio/visual devices in the collaboration environment;
receiving input from users in the shared environment on the virtual object, the input manipulating a change in the virtual object; and
outputting the change in the virtual object in the apparatus.

### Claim 11 (depends on 10)

The method of claim 10 further including:
determining an associated audio/visual device for each user in the collaboration environment, including determining the audio/visual capabilities of the audio visual device associated with each user;
translating changes for a virtual object in received input from users to a format suitable for the audio/visual capability of each device.

### Claim 12 (depends on 11)

The method of claim 11 wherein translating includes providing a representation of a virtual object manipulated in a see through head mounted display to a two dimensional display of an audio/visual device.

### Claim 13 (depends on 11)

The method of claim 11 further including authenticating users and associated audio/visual devices in the collaboration environment by a combination of biometric factors.

### Claim 14 (depends on 11)

The method of claim 11 further including managing permissions for shared experience by allowing or blocking applications or use of particular functions of the audio/visual device.

### Claim 15

An see through head mounted display apparatus presenting a collaboration environment to a wearer's field of view, comprising:
a see-through, near-eye, augmented reality display that is worn by a wearer;
one or more processing devices in communication with apparatus, the one or more processing devices automatically determine that the wearer is at a location, the one or more processing devices determine real objects and other persons in the field of view of the wearer, the one or more processing devices determine a collaboration environment in the field of view of the wearer, the environment including users authorized to see collaboration information in the form of virtual objects in the environment, the one or more processing devices rendering virtual objects and user actions in the collaboration environment including managing information provided to users in the collaboration environment;
the collaboration environment including a plurality of audio/visual devices capable of rendering information, including see through head mounted display devices and other devices.

### Claim 16 (depends on 15)

The apparatus of claim 15 wherein the one or more processing devices render an auditory or visual representation of collaboration permissions assigned to each users relative to the collaboration information.

### Claim 17 (depends on 16)

The apparatus of claim 16 wherein the one or more processing devices render a an auditory or visual representation of user identity and associated processing devices and device capability.

### Claim 18 (depends on 17)

The apparatus of claim 17 wherein the one or more processing devices render a an auditory or visual representation of user sharing links between users and user devices.

### Claim 19 (depends on 18)

The apparatus of claim 18 wherein the one or more processing devices render a an auditory or visual representation of a set of virtual objects for the collaboration environment and receiving a selection of the virtual objects.

### Claim 20 (depends on 19)

The apparatus of claim 19 wherein the one or more processing devices render a set of object controls in the field of view of the wearer, the object controls including at least collaboration permission controls for users in the collaboration environment.
