# US 11,305,195 B2 · Extended environmental using real-world environment data

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,305,195 B2
- Document ID: US11305195B2
- Status: Granted
- Kind: B2
- Application number: 16/870,321
- Filed: 2020-05-08
- Published: 2021-11-11
- Granted: 2022-04-19
- Earliest claimed: 2020-05-08
- Assignees: T-Mobile USA, Inc.
- Applicants: T-Mobile USA, Inc.
- Inventors: Ali Daniali
- Web page: https://patentgrep.com/patent/US11305195B2

## CPC

- A63F 13/65: automatically by game devices or servers from real world data, e.g. measurement in live racing competition
- G09B 9/02: for teaching control of vehicles or other craft
- A63F 13/216: using geographical information, e.g. location of the game device or player using GPS
- A63F 13/23: for interfacing with the game device, e.g. specific interfaces between game controller and console
- A63F 13/332: using wireless networks, e.g. cellular phone networks
- A63F 13/352: involving special game server arrangements, e.g. regional servers connected to a national server or a plurality of servers managing partitions of the game world
- A63F 13/803: Driving vehicles or craft, e.g. cars, airplanes, ships, robots or tanks
- B60R 16/023: for transmission of signals between vehicle parts or subsystems
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/012: {Head tracking input arrangements}
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G09B 5/02: with visual presentation of the material to be studied, e.g. using film strip
- A63F 2300/1025: details of the interface with the game device, e.g. USB version detection
- A63F 2300/69: Involving elements of the real world in the game world, e.g. measurement in live races, real video
- A63F 2300/8017: Driving on land or water; Flying
- A63F 2300/8082: Virtual reality
- G06F 2203/011: Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
- G09B 9/05: the view from a vehicle being simulated (G09B9/052, G09B9/058 take precedence)
- A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B)
- B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Techniques are described herein for presenting an extended environment based on real data obtained from a real-world environment. The techniques include pairing an XR capable user device and an onboard diagnostics (OBD) accessory device. Upon pairing, the XR capable user device receives vehicle data of a vehicle equipped with the OBD accessory device in a real-world environment. Based at least on the vehicle data, a virtual location and a virtual position of a user in an extended environment are determined, wherein the user operates the XR capable user device in the real-world environment represented in the extended environment. Upon determining the virtual location and the virtual position, the extended environment is presented via the XR capable user device.

## Claims

### Claim 1

One or more non-transitory computer-readable media storing computer-executable instructions that upon execution cause one or more processors to perform acts comprising:
pairing an extended reality (XR) capable user device and an onboard diagnostics (OBD) accessory device;
receiving vehicle data of a vehicle equipped with the OBD accessory device in a real-world environment;
determining a virtual location and a virtual position of a user operating the XR capable user device in an extended environment representing the real-world environment based at least on the vehicle data;
presenting, via the XR capable user device, the extended environment that represents the real-world environment based at least on the virtual location and the virtual position of the user;
receiving a first user input to add a virtual object into the extended environment;
adding the virtual object to the extended environment at the virtual location and the virtual position;
receiving a second user input to invite a second user operating a second XR capable user device to access the extended environment; and
providing the second XR capable user device with access to the virtual object within the extended environment.

### Claim 2 (depends on 1)

The one or more non-transitory computer-readable media of claim 1, wherein the acts further comprise:
receiving an additional user input to interact with the virtual object; and
executing a function that is associated with the virtual object.

### Claim 3 (depends on 2)

The one or more non-transitory computer-readable media of claim 2, wherein the acts further comprise:
determining if the user input is associated with an authorization to interact with the virtual object; and
if the user input is associated with the authorization, performing an authorized operation associated with the function.

### Claim 4 (depends on 1)

The one or more non-transitory computer-readable media of claim 1, wherein the vehicle data comprises location data and orientation data of the vehicle, and wherein the vehicle data is collected from one or more sensors of the vehicle communicatively coupled to the OBD accessory device.

### Claim 5 (depends on 4)

The one or more non-transitory computer-readable media of claim 4, wherein the one or more sensors comprises a Global Positioning System (GPS) and the OBD accessory device receives GPS coordinates from the GPS, and wherein the location data is determined by the GPS coordinates.

### Claim 6 (depends on 4)

The one or more non-transitory computer-readable media of claim 4, wherein the one or more sensors comprise a camera and the OBD accessory device receives images captured by the camera, and wherein the location data and the orientation data are determined by the individual images.

### Claim 7

A computer-implemented method, comprising:
pairing an XR capable user device and an onboard diagnostics (OBD) accessory device;
receiving vehicle data of a vehicle equipped with the OBD accessory device in a real-world environment;
determining a virtual location and a virtual position of a user operating the XR capable user device in an extended environment representing the real-world environment based at least on the vehicle data;
presenting, via the XR capable user device, the extended environment that represents the real-world environment based at least on the virtual location and the virtual position of the user
receiving a first user input to add a virtual object into the extended environment;
addinq the virtual object to the extended environment at the virtual location and the virtual position;
receivinq a second user input to invite a second user operating a second XR capable user device to access the extended environment; and
providinq the second XR capable user device with access to the virtual object within the extended environment.

### Claim 8 (depends on 7)

The computer-implemented method of claim 7, wherein the XR capable user device comprises a game user interface, the game user interface displaying the extended environment.

### Claim 9 (depends on 7)

The computer-implemented method of claim 7, further comprising:
receiving a user input to remove a second virtual object from the extended environment; and
removing the second virtual object from the extended environment.

### Claim 10 (depends on 7)

The computer-implemented method of claim 7, further comprising:
associating the virtual object with one or more functions;
receiving an additional user input to interact with the virtual object; and
executing the one or more functions that are associated with the virtual object.

### Claim 11 (depends on 7)

The computer-implemented method of claim 7, wherein the extended environment comprises an augmented environment or a virtual environment.

### Claim 12

A system, comprising:
one or more non-transitory storage mediums configured to provide stored computer-readable instructions, the one or more non-transitory storage mediums coupled to one or more processors, the one or more processors configured to execute the computer-readable instructions to cause the one or more processors to:
pair an XR capable user device and an onboard diagnostics (OBD) accessory device;
receive vehicle data of a vehicle equipped with the OBD accessory device in a real-world environment;
determine a virtual location and a virtual position of a user operating the XR capable user device in an extended environment representing the real-world environment based at least on the vehicle data;
present, via the XR capable user device, the extended environment that represents the real-world environment based at least on the virtual location and the virtual position of the user
receive a first user input to add a virtual object into the extended environment;
add the virtual object to the extended environment at the virtual location and the virtual position;
receive a second user input to invite a second user operating a second XR capable user device to access the extended environment; and
provide the second XR capable user device with access to the virtual object within the extended environment.

### Claim 13 (depends on 12)

The system of claim 12, wherein the one or more processor is further configured to:
receive an additional user input to interact with the virtual object; and
execute a function that is associated with the virtual object.

### Claim 14 (depends on 13)

The system of claim 13, wherein the one or more processor is further configured to:
determine if the additional user input is associated with an authorization to interact with the virtual object; and
if the additional user input is associated with the authorization, perform an authorized operation associated with the function.

### Claim 15 (depends on 12)

The system of claim 12, wherein the XR capable user device comprises a game user interface, the game user interface displaying the extended environment, and wherein the extended environment comprises an augmented environment.

### Claim 16 (depends on 1)

The one or more non-transitory computer-readable media of claim 1, wherein the virtual object comprises one or more of images, texts, symbols, annotations, or markers.