# US 2024/0201805 A1 · Synchronizing Augmented or Virtual Reality (AR/VR) Applications with Companion Device Interfaces > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2024/0201805 A1 - Document ID: US20240201805A1 - Status: Published application - Kind: A1 - Application number: 18/589,396 - Filed: 2024-02-27 - Published: 2024-06-20 - Earliest claimed: 2019-03-14 - Assignees: eBay Inc. - Applicants: eBay Inc. - Inventors: Joshua Reino Timonen; Andrew Chalkley - Web page: https://patentgrep.com/patent/US20240201805A1 ## CPC - G06F 3/038: Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06F 3/012: {Head tracking input arrangements} - G06F 3/0428: {by sensing at the edges of the touch surface the interruption of optical paths, e.g. an illumination plane, parallel to the touch surface which may be virtual (sensing beam interruptions in a planar beam grid of an optical touch-screen G06F3/0421)} - G06F 3/0484: for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range - G06F 9/451: Execution arrangements for user interfaces - H04L 67/06: specially adapted for file transfer, e.g. file transfer protocol [FTP] - H04W 76/14: Direct-mode setup - G06F 2203/0384: Wireless input, i.e. hardware and software details of wireless interface arrangements for pointing devices - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts - H04L 67/131: Protocols for games, networked simulations or virtual reality - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract An augmented reality or virtual reality (AR/VR) device pairs with a companion device to augment input interfaces associated with an AR/VR application at the AR/VR device. In implementations, an AR/VR device determines a portion of a markup file that corresponds to an AR/VR scene of a plurality of AR/VR scenes in an AR/VR environment, and communicates the portion of the markup file to the companion device to cause the companion device to configure a companion user interface associated with initiating an action as part of the AR/VR scene. In response to receiving user input via the companion user interface, the companion device communicates the action to the AR/VR device to initiate the action. The AR/VR device receives input data from the companion device, and initiates the action for the AR/VR scene. ## Claims ### Claim 1 A method as implemented by a computing device, the method comprising: receiving, by the computing device, one or more inputs corresponding to an augmented or virtual reality (AR/VR) application, the AR/VR application including one or more AR/VR scenes; based on the one or more inputs, generating, by the computing device, a portion of a markup file corresponding to an AR/VR scene associated with the AR/VR application; and storing, by the computing device, the portion of the markup file as part of a markup file associated with the AR/VR application. ### Claim 2 (depends on 1) The method as described in claim 1, wherein the portion of the markup file is configured to support communication of one or more linkages to a companion device. ### Claim 3 (depends on 1) The method as described in claim 1, further comprising communicating, by the computing device, the portion of the markup file to a companion device to cause the companion device to configure a companion user interface that corresponds to the AR/VR scene. ### Claim 4 (depends on 1) The method as described in claim 1, wherein the portion of the markup file includes markup language that links an action API to an action identifier. ### Claim 5 (depends on 1) The method as described in claim 1, wherein the portion of the markup file indicates controls. ### Claim 6 (depends on 1) The method as described in claim 1, wherein the portion of the markup file is configured for execution within a browser of a companion device. ### Claim 7 A system comprising: a memory component; and a processing device coupled to the memory component, the processing device to perform operations comprising: receiving one or more inputs corresponding to an augmented or virtual reality (AR/VR) application, the AR/VR application including one or more AR/VR scenes; based on the one or more inputs, generating a portion of a markup file corresponding to an AR/VR scene associated with the AR/VR application; and storing the portion of the markup file as part of a markup file associated with the AR/VR application. ### Claim 8 (depends on 7) The system as described in claim 7, wherein the portion of the markup file includes markup language that specifies a control to display at a companion user interface that links the control to an action identifier. ### Claim 9 (depends on 7) The system as described in claim 7, wherein the portion of the markup file includes markup language that links a first control at a companion user interface to a second control. ### Claim 10 (depends on 7) The system as described in claim 7, wherein the portion of the markup file includes scene identifiers. ### Claim 11 (depends on 7) The system as described in claim 7, wherein the portion of the markup file includes markup language indicating types of controls to render at a companion user interface. ### Claim 12 (depends on 7) The system as described in claim 7, wherein the portion of the markup file includes markup language indicating APIs that correspond to one or more actions. ### Claim 13 A non-transitory computer-readable storage medium storing executable instructions, which when executed by a processing device, cause the processing device to perform operations comprising: receiving one or more inputs corresponding to an augmented or virtual reality (AR/VR) application, the AR/VR application including one or more AR/VR scenes; based on the one or more inputs, generating a portion of a markup file corresponding to an AR/VR scene associated with the AR/VR application; and storing the portion of the markup file as part of a markup file associated with the AR/VR application. ### Claim 14 (depends on 13) The non-transitory computer-readable storage medium as described in claim 13, wherein the portion of the markup file indicates controls. ### Claim 15 (depends on 13) The non-transitory computer-readable storage medium as described in claim 13, wherein the portion of the markup file is configured to support communication of one or more linkages to a companion device. ### Claim 16 (depends on 13) The non-transitory computer-readable storage medium as described in claim 13, further comprising communicating the portion of the markup file to a companion device to cause the companion device to configure a companion user interface that corresponds to the AR/VR scene. ### Claim 17 (depends on 13) The non-transitory computer-readable storage medium as described in claim 13, wherein the portion of the markup file is configured for execution within a browser of a companion device. ### Claim 18 (depends on 13) The non-transitory computer-readable storage medium as described in claim 13, wherein the portion of the markup file includes scene identifiers. ### Claim 19 (depends on 13) The non-transitory computer-readable storage medium as described in claim 13, wherein the portion of the markup file identifies an action that is used by a companion device to receive user input. ### Claim 20 (depends on 13) The non-transitory computer-readable storage medium as described in claim 13, wherein the portion of the markup file is included as part of an AR/VR application that is executed to implement a plurality of AR/VR scenes as part of an AR/VR environment.