# US 2022/0286488 A1 · Systems and methods for moving content between virtual and physical displays

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0286488 A1
- Document ID: US20220286488A1
- Status: Published application
- Kind: A1
- Application number: 17/712,812
- Filed: 2022-04-04
- Published: 2022-09-08
- Earliest claimed: 2021-02-08
- Assignees: Multinarity Ltd
- Applicants: Multinarity Ltd
- Inventors: Tamir Berliner; Tomer Kahan; Orit Dolev
- Web page: https://patentgrep.com/patent/US20220286488A1

## CPC

- G06F 1/163: {Wearable computers, e.g. on a belt}
- G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking}
- G02B 27/01: Head-up displays
- G02B 27/0101: {characterised by optical features (G02B27/0172 takes precedence)}
- G02B 27/017: {Head mounted}
- G02B 27/0172: {characterised by optical features}
- G06F 1/1616: {with folding flat displays, e.g. laptop computers or notebooks having a clamshell configuration, with body parts pivoting to an open position around an axis parallel to the plane they define in closed position}
- G06F 1/1632: {External expansion units, e.g. docking stations}
- G06F 1/1656: {Details related to functional adaptations of the enclosure, e.g. to provide protection against EMI, shock, water, or to host detachable peripherals like a mouse or removable expansions units like PCMCIA cards, or to provide access to internal components for maintenance or to removable storage supports like CDs or DVDs, or to mechanically mount accessories (mounting of accessories to a computer display G06F1/1607; display hoods G06F1/1603; cooling arrangements for portable computers G06F1/203)}
- G06F 1/1662: {Details related to the integrated keyboard}
- G06F 1/1669: {Detachable keyboards}
- G06F 1/1686: {the I/O peripheral being an integrated camera}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/012: {Head tracking input arrangements}
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06F 3/014: {Hand-worn input/output arrangements, e.g. data gloves}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/02: Input arrangements using manually operated switches, e.g. using keyboards or dials
- G06F 3/021: {Arrangements integrating additional peripherals in a keyboard, e.g. card or barcode reader, optical scanner}
- G06F 3/023: Arrangements for converting discrete items of information into a coded form, e.g. arrangements for interpreting keyboard generated codes as alphanumeric codes, operand codes or instruction codes
- G06F 3/0231: {Cordless keyboards}
- G06F 3/0233: {Character input methods}
- G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)}
- G06F 3/033: Pointing devices displaced or positioned by the user {, e.g. mice, trackballs, pens or joysticks}; Accessories therefor (digitisers characterised by the transducing means G06F3/041)
- G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
- G06F 3/03547: {Touch pads, in which fingers can move on a surface}
- G06F 3/041: Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
- G06F 3/0481: based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F 3/04812: Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
- G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object
- G06F 3/04817: using icons (graphical or visual programming using iconic symbols G06F8/34)
- G06F 3/0482: Interaction with lists of selectable items, e.g. menus
- G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence)
- G06F 3/04847: Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
- G06F 3/0486: Drag-and-drop
- G06F 3/0488: using a touch-screen or digitiser, e.g. input of commands through traced gestures
- G06F 3/04883: for inputting data by handwriting, e.g. gesture or text
- G06F 3/04886: by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
- G06F 3/0489: using dedicated keyboard keys or combinations thereof
- G06F 3/04892: Arrangements for controlling cursor position based on codes indicative of cursor displacements from one discrete location to another, e.g. using cursor control keys associated to different directions or using the tab key (arrangements for controlling cursor position based on coordinate signals G06F3/038)
- G06F 3/14: Digital output to display device {; Cooperation and interconnection of the display device with other functional units}
- G06F 3/1423: {controlling a plurality of local displays, e.g. CRT and flat panel display}
- G06F 3/1454: {involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay}
- G06F 3/147: using display panels
- G06F 3/165: {Management of the audio stream, e.g. setting of volume, audio stream path}
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G06K 7/1417: {2D bar codes}
- G06Q 10/0631: Resource planning, allocation, distributing or scheduling for enterprises or organisations
- G06T 3/40: Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06T 11/00: Two-dimensional [2D] image generation
- G06T 15/20: Perspective computation
- G06T 19/003: {Navigation within 3D models or images}
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- G06V 20/20: in augmented reality scenes
- G06V 30/14: Image acquisition
- G06V 30/228: of three-dimensional [3D] handwriting, e.g. writing in the air
- G06V 40/18: Eye characteristics, e.g. of the iris
- G06V 40/25: {Recognition of walking or running movements, e.g. gait recognition}
- G06V 40/28: {Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113)}
- G09G 3/003: {to produce spatial visual effects}
- G09G 5/10: Intensity circuits
- G09G 5/38: with means for controlling the display position
- H04L 12/1831: {Tracking arrangements for later retrieval, e.g. recording contents, participants activities or behavior, network status}
- H04L 51/222: using geographical location information, e.g. messages transmitted or received in proximity of a certain spot or area
- H04L 51/58: Message adaptation for wireless communication
- H04L 63/102: {Entity profiles}
- H04L 65/4015: {where at least one of the additional parallel sessions is real time or time sensitive, e.g. white board sharing, collaboration or spawning of a subconference}
- H04L 65/403: Arrangements for multi-party communication, e.g. for conferences (data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15)
- H04L 67/131: Protocols for games, networked simulations or virtual reality
- H04M 1/724097: {Worn on the head}
- H04N 23/45: for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
- H04N 23/51: Housings
- H04N 23/57: Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
- H04W 4/02: Services making use of location information
- H04W 4/70: Services for machine-to-machine communication [M2M] or machine type communication [MTC]
- H04W 4/80: Services using short range communication, e.g. near-field communication [NFC], radio-frequency identification [RFID] or low energy communication
- H04W 12/08: Access security
- G02B 2027/0138: {comprising image capture systems, e.g. camera}
- G02B 2027/014: {comprising information/image processing systems}
- G02B 2027/0141: {characterised by the informative content of the display}
- G02B 2027/0167: {Emergency system, e.g. to prevent injuries}
- G02B 2027/0178: {Eyeglass type (eyeglass details G02C)}
- G02B 2027/0187: {slaved to motion of at least a part of the body of the user, e.g. head, eye}
- G06F 3/0484: for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F 2200/1633: Protecting arrangement for the entire housing of the computer
- G06F 2203/04104: Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
- G06F 2203/04801: Cursor retrieval aid, i.e. visual aspect modification, blinking, colour changes, enlargement or other visual cues, for helping user do find the cursor in graphical user interfaces
- G06F 2203/04808: Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
- G06T 2200/24: involving graphical user interfaces [GUIs]
- G06T 2207/30196: Human being; Person
- G06T 2219/012: Dimensioning, tolerancing
- G06T 2219/024: Multi-user, collaborative environment
- G09G 3/03: {specially adapted for displays having non-planar surfaces, e.g. curved displays}
- G09G 5/14: Display of multiple viewports
- G09G 2320/08: Arrangements within a display terminal for setting, manually or automatically, display parameters of the display terminal
- G09G 2340/04: Changes in size, position or resolution of an image
- G09G 2340/0492: Change of orientation of the displayed image, e.g. upside-down, mirrored
- G09G 2354/00: Aspects of interface with display user
- G09G 2370/16: Use of wireless transmission of display information
- H04L 67/12: specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
- H04L 67/52: specially adapted for the location of the user terminal
- H04M 1/72454: according to context-related or environment-related conditions
- H04W 4/026: {using orientation information, e.g. compass}
- H04W 4/38: for collecting sensor information
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06K: GRAPHICAL DATA READING (image or video recognition or understanding G06V); PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72)

## Abstract

Systems, methods, and non-transitory computer readable media for transferring virtual content to a physical display device are disclosed. An extended reality environment may be presented in a room via a wearable extended reality appliance configured to be paired with multiple display devices located in the room. Each display device may be associated with a unique network identifier. Input to cause presentation of a specific virtual object in the extended reality environment on a target display device and image data depicting the target display device may be received. The image data may be analyzed to identify the target display device. A network identifier of the target display device may be determined. A communications link with the target display device may be established. Data representing the specific virtual object may be transmitted to the target display device, to enable the target display device to present the specific virtual object.

## Claims

### Claim at position 1

1-60. (canceled)

### Claim 61

A non-transitory computer readable medium containing instructions that when executed by at least one processor cause the at least one processor to perform operations for transferring virtual content to a physical display device, the operations comprising:
presenting an extended reality environment in a room via a wearable extended reality appliance, the wearable extended reality appliance being configured to be paired with multiple display devices located in the room, wherein each display device is associated with a unique network identifier;
receiving input associated with the wearable extended reality appliance to cause presentation of a specific virtual object in the extended reality environment on a target display device;
receiving image data from an image sensor associated with the wearable extended reality appliance, the image data depicting the target display device;
analyzing the image data to identify the target display device;
upon identifying the target display device, determining a network identifier of the target display device;
using the determined network identifier of the target display device to establish a communications link with the target display device; and
transmitting data representing the specific virtual object to the target display device, wherein the transmitted data enables the target display device to present the specific virtual object.

### Claim 62 (depends on 61)

The non-transitory computer readable medium of claim 61, wherein the input is received from the image sensor associated with the wearable extended reality appliance.

### Claim 63 (depends on 62)

The non-transitory computer readable medium of claim 62, wherein the operations further include analyzing the image data to identify a gesture initiated by a user of the wearable extended reality appliance that triggers a virtual collision between the specific virtual object and the target display device.

### Claim 64 (depends on 61)

The non-transitory computer readable medium of claim 61, wherein the input is received from a pointing device associated with the wearable extended reality appliance.

### Claim 65 (depends on 61)

The non-transitory computer readable medium of claim 61, wherein the operations further include analyzing the image data to recognize content displayed by the target display device and to thereby determine an identity of the target display device.

### Claim 66 (depends on 61)

The non-transitory computer readable medium of claim 61, wherein the operations further include determining that the specific virtual object is classified as private, requesting a permission for displaying the specific virtual object through the target display device, and transmitting the data representing the specific virtual object to the target display device only after receiving the permission.

### Claim 67 (depends on 61)

The non-transitory computer readable medium of claim 61, wherein the operations further include determining that the specific virtual object is classified as private, determining that the target display device is authorized for presenting the private information, and in response to the determination that the target display device is authorized for presenting the private information, transmitting the data representing the specific virtual object to the target display device.

### Claim 68 (depends on 61)

The non-transitory computer readable medium of claim 61, wherein the operations further include determining that the specific virtual object is classified as private, analyzing the image data to determine presence of individuals other than a user of the wearable extended reality appliance exposed to the target display device, in response to a determination of no individuals other than the user exposed to the target display device, transmitting the data representing the specific virtual object to the target display device, and in response to a determination of more than one individual is exposed to the target display device, avoiding transmission of the data representing the specific virtual object to the target display device.

### Claim 69 (depends on 68)

The non-transitory computer readable medium of claim 68, wherein the operations further include, after transmitting the data representing the specific virtual object to the target display device: determining that an individual other than the user is likely to be exposed to the target display device; and in response to the determination that the individual other than the user is likely to be exposed to the target display device, causing the target display device to cease presentation of the specific virtual object.

### Claim 70 (depends on 61)

The non-transitory computer readable medium of claim 61, wherein the specific virtual object is a virtual screen playing media content, and the transmission of the data causes the media content to be presented through the target display device.

### Claim 71 (depends on 61)

The non-transitory computer readable medium of claim 61, wherein the operations further include causing the specific virtual object to disappear from the extended reality environment once the specific virtual object is presented through the target display device.

### Claim 72 (depends on 61)

The non-transitory computer readable medium of claim 61, wherein the operations further include displaying via the wearable extended reality appliance a virtual indicator indicating that the specific virtual object is presented through the target display device.

### Claim 73 (depends on 61)

The non-transitory computer readable medium of claim 61, wherein the operations further include generating a virtual controller associated with the target display device, the virtual controller enabling modification of display parameters of the target display device.

### Claim 74 (depends on 61)

The non-transitory computer readable medium of claim 61, wherein the operations further include, after the target display device begins presenting the specific virtual object, receiving additional input, and modifying the specific virtual object on the target display device based on the additional input.

### Claim 75 (depends on 61)

The non-transitory computer readable medium of claim 61, wherein the operations further include, after the target display device begins presenting the specific virtual object, receiving an additional input indicating a desire of a user of the wearable extended reality appliance to stop presenting the specific virtual object through the target display device; and in response to receiving the additional input, causing the target display device to cease presentation of the specific virtual object by the target display device.

### Claim 76 (depends on 75)

The non-transitory computer readable medium of claim 75, wherein the additional input indicating the desire of the user of the wearable extended reality appliance to stop presenting the specific virtual object through the target display device includes an indication that the wearable extended reality appliance left the room.

### Claim 77 (depends on 76)

The non-transitory computer readable medium of claim 76, wherein the operations further include, after ceasing the presentation of the specific virtual object, receiving an indication that the wearable extended reality appliance reentered the room, and causing a renewal of the presentation of the specific virtual object by the target display device in response to the reentrance.

### Claim 78 (depends on 61)

The non-transitory computer readable medium of claim 61, wherein the operations further include:
receiving additional input to cause presentation of content from a second display device through the wearable extended reality appliance;
receiving second image data from the image sensor, the second image data depicting the second display device;
analyzing the second image data to identify the second display device;
upon identifying the second display device, determining a network identifier of the second display device;
using the determined network identifier of the second display device to establish a communications link with the second display device;
receiving data representing the content from the second display device; and
displaying through the wearable extended reality appliance the content from the second display device.

### Claim 79

A method for transferring virtual content to a physical display device, the method comprising:
presenting an extended reality environment in a room via a wearable extended reality appliance, the wearable extended reality appliance being configured to be paired with multiple display devices located in the room, wherein each display device is associated with a unique network identifier;
receiving input associated with the wearable extended reality appliance to cause presentation of a specific virtual object in the extended reality environment on a target display device;
receiving image data from an image sensor associated with the wearable extended reality appliance, the image data depicting the target display device;
analyzing the image data to identify the target display device;
upon identifying the target display device, determining a network identifier of the target display device;
using the determined network identifier of the target display device to establish a communications link with the target display device; and
transmitting data representing the specific virtual object to the target display device, wherein the transmitted data enables the target display device to present the specific virtual object.

### Claim 80

A system for transferring virtual content to a physical display device, the system comprising:
at least one processing device configured to:
present an extended reality environment in a room via a wearable extended reality appliance, the wearable extended reality appliance being configured to be paired with multiple display devices located in the room, wherein each display device is associated with a unique network identifier;
receive input associated with the wearable extended reality appliance to cause presentation of a specific virtual object in the extended reality environment on a target display device;
receive image data from an image sensor associated with the wearable extended reality appliance, the image data depicting the target display device;
analyze the image data to identify the target display device;
upon identifying the target display device, determine a network identifier of the target display device;
use the determined network identifier of the target display device to establish a communications link with the target display device; and
transmit data representing the specific virtual object to the target display device, wherein the transmitted data enables the target display device to present the specific virtual object.

### Claim at position 22

81-260. (canceled)
