# US 2019/0378476 A1 · Multimedia device and method for controlling the same > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0378476 A1 - Document ID: US20190378476A1 - Status: Published application - Kind: A1 - Application number: 16/549,282 - Filed: 2019-08-23 - Published: 2019-12-12 - Earliest claimed: 2019-08-08 - Assignees: LG Electronics Inc. - Applicants: LG Electronics Inc. - Inventors: Chanyong JEON - Web page: https://patentgrep.com/patent/US20190378476A1 ## CPC - G09G 3/002: {to project the image of a two-dimensional display, such as an array of light emitting or modulating elements or a CRT} - G09G 5/37: Details of the operation on graphic patterns (G09G5/38 takes precedence) - G06F 3/14: Digital output to display device {; Cooperation and interconnection of the display device with other functional units} - G06F 3/147: using display panels - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/08: Learning methods - G06N 3/09: Supervised learning - G06N 20/00: Machine learning - G09G 5/14: Display of multiple viewports - G09G 2340/0464: Positioning - G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract Proposed is an extended reality (XR) device for providing augmented reality (AR) mode and virtual reality (VR) mode and a method of controlling the same. The extended reality (XR) device may comprises: a camera; a communication unit for communicating with at least one of an external sensor, an external device, and an external server; a display for displaying an extended reality object; and a controller configured to control the display to change at least one of an appearance and a position of the displayed extended reality object based on data sensed by the external sensor. ## Claims ### Claim 1 An extended reality (XR) device comprising: a camera; a communication unit for communicating with at least one of an external sensor, an external device, and an external server; a display for displaying an extended reality object; and a controller configured to control the display to change at least one of an appearance and a position of the displayed extended reality object based on data sensed by the external sensor. ### Claim 2 (depends on 1) The XR device of claim 1, wherein the external sensor includes a home IoT (Internet of Things) sensor. ### Claim 3 (depends on 1) The XR device of claim 1, wherein the extended reality device includes an extended reality engine for generating an extended reality object based on the data. ### Claim 4 (depends on 1) The XR device of claim 1, wherein the external sensor includes at least two temperature sensors, wherein the controller is further configured to: calculate a humidity based on a difference between temperatures sensed by the temperature sensors; and control the display to change at least one of an appearance and a position of the displayed extended reality object based on the calculated humidity. ### Claim 5 (depends on 1) The XR device of claim 1, wherein the external sensor includes at least one sound sensor, wherein the controller is further configured to: calculate at least one of a position and an intensity of a sound of a predefined real object detected via the at least one sound sensor; and control the display to change at least one of an appearance and a position of the displayed extended reality object based on at least one of the position and intensity of the sound of the predefined real object. ### Claim 6 (depends on 1) The XR device of claim 1, wherein the external sensor includes at least one air flow sensor, wherein the controller is further configured to control the display to change at least one of an appearance and a position of the displayed extended reality object based on at least one of an air flow direction or speed detected by the air flow sensor. ### Claim 7 (depends on 1) The XR device of claim 1, wherein the extended reality device further includes an AI (Artificial Intelligence) engine for analyzing the sensed data, wherein the controller is further configured to allow the AI engine to learn positions of objects received in a predefined receiving space captured in real time by the camera and to calculate a receiving position of each object. ### Claim 8 (depends on 7) The XR device of claim 7, wherein when an unreceived object in the space is captured in real time via the camera, the controller is further configured to identify a type of the unreceived object, and to control the display to display the extended reality object to indicate a receiving position in the receiving space corresponding to the identified type. ### Claim 9 (depends on 8) The XR device of claim 8, wherein the controller is further configured to control the display to display the extended reality object when the object and the receiving space are concurrently captured in real time via the camera. ### Claim 10 (depends on 9) The XR device of claim 9, wherein the controller is further configured to control the display to display different-shaped extended reality objects corresponding to different object types. ### Claim 11 A method for operating an extended reality (XR) device, the method comprising: communicating with at least one of an external sensor, an external device, and an external server; displaying an extended reality object; and changing at least one of an appearance and a position of the displayed extended reality object based on data sensed by the external sensor. ### Claim 12 (depends on 11) The method of claim 11, wherein the external sensor includes a home IoT (Internet of Things) sensor. ### Claim 13 (depends on 11) The method of claim 11, wherein the method further includes generating an extended reality object based on the data. ### Claim 14 (depends on 11) The method of claim 11, wherein the external sensor includes at least two temperature sensors, wherein the method further includes: calculating a humidity based on a difference between temperatures sensed by the temperature sensors; and changing at least one of an appearance and a position of the displayed extended reality object based on the calculated humidity. ### Claim 15 (depends on 11) The method of claim 11, wherein the external sensor includes at least one sound sensor, wherein the method further includes: calculating at least one of a position and an intensity of a sound of a predefined real object detected via the at least one sound sensor; and changing at least one of an appearance and a position of the displayed extended reality object based on at least one of the position and intensity of the sound of the predefined real object. ### Claim 16 (depends on 11) The method of claim 11, wherein the external sensor includes at least one air flow sensor, wherein the method further includes changing at least one of an appearance and a position of the displayed extended reality object based on at least one of an air flow direction or speed detected by the air flow sensor. ### Claim 17 (depends on 11) The method of claim 11, wherein the extended reality device further includes an AI (Artificial Intelligence) engine for analyzing the sensed data, wherein the method further includes allowing the AI engine to learn positions of objects received in a predefined receiving space captured in real time by a camera, and calculating a receiving position of each object. ### Claim 18 (depends on 17) The method of claim 17, wherein the method further includes: when an unreceived object in the space is captured in real time via the camera, identifying a type of the unreceived object, and displaying the extended reality object to indicate a receiving position in the receiving space corresponding to the identified type. ### Claim 19 (depends on 18) The method of claim 18, wherein the method further includes: displaying the extended reality object when the object and the receiving space are concurrently captured in real time via the camera. ### Claim 20 (depends on 19) The method of claim 19, wherein the method further includes displaying different-shaped extended reality objects corresponding to different object types.