# US 10,768,881 B2 · Multi-screen interaction method and system in augmented reality scene > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,768,881 B2 - Document ID: US10768881B2 - Status: Granted - Kind: B2 - Application number: 16/374,559 - Filed: 2019-04-03 - Published: 2019-07-25 - Granted: 2020-09-08 - Earliest claimed: 2016-12-13 - Assignees: Tencent Technology (Shenzhen) Company Limited - Applicants: Tencent Technology (Shenzhen) Company Limited - Inventors: Keqing Liu; Wenhao Zhang; Libo Li; Zhizhi Zhou; Zelong Deng; Xueyu Wang; Mengcheng Zhang; Xikun Yang; Zhihui Wang; Chao Yang; Shuya Feng; Huamei Yang; Jing Xu; Xin Li; Fangji Zhang; Xiao Xiao; Xicheng Liu - Web page: https://patentgrep.com/patent/US10768881B2 ## CPC - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)} - G06F 3/0484: for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range - G06F 3/0488: using a touch-screen or digitiser, e.g. input of commands through traced gestures - G06F 3/14: Digital output to display device {; Cooperation and interconnection of the display device with other functional units} - G06F 3/1423: {controlling a plurality of local displays, e.g. CRT and flat panel display} - G06K 7/10722: {Photodetector array or CCD scanning} - G06K 7/1417: {2D bar codes} - G06V 10/225: {based on a marking or identifier characterising the area} - G06V 20/20: in augmented reality scenes - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06K: GRAPHICAL DATA READING (image or video recognition or understanding G06V); PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A method for a multi-screen interaction system for an augmented reality scene, information processing devices, and a non-transitory computer-readable storage medium are provided. In the method, a resource object of the augmented reality scene, in which an augmented reality marker is embedded, is displayed. An augmented reality effect play message is received from a second device of the multi-screen interaction system for the augmented reality scene through a communication connection. An augmented reality resource associated with the augmented reality effect play message is displayed in response to the received augmented reality effect play message. ## Claims ### Claim 1 A multi-screen interaction method for a multi-screen interaction system for an augmented reality scene, the method comprising: displaying, by processing circuitry of a first device of the multi-screen interaction system for the augmented reality scene, a resource object of the augmented reality scene, in which an augmented reality marker is embedded; receiving, by the processing circuitry of the first device, an augmented reality effect play message from a second device of the multi-screen interaction system for the augmented reality scene through a communication connection; and displaying, by the processing circuitry of the first device, an augmented reality resource associated with the augmented reality effect play message in response to the received augmented reality effect play message, wherein the augmented reality resource is displayed by the processing circuitry of the first device concurrently with an augmented reality effect displayed by the second device. ### Claim 2 (depends on 1) The method of claim 1, wherein the displaying the resource object of the augmented reality scene comprises: using, by the processing circuitry of the first device, one of an application and a web page of a browser to display the resource object of the augmented reality scene, in which the augmented reality marker is embedded. ### Claim 3 (depends on 1) The method of claim 1, further comprising: sending, by the processing circuitry of the first device, a touch-and-hold identification request through the communication connection to the second device after a display screen of the first device is touched. ### Claim 4 (depends on 3) The method of claim 3, further comprising: receiving, by the processing circuitry of the second device of the multi-screen interaction system for the augmented reality scene, the touch-and-hold identification request from the first device through the communication connection after the display screen of the first device is touched; and capturing, by the processing circuitry of the second device, the augmented reality marker embedded in the resource object from the display screen of the first device when the touch-and-hold identification request is received. ### Claim 5 (depends on 4) The method of claim 4, further comprising: sending, by the processing circuitry of the second device, the augmented reality effect play message to the first device through the communication connection. ### Claim 6 (depends on 4) The method of claim 4, further comprising: obtaining, by the processing circuitry of the second device, the augmented reality resource corresponding to the augmented reality marker; and displaying, by the processing circuitry of the second device, the augmented reality effect corresponding to the augmented reality resource on a display screen of the second device. ### Claim 7 (depends on 1) The method of claim 1, further comprising: using a camera, by the processing circuitry of the second device of the multi-screen interaction system for the augmented reality scene, to scan or photograph a display screen of the first device, to capture the augmented reality marker embedded in the resource object from the first device. ### Claim 8 (depends on 7) The method of claim 7, further comprising: sending, by the processing circuitry of the second device, the augmented reality effect play message to the first device through the communication connection. ### Claim 9 (depends on 7) The method of claim 7, further comprising: obtaining, by the processing circuitry of the second device, the augmented reality resource corresponding to the augmented reality marker; and displaying, by the processing circuitry of the second device, the augmented reality effect corresponding to the augmented reality resource on a display screen of the second device. ### Claim 10 A device for interacting with a second device for a multi-screen interaction system for an augmented reality scene, the device comprising: interface circuitry configured to transmit and receive signals carrying media data to and from the second device through a communication connection; and processing circuitry configured to: display a resource object of the augmented reality scene, in which an augmented reality marker is embedded; receive an augmented reality effect play message corresponding to augmented reality marker from the second device through the communication connection; and display an augmented reality resource associated with the augmented reality effect play message in response to the received augmented reality effect play message, wherein the augmented reality resource is displayed by the processing circuitry of the device concurrently with an augmented reality effect displayed by the second device. ### Claim 11 (depends on 10) The device of claim 10, wherein the augmented reality marker causes the second device to display the augmented reality effect of the augmented reality scene corresponding to the augmented reality resource when the augmented reality marker is captured by the second device from the displayed resource object. ### Claim 12 (depends on 10) The device of claim 10, wherein the processing circuitry is configured to use one of an application or a web page of a browser to display the resource object of the augmented reality scene, in which the augmented reality marker is embedded. ### Claim 13 (depends on 10) The device of claim 10, wherein the processing circuitry is further configured to: send a touch-and-hold identification request to the second device through the communication connection when a display screen of the device is touched. ### Claim 14 (depends on 13) The device of claim 13, wherein the augmented reality marker embedded in the resource object from the display screen of the device is captured by the second device when the touch-and-hold identification request is received by the second device through the communication connection. ### Claim 15 (depends on 14) The device of claim 14, wherein the augmented reality effect play message is sent by the second device through the communication connection to the device when the second device obtains the augmented reality resource corresponding to the augmented reality marker. ### Claim 16 (depends on 10) The device of claim 10, wherein the augmented reality marker embedded in the resource object from a display screen of the device is captured by the second device by using a camera to scan or photograph the display screen of the device. ### Claim 17 (depends on 16) The device of claim 16, wherein the augmented reality effect play message is sent by the second device through the communication connection to the device when the second device obtains the augmented reality resource corresponding to the augmented reality marker. ### Claim 18 (depends on 10) The device of claim 10, wherein the second device is an augmented reality head-mounted device. ### Claim 19 (depends on 10) The device of claim 10, wherein a display screen of the device or a display screen of the second device is a touch screen. ### Claim 20 A non-transitory computer-readable storage medium storing instructions which when executed by at least one processor of a first device cause the at least one processor to perform: displaying a resource object of an augmented reality scene, in which an augmented reality marker is embedded; receiving an augmented reality effect play message from a second device of a multi-screen interaction system for the augmented reality scene through a communication connection; and displaying an augmented reality resource associated with the augmented reality effect play message in response to the received augmented reality effect play message, wherein the augmented reality resource is displayed by the first device concurrently with an augmented reality effect displayed by the second device.