# US 11,122,220 B2 · Augmented video reality > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,122,220 B2 - Document ID: US11122220B2 - Status: Granted - Kind: B2 - Application number: 16/526,491 - Filed: 2019-07-30 - Published: 2019-12-12 - Granted: 2021-09-14 - Earliest claimed: 2017-02-03 - Assignees: Zyetric System Limited - Applicants: Zyetric System Limited - Inventors: Pak Kit Lam; Peter Han Joo Chong - Web page: https://patentgrep.com/patent/US11122220B2 ## CPC - H04N 5/272: Means for inserting a foreground image in a background image, i.e. inlay, outlay - G06F 3/14: Digital output to display device {; Cooperation and interconnection of the display device with other functional units} - H04N 5/2628: {Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation} - H04N 5/265: Mixing - H04N 7/142: {Constructional details of the terminal equipment, e.g. arrangements of the camera and the display} - H04N 7/147: {Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals (selecting H04Q)} - H04N 7/15: Conference systems - G16H 80/00: ICT specially adapted for facilitating communication between medical practitioners or patients, e.g. for collaborative diagnosis, therapy or health monitoring - H04N 2005/2726: {for simulating a person's appearance, e.g. hair style, glasses, clothes} - H04N 2007/145: {Handheld terminals} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA ## Abstract At an electronic device coupled to a display, a communication interface, and a first image sensor, first image data is captured from the first image sensor. Second image data is received over the communication interface from a first remote device. The display displays a portion of the second image data with a portion of the first image data. The displayed portion of the second image data obscures some of the first image data. ## Claims ### Claim 1 A method, comprising: at an electronic device coupled to a display facing toward front of the electronic device, a communication interface, and a first image sensor facing toward back of the electronic device: capturing first image data from the first image sensor, the first image data representing the environment surrounding the electronic device; receiving second image data over the communication interface from a first remote device, wherein the second image data is real-time video data of a first remote user of the first remote device; cropping the second image data to obtain a cropped image data; and displaying on the display the cropped image data with a portion of the first image data, wherein the cropped image data obscures some of the first image data such that the display shows the first remote user of the first remote device appearing in the environment surrounding the electronic device; receiving third image data over the communication interface from a second remote device, wherein the third image data is real-time video data of a second remote user of the second remote device; and detecting a position change of the electronic device using a sensor; capturing updated first image data form the first image sensor; and in response to detecting the position change, updating the display to display the third image data overlaid on the updated first image data such that the display shows the second remote user of the second remote device appearing in the environment surrounding the electronic device. ### Claim 2 (depends on 1) The method of claim 1, wherein the electronic device is further coupled to a second image sensor pointing in a different direction than a direction of the first image sensor, the method further comprising: capturing fourth image data from the second image sensor; and sending the fourth image data to the first remote device over the communication interface. ### Claim 3 (depends on 2) The method of claim 2, wherein the second image sensor faces the same direction as the display faces. ### Claim 4 (depends on 1) The method of claim 1 wherein the display, the communication interface, and the first image sensor are all contained within the same electronic device housing. ### Claim 5 (depends on 1) The method of claim 1 wherein at least one of the display, the communication interface, or the first image sensor are not contained within the same housing. ### Claim 6 A non-transitory computer-readable storage medium encoded with a computer program executable by an electronic device coupled to a display facing toward front of the electronic device, memory, and an image sensor facing toward back of the electronic device, the computer program comprising instructions for: capturing first image data from the first image sensor, the first image data representing the environment surrounding the electronic device; receiving second image data over the communication interface from a first remote device, wherein the second image data is real-time video data of a first remote user of the first remote device; cropping the second image data to obtain a cropped image data; and displaying on the display the cropped image data with a portion of the first image data, wherein the cropped image data obscures some of the first image data such that the display shows the first remote user of the first remote device appearing in the environment surrounding the electronic device; receiving third image data over the communication interface from a second remote device, wherein the third image data is real-time video data of a second remote user of the second remote device; and detecting a position change of the electronic device using a sensor; capturing updated first image data form the first image sensor; and in response to detecting the position change, updating the display to display the third image data overlaid on the updated first image data such that the display shows the second remote user of the second remote device appearing in the environment surrounding the electronic device. ### Claim 7 (depends on 6) The computer-readable storage medium of claim 6, wherein the electronic device is further coupled to a second image sensor pointing in a different direction than a direction of the first image sensor, the computer program comprising instructions for: capturing fourth image data from the second image sensor; and sending the fourth image data to the first remote device over the communication interface. ### Claim 8 (depends on 7) The computer-readable storage medium of claim 7, wherein the second image sensor faces the same direction as the display faces. ### Claim 9 (depends on 6) The computer-readable storage medium of claim 6 the computer program comprising instructions for: merging the cropped image with the first image data. ### Claim 10 (depends on 6) The computer-readable storage medium of claim 6 wherein the display, the communication interface, and the first image sensor are all contained within the same electronic device housing. ### Claim 11 (depends on 6) The computer-readable storage medium of claim 6 wherein at least one of the display, the communication interface, or the first image sensor are not contained within the same housing. ### Claim 12 A system comprising: a display facing toward front of the electronic device; an image sensor facing toward back of the electronic device; and memory encoded with a computer program executable by an electronic device coupled to a display, memory, and an image sensor, the computer program having instructions for: capturing first image data from the first image sensor, the first image data representing the environment surrounding the electronic device; receiving second image data over the communication interface from a first remote device, wherein the second image data is real-time video data of a first remote user of the first remote device; cropping the second image data to obtain a cropped image data; displaying on the display the cropped image data with a portion of the first image data, wherein the cropped image data obscures some of the first image data such that the display shows the first remote user of the first remote device appearing in the environment surrounding the electronic device; receiving third image data over the communication interface from a second remote device, wherein the third image data is real-time video data of a second remote user of the second remote device; and detecting a position change of the electronic device using a sensor; capturing updated first image data form the first image sensor; and in response to detecting the position change, updating the display to display the third image data overlaid on the updated first image data such that the display shows the second remote user of the second remote device appearing in the environment surrounding the electronic device. ### Claim 13 (depends on 12) The system of claim 12, wherein the electronic device is further coupled to a second image sensor pointing in a different direction than a direction of the first image sensor, the computer program comprising instructions for: capturing fourth image data from the second image sensor; and sending the fourth image data to the first remote device over the communication interface. ### Claim 14 (depends on 13) The system of claim 13, wherein the second image sensor faces the same direction as the display faces. ### Claim 15 (depends on 12) The system of claim 12 wherein the display, the communication interface, and the first image sensor are all contained within the same electronic device housing.