# US 8,542,266 B2 · Method and system for adapting a CP layout according to interaction between conferees > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 8,542,266 B2 - Document ID: US8542266B2 - Status: Granted - Kind: B2 - Application number: 12/750,851 - Filed: 2010-03-31 - Published: 2011-04-21 - Granted: 2013-09-24 - Earliest claimed: 2001-01-07 - Assignees: Polycom, Inc. - Applicants: Eyal Leviav; Niv Wagner; Efrat Be'ery - Inventors: Eyal Leviav; Niv Wagner; Efrat Be'ery - Web page: https://patentgrep.com/patent/US8542266B2 ## CPC - H04N 7/152: {Multipoint control units therefor} - G06T 11/60: Creating or editing images; Combining images with text - H04N 7/15: Conference systems - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A system and method is disclosed for adapting a continuous presence videoconferencing layout according to interactions between conferees. Using regions of interest found in video images, the arrangement of images of conferees may be dynamically arranged as displayed by endpoints. Arrangements may be responsive to various metrics, including the position of conferees in a room and dominant conferees in the videoconference. Video images may be manipulated as part of the arrangement, including cropping and mirroring the video image. As interactions between conferees change, the layout may be automatically rearranged responsive to the changed interactions. ## Claims ### Claim 1 A method, comprising: designing a continuous presence video image automatically for a first endpoint of a continuous presence videoconference responsive to an interaction between a plurality of conferees, comprising: determining automatically an interaction between a first conferee and a second conferee of the plurality of conferees; positioning a first video image corresponding to the first conferee in the continuous presence video image; and positioning a second video image corresponding to the second conferee relative to the first video image in the continuous presence video image, responsive to the interaction between the first conferee and the second conferee; and displaying the continuous presence video image at the first endpoint of the continuous presence videoconference. ### Claim 2 (depends on 1) The method of claim 1, wherein the act of designing a continuous presence video image further comprises: receiving content from an endpoint corresponding to the first conferee; and presenting the content in the continuous presence video image, wherein the act of positioning a first video image comprises: positioning the first video image on a first side of the content, and wherein the act of positioning a second video image comprises: positioning the second video image on a second side of the content. ### Claim 3 (depends on 2) The method of claim 2, further comprising: processing the first video image and the second video image such that the first conferee and the second conferee are presented looking toward the content. ### Claim 4 (depends on 1) The method of claim 1, wherein the act of designing a continuous presence video image further comprises: generating instructions for a first endpoint of the continuous presence videoconference for creating the continuous presence video image at the first endpoint responsive to the interaction between the first conferee and the second conferee; sending the instructions to the first endpoint of the continuous presence videoconference; and creating the continuous presence video image at the first endpoint responsive to the instructions. ### Claim 5 (depends on 1) The method of claim 1, wherein the act of designing a continuous presence video image further comprises: creating the continuous presence video image for the first endpoint responsive to the interaction between the first conferee and the second conferee; and sending the continuous presence video image to the first endpoint. ### Claim 6 (depends on 1) The method of claim 1, further comprising: designing a continuous presence video image for a second endpoint of the continuous presence videoconference automatically responsive to the interaction between the plurality of conferees, the continuous presence video image for the second endpoint different from the continuous presence video image for the first endpoint; and displaying the continuous presence video image for the second endpoint at the second endpoint of the continuous presence videoconference. ### Claim 7 (depends on 1) The method of claim 1, wherein the act of designing a continuous presence video image further comprises: analyzing a first video image corresponding to the first conferee of the plurality of conferees; and analyzing a second video image corresponding to the second conferee of the plurality of conferees; and wherein determining automatically an interaction between the first conferee and the second conferee comprises determining the interaction responsive to the act of analyzing a first video image and the act of analyzing a second video image. ### Claim 8 (depends on 7) The method of claim 7, further comprising: sending the continuous presence video image for the first endpoint to the first endpoint. ### Claim 9 (depends on 7) The method of claim 7, further comprising: generating instructions for creating the continuous presence video image for the first endpoint; and sending the instructions for creating the continuous presence video image for the first endpoint to the first endpoint. ### Claim 10 (depends on 7) The method of claim 7, wherein the act of designing a continuous presence video image further comprises: modifying the first video image responsive to the interaction between the first conferee and the second conferee. ### Claim 11 (depends on 10) The method of claim 10, wherein the act of modifying the first video image comprises: mirroring the first video image. ### Claim 12 (depends on 1) The method of claim 1, wherein determining automatically an interaction between a first conferee and a second conferee comprises: determining a dominant conferee of the plurality of conferees; and determining a non-dominant conferee of the plurality of conferees; and wherein positioning a first video image corresponding to the first conferee in the continuous presence video image comprises: positioning automatically a first video image corresponding to the dominant conferee in the continuous presence video image, and wherein positioning a second video image corresponding to the second conferee relative to the first video image comprises: positioning automatically a second video image corresponding to the non-dominant conferee relative to the first video image in the continuous presence video image. ### Claim 13 (depends on 1) The method of claim 1, wherein the act of designing a continuous presence video image further comprises: designing a first continuous presence video image automatically responsive to a first interaction between the plurality of conferees; and designing a second continuous presence video image automatically responsive to a second interaction between the plurality of conferees subsequent to the first interaction, and wherein the act of displaying the continuous presence video image comprises: displaying the first continuous presence video image; and displaying the second continuous presence video image subsequent to the first continuous presence video image. ### Claim 14 (depends on 1) The method of claim 1, wherein the act of designing a continuous presence video image further comprises: analyzing a first audio energy corresponding to the first conferee of the plurality of conferees; analyzing a second audio energy corresponding to the second conferee of the plurality of conferees; and wherein determining automatically an interaction between the first conferee and the second conferee comprises determining the interaction responsive to the act of analyzing a first audio energy and the act of analyzing a second audio energy. ### Claim 15 (depends on 1) The method of claim 1, wherein the act of designing a continuous presence video image further comprises: identifying a first relative location of a first region of interest within a first video image corresponding to the first conferee of the plurality of conferees; identifying a second relative location of a second region of interest within a second video image corresponding to the second conferee of the plurality of conferees; and wherein determining automatically an interaction between a first conferee and a second conferee comprises determining an interaction between the first conferee and the second conferee responsive to the first relative location of the first region of interest within the first video image and the second relative location of the second region of interest within the second video image. ### Claim 16 (depends on 1) The method of claim 1, wherein positioning a second video image corresponding to the second conferee relative to the first video image in the continuous presence video image comprises: positioning a first video image corresponding to a first conferee relative to a second video image corresponding to a second conferee so that the first conferee and the second conferee appear to be facing each other. ### Claim 17 (depends on 16) The method of claim 16, wherein the act of designing a continuous presence video image further comprises: mirroring the first video image. ### Claim 18 (depends on 16) The method of claim 16, further comprising: designating a conferee of the plurality of conferees at a first site as an active conferee; copying an image of the active conferee from a video image received from the first site; and presenting the image of the active conferee and the video image received from the first site in the continuous presence video image. ### Claim 19 (depends on 16) The method of claim 16, further comprising: designating a conferee of the plurality of conferees at a first site as an active conferee; copying an image of the active conferee from a video image received from the first site; and presenting the image of the active conferee in the continuous presence video image instead of the video image received from the first site. ### Claim 20 A videoconferencing system, comprising: a plurality of endpoints, comprising: a first endpoint, corresponding to a first conferee; and a second endpoint, corresponding to a second conferee; and a layout controller adapted to design a continuous presence video image responsive to an interaction between the first conferee and the second conferee, wherein the layout controller is adapted to: determine automatically an interaction between the first conferee and the second conferee; position a first video image corresponding to the first conferee in the continuous presence video image; and position a second video image corresponding to the second conferee relative to the first video image in the continuous presence video image, responsive to the interaction between the first conferee and the second conferee. ### Claim 21 (depends on 20) The videoconferencing system of claim 20, wherein the layout controller is a module of a multipoint control unit (MCU). ### Claim 22 (depends on 20) The videoconferencing system of claim 20, further comprising: an editor module adapted to manipulate a first video image received from the first endpoint and a second video image responsive to the layout controller. ### Claim 23 (depends on 20) The videoconferencing system of claim 20, wherein the layout controller is further adapted to send instructions for displaying the continuous presence video image designed by the layout controller to the plurality of endpoints. ### Claim 24 (depends on 20) The videoconferencing system of claim 20, further comprising: a video module adapted to encode the continuous presence video image responsive to the layout controller. ### Claim 25 (depends on 20) The videoconferencing system of claim 20, further comprising: a video layout editor module, adapted to position in the continuous presence video image a first video image corresponding to the first conferee and a second video image corresponding to the second conferee relative to the first video image, responsive to the layout controller. ### Claim 26 (depends on 25) The videoconferencing system of claim 25, wherein the video layout editor module is further adapted to manipulate the first video image. ### Claim 27 (depends on 25) The videoconferencing system of claim 25, wherein the video layout editor module is further adapted to mirror the first video image. ### Claim 28 (depends on 20) The videoconferencing system of claim 20, further comprising: an interaction decoder, adapted to detect the interaction between the first conferee and the second conferee and to inform the layout controller of the interaction. ### Claim 29 (depends on 28) The videoconferencing system of claim 28, wherein the interaction decoder comprises a video interaction decoder adapted to identify a relative location of a region of interest in a first video image corresponding to the first conferee. ### Claim 30 (depends on 28) The videoconferencing system of claim 28, wherein the interaction decoder is adapted to detect that the first conferee is a dominant conferee. ### Claim 31 (depends on 28) The videoconferencing system of claim 28, wherein the interaction decoder comprises an audio interaction decoder adapted to analyze audio energy and to inform the layout controller of the interaction responsive to the audio energy. ### Claim 32 (depends on 20) The videoconferencing system of claim 20, wherein the layout controller is adapted to design the continuous presence video image so that the first conferee and the second conferee appear to be facing each other. ### Claim 33 An apparatus for processing a first video image from a first endpoint corresponding to a first conferee in a continuous presence video conference and a second video image from a second endpoint corresponding to a second conferee in the continuous presence video conference, comprising: a control module adapted to design a continuous presence video conference video image responsive to an interaction between the first conferee and the second conferee, wherein the control module is adapted to: determine automatically an interaction between the first conferee and the second conferee; position a first video image corresponding to the first conferee in the continuous presence video image; and position a second video image corresponding to the second conferee relative to the first video image in the continuous presence video image, responsive to the interaction between the first conferee and the second conferee; and a first video module adapted to create the continuous presence video conference video image designed by the control module. ### Claim 34 (depends on 33) The apparatus of claim 33, further comprising: an interaction decoder, adapted to detect the interaction between the first conferee and the second conferee and to inform the control module of the interaction. ### Claim 35 (depends on 34) The apparatus of claim 34, wherein the interaction decoder is a video interaction decoder, adapted to identify a relative location of a region of interest in the first video image. ### Claim 36 (depends on 34) The apparatus of claim 34, wherein the interaction decoder is an audio interaction decoder, adapted to analyze a first audio energy corresponding to the first conferee and a second audio energy corresponding to the second conferee. ### Claim 37 (depends on 34) The apparatus of claim 34, wherein the interaction decoder is adapted to detect that the first conferee is a dominant conferee. ### Claim 38 (depends on 33) The apparatus of claim 33, wherein the apparatus is a multipoint control unit. ### Claim 39 (depends on 33) The apparatus of claim 33, wherein the first video module is associated with the first endpoint. ### Claim 40 (depends on 33) The apparatus of claim 33, further comprising: an editor module adapted to manipulate the first video image responsive to the control module. ### Claim 41 (depends on 40) The apparatus of claim 40, wherein the editor module is adapted to mirror the first video image responsive to the control module.