# US 12,603,971 B2 · Providing awareness of who can hear audio in a virtual conference, and applications thereof > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,603,971 B2 - Document ID: US12603971B2 - Status: Granted - Kind: B2 - Application number: 18/218,413 - Filed: 2023-07-05 - Published: 2023-11-02 - Granted: 2026-04-14 - Earliest claimed: 2021-05-06 - Assignees: Katmai Tech Inc. - Applicants: Katmai Tech Inc. - Inventors: Gerard Cornelis Krol; Erik Stuart Braund - Web page: https://patentgrep.com/patent/US12603971B2 ## CPC - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - H04N 7/157: {defining a virtual conference space and using avatars or agents (computer conference optimisation or adaptation H04L12/1827)} - G06T 15/04: Texture mapping - G06T 15/20: Perspective computation - G10L 25/51: for comparison or discrimination - G06T 2219/004: Annotating, labelling - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract Disclosed herein is a computer-implemented method, system, device, and computer program product for providing awareness of who are able to hear audio in a virtual conference. For each of the users in the virtual conference, a device of the speaking user or a server determines whether a respective user is able to hear the speaking user based on whether a respective sound volume at which the respective user is able hear the speaking user exceeds a threshold amount. The speaking user's device or the server outputs a notification indicating the users that are able to hear the speaking user. ## Claims ### Claim 1 A computer-implemented method for providing awareness of who can hear audio in a virtual conference, comprising: (a) from a perspective of a virtual camera of a speaking user, rendering for display to the speaking user a three-dimensional virtual space including a plurality of avatars, wherein each user of a plurality of users is represented by a respective avatar of the plurality of avatars; for each user of the plurality of users: (b) determining a sound volume based on a position in the three-dimensional virtual space of the respective avatar relative to a position of the virtual camera in the three-dimensional virtual space; (c) determining whether a user is able to audibly hear the speaking user based on whether the sound volume exceeds a threshold amount, wherein the sound volume can be higher or lower than the threshold amount; and (d) outputting a single visual notification comprising a username identifying each user of a list of users for display to the speaking user, wherein the single visual notification of the list of users excludes users from the plurality of users that are unable to audibly hear the speaking user. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, wherein the single visual notification includes a total count of the list of users determined in (c) that are able to audibly hear the speaking user. ### Claim 3 (depends on 1) The computer-implemented method of claim 1, further comprising: (e) receiving position and direction information of at least one user of the plurality of users in the three-dimensional virtual space; (f) receiving a video stream captured from a camera of a device of the at least one user, the camera positioned to capture photographic images of the at least one user; (g) texture mapping the video stream onto a three-dimensional model of an avatar of the at least one user; and (h) rendering, from the perspective of the virtual camera of the speaking user, for display to the speaking user, the three-dimensional virtual space including the texture-mapped three-dimensional model of the avatar located at a position and oriented at a direction based on the position and direction information of the at least one user. ### Claim 4 (depends on 1) The computer-implemented method of claim 1, wherein an application downloaded on a web-browser renders the three-dimensional virtual space in (a). ### Claim 5 (depends on 1) The computer-implemented method of claim 1, wherein the three-dimensional virtual space is segmented into a plurality of areas and each area in the plurality of areas comprises a wall transmission factor and a distance transmission factor. ### Claim 6 (depends on 5) The computer-implemented method of claim 5, wherein the determining of the sound volume in (b) is based on one or more of: i) a distance between the virtual camera and the respective avatar, and the wall transmission factor of all areas between the virtual camera and the respective avatar; or ii) the distance transmission factor of all areas between the virtual camera and the respective avatar. ### Claim 7 (depends on 1) The computer-implemented method of claim 1, further comprising: (e) determining whether a new user is able to audibly hear the speaking user based on a new sound volume for the new user exceeding the threshold amount; and (f) adding information about the new user in the visual notification. ### Claim 8 (depends on 1) The computer-implemented method of claim 1, further comprising: (e) determining that at least one sound volume associated with at least one user in the plurality of users has dropped lower than the threshold amount; and (f) updating information about the at least one user from the visual notification. ### Claim 9 A non-transitory, tangible computer-readable device having instructions stored thereon that, when executed by at least one computing device, causes the at least one computing device to perform operations for providing awareness of who can hear audio in a virtual conference, comprising: (a) from a perspective of a virtual camera of a speaking user, rendering for display to the speaking user a three-dimensional virtual space, including a plurality of avatars, wherein each user of a plurality of users is represented by a respective avatar of the plurality of avatars; for each user of the plurality of users: (b) determining a sound volume based on a position in the three-dimensional virtual space of the respective avatar relative to a position of the virtual camera in the three-dimensional virtual space; (c) determining whether a user is able to audibly hear the speaking user based on whether the sound volume exceeds a threshold amount, wherein the sound volume can be higher or lower than the threshold amount; and (d) outputting a single visual notification comprising a username identifying each user of a list of users for display to the speaking user, wherein the single visual notification of the list of users excludes users from the plurality of users that are unable to audibly hear the speaking user. ### Claim 10 (depends on 9) The non-transitory, tangible computer-readable device of claim 9, wherein the single visual notification includes a total count of the list of users determined in (c) that are able to audibly hear the speaking user. ### Claim 11 (depends on 9) The non-transitory, tangible computer-readable device of claim 9, wherein the operations further comprise: (e) receiving position and direction information of at least one user of the plurality of users in the three-dimensional virtual space; (f) receiving a video stream captured from a camera of a device of the at least one user, the camera positioned to capture photographic images of the at least one user; (g) texture mapping the video stream onto a three-dimensional model of an avatar of the at least one user; and (h) rendering, from the perspective of the virtual camera of the speaking user, for display to the speaking user, the three-dimensional virtual space including the texture-mapped three-dimensional model of the avatar located at a position and oriented at a direction based on the position and direction information of the at least one user. ### Claim 12 (depends on 9) The non-transitory, tangible computer-readable device of claim 9, wherein an application downloaded on a web-browser renders the three-dimensional virtual space in (a). ### Claim 13 (depends on 9) The non-transitory, tangible computer-readable device of claim 9, wherein the three-dimensional virtual space is segmented into a plurality of areas and each area in the plurality of areas comprises a wall transmission factor and a distance transmission factor. ### Claim 14 (depends on 13) The non-transitory, tangible computer-readable device of claim 13, wherein the determining of the sound volume in (b) is based on one or more of: i) a distance between the virtual camera and the respective avatar, and the wall transmission factor of all areas between the virtual camera and the respective avatar; or ii) the distance transmission factor of all areas between the virtual camera and the respective avatar. ### Claim 15 (depends on 9) The non-transitory, tangible computer-readable device of claim 9, wherein the operations further comprise: (e) determining whether a new user is able to audibly hear the speaking user based on a new sound volume for the new user exceeding the threshold amount; and (f) adding information about the new user in the visual notification. ### Claim 16 (depends on 9) The non-transitory, tangible computer-readable device of claim 9, wherein the operations further comprise: (e) determining that at least one sound volume associated with at least one user in the plurality of users has dropped lower than the threshold amount; and (f) updating information about the at least one user from the single visual notification. ### Claim 17 A system for providing awareness of who can hear audio in a virtual conference, the system comprising: a memory; and a processor coupled to the memory, the processor configured to: (a) from a perspective of a virtual camera of a speaking user, render for display to the speaking user a three-dimensional virtual space, including a plurality of avatars, wherein each user of a plurality of users is represented by a respective avatar of the plurality of avatars; for each user of the plurality of users: (b) determine a sound volume based on a position in the three-dimensional virtual space of the respective avatar relative to a position of the virtual camera in the three-dimensional virtual space; (c) determine whether a user is able to audibly hear the speaking user based on whether the respective sound volume exceeds a threshold amount, wherein the sound volume can be higher or lower than the threshold amount; and (d) output a single visual notification comprising a username identifying each user of a list of users for display to the speaking user, wherein the single visual notification of the list of users excludes users from the plurality of users that are unable to audibly hear the speaking user. ### Claim 18 (depends on 17) The system of claim 17, wherein the processor is further configured to: (e) receive position and direction information of at least one user of the plurality of users in the three-dimensional virtual space; (f) receive a video stream captured from a camera of a device of the at least one user, the camera positioned to capture photographic images of the at least one user; (g) texture map the video stream onto a three-dimensional model of an avatar of the at least one user; and (h) render, from the perspective of the virtual camera of the speaking user, for display to the speaking user, the three-dimensional virtual space including the texture-mapped three-dimensional model of the avatar located at a position and oriented at a direction based on the position and direction information of the at least one user. ### Claim 19 (depends on 17) The system of claim 17, wherein an application downloaded on a web-browser renders the three-dimensional virtual space in (a). ### Claim 20 (depends on 17) The system of claim 17, wherein the three-dimensional virtual space is segmented into a plurality of areas and each area in the plurality of areas has a wall transmission factor and a distance transmission factor, and wherein the determining of the sound volume in (b) is based on one or more of: i) a distance between the virtual camera and the respective avatar, and the wall transmission factor of all areas between the virtual camera and the respective avatar; or ii) the distance transmission factor of all areas between the virtual camera and the respective avatar.