# US 6,192,395 B1 · System and method for visually identifying speaking participants in a multi-participant networked event

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 6,192,395 B1
- Document ID: US6192395B1
- Status: Granted
- Kind: B1
- Application number: 09/465,717
- Filed: 1999-12-17
- Granted: 2001-02-20
- Earliest claimed: 1998-12-23
- Assignees: Multitude Inc.
- Inventors: Edward A. Lerner; Arthur W. Min; James E. G. Morris
- Web page: https://patentgrep.com/patent/US6192395B1

## CPC

- H04M 3/56: Arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities (video conference systems H04N7/15)
- H04M 3/493: Interactive information services, e.g. directory enquiries {; Arrangements therefor, e.g. interactive voice response [IVR] systems or voice portals}
- H04M 7/006: {Networks other than PSTN/ISDN providing telephone service, e.g. Voice over Internet Protocol (VoIP) , including next generation networks with a packet-switched transport layer (H04L65/00 takes precedence; aspects not specific to the type of network H04M3/00; special services in those networks H04M3/42)}
- H04M 3/42059: {Making use of the calling party identifier}
- H04M 3/42365: {Presence services providing information on the willingness to communicate or the ability to communicate in terms of media capability or network connectivity}
- H04M 3/563: {User guidance or feature selection}
- H04M 2201/38: Displays
- H04M 2201/42: Graphical user interfaces
- H04M 2203/1066: Game playing
- H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08)

## Abstract

A method of visually identifying speaking participants in a multi-participant event such as an audio conference or an on-line game includes the step of receiving packets of digitized sound from a network connection. The identity of the participant associated with each packet is used to route the packet to a channel buffer or an overflow buffer. Each channel buffer may be assigned to a single participant in the multi-participant. A visual identifier module updates the visual identifier associated with participants that have been assigned a channel buffer. In some embodiments, the appearance of the visual identifier associated with the participant is dependent upon the differential of an acoustic parameter derived from content in the associated buffer channel and a reference value stored in a participant record.

## Claims

### Claim 1

A computer product for use in conjunction with a computer system, the computer program product comprising a computer readable storage medium and a computer program mechanism embedded therein, the computer program mechanism comprising:
(i) a participant data structure comprising a plurality of participant records, each participant record associated with a different participant in a multi-participant event;
(ii) an application module for providing a user interface to said multi-participant event;
(iii) a sound control module for receiving a plurality of packets from a network connection, each said packet associated with a participant in said multi-participant event and including digitized speech from said participant, said sound controller comprising:
a plurality of buffers, each buffer including instructions for managing a subset of said packets;
a packet controller that includes instructions for determining said participant associated with said packet and instructions for routing said packet to a buffer, wherein when said packet is routed to said buffer, said packet is managed by said buffer; and
a visual identification module that includes instructions for visually identifying said participant in said multi-participant event and a characteristic associated with said participant; and
(iv) a sound mixer that includes instructions for mixing digitized speech from at least one of said buffers to produce a signal that is presented to an output device.

### Claim 2 (depends on 1)

The computer product of claim 1 wherein said participant record further includes a participant source identifier, a visual identifier associated with said participant, a position of said visual identifier on an output device, and a visual identifier state.

### Claim 3 (depends on 2)

The computer product of claim 2 wherein said visual identification module further includes instructions for updating said visual identifier state in a participant record associated with said participant, wherein:
when said participant is speaking in said multi-participant event, said visual identifier is set to a first state; and
when said participant is not speaking in said multi-participant event, said visual identifier is set to a second state.

### Claim 4 (depends on 3)

The computer product of claim 3 wherein said application module includes instructions for displaying said visual identifier on an output device based upon said characteristic associated with said participant.

### Claim 5 (depends on 2)

The computer product of claim 2 wherein said application module includes instructions for displaying said visual identifier on an output device, wherein:
when said participant associated with said visual identifier is speaking in said multi-participant event said unique visual identifier is displayed in a first state; and
when said participant associated with said visual identifier is not speaking in said multi-participant event said visual identifier is displayed in a second state.

### Claim 6 (depends on 2)

The computer product of claim 2 wherein:
(i) said participant record further includes a reference speech amplitude associated with said participant; and
(ii) said visual identification module further includes:
instructions for determining a buffered speech amplitude based upon a characteristic of digitized speech in at least one packet, associated with said participant, that is managed by a buffer;
instructions for computing a speech amplitude differential based on said buffered speech amplitude and said reference speech amplitude;
instructions for updating said visual identifier associated with said participant based on said speech amplitude differential; and
instructions for storing said buffered speech amplitude as said reference speech amplitude in said participant record.

### Claim 7 (depends on 6)

The computer product of claim 6 wherein said application module includes instructions for displaying said visual identifier on an output device based on a function of said visual identifier state in said participant record; and
said instructions for updating said visual identifier associated with said participant based on said speech amplitude differential includes instructions for updating said visual identifier state based upon a value of said speech amplitude differential.

### Claim 8 (depends on 1)

The computer product of claim 1 wherein said multi-participant event is selected from the group consisting of an audio conference and an on-line game.

### Claim 9 (depends on 1)

The computer product of claim 1 wherein:
said plurality of buffers includes an overflow buffer and a plurality of channel buffers, wherein:
(i) when a packet is present in a channel buffer, said channel buffer is characterized by an identity of said packet; and
(ii) when no packet is present in a channel buffer said channel buffer is available; and
said instructions for routing said packet to said buffer includes instructions for comparing an identity of the participant associated with said packet with said identity that characterizes said channel buffer, wherein:
(a) when said identity of the participant associated with said packet matches said identity characterizing said channel buffer, said packet is routed to said channel buffer; and
(b) when said identity of the participant associated with said packet does not match said identity that characterizes a channel buffer, said packet is routed to an available channel buffer, and when no channel buffer in said plurality of channel buffers is available, said packet is routed to said overflow buffer.

### Claim 10 (depends on 1)

The computer product of claim 1 wherein said instructions for managing said subset of said packets is first in first out.

### Claim 11 (depends on 2)

The computer product of claim 2 wherein said participant source identifier is a temporary unique number assigned to said participant for the duration of said multi-participant event.

### Claim 12 (depends on 2)

The computer product of claim 2 wherein said packet comprises a packet header and a formatted payload and said formatted payload includes said participant source identifier, a packet data size, and said digitized speech from said participant.

### Claim 13 (depends on 9)

The computer product of claim 9 wherein said sound mixer further includes:
instructions for retrieving a portion of said digitized speech from a first packet in each said channel buffer; and
instructions for combining each said portion of said digitized speech into said mixed digitized signal.

### Claim 14 (depends on 13)

The computer product of claim 13 wherein said portion of said digitized speech is ten milliseconds.

### Claim 15 (depends on 1)

The computer product of claim 1 wherein said characteristic associated with said participant is selected from the group consisting of (i) whether said participant is associated with a channel buffer, (ii) whether said participant has moderation privileges, (iii) whether said participant has placed said multi-participant event on hold and (iv) whether said participant has specified that he is away from the keyboard.

### Claim 16

A method for visually identifying speaking participants in a multi-participant event, said method comprising the steps of:
receiving a packet from a remote source;
determining an identity associated with said packet;
comparing said identity of said packet with an identity associated with a channel buffer selected from a plurality of channel buffers; wherein, said identity associated with said channel buffer is determined by an identity of a packet stored by said channel buffer when said channel buffer is storing a packet, and said channel buffer is available when no packet is stored by said channel buffer;
routing said packet to:
(i) a channel buffer when said identity of said packet matches said identity associated with said channel buffer;
(ii) an available channel buffer when said identity of said packet does not match an identity of a channel buffer; and
(iii) an overflow buffer when said identity of said packet does not match an identity of a channel buffer and there is no available channel buffer; and
associating a different visual identifier with each participant in said multi-participant event;
displaying each said different visual identifier on an output device; wherein said different visual identifier is determined by a characteristic associated with said participant.

### Claim 17 (depends on 16)

The method of claim 16 further comprising the step of:
updating a visual identifier state in a participant record, said visual identifier state determined by whether an identity of a participant corresponding to said participant record matches an identity associated with a channel buffer.

### Claim 18 (depends on 17)

The method of claim 17 wherein said updating step further comprises:
determining a difference between a characteristic of said packet and a reference characteristic stored in said participant record; and
setting said visual identifier state based upon said difference.

### Claim 19 (depends on 16)

The method of claim 16 wherein said multi-participant event includes a local participant and at least one remote participant, said method further comprising the steps of:
accepting a frame of sound from an input device;
deriving acoustic parameters from the content of said frame of sound;
performing an acoustic function using said acoustic parameters to determine whether said frame of sound includes speech from said local participant;
updating a visual identifier state in a participant record associated with said local participant, said visual identifier state determined by whether said frame of sound includes speech from said local participant.

### Claim 20 (depends on 16)

The method of claim 16 wherein said multi-participant event is selected from the group consisting of an audio conference and an on-line game.

### Claim 21 (depends on 16)

The method of claim 16 further including the step of assigning a temporary number to a participant for the duration of said multi-participant event; wherein said temporary number provides an identity to said participant.

### Claim 22 (depends on 16)

The method of claim 16 further including the steps of:
mixing sound from each channel buffer; and
presenting said mixed sound to an output device.

### Claim 23 (depends on 22)

The method of claim 22 wherein said mixing step further includes the steps of:
retrieving a portion of digitized speech from a first packet in each said channel buffer; and
combining each said portion of said digitized speech into a mixed digitized signal.

### Claim 24 (depends on 23)

The method of claim 23 wherein said portion of said digitized speech is ten milliseconds.

### Claim 25 (depends on 16)

The method of claim 16 wherein said characteristic associated with said participant is selected from the group consisting of (i) whether said participant is associated with a channel buffer, (ii) whether said participant has moderation privileges, (iii) whether said participant has placed said multi-participant event on hold and (iv) whether said participant has specified that he is away from the keyboard.