# US 10,372,298 B2 · User interface for multi-user communication session

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,372,298 B2
- Document ID: US10372298B2
- Status: Granted
- Kind: B2
- Application number: 16/035,422
- Filed: 2018-07-13
- Published: 2019-04-04
- Granted: 2019-08-06
- Earliest claimed: 2017-09-29
- Assignees: Apple Inc.
- Applicants: Apple Inc.
- Inventors: Freddy Allen Anzures; Nicholas V. King; Stephen O. Lemay; Hoan Pham; Giancarlo Yerkes
- Web page: https://patentgrep.com/patent/US10372298B2

## CPC

- G06F 3/0482: Interaction with lists of selectable items, e.g. menus
- G06F 3/0488: using a touch-screen or digitiser, e.g. input of commands through traced gestures
- H04L 51/04: Real-time or near real-time messaging, e.g. instant messaging [IM]
- H04L 51/10: Multimedia information
- H04M 1/7243: with interactive means for internal management of messages
- H04N 7/147: {Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals (selecting H04Q)}
- H04N 7/15: Conference systems
- H04M 1/27457: Management thereof, e.g. manual editing of data
- H04M 1/27475: using interactive graphical means or pictorial representations
- H04M 1/72433: for voice messaging, e.g. dictaphones (for answering incoming calls H04M1/64)
- H04M 1/72439: for image or video messaging
- H04M 2250/62: user interface aspects of conference calls
- H04N 2007/145: {Handheld terminals}
- H04S 7/30: {Control circuits for electronic adaptation of the sound field}
- H04S 2400/11: Positioning of individual sound objects, e.g. moving airplane, within a sound field (H04S2420/13 takes precedence)
- H04S 2420/01: Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04S: STEREOPHONIC SYSTEMS

## Abstract

The present disclosure generally relates to user interfaces for multi-user communication sessions. In some examples, a device initiates a live stream in a communication session. In some examples, a device transitions between streaming live audio and live video. In some examples, a device enables synchronizing media playback during a live stream.

## Claims

### Claim 1

An electronic device, comprising:
a display;
one or more camera sensors;
one or more microphones;
one or more processors; and
memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for:
receiving user input identifying one or more contacts to include as one or more participants in a communication session;
while in the communication session, concurrently displaying, on the display:
a first affordance for transmitting a live media stream that includes live audio and does not include live video, and
a second affordance for transmitting a live media stream that includes live audio and live video;
receiving user input activating one of the first affordance and the second affordance; and
in response to receiving user input activating one of the first affordance and the second affordance:
in accordance with receiving user input activating the first affordance, concurrently:
detecting, using the one or more microphones, audio; and
transmitting live audio in a live media stream to the one or more participants of the communication session, wherein transmitting the live audio occurs without transmission of live video;
in accordance with receiving user input activating the second affordance, concurrently:
detecting, using the one or more microphones, audio;
detecting, using the one or more camera sensors, a plurality of images for a video; and
transmitting the live audio and the live video in a live media stream to the one or more participants of the communication session; and
displaying one or more respective visual indicators for one or more participants of the communication session, wherein a characteristic of a respective visual indicator is indicative of whether a respective participant is currently transmitting a live media stream to participants of the communication session, and
wherein the characteristic of the respective visual indicator is a size of the respective visual indicator and the respective visual indicator of the respective participant varies in accordance with a volume of audio received from the respective participant.

### Claim 2 (depends on 1)

The electronic device of claim 1, the one or more programs further including instructions for:
receiving user input for initiating a live streaming session,
wherein concurrently displaying the first and second affordances is in response to receiving the user input for initiating the live streaming session.

### Claim 3 (depends on 2)

The electronic device of claim 2, the one or more programs further including instructions for:
in response to receiving user input activating one of the first affordance and the second affordance, initiating display of a countdown for initiating the live streaming session,
wherein transmitting the audio or video in the live media stream to the one or more participants of the communication session occurs subsequent to completion of the displayed countdown.

### Claim 4 (depends on 1)

The electronic device of claim 1, the one or more programs further including instructions for:
in accordance with receiving the user input activating the first affordance:
displaying, in a transcript area of the communication session, an indication that live audio streaming has begun; and
in accordance with receiving the user input activating the second affordance:
displaying, in the transcript area of the communication session, an indication that live video streaming has begun.

### Claim 5 (depends on 1)

The electronic device of claim 1, the one or more programs further including instructions for:
in response to receiving the user input identifying one or more contacts, transmitting requests to the one or more contacts to join the communication session.

### Claim 6 (depends on 1)

The electronic device of claim 1, the one or more programs further including instructions for:
receiving a request to transmit media to participants of the communication session; and
in response to receiving the request to transmit media to participants of the communication session:
in accordance with a determination that the communication session does not currently include transmitting a live media stream, transmitting a first type of notification to one or more of the first participants without transmitting a second type of notification to the one or more of the first participants; and
in accordance with a determination that the communication session does currently include transmitting a live media stream, transmitting the second type of notification to one or more of the first participants without transmitting the first type of notification to the one or more of the first participants.

### Claim 7 (depends on 1)

The electronic device of claim 1, the one or more programs further including instructions for:
displaying one or more respective avatars for one or more of the participants of the communication session.

### Claim 8 (depends on 7)

The electronic device of claim 7, the one or more programs further including instructions for:
in response to detecting the first gesture, transitioning display of one or more avatars of one or more participants of the communication session by concurrently:
reducing sizes of the displayed one or more avatars of the one or more participants;
changing shapes of the displayed avatars of the one or more participants; and
changing locations of the displayed avatars of the one or more participants.

### Claim 9 (depends on 1)

The electronic device of claim 1, the one or more programs further including instructions for:
displaying one or more respective status indicators for one or more of the participants of the communication session, wherein respective status indicators include respective indications of whether the respective participant is currently transmitting a live media stream to participants of the communication session.

### Claim 10 (depends on 9)

The electronic device of claim 9, the one or more programs further including instructions for:
displaying respective avatars of participants of the communication session,
wherein a respective status indicator of the respective participant includes the respective visual indicator around a respective avatar of the respective participant.

### Claim 11 (depends on 1)

The electronic device of claim 1, the one or more programs further including instructions for:
while in the communication session and not displaying a keyboard on the display; detecting a first gesture; and
in response to detecting the first gesture, displaying a keyboard.

### Claim 12 (depends on 1)

The electronic device of claim 1, the one or more programs further including instructions for:
subsequent to transitioning display of one or more avatars of one or more participants in response to the first gesture, detecting a second gesture; and
in response to detecting the second gesture, transitioning display of the one or more avatars of the one or more participants by concurrently:
enlarging sizes of the displayed one or more avatars of the one or more participants;
changing shapes of the displayed avatars of the one or more participants; and
changing locations of the displayed avatars of the one or more participants.

### Claim 13 (depends on 1)

The electronic device of claim 1, the one or more programs further including instructions for:
detecting a third gesture at a location corresponding to a displayed avatar of a participant of the communication session; and
in response to detecting the third gesture, enlarging the respective avatar of the participant.

### Claim 14 (depends on 1)

The electronic device of claim 1, the one or more programs further including instructions for:
detecting a user input for enabling captions; and
in response to detecting the user input for enabling captions, displaying captions of audio feeds of one or more participants of the communication session.

### Claim 15

A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device with a display, one or more camera sensors, and one or more microphones, the one or more programs including instructions for:
receiving user input identifying one or more contacts to include as one or more participants in a communication session;
while in the communication session, concurrently displaying, on the display:
a first affordance for transmitting a live media stream that includes live audio and does not include live video, and
a second affordance for transmitting a live media stream that includes living audio and live video;
receiving user input activating one of the first affordance and the second affordance; and
in response to receiving user input activating one of the first affordance and the second affordance:
in accordance with receiving user input activating the first affordance, concurrently:
detecting, using the one or more microphones, audio; and
transmitting live audio in a live media stream to the one or more participants of the communication session, wherein transmitting the live audio occurs without transmission of live video;
in accordance with receiving user input activating the second affordance, concurrently:
detecting, using the one or more microphones, audio;
detecting, using the one or more camera sensors, a plurality of images for a video; and
transmitting the live audio and the live video in a live media stream to the one or more participants of the communication session; and
displaying one or more respective visual indicators for one or more participants of the communication session, wherein a characteristic of a respective visual indicator is indicative of whether a respective participant is currently transmitting a live media stream to participants of the communication session, and
wherein the characteristic of the respective visual indicator is a size of the respective visual indicator and the respective visual indicator of the respective participant varies in accordance with a volume of audio received from the respective participant.

### Claim 16 (depends on 15)

The non-transitory computer-readable storage medium of claim 15, the one or more programs further including instructions for:
receiving user input for initiating a live streaming session,
wherein concurrently displaying the first and second affordances is in response to receiving the user input for initiating the live streaming session.

### Claim 17 (depends on 16)

The non-transitory computer-readable storage medium of claim 16, the one or more programs further including instructions for:
in response to receiving user input activating one of the first affordance and the second affordance, initiating display of a countdown for initiating the live streaming session,
wherein transmitting the audio or video in the live media stream to the one or more participants of the communication session occurs subsequent to completion of the displayed countdown.

### Claim 18 (depends on 15)

The non-transitory computer-readable storage medium of claim 15, the one or more programs further including instructions for:
in accordance with receiving the user input activating the first affordance:
displaying, in a transcript area of the communication session, an indication that live audio streaming has begun; and
in accordance with receiving the user input activating the second affordance:
displaying, in the transcript area of the communication session, an indication that live video streaming has begun.

### Claim 19 (depends on 15)

The non-transitory computer-readable storage medium of claim 15, the one or more programs further including instructions for:
in response to receiving the user input identifying one or more contacts, transmitting requests to the one or more contacts to join the communication session.

### Claim 20 (depends on 15)

The non-transitory computer-readable storage medium of claim 15, the one or more programs further including instructions for:
receiving a request to transmit media to participants of the communication session; and
in response to receiving the request to transmit media to participants of the communication session:
in accordance with a determination that the communication session does not currently include transmitting a live media stream, transmitting a first type of notification to one or more of the first participants without transmitting a second type of notification to the one or more of the first participants; and
in accordance with a determination that the communication session does currently include transmitting a live media stream, transmitting the second type of notification to one or more of the first participants without transmitting the first type of notification to the one or more of the first participants.

### Claim 21 (depends on 15)

The non-transitory computer-readable storage medium of claim 15, the one or more programs further including instructions for:
displaying one or more respective avatars for one or more of the participants of the communication session.

### Claim 22 (depends on 21)

The non-transitory computer-readable storage medium of claim 21, the one or more programs further including instructions for:
in response to detecting the first gesture, transitioning display of one or more avatars of one or more participants of the communication session by concurrently:
reducing sizes of the displayed one or more avatars of the one or more participants;
changing shapes of the displayed avatars of the one or more participants; and
changing locations of the displayed avatars of the one or more participants.

### Claim 23 (depends on 15)

The non-transitory computer-readable storage medium of claim 15, the one or more programs further including instructions for:
displaying one or more respective status indicators for one or more of the participants of the communication session, wherein respective status indicators include respective indications of whether the respective participant is currently transmitting a live media stream to participants of the communication session.

### Claim 24 (depends on 23)

The non-transitory computer-readable storage medium of claim 23, the one or more programs further including instructions for:
displaying respective avatars of participants of the communication session,
wherein a respective status indicator of the respective participant includes the respective visual indicator around a respective avatar of the respective participant.

### Claim 25 (depends on 15)

The non-transitory computer-readable storage medium of claim 15, the one or more programs further including instructions for:
while in the communication session and not displaying a keyboard on the display, detecting a first gesture; and
in response to detecting the first gesture, displaying a keyboard.

### Claim 26 (depends on 15)

The non-transitory computer-readable storage medium of claim 15, the one or more programs further including instructions for:
subsequent to transitioning display of one or more avatars of one or more participants in response to the first gesture, detecting a second gesture; and
in response to detecting the second gesture, transitioning display of the one or more avatars of the one or more participants by concurrently:
enlarging sizes of the displayed one or more avatars of the one or more participants;
changing shapes of the displayed avatars of the one or more participants; and
changing locations of the displayed avatars of the one or more participants.

### Claim 27 (depends on 15)

The non-transitory computer-readable storage medium of claim 15, the one or more programs further including instructions for:
detecting a third gesture at a location corresponding to a displayed avatar of a participant of the communication session; and
in response to detecting the third gesture, enlarging the respective avatar of the participant.

### Claim 28 (depends on 15)

The non-transitory computer-readable storage medium of claim 15, the one or more programs further including instructions for:
detecting a user input for enabling captions; and
in response to detecting the user input for enabling captions, displaying captions of audio feeds of one or more participants of the communication session.

### Claim 29

A method, comprising:
at an electronic device with a display, one or more camera sensors, and one or more microphones:
receiving user input identifying one or more contacts to include as one or more participants in a communication session;
while in the communication session, concurrently displaying, on the display:
a first affordance for transmitting a live media stream that includes live audio and does not include live video, and
a second affordance for transmitting a live media stream that includes live audio and live video;
receiving user input activating one of the first affordance and the second affordance; and
in response to receiving user input activating one of the first affordance and the second affordance:
in accordance with receiving user input activating the first affordance, concurrently:
detecting, using the one or more microphones, audio; and
transmitting live audio in a live media stream to the one or more participants of the communication session, wherein transmitting the live audio occurs without transmission of live video;
in accordance with receiving user input activating the second affordance, concurrently:
detecting, using the one or more microphones, audio;
detecting, using the one or more camera sensors, a plurality of images for a video; and
transmitting the live audio and the live video in a live media stream to the one or more participants of the communication session; and
displaying one or more respective visual indicators for one or more participants of the communication session, wherein a characteristic of a respective visual indicator is indicative of whether a respective participant is currently transmitting a live media stream to participants of the communication session, and
wherein the characteristic of the respective visual indicator is a size of the respective visual indicator and the respective visual indicator of the respective participant varies in accordance with a volume of audio received from the respective participant.

### Claim 30 (depends on 29)

The method of claim 29, further comprising:
receiving user input for initiating a live streaming session,
wherein concurrently displaying the first and second affordances is in response to receiving the user input for initiating the live streaming session.

### Claim 31 (depends on 30)

The method of claim 30, further comprising:
in response to receiving user input activating one of the first affordance and the second affordance, initiating display of a countdown for initiating the live streaming session,
wherein transmitting the audio or video in the live media stream to the one or more participants of the communication session occurs subsequent to completion of the displayed countdown.

### Claim 32 (depends on 29)

The method of claim 29, further comprising:
in accordance with receiving the user input activating the first affordance:
displaying, in a transcript area of the communication session, an indication that live audio streaming has begun; and
in accordance with receiving the user input activating the second affordance:
displaying, in the transcript area of the communication session, an indication that live video streaming has begun.

### Claim 33 (depends on 29)

The method of claim 29, further comprising:
in response to receiving the user input identifying one or more contacts, transmitting requests to the one or more contacts to join the communication session.

### Claim 34 (depends on 29)

The method of claim 29, further comprising:
receiving a request to transmit media to participants of the communication session; and
in response to receiving the request to transmit media to participants of the communication session:
in accordance with a determination that the communication session does not currently include transmitting a live media stream, transmitting a first type of notification to one or more of the first participants without transmitting a second type of notification to the one or more of the first participants; and
in accordance with a determination that the communication session does currently include transmitting a live media stream, transmitting the second type of notification to one or more of the first participants without transmitting the first type of notification to the one or more of the first participants.

### Claim 35 (depends on 29)

The method of claim 29, further comprising:
displaying one or more respective avatars for one or more of the participants of the communication session.

### Claim 36 (depends on 35)

The method of claim 35, further comprising:
in response to detecting the first gesture, transitioning display of one or more avatars of one or more participants of the communication session by concurrently:
reducing sizes of the displayed one or more avatars of the one or more participants;
changing shapes of the displayed avatars of the one or more participants; and
changing locations of the displayed avatars of the one or more participants.

### Claim 37 (depends on 29)

The method of claim 29, further comprising:
displaying one or more respective status indicators for one or more of the participants of the communication session, wherein respective status indicators include respective indications of whether the respective participant is currently transmitting a live media stream to participants of the communication session.

### Claim 38 (depends on 37)

The method of claim 37, further comprising:
displaying respective avatars of participants of the communication session,
wherein a respective status indicator of the respective participant includes the respective visual indicator around a respective avatar of the respective participant.

### Claim 39 (depends on 29)

The method of claim 29, further comprising:
while in the communication session and not displaying a keyboard on the display, detecting a first gesture; and
in response to detecting the first gesture, displaying a keyboard.

### Claim 40 (depends on 29)

The method of claim 29, further comprising:
subsequent to transitioning display of one or more avatars of one or more participants in response to the first gesture, detecting a second gesture; and
in response to detecting the second gesture, transitioning display of the one or more avatars of the one or more participants by concurrently:
enlarging sizes of the displayed one or more avatars of the one or more participants;
changing shapes of the displayed avatars of the one or more participants; and
changing locations of the displayed avatars of the one or more participants.

### Claim 41 (depends on 29)

The method of claim 29, further comprising:
detecting a third gesture at a location corresponding to a displayed avatar of a participant of the communication session; and
in response to detecting the third gesture, enlarging the respective avatar of the participant.

### Claim 42 (depends on 29)

The method of claim 29, further comprising:
detecting a user input for enabling captions; and
in response to detecting the user input for enabling captions, displaying captions of audio feeds of one or more participants of the communication session.