# US 11,308,427 B2 · Event transcript presentation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,308,427 B2
- Document ID: US11308427B2
- Status: Granted
- Kind: B2
- Application number: 16/162,127
- Filed: 2018-10-16
- Published: 2020-04-02
- Granted: 2022-04-19
- Earliest claimed: 2018-09-28
- Assignees: Evernote Corporation
- Applicants: Evernote Corporation
- Inventors: Anirban Kundu; Andrew Henderson Malcolm; Amir Doron
- Web page: https://patentgrep.com/patent/US11308427B2

## CPC

- G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence)
- G06F 3/0484: for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F 3/04847: Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
- G06Q 10/063112: {Skill-based matching of a person or a group to a task}
- G06Q 10/063114: {Status monitoring or status determination for a person or group}
- G06Q 10/103: {Workflow collaboration or project management}
- G06Q 10/1093: Calendar-based scheduling for persons or groups
- G06F 16/248: Presentation of query results
- G10L 15/26: Speech to text systems (G10L15/08 takes precedence)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

A computer system receives data that includes indications of a plurality of agenda items that correspond to the group event. During the group event, the system captures, by a media recording device, media data, converts the captured media data to a searchable version of the media data and searches the searchable version of the media data to determine a first time frame of the media data that corresponds to a first agenda item of the plurality of agenda items. The system displays, in a user interface that is accessible to at least one user associated with the group event, the representation of the media data. A first portion of the representation of the media data that corresponds to the first time frame is visually distinguished from at least a portion of the representation of the media data that does not correspond to the first time frame.

## Claims

### Claim 1

A method for displaying a representation of media data that corresponds to a group event, the method comprising, at a computing device:
obtaining data that includes indications of a plurality of agenda items from an agenda that correspond to the group event; and
during the group event:
obtaining media data captured by a media recording device;
converting the captured media data to a searchable version of the media data;
determining non-consecutive time frames of the media data during which at least one of a first agenda item and a second distinct agenda item of the plurality of agenda items is discussed; and
enabling display of the representation of the media data in a user interface that is accessible to at least one user associated with the group event, wherein the representation comprises an audio waveform, including:
arranging a plurality of portions of the representation along a time axis, each portion representing a respective subset of the media data captured for a respective one of the plurality of agenda items during a respective time frame;
determining that a plurality of first non-contiguous portions of the representation of the media data discuss the first agenda item and that a second portion of the representation of the media data discusses the second agenda item, each of the first non-contiguous portions and second portion of the representation of the media data captured during a respective time frame of the non-consecutive time frames, wherein on the time axis, the second portion of the second agenda item separates two of the first non-contiguous portions of the first agenda item, wherein the plurality of portions, the plurality of first non-contiguous portions, and the second portion of the representation include respective portions of the audio waveform, wherein the non-consecutive time frames include a first time frame and a second time frame that is distinct from and non-consecutive to the first time frame of the two of the first non-contiguous portions;
determining a third time frame of the media data during which a second agenda item of the plurality of agenda items is discussed, the second agenda item being distinct from the first agenda item;
determining that the second portion of the representation of the media data, distinct from the plurality of first non-contiguous portions of the media data, discusses the second agenda item during the third time frame, wherein arranging the plurality of portions of the representation along the time axis including arranging the third time frame of the second portion of the representation between the first time frame and the second time frame and separating the two of the first non-contiguous portions;
detecting a user input action at any one of the first non-contiguous portions of the representation discussing the first agenda item; and
in response to detecting a user input action at any one of the first non-contiguous portions of the representation discussing the first agenda item:
visually emphasizing all of the plurality of the first non-contiguous portions of the representation of the media data captured during respective time frames of the non-consecutive time frames discussing the first agenda item including the first time frame and the second time frame;
displaying text describing the first agenda item with the plurality of the first non-contiguous portions of the representation of the media data; and
displaying the second portion of the representation of the media data that separates the two of the first non-contiguous portions without the visual emphasis of all of the plurality of the first non-contiguous portions of the representation of the media data.

### Claim 2 (depends on 1)

The method of claim 1, wherein the plurality of agenda items is automatically generated based on information stored by an organization data structure.

### Claim 3 (depends on 1)

The method of claim 1, wherein the plurality of agenda items is generated based on information from a plurality of agendas, the method further comprising:
determining, from the media data, a respective agenda of the plurality of agendas that includes to the first agenda item,
wherein displaying the representation of the media data includes displaying an indication of a source of the respective agenda of the plurality of agendas that includes the first agenda item, wherein the source is one or more of: a project within an organization, a calendar invitation, or a group event invitation.

### Claim 4 (depends on 1)

The method of claim 1, wherein the obtained data includes indications of a plurality of group event attendees, the method further comprising:
identifying a first attendee of the plurality of group event attendees, wherein the first agenda item is assigned to the first attendee and a task is assigned to the first attendee; and
determining that the first agenda item of the plurality of agenda items is associated to the task based on the first agenda item and the task being assigned to the first attendee.

### Claim 5 (depends on 1)

The method of claim 1, wherein converting the captured media data to the searchable version of the media data includes performing speech-to-text processing and comparing text obtained from the speech-to-text processing with text that corresponds to the plurality of agenda items.

### Claim 6 (depends on 1)

The method of claim 1, wherein the representation comprises a set of frames of video, and one of the plurality of first non-contiguous portions of the representation of the media data includes a first frame of the set of frames of video, the first frame being visually emphasized relative to other frames of the video.

### Claim 7 (depends on 1)

The method of claim 1, wherein the media data includes data recorded at a plurality of personal devices of respective attendees of the group event.

### Claim 8 (depends on 1)

The method of claim 1, wherein enabling display of the representation of the media data in a user interface further includes:
providing the representation of the media data and the plurality of agenda items for simultaneous display, including displaying one or more markers to visually distinguish portions of the representation that correspond to the displayed agenda items.

### Claim 9 (depends on 1)

The method of claim 1, wherein received data includes indications of a plurality of group event attendees, the method further comprising:
identifying a first attendee, of the plurality of group event attendees, that is associated with a project, wherein the first agenda item is assigned to the first attendee; and
determining that the first agenda item of the plurality of agenda items is associated to the project based on: (i) an association of the first attendee with the project and (ii) the first agenda item being assigned to the first attendee.

### Claim 10

An electronic device, comprising:
one or more processors; and
memory storing one or more programs for execution by the one or more processors, the one or more programs including instructions for:
obtaining data that includes indications of a plurality of agenda items from an agenda that correspond to a group event; and
during the group event:
obtaining media data captured by a media recording device;
converting the captured media data to a searchable version of the media data;
determining non-consecutive time frames of the media data during which at least one of a first agenda item and a second distinct agenda item of the plurality of agenda items is discussed; and
enabling display of a representation of the media data in a user interface that is accessible to at least one user associated with the group event, wherein the representation comprises an audio waveform, including:
arranging a plurality of portions of the representation along a time axis, each portion representing a respective subset of the media data captured for a respective one of the plurality of agenda items during a respective time frame;
determining that a plurality of first non-contiguous portions of the representation of the media data discuss the first agenda item and that a second portion of the representation of the media data discusses the second agenda item, each of the first non-contiguous portions and second portion of the representation of the media data captured during a respective time frame of the non-consecutive time frames, wherein on the time axis, the second portion of the second agenda item separates two of the first non-contiguous portions of the first agenda item, wherein the plurality of portions, the plurality of first non-contiguous portions, and the second portion of the representation include respective portions of the audio waveform, wherein the non-consecutive time frames include a first time frame and a second time frame that is distinct from and non-consecutive to the first time frame of the two of the first non-contiguous portions;
determining a third time frame of the media data during which a second agenda item of the plurality of agenda items is discussed, the second agenda item being distinct from the first agenda item;
determining that the second portion of the representation of the media data, distinct from the plurality of first non-contiguous portions of the media data, discusses the second agenda item during the third time frame, wherein arranging the plurality of portions of the representation along the time axis including arranging the third time frame of the second portion of the representation between the first time frame and the second time frame and separating the two of the first non-contiguous portions;
detecting a user input action at any one of the first non-contiguous portions of the representation discussing the first agenda item; and
in response to detecting a user input action at any one of the first non-contiguous portions of the representation discussing the first agenda item:
visually emphasizing all of the plurality of the first non-contiguous portions of the representation of the media data captured during respective time frames of the non-consecutive time frames discussing the first agenda item including the first time frame and the second time frame;
displaying text describing the first agenda item with the plurality of the first non-contiguous portions of the representation of the media data; and
displaying the second portion of the representation of the media data that separates the two of the first non-contiguous portions without the visual emphasis of all of the plurality of the first non-contiguous portions of the representation of the media data.

### Claim 11 (depends on 10)

The electronic device of claim 10, wherein information corresponding to a speaker at the group event is displayed at a location in the user interface that corresponds to a portion of the representation associated with the speaker.

### Claim 12 (depends on 10)

The electronic device of claim 10, wherein information identifying at least one person associated with the first agenda item is displayed at a location in the user interface that corresponds to the first non-contiguous portions of the representation.

### Claim 13 (depends on 10)

The electronic device of claim 10, wherein the representation of the media data is updated as the media data is captured.

### Claim 14 (depends on 10)

The electronic device of claim 10, wherein the obtained data includes indications of a plurality of group event attendees, the one or more programs further comprising instructions for:
identifying a first attendee of the plurality of group event attendees, wherein the first agenda item is assigned to the first attendee and a task is assigned to the first attendee; and
determining that the first agenda item of the plurality of agenda items is associated to the task based on the first agenda item and the task being assigned to the first attendee.

### Claim 15 (depends on 10)

The electronic device of claim 10, wherein converting the captured media data to the searchable version of the media data includes performing speech-to-text processing and comparing text obtained from the speech-to-text processing with text that corresponds to the plurality of agenda items.

### Claim 16

A non-transitory computer-readable storage medium storing one or more programs for execution by one or more processors of an electronic device, the one or more programs comprising instructions for:
obtaining data that includes indications of a plurality of agenda items from an agenda that correspond to a group event; and
during the group event:
obtaining media data captured by a media recording device;
converting the captured media data to a searchable version of the media data;
determining non-consecutive time frames of the media data during which at least one of a first agenda item and a second distinct agenda item of the plurality of agenda items is discussed; and
enabling display of a representation of the media data in a user interface that is accessible to at least one user associated with the group event, wherein the representation comprises an audio waveform, including:
arranging a plurality of portions of the representation along a time axis, each portion representing a respective subset of the media data captured for a respective one of the plurality of agenda items during a respective time frame;
determining that a plurality of first non-contiguous portions of the representation of the media data discuss the first agenda item and that a second portion of the representation of the media data discusses the second agenda item, each of the first non-contiguous portions and second portion of the representation of the media data captured during a respective time frame of the non-consecutive time frames, wherein on the time axis, the second portion of the second agenda item separates two of the first non-contiguous portions of the first agenda item, wherein the plurality of portions, the plurality of first non-contiguous portions, and the second portion of the representation include respective portions of the audio waveform, wherein the non-consecutive time frames include a first time frame and a second time frame that is distinct from and non-consecutive to the first time frame of the two of the first non-contiguous portions;
determining a third time frame of the media data during which a second agenda item of the plurality of agenda items is discussed, the second agenda item being distinct from the first agenda item;
determining that the second portion of the representation of the media data, distinct from the plurality of first non-contiguous portions of the media data, discusses the second agenda item during the third time frame, wherein arranging the plurality of portions of the representation along the time axis including arranging the third time frame of the second portion of the representation between the first time frame and the second time frame and separating the two of the first non-contiguous portions;
detecting a user input action at any one of the first non-contiguous portions of the representation discussing the first agenda item; and
in response to detecting a user input action at any one of the first non-contiguous portions of the representation discussing the first agenda item:
visually emphasizing all of the plurality of the first non-contiguous portions of the representation of the media data captured during respective time frames of the non-consecutive time frames discussing the first agenda item including the first time frame and the second time frame;
displaying text describing the first agenda item with the plurality of the first non-contiguous portions of the representation of the media data; and
displaying the second portion of the representation of the media data that separates the two of the first non-contiguous portions without the visual emphasis of all of the plurality of the first non-contiguous portions of the representation of the media data.

### Claim 17 (depends on 16)

The non-transitory computer-readable storage medium of claim 16, the one or more programs further comprising instructions for, searching the searchable version of the media data to determine a second time frame of the media data that corresponds to a key term associated with the group event, wherein displaying the representation of the media data includes displaying a portion of the representation of the media data that corresponds to the second time frame.

### Claim 18 (depends on 16)

The non-transitory computer-readable storage medium of claim 16, the one or more programs further comprising instructions for, during the group event, displaying a prompt during the group event based on the captured media data and the plurality of agenda items, wherein the prompt includes any of:
an indication of a remaining time for the group event; and
information indicating agenda items of the plurality of agenda items that have not yet been captured.

### Claim 19 (depends on 16)

The non-transitory computer-readable storage medium of claim 16, the one or more programs further comprising instructions for, during the group event, displaying a prompt during the group event based on the captured media data and the plurality of agenda items, wherein the prompt is displayed in response to any of and conveys information related to:
a length of time of the group event satisfying a threshold; and
a determination of a threshold time remaining with respect to a scheduled amount of time for the group event.

### Claim 20 (depends on 16)

The non-transitory computer-readable storage medium of claim 16, wherein the plurality of agenda items is automatically generated based on information stored by an organization data structure.

### Claim 21 (depends on 16)

The non-transitory computer-readable storage medium of claim 16, wherein the plurality of agenda items is generated based on information from a plurality of agendas, the one or more programs further comprising instructions for:
determining, from the media data, a respective agenda of the plurality of agendas that includes to the first agenda item,
wherein displaying the representation of the media data includes displaying an indication of a source of the respective agenda of the plurality of agendas that includes the first agenda item, wherein the source is one or more of: a project within an organization, a calendar invitation, or a group event invitation.